One of the most promising applications for the predictive capabilities of artificial intelligence (AI) is in digital healthcare, particularly in precision and personalized medicine. As AI algorithms gain traction in the healthcare and precision medicine fields, it is important to understand not only the strengths of machine learning, but also its potential weaknesses. new research A study led by Yale School of Medicine shows that AI algorithms used to predict patient outcomes lack generalizability. AI algorithms have only performed well in the specific clinical trials in which they were developed, and have not performed well in various clinical trials for schizophrenia treatment.
Using AI machine learning to predict whether a particular patient will respond to drug treatment is an important aspect of precision medicine. Antipsychotic drugs to treat schizophrenia are clinically sufficient for more than 50 percent of relapse patients and up to 20 to 30 percent of first-time patients, depending on the definition of clinical outcome, according to Yale University-led researchers. It is said to cause no reaction.
In the field of artificial intelligence, a key indicator of the robustness of machine learning algorithms is generalizability, or the ability of an AI model to perform with high accuracy on new data that the algorithm has never seen before. Ideally, the AI algorithms used to predict precision medicine treatment outcomes are robust. This new research is important because it examines the inner workings of AI algorithms at a mathematical level.
“The entire field (including the current authors) is hopeful that machine learning approaches can ultimately improve treatment allocation in healthcare. However, predictions that lack independent samples for validation are “A priori one should remain skeptical about the model’s results,” wrote corresponding author Dr. Adam Chekroed. forbes An experienced interdisciplinary team of 30 Under 30 2018: Consumer Technology honorees, adjunct assistant professor of psychiatry at Yale, co-founders of Spring Health, and notable co-authors in the fields of medicine, psychiatry, and data science and neurology. Science.
Yale researchers who participated in the study included John Crystal, MD, chair of the Yale Department of Psychiatry, Philip Corlett, PhD, associate professor of psychiatry, and Harlan Krumholtz, MD, SM professor and director of the Yale New Haven Hospital Center. is included. In collaboration with Alkomiet Hasan, MD, Professor and Chair of the Department of Psychiatry and Psychotherapy at Augsburg University, and in collaboration with Outcomes Research and Evaluation (CORE) researchers Matt Haurylenko, Ph.D., Laritsa Georgieva, Ph.D., and Hieronymus Rojo, Ph.D. conducted research.Nikolaos Kousleris, Professor and Professor of Precision Psychiatry, College London, Joseph Kambaits, MD, Professor of Biological Psychiatry, University of Cologne and Cologne University Hospital, Julia Bondar, Spring Health Data Scientist, Scientific Director, Laureate Brain Institute Co-Director Martin Paulus, MD
“Clinical responses to pharmacological interventions are heterogeneous and depend on many environmental factors, such as individual and family-related stress, substance abuse, homelessness, and social isolation to predict treatment outcome in schizophrenia. may be particularly beneficial,” the researchers wrote.
Schizophrenia is a severe chronic brain disease that affects an estimated 24 million people worldwide, according to the World Health Organization (WHO). The National Institute of Mental Health (NIMH) defines psychosis as “a collection of symptoms that affect the mind, resulting in some degree of loss of contact with reality.” According to the NIMH, psychotic symptoms of schizophrenia may include hallucinations, delusions, abnormal or illogical thinking, and abnormal body movements. Other symptoms of schizophrenia include disorganized speech, lack of emotion, withdrawal from others, high self-esteem, and paranoia, according to the Johns Hopkins University School of Medicine. Treatment for schizophrenia for which there is no cure may include antidepressants and mood stabilizers, cognitive therapy, behavioral therapy, training, support groups, and antipsychotic medications.
To understand exactly how well an AI machine learning model predicts outcomes for schizophrenia patients across independent clinical trials of antipsychotic drugs, the team will use initial training data and the current DSM. We evaluated the performance of the AI model based on data from an independent clinical trial on patients. Diagnosed with schizophrenia at the start of the study. Specifically, he conducted his five multicenter randomized controlled trials of the Yale University Open Data Access (YODA) project, which included over 1,500 patients from over 190 sites in North America, Europe, Africa, and Asia. Data from the study was used.
The researchers used an elastic net regression algorithm. This is because this type of penalized regression technique has been used successfully in other psychiatric studies to predict the outcome of psychiatric treatment. In statistics, elastic net regression combines variations of his two types of linear regression: Ridge (L2 regularization) and LASSO (least absolute shrinkage and selection operator – L1 regularization). In machine learning, elastic net regression is used to reduce overfitting and improve AI prediction accuracy. In mathematics and statistics, a linear regression model describes the relationship between a dependent variable (y), also called the response variable, and one or more independent variables (X), also called predictor variables.
Artificial Intelligence Essentials
The researchers hypothesize that there are three potential contributing factors to this lack of generalizability. These include differences in patient populations between trials, the amount and type of data, and the context-specific nature of patient outcomes.
Patient populations with the same diagnostic category may differ from study to study. For example, patients currently diagnosed with schizophrenia in the DSM-5 may be at different stages of the disease’s progression.
“If the data does not capture important information that differentiates patients, or if the dataset used to develop the model has a more limited scope of that information compared to the trial of interest, predictions will be inaccurate. “The precise amount and type of data, as well as data dependencies, can determine patient outcomes in context,” the researchers wrote.
AI machine learning requires large amounts of training data for the algorithm to “learn” features from the data. The amount of data can affect generalizability. Additionally, the specific type of data collected can impact the predictive quality of an AI algorithm. The study used patient information, including sociodemographics, biomarkers, and clinical data, but not psychosocial and social determinants of health.researchers cite another study was announced on lancet psychiatry 2016, by Professor Koutsouleris, MD otherThey also worked on this study and found that psychosocial and social determinants of health are useful in AI machine learning to predict treatment outcomes for first-episode psychosis.
Interestingly, the researchers do not recommend any type of genetic data or brain imaging data to improve AI accuracy. The exact cause of schizophrenia is unknown, but a family history of schizophrenia is a risk factor, and brain imaging studies have shown differences in the structure of the brain and central nervous system, according to the Mayo Clinic. .
“While some have suggested the use of neuroimaging or genetic data, there is currently little evidence to suggest that such data improves predictions. Furthermore, collecting these data will “This would create additional barriers to implementation,” the Yale-led researchers wrote.
Ultimately, the outcome of antipsychotic drugs in patients with schizophrenia “may be too context-dependent.” For example, there may be trial-level characteristics in treatment protocols or recruitment and participation criteria that influence patient outcomes.
“Our modeling scenario using antipsychotic treatment outcome prediction in schizophrenia shows that predictive models are weak and that good performance in one clinical situation is not a strong indicator of future patient performance. “This suggests that,” the researchers concluded.
Copyright © 2024 Kami Rosso. All rights reserved.