TY - JOUR
T1 - Multimodal Machine Learning Workflows for Prediction of Psychosis in Patients with Clinical High-Risk Syndromes and Recent-Onset Depression
AU - Koutsouleris, Nikolaos
AU - Dwyer, Dominic B.
AU - Degenhardt, Franziska
AU - Maj, Carlo
AU - Urquijo-Castro, Maria Fernanda
AU - Sanfelici, Rachele
AU - Popovic, David
AU - Oeztuerk, Oemer
AU - Haas, Shalaila S.
AU - Weiske, Johanna
AU - Ruef, Anne
AU - Kambeitz-Ilankovic, Lana
AU - Antonucci, Linda A.
AU - Neufang, Susanne
AU - Schmidt-Kraepelin, Christian
AU - Ruhrmann, Stephan
AU - Penzel, Nora
AU - Kambeitz, Joseph
AU - Haidl, Theresa K.
AU - Rosen, Marlene
AU - Chisholm, Katharine
AU - Riecher-Rössler, Anita
AU - Egloff, Laura
AU - Schmidt, André
AU - Andreou, Christina
AU - Hietala, Jarmo
AU - Schirmer, Timo
AU - Romer, Georg
AU - Walger, Petra
AU - Franscini, Maurizia
AU - Traber-Walker, Nina
AU - Schimmelmann, Benno G.
AU - Flückiger, Rahel
AU - Michel, Chantal
AU - Rössler, Wulf
AU - Borisov, Oleg
AU - Krawitz, Peter M.
AU - Heekeren, Karsten
AU - Buechler, Roman
AU - Pantelis, Christos
AU - Falkai, Peter
AU - Salokangas, Raimo K.R.
AU - Lencer, Rebekka
AU - Bertolino, Alessandro
AU - Borgwardt, Stefan
AU - Noethen, Markus
AU - Brambilla, Paolo
AU - Wood, Stephen J.
AU - Upthegrove, Rachel
AU - Schultze-Lutter, Frauke
AU - Theodoridou, Anastasia
AU - Meisenzahl, Eva
N1 - Publisher Copyright:
© 2020 American Medical Association. All rights reserved.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2020/12/2
Y1 - 2020/12/2
N2 - Importance: Diverse models have been developed to predict psychosis in patients with clinical high-risk (CHR) states. Whether prediction can be improved by efficiently combining clinical and biological models and by broadening the risk spectrum to young patients with depressive syndromes remains unclear. Objectives: To evaluate whether psychosis transition can be predicted in patients with CHR or recent-onset depression (ROD) using multimodal machine learning that optimally integrates clinical and neurocognitive data, structural magnetic resonance imaging (sMRI), and polygenic risk scores (PRS) for schizophrenia; to assess models' geographic generalizability; to test and integrate clinicians' predictions; and to maximize clinical utility by building a sequential prognostic system. Design, Setting, and Participants: This multisite, longitudinal prognostic study performed in 7 academic early recognition services in 5 European countries followed up patients with CHR syndromes or ROD and healthy volunteers. The referred sample of 167 patients with CHR syndromes and 167 with ROD was recruited from February 1, 2014, to May 31, 2017, of whom 26 (23 with CHR syndromes and 3 with ROD) developed psychosis. Patients with 18-month follow-up (n = 246) were used for model training and leave-one-site-out cross-validation. The remaining 88 patients with nontransition served as the validation of model specificity. Three hundred thirty-four healthy volunteers provided a normative sample for prognostic signature evaluation. Three independent Swiss projects contributed a further 45 cases with psychosis transition and 600 with nontransition for the external validation of clinical-neurocognitive, sMRI-based, and combined models. Data were analyzed from January 1, 2019, to March 31, 2020. Main Outcomes and Measures: Accuracy and generalizability of prognostic systems. Results: A total of 668 individuals (334 patients and 334 controls) were included in the analysis (mean [SD] age, 25.1 [5.8] years; 354 [53.0%] female and 314 [47.0%] male). Clinicians attained a balanced accuracy of 73.2% by effectively ruling out (specificity, 84.9%) but ineffectively ruling in (sensitivity, 61.5%) psychosis transition. In contrast, algorithms showed high sensitivity (76.0%-88.0%) but low specificity (53.5%-66.8%). A cybernetic risk calculator combining all algorithmic and human components predicted psychosis with a balanced accuracy of 85.5% (sensitivity, 84.6%; specificity, 86.4%). In comparison, an optimal prognostic workflow produced a balanced accuracy of 85.9% (sensitivity, 84.6%; specificity, 87.3%) at a much lower diagnostic burden by sequentially integrating clinical-neurocognitive, expert-based, PRS-based, and sMRI-based risk estimates as needed for the given patient. Findings were supported by good external validation results. Conclusions and Relevance: These findings suggest that psychosis transition can be predicted in a broader risk spectrum by sequentially integrating algorithms' and clinicians' risk estimates. For clinical translation, the proposed workflow should undergo large-scale international validation.
AB - Importance: Diverse models have been developed to predict psychosis in patients with clinical high-risk (CHR) states. Whether prediction can be improved by efficiently combining clinical and biological models and by broadening the risk spectrum to young patients with depressive syndromes remains unclear. Objectives: To evaluate whether psychosis transition can be predicted in patients with CHR or recent-onset depression (ROD) using multimodal machine learning that optimally integrates clinical and neurocognitive data, structural magnetic resonance imaging (sMRI), and polygenic risk scores (PRS) for schizophrenia; to assess models' geographic generalizability; to test and integrate clinicians' predictions; and to maximize clinical utility by building a sequential prognostic system. Design, Setting, and Participants: This multisite, longitudinal prognostic study performed in 7 academic early recognition services in 5 European countries followed up patients with CHR syndromes or ROD and healthy volunteers. The referred sample of 167 patients with CHR syndromes and 167 with ROD was recruited from February 1, 2014, to May 31, 2017, of whom 26 (23 with CHR syndromes and 3 with ROD) developed psychosis. Patients with 18-month follow-up (n = 246) were used for model training and leave-one-site-out cross-validation. The remaining 88 patients with nontransition served as the validation of model specificity. Three hundred thirty-four healthy volunteers provided a normative sample for prognostic signature evaluation. Three independent Swiss projects contributed a further 45 cases with psychosis transition and 600 with nontransition for the external validation of clinical-neurocognitive, sMRI-based, and combined models. Data were analyzed from January 1, 2019, to March 31, 2020. Main Outcomes and Measures: Accuracy and generalizability of prognostic systems. Results: A total of 668 individuals (334 patients and 334 controls) were included in the analysis (mean [SD] age, 25.1 [5.8] years; 354 [53.0%] female and 314 [47.0%] male). Clinicians attained a balanced accuracy of 73.2% by effectively ruling out (specificity, 84.9%) but ineffectively ruling in (sensitivity, 61.5%) psychosis transition. In contrast, algorithms showed high sensitivity (76.0%-88.0%) but low specificity (53.5%-66.8%). A cybernetic risk calculator combining all algorithmic and human components predicted psychosis with a balanced accuracy of 85.5% (sensitivity, 84.6%; specificity, 86.4%). In comparison, an optimal prognostic workflow produced a balanced accuracy of 85.9% (sensitivity, 84.6%; specificity, 87.3%) at a much lower diagnostic burden by sequentially integrating clinical-neurocognitive, expert-based, PRS-based, and sMRI-based risk estimates as needed for the given patient. Findings were supported by good external validation results. Conclusions and Relevance: These findings suggest that psychosis transition can be predicted in a broader risk spectrum by sequentially integrating algorithms' and clinicians' risk estimates. For clinical translation, the proposed workflow should undergo large-scale international validation.
UR - http://www.scopus.com/inward/record.url?scp=85097418926&partnerID=8YFLogxK
U2 - 10.1001/jamapsychiatry.2020.3604
DO - 10.1001/jamapsychiatry.2020.3604
M3 - Journal articles
C2 - 33263726
AN - SCOPUS:85097418926
SN - 2168-622X
JO - JAMA Psychiatry
JF - JAMA Psychiatry
ER -