Multivariable prediction of functional outcome after first-episode psychosis: a crossover validation approach in EUFEST and PSYSCAN
Margot I. E. Slot,Maria F. Urquijo Castro,Inge Winter - van Rossum,Hendrika H. van Hell,Dominic Dwyer,Paola Dazzan,Arija Maat,Lieuwe De Haan,Benedicto Crespo-Facorro,Birte Y. Glenthøj,Stephen M. Lawrie,Colm McDonald,Oliver Gruber,Thérèse van Amelsvoort,Celso Arango,Tilo Kircher,Barnaby Nelson,Silvana Galderisi,Mark Weiser,Gabriele Sachs,Matthias Kirschner,the PSYSCAN Consortium,London,Philip McGuire,Stefania Tognin,Paolo Fusar-Poli,Matthew Kempton,Alexis E. Cullen,Gemma Modinos,Kate Merritt,Andrea Mechelli,George Gifford,Natalia Petros,Mathilde Antoniades,Andrea De Micheli,Sandra Vieira,Tom Spencer,Zhaoying Yu,Dominic Oliver,Fiona Coutts,Emily Hird,Helen Baldwin,Utrecht,Rene Kahn,Erika van Hell,Inge Winter,Amsterdam,Lieuwe de Haan,Frederike Schirmbeck,Cantabria,Diana Tordesillas-Gutierrez,Esther Setien-Suero,Rosa Ayesa-Arriola,Paula Suarez-Pinilla,Victor Ortiz Garcia-de la foz,Copenhagen,Birte Glenthøj,Mikkel Erlang Sørensen,Bjørn H. Ebdrup,Jayachandra Mitta Raghava,Egill Rostrup,Edinburgh,Galway,Brian Hallahan,Dara M. Cannon,James McLoughlin,Martha Finnegan,Heidelberg,Anja Richter,Bernd Krämer,Maastricht,Thérèse van Amelsvoort,Bea Campforts,Machteld Marcelis,Claudia Vingerhoets,Madrid,Covadonga M. Díaz-Caneja,Miriam Ayora,Joost Janssen,Mara Parellada,Jessica Merchán-Naranjo,Roberto Rodríguez-Jiménez,Marina Díaz-Marsá,Marburg,Irina Falkenberg,Florian Bitsch,Jens Sommer,Melbourne,Patrick McGorry,Paul Amminger,Christos Pantelis,Meredith McHugh,Jessica Spark,Naples,Armida Mucci,Paola Bucci,Giuseppe Piegari,Daria Pietrafesa,Alessia Nicita,Sara Patriarca,Tel Hashomer,Linda Levi,Yoav Domany,Vienna,Matthäus Willeit,Marcena Lenczowska,Ullrich Sauerzopf,Ana Weidenauer,Julia Furtner,Daniela Prayer,Zurich,Anke Maatz,Achim Burrer,Philipp Stämpfli,Naemi Huber,Stefan Kaiser,Wolfram Kawohl,Sao Paulo,Rodrigo Bressan,André Zugman,Ary Gadelha,Graccielle Rodrigues da Cunha,Seoul,Jun Soo Kwon,Kang Ik Kevin Cho,Tae Young Lee,Minah Kim,Sun-Young Moon,Silvia Kyungjin Lho,Toronto,Romina Mizrahi,Michael Kiang,Cory Gerritsen,Margaret Maheandiran,Sarah Ahmed,Ivana Prce,Jenny Lepock,W. Wolfgang Fleischhacker,Philip McGuire,Nikolaos Koutsouleris,René S. Kahn
DOI: https://doi.org/10.1038/s41537-024-00505-w
2024-10-09
Schizophrenia
Abstract:Several multivariate prognostic models have been published to predict outcomes in patients with first episode psychosis (FEP), but it remains unclear whether those predictions generalize to independent populations. Using a subset of demographic and clinical baseline predictors, we aimed to develop and externally validate different models predicting functional outcome after a FEP in the context of a schizophrenia-spectrum disorder (FES), based on a previously published cross-validation and machine learning pipeline. A crossover validation approach was adopted in two large, international cohorts (EUFEST, n = 338, and the PSYSCAN FES cohort, n = 226). Scores on the Global Assessment of Functioning scale (GAF) at 12 month follow-up were dichotomized to differentiate between poor (GAF current < 65) and good outcome (GAF current ≥ 65). Pooled non-linear support vector machine (SVM) classifiers trained on the separate cohorts identified patients with a poor outcome with cross-validated balanced accuracies (BAC) of 65-66%, but BAC dropped substantially when the models were applied to patients from a different FES cohort (BAC = 50–56%). A leave-site-out analysis on the merged sample yielded better performance (BAC = 72%), highlighting the effect of combining data from different study designs to overcome calibration issues and improve model transportability. In conclusion, our results indicate that validation of prediction models in an independent sample is essential in assessing the true value of the model. Future external validation studies, as well as attempts to harmonize data collection across studies, are recommended.
psychiatry