AUTHOR=Gaubert Malo , Dell’Orco Andrea , Lange Catharina , Garnier-Crussard Antoine , Zimmermann Isabella , Dyrba Martin , Duering Marco , Ziegler Gabriel , Peters Oliver , Preis Lukas , Priller Josef , Spruth Eike Jakob , Schneider Anja , Fliessbach Klaus , Wiltfang Jens , Schott Björn H. , Maier Franziska , Glanz Wenzel , Buerger Katharina , Janowitz Daniel , Perneczky Robert , Rauchmann Boris-Stephan , Teipel Stefan , Kilimann Ingo , Laske Christoph , Munk Matthias H. , Spottke Annika , Roy Nina , Dobisch Laura , Ewers Michael , Dechent Peter , Haynes John Dylan , Scheffler Klaus , Düzel Emrah , Jessen Frank , Wirth Miranka , for the DELCODE study group , Holger Amthauer , Can Cetindag Arda , Carmen Cosma Nicoleta , Dominik Diesing , Marie Ehrlich , Frederike Fenski , Dawn Freiesleben Silka , Manuel Fuentes , Dietmar Hauser , Nicole Hujer , Irem Incesoy Enise , Christian Kainz , Catharina Lange , Katja Lindner , Herlind Megges , Oliver Peters , Lukas Preis , Slawek Altenstein , Andrea Lohse , Christiana Franke , Josef Priller , Eike Spruth , Irene Villar Munoz , Miriam Barkhoff , Henning Boecker , Frederic Brosseron , Marcel Daamen , Tanja Engels , Jennifer Faber , Klaus Fließbach , Ingo Frommann , Marcus Grobe-Einsler , Guido Hennes , Gabi Herrmann , Lorraine Jost , Pascal Kalbhen , Okka Kimmich , Xenia Kobeleva , Barbara Kofler , Cornelia McCormick , Lisa Miebach , Carolin Miklitz , Anna Müller , Demet Oender , Alexandra Polcher , Veronika Purrer , Sandra Röske , Christine Schneider , Anja Schneider , Annika Spottke , Ina Vogt , Michael Wagner , Steffen wolfsgruber , Sagik Yilmaz , Claudia Bartels , Peter Dechent , Niels Hansen , Lina Hassoun , Sina Hirschel , Sabine Nuhn , Ilona Pfahlert , Lena Rausch , Björn Schott , Charles Timäus , Christine Werner , Jens Wiltfang , Lioba Zabel , Heike Zech , Abdelmajid Bader , Carlos Baldermann Juan , Britta Dölle , Alexander Drzezga , Claus Escher , Roshan Ghiasi Nasim , Katja Hardenacke , Frank Jessen , Hannah Lützerath , Franziska Maier , Benjamin Marquardt , Anja Martikke , Dix Meiberth , Snjezana Petzler , Ayda Rostamzadeh , Lena Sannemann , Ann-Katrin Schild , Susanne Sorgalla , Simone Stockter , Manuela Thelen , Maike Tscheuschler , Franziska Uhle , Philip Zeyen , Daniel Bittner , Arturo Cardenas-Blanco , Laura Dobisch , Emrah Düzel , Doreen Grieger-Klose , Deike Hartmann , Coraline Metzger , Peter Nestor , Christin Ruß , Franziska Schulze , Oliver Speck , Glanz Wenzel , Renat Yakupov , Gabriel Ziegler , Christine Brauneis , Katharina Bürger , Cihan Catak , Lisa Coloma Andrews , Martin Dichgans , Angelika Dörr , Birgit Ertl-Wagner , Daniela Frimmer , Brigitte Huber , Daniel Janowitz , Max Kreuzer , Eva Markov , Claudia Müller , Axel Rominger , Jennifer Schmid (ehemals Spreider) , Anna Seegerer , Julia Stephan , Adelgunde Zollver , Lena Burow , Sylvia de Jonge , Peter Falkai , Natalie Garcia Angarita , Thomas Görlitz , Üstün Gürsel Selim , Ildiko Horvath , Carolin Kurz , Eva Meisenzahl-Lechner , Robert Perneczky , Julia Utecht , Martin Dyrba , Heike Janecek-Meyer , Ingo Kilimann , Chris Lappe , Esther Lau , Henrike Pfaff , Heike Raum , Petr Sabik , Monika Schmidt , Heike Schulz , Sarah Schwarzenboeck , Stefan Teipel , Marc-Andre Weber , Martina Buchmann , Tanja Heger , Petra Hinderer , Elke Kuder-Buletta , Christoph Laske , Matthias Munk , Christian Mychajliw , Surjo Soekadar , Patricia sulzer , Theresia Trunk TITLE=Performance evaluation of automated white matter hyperintensity segmentation algorithms in a multicenter cohort on cognitive impairment and dementia JOURNAL=Frontiers in Psychiatry VOLUME=13 YEAR=2023 URL=https://www.frontiersin.org/journals/psychiatry/articles/10.3389/fpsyt.2022.1010273 DOI=10.3389/fpsyt.2022.1010273 ISSN=1664-0640 ABSTRACT=Background

White matter hyperintensities (WMH), a biomarker of small vessel disease, are often found in Alzheimer’s disease (AD) and their advanced detection and quantification can be beneficial for research and clinical applications. To investigate WMH in large-scale multicenter studies on cognitive impairment and AD, appropriate automated WMH segmentation algorithms are required. This study aimed to compare the performance of segmentation tools and provide information on their application in multicenter research.

Methods

We used a pseudo-randomly selected dataset (n = 50) from the DZNE-multicenter observational Longitudinal Cognitive Impairment and Dementia Study (DELCODE) that included 3D fluid-attenuated inversion recovery (FLAIR) images from participants across the cognitive continuum. Performances of top-rated algorithms for automated WMH segmentation [Brain Intensity Abnormality Classification Algorithm (BIANCA), lesion segmentation toolbox (LST), lesion growth algorithm (LGA), LST lesion prediction algorithm (LPA), pgs, and sysu_media] were compared to manual reference segmentation (RS).

Results

Across tools, segmentation performance was moderate for global WMH volume and number of detected lesions. After retraining on a DELCODE subset, the deep learning algorithm sysu_media showed the highest performances with an average Dice’s coefficient of 0.702 (±0.109 SD) for volume and a mean F1-score of 0.642 (±0.109 SD) for the number of lesions. The intra-class correlation was excellent for all algorithms (>0.9) but BIANCA (0.835). Performance improved with high WMH burden and varied across brain regions.

Conclusion

To conclude, the deep learning algorithm, when retrained, performed well in the multicenter context. Nevertheless, the performance was close to traditional methods. We provide methodological recommendations for future studies using automated WMH segmentation to quantify and assess WMH along the continuum of cognitive impairment and AD dementia.