Development of an Al-Based Web Diagnostic System for Phenotyping Psychiatric Disorders

Background: Artificial intelligence (AI)-based medical diagnostic applications are on the rise. Our recent study has suggested an explainable deep neural network (EDNN) framework for identifying key structural deficits related to the pathology of schizophrenia. Here, we presented an AI-based web diagnostic system for schizophrenia under the EDNN framework with three-dimensional (3D) visualization of subjects' neuroimaging dataset.

Methods: This AI-based web diagnostic system consisted of a web server and a neuroimaging diagnostic database. The web server deployed the EDNN algorithm under the Node.js environment. Feature selection and network model building were performed on the dataset obtained from two hundred schizophrenic patients and healthy controls in the Taiwan Aging and Mental Illness (TAMI) cohort. We included an independent cohort with 88 schizophrenic patients and 44 healthy controls recruited at Tri-Service General Hospital Beitou Branch for validation purposes.

Results: Our AI-based web diagnostic system achieved 84.00% accuracy (89.47% sensitivity, 80.62% specificity) for gray matter (GM) and 90.22% accuracy (89.21% sensitivity, 91.23% specificity) for white matter (WM) on the TAMI cohort. For the Beitou cohort as an unseen test set, the model achieved 77.27 and 70.45% accuracy for GM and WM. Furthermore, it achieved 85.50 and 88.20% accuracy after model retraining to mitigate the effects of drift on the predictive capability. Moreover, our system visualized the identified voxels in brain atrophy in a 3D manner with patients' structural image, optimizing the evaluation process of the diagnostic results.

Discussion: Together, our approach under the EDNN framework demonstrated the potential future direction of making a schizophrenia diagnosis based on structural brain imaging data. Our deep learning model is explainable, arguing for the accuracy of the key information related to the pathology of schizophrenia when using the AI-based web assessment platform. The rationale of this approach is in accordance with the Research Domain Criteria suggested by the National Institute of Mental Health.

Introduction

Advancement in computational strategy and the collection of big medical data have been contributing to an increasing interest in applying artificial intelligence (AI) to the medical field nowadays. Recent surge in the automated assessment of various medical diseases has suggested promising direction of improving the conventional workflow of the clinical practice. For example, AI-based medical applications have been consistently shown to successfully identify pathological patterns comparable or even superior to trained physicians (1, 2). The powerful pattern recognition adopted in these state-of-art AI algorithms can be crucial in delineating the complex pathophysiology of various medical diseases. Specifically, the AI algorithm may help identify subtle changes in brain images acquired from patients with mental disorders, which would tremendously support clinicians in the daily practice (3).

For diagnosis of major psychiatric disorder, the traditional approach is still exclusively based on phenomenology (4), leading to increasing interests in searching for objective biomarkers to different types of mental illnesses (5). Magnetic resonance imaging (MRI), which provides quantitative information of brain tissues, is one of the most used techniques for studying major psychiatric disorders. Conventional applications of machine learning techniques [e.g., support vector machines (SVM), random forest (RF)] are useful to perform prediction and discrimination tasks and have been commonly used for neuroanatomical investigations (6–10). However, these algorithms are unable to locate widespread patterns of abnormalities across the brain. Furthermore, limitations such as region-of-interest approaches or univariate analyses in conventional machine-learning models constantly lead to concerns about interpretability and generalizability (11). To tackle these difficulties, Chin et al. (12) introduced an anatomical regularized SVM classifier which yielded accuracy of 89.4% and demonstrated the utility of spatial and anatomical priors for structural MRI analyses. Apart from the conventional machine learning techniques, the neural network-based approaches have shown promising classification results when it is used with structural MRI, functional MRI, and diffusion tensor imaging (DTI) together (13–16). However, to our current knowledge, the deep-learning-based model with mere structural MRI data has not been fully developed so far, leaving it unclear about the classification capability for identifying psychiatric disorders and brain structural abnormalities as well as visualization of key deficits (17–19). Therefore, we applied the deep learning approach to identify diagnosis-specific brain region that can help differentiate schizophrenia and healthy controls. Given that brain imaging pathology in mental disorders is inconspicuous, developing explainable machine-learning models is inherently crucial, especially for indexing brain biomarkers in clinical practice with neuroimaging data (20).

In this study, we built an online website service for assessment of brain images based on an explainable deep learning algorithm (21). This AI-based web assessment system was developed for the purpose of classifying schizophrenia patients and healthy controls with structural MRI brain images. Furthermore, we aim to provide better interpretation from physician's point of view by visualizing the brain imaging pathology through web browser along with the analysis of structural deficits based on deep-learning findings. We anticipated this system to facilitate the exploitation of the promising AI technique for psychiatric applications.

Methods and Materials

Participants

In this study, two independent cohorts with a total of 532 subjects from different institutions were included for model building and model transfer validation. First, the dataset for model building consisted of 200 patients with schizophrenia and 200 age- and sex-matched healthy controls from the discovery cohort “Taiwan Aging and Mental Illness” (TAMI). Data from 400 participants were divided into a training set consisting of 350 participants (87.50%) and a test set with 50 participants (12.50%). Second, the dataset for model evaluation and transfer learning comprised 88 patients and 44 health controls at Tri-Service General Hospital Beitou Branch, Taiwan. For clinical data evaluation, we applied the abovementioned model on the unseen data from 132 participants. The transfer learning was conducted by using the unseen dataset for model retraining. In line with the approach, data from 132 participants were divided into a training set consisting of 92 participants (69.69%) and a test set with 40 participants (30.31%). The training and test sets were randomly selected by matching participants based on diagnosis for the partitioning of data (Figure 1).

FIGURE 1

Figure 1. Framework of the train and test procedure: In the top left, the cohort is used to train the EDNN for the classification model. In the lower right box, a cartoon figure shows a trained model being deployed on a web server. The diagnostic result of the imaging data is visualized on the web browser. In the top right, the independent cohort is used to validate the model by transfer learning.

All schizophrenic patients were evaluated by the same protocol described previously (22). Healthy controls were recruited from the local communities through advertisements, and none of the them reported a history of neurological or psychiatric disorders. The study was approved by the Institutional Review Board of Taipei Veterans General Hospital (TVGH). Informed consent was obtained from all the participants before the MRI scanning. Tables 1, 2 shows demographic features of the participants.

TABLE 1

Table 1. Demographics and clinical characteristics of participants.

TABLE 2

Table 2. Further detailed information of schizophrenic participants from TAMI cohort (N = 200).

MRI Data Acquisition and Imaging Processing

Structural T1-weighted images were acquired at National Yang-Ming University with a 3.0-T Siemens Magnetom Tim Trio Scanner (Siemens AG, Erlangen, Germany) using a 3D magnetization-prepared rapid gradient echo (MPRAGE) sequence (TR = 2,530 ms, TE = 3.5 ms, TI = 1,100 ms, FOV = 256 mm, flip angle = 7) and a 12-channel head coil. The scanning protocol was the same as our prior reports (22–24).

Data were analyzed with DPARSF (V4.3) and SPM12 (Wellcome Trust Center for Neuroimaging, London, UK). T1 images were resliced (2 mm isotropic) and segmented into gray matter (GM), white matter (WM), and CSF tissue. Data preprocessing included slice timing, segmentation, and normalization into the Montreal Neurological Institute (MNI) stereotactic space.

The normalized GM and WM images served as the inputs per case for the web diagnostic system under the EDNN framework. In this approach, each subject's input images are considered as a data point in a high-dimensional space of anatomical information defined by GM or WM volumes.

Neural Network and Web Diagnostic System

An explainable deep neural network (EDNN) with KL-L1 regularization is a classifier capable of identifying key structural deficits in schizophrenia (21). In this study, we adapted the EDNN framework to classify schizophrenia patients and healthy controls based on whole-brain gray matter and white matter densities (voxel-based morphometry, VBM).

The EDNN model is composed of a feature selection process and two fully connected layers with the KL-L1 regularization method.

$\begin{array}{l}{L}_{KL-L1}(\theta )={\displaystyle \sum _i}\Vert \overline{W_{i·}}{\Vert }_{1}log\frac{\Vert \overline{W_{i·}}{\Vert }_1}{\epsilon }+(1-\Vert \overline{W_{i·}}{\Vert }_1)\\ log\frac{1-\Vert \overline{W_{i·}}{\Vert }_1}{1-\epsilon }\\ I(X,Y)={\displaystyle \sum _{xϵX}}{\displaystyle \sum _{yϵY}}p(x,y)logp(y|x)-{\displaystyle \sum _{yϵY}}p(y)logp(y)\end{array}$

mutual information analysis, I(X, Y), served as a feature selection process in the EDNN. This analysis was employed to reduce the dimensionality of neuroimaging data and to identify the key voxels that contribute to classification of schizophrenia. After implementing this selection of features, corresponding identified brain voxels that were associated with diagnosis label are then used to perform deep neural network learning and to investigate the performance for later classification.

To develop a systemic diagnostic platform for differentiating the information of brain imaging data in healthy and schizophrenia patients, we have proposed an AI-based web diagnostic system (Figure 2). The system consists of the web server and the imaging diagnostic database.

FIGURE 2

Figure 2. Flowchart of the assessment system. The evaluation of brain imaging data is divided separately into two-fold regarding the types of brain tissue. The diagnostic system individually evaluates the uploading imaging data in case of the uploading imaging data being identified as a gray matter or white matter imaging data, and then the system provides the analysis in brain morphometry to the front-end accordingly. As soon as both of the white matter and gray matter are uploaded and evaluated, a classification diagnosis of healthy and schizophrenia is intuitively obtained and presented to the end user.

The server is constructed with Node.js and Tensorflow.js. Node.js is a platform on Chrome's JavaScript runtime for building scalable network applications. Node.js has been tested to yield better efficiency than PHP and Python-Web (25). Tensorflow.js provides flexible API in JavaScript, and it is part of the TensorFlow ecosystem. On the server side, we integrated feature selection and deep-learning framework (21) for the EDNN model. The model is deployed to communicate with the imaging diagnostic database and to perform a diagnosis task for schizophrenia classification along with brain morphometry identification. Apart from the back-end, the front-end interface is composed of JavaScript (js) and WebGL with plug-in applications “Papaya” in Mango image processing software (Lancaster, Martinez; www.ric.uthscsa.edu/mango) for visualization of brain structural abnormality.

Algorithm Validation and Statistical Analysis

Six quantities are reported in this study for validating the EDNN algorithm: specificity, sensitivity, accuracy, precision (PPV), negative predictive value (NPV) and Number Needed to Predict (NNP) (10). To assist with parameter tuning, we validated these quantities and the receiver operating characteristic (ROC) curves on two independent cohorts using the EDNN framework, which gives an estimate of diagnostic utility and how well the model will perform in a new dataset.

Results

Classifier Performance

The classification results of the models are given in Table 3. Through application of the EDNN on both TAMI (400 subjects) and Beitou (132 subjects) cohorts, the training accuracy reached 100.00 % in the classification of schizophrenia and healthy controls. On the other hand, the EDNN classifier discriminated schizophrenia cases and healthy controls with 84.00% (95% confidence interval CI: 73.97–84.36) accuracy and 90.22% (95% CI: 85.40–91.40) accuracy for GM and WM, respectively in the TAMI cohort. Using the Beitou cohort, it yielded classification accuracy of 77.27% (95% CI: 75.23–81.32) for GM and accuracy of 70.45% (95% CI: 68.58–74.29) for WM. After performing transfer learning by retraining the model, it achieved 85.50% (95% CI: 85.23–88.07) for GM and accuracy of 88.20% (95% CI: 86.52–88.80) for WM. The ROC curves in Figure 3 showed the area under the curve (AUC) scores indicating the competitive performance using our models.

TABLE 3

Table 3. The performance of the EDNN model in two independent cohorts with GM and WM imaging data.

FIGURE 3

Figure 3. Quantitative performance of our model in ROC curves and AUC scores.

Framework of Web Diagnostic System

As the framework is illustrated in previous section, the EDNN produced a classifier for the AI-based diagnostic system from our cohorts, and the web-platform provided a fast and easy access to the machine-learning technique we developed. Taken together, we presented an online AI diagnostic system (www.brain-diagnosis.com) which is a fully open cloud service to both clinical and academic end users, aiming to help classify and diagnose schizophrenia patients via the internet.

Figure 4 illustrates a schizophrenia classification and diagnostic result using our AI-based system. When the end user uploads a structural MRI image to our web platform, the backend will perform the following tasks. Brain tissues necessary to perform voxel-based morphometry are first extracted from the DICOM image taken by an MRI scanner. The task of classification and resulting diagnosis are presented on the diagnostic system. The diagnostic system is capable of converting all the brain image data and diagnosis result for graphical preparations and visualizations through the user's local computing resources for the graphical abilities. The end-user can simply upload the NIFTI files of GM and WM images, and then the classification of schizophrenia along with pragmatic diagnosis will be delivered in a browser instantaneously.

FIGURE 4

Figure 4. Schematic depiction of our prediction and evaluation workflow.

Furthermore, our web platform is capable of visualizing the deficits of voxel morphometry identified by the EDNN model. A voxelwise brain volume map of the GM densities was transformed to the z-score with respect to the MRI data in our cohorts.

$\begin{array}{l}\text{ zscore}\_\text{HC}=\frac{\text{value of voxel}-\text{mean value of voxels from HC}}{\text{std value of voxels from HC}}\\ \text{zscore}\_\text{SCZ}=\frac{\text{value of voxel}-\text{mean value of voxels from SCZ}}{\text{std value of voxels from SCZ}}\\ \text{ z}-\text{score}=\frac{\text{zscore}\_\text{SCZ}}{\text{zscore}\_\text{SCZ}+\text{zscore}\_\text{HC}}\end{array}$

In Figure 5, a 3D saliency map of one demonstrative case with the highlighted brain regions shows significant GM reduction in the insula, anterior, and anterior cortex. In doing so, the platform based on the EDNN model gives us a new straightforward approach on the automated individual-level assessment of schizophrenia.

FIGURE 5

Figure 5. Individualized visualization and z-score analysis delivered by our diagnostic platform for which the volumes were significantly reduced in a single schizophrenia patient compared with healthy subjects.

Discussion

Summary of Findings

Adopting EDNN to investigate schizophrenia shows a practical advantage without requiring manual feature selection as other conventional machine learning approaches do. With the aim to preserving the 3D structure in the classification pipeline and elucidating the neural correlates of mental disorders, it is critical both to identify schizophrenia and to find relevant brain-based imaging markers simultaneously. In line with the thought, we applied an EDNN model to visualize the brain structural deficits and to distinguish patients with schizophrenia from healthy controls in two large independent samples using structural MRI data.

EDNN is a particular type of the deep neural network showing its data-driven interpretation along with promising recognition accuracy. The strength of our approach was its novel use of an EDNN model to distinguish patients with schizophrenia from healthy controls without anatomical priors. The model's discriminative pattern showed decreased GM density in the insula and frontal and superior temporal lobes as well as reduced intensity of WM in the cingulum/hippocampus, splenium of corpus callosum, and posterior corona radiate tracts. These deep-learning-based findings provide important brain-based imaging markers for subsequent neuroimaging analyses and the development of our web platform.

Evaluation of Our Results With Others

The 89.63% average classification accuracies of our model obtained in two large independent cohorts were relatively compelling, compared to relevant studies in literature (8–10, 19). Besides, the AUC score indicates that EDNN performed well for classifying patients from controls (19). The current finding of NNP suggests that the EDNN classifier is capable of providing reproducible results across different large cohorts. Even though some studies (7, 12, 26, 27) asserted the decrease in accuracy being attributed to greater variability in brain endophenotypes with sample size increase, the EDNN framework in this study appears to be promising in discrimination of schizophrenia by integrating rich information in VBM.

Research Domain Criteria

Visualization of the salient brain region provides important clinical information by using our AI-based diagnostic platform. The rationale as “explainable” deep neural network might aid in the development of neuroimaging biomarkers for psychiatric disorders. As suggested by the Research Domain Criteria (RDoC) approach, our approach is directly relevant to the search for biomarkers because our framework yields more robust feature representation and reliable performance on classification. Therefore, we expect this web-based RDoC approach may improve the reliability of psychiatric diagnosis as well as the objectiveness of clinical evaluation.

An increasing amount of neuroimaging data has been established in recent years to seek for machine learning-based classification of neuropsychiatric diseases, such as Autism Brain Imaging Data Exchange (28), Alzheimer's disease neuroimaging initiative (29), or Bipolar & Schizophrenia Network on Intermediate Phenotypes (30). These databases have been released to the public or have plans to be public in the near future, facilitating the process of developing AI-based neuroimaging analysis. Future validation of our EDNN method with related and larger databases that contains heterogeneous cohorts could help evaluate the adaptability and improve the performance of our web-based diagnostic platform. Furthermore, our EDNN approach has identified certain brain regions related to schizophrenia, which could help generate hypothesis-driven research of pathophysiology of schizophrenia. These brain-based imaging markers may augment the doctor to improve the decision-making and assessment process in the workflow of the clinical practice. Importantly, this EDNN algorithm is accessible publicly through our web-based diagnostic platform.

Limitations

There are several limitations to our study. First, it should be noted that the imbalanced dataset and different cohorts might have affected the performance when using the model. While our model achieved approximately 90% within a single cohort, it became <75% when we applied the trained model with a completely unseen dataset (i.e., Beitou cohort). Furthermore, the uneven number of classes, i.e., 88 schizophrenia and 44 healthy controls, might have also caused the drop of predictive performance. To overcome the issue, data sampling and ensemble system may be employed (31). Note that with the two approaches, altering the distribution of the training data may cause biasing.

Second, psychosis is still a heterogeneous entity. Although our model can get promising accuracy in the binary classification, it is difficult to directly generalize the mutual information analysis to a multi-class, multi-label classification task across other psychotic disorders such as schizoaffective, depression, and bipolar disorders.

Therefore, further investigation may be focused on generalization to other neuropsychiatric disorders (e.g., Alzheimer's disease, Parkinson's disease). In addition, more examination is needed on the feasibility of identifying biological subtypes of psychotic disorder. By using our web-based platform, we aim to come up with persuasive phenotypes based on the brain anatomic abnormalities.

Conclusion

This paper presented a new platform for a publicly available brain diagnostic system over browsers called “The AI-based online brain diagnosis system” which shows the potential diagnostic ability for schizophrenia using a T1-weighted image. This tool is to provide an objective quantification of brain pathology rather than to replace the conventional diagnostic interview in the clinical practice.

Schizophrenia is a heterogeneous disorder with respect to the involvement of many co-occurring symptoms. Traditionally, the diagnosis of schizophrenia is mainly based on symptom profile with descriptors of the course of illness. Yet, there have been cases that missed to be identified with such approach. More generally, it has long been a debate on the underlying principles of diagnosing schizophrenia. The platform proposed in our study is a proof of concept by demonstrating the use of deep-learning models to differentiating schizophrenia from healthy controls based solely on imaging markers. We anticipate that this platform can be generalized to the differential diagnosis of multiple psychiatric disorders such as schizophrenia, bipolar disorder, or unipolar depression. The main motivations for developing this platform are to facilitate objective diagnosis by the user-friendly website interface and to promote the visibility and availability of newly developed state-of-the-art algorithms for the diagnosis of schizophrenia. An overview of the main user interface is currently available on “brain-diagnosis.com.” We envisage that this platform can bring improvements in the effectiveness and impact of the deep learning applications in neuroimaging.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by the Institutional Review Board in Taipei Veterans General Hospital, National Yang-Ming University and Tri-Service Military General Hospital, Beitou Branch. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

Y-WC: manuscript writing, programming, and data analysis. S-JT: manuscript discussion and statistical analysis. Y-FW: study design, participants selection, and collection. AY: study design, data collection, development of EDNN model, and supervision of the project.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

AY and S-JT were supported by grants (109-2628-B-010-011; 109-2321-B-010-006; and 109-2634-F-075-001) from the Ministry of Science and Technology of Taiwan. AY was also supported by Mt. Jade Young Scholarship Award from Ministry of Education, Taiwan as well as Brain Research Center, National Yang-Ming University and the Ministry of Education (Aim for the Top University Plan), Taipei, Taiwan. This work was also supported by Tri-Service General Hospital by grant (TSGH-BT-107-010, TSGH-BT-107-011).

References

1. Kuo W, Häne C, Mukherjee P, Malik J, Yuh EL. Expert-level detection of acute intracranial hemorrhage on head computed tomography using deep learning. Proc Natl Acad Sci U S A. (2019) 116:22737–45. doi: 10.1073/pnas.1908021116

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Arbabshirani MR, Fornwalt BK, Mongelluzzo GJ, Suever JD, Geise BD, Patel AA, et al. Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration. NPJ Digit Med. (2018) 1:1–7. doi: 10.1038/s41746-017-0015-z

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Lundervold AS, Lundervold A. An overview of deep learning in medical imaging focusing on MRI. Z Med Phys. (2019) 29:102–27. doi: 10.1016/j.zemedi.2018.11.002

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Abi-Dargham A, Horga G. The search for imaging biomarkers in psychiatric disorders. Nat Med. (2016) 22:1248–55. doi: 10.1038/nm.4190

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Aydin O, Unal Aydin P, Arslan A. Development of neuroimaging-based biomarkers in psychiatry. Adv Exp Med Biol. (2019) 1192:159–95. doi: 10.1007/978-981-32-9721-0_9

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Xiao Y, Yan Z, Zhao Y, Tao B, Sun H, Li F, et al. Support vector machine-based classification of first episode drug-naïve schizophrenia patients and healthy controls using structural MRI. Schizophr Res. (2019) 214:11–7. doi: 10.1016/j.schres.2017.11.037

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Nieuwenhuis M, van Haren NEM, Hulshoff Pol HE, Cahn W, Kahn RS, Schnack HG. Classification of schizophrenia patients and healthy controls from structural MRI scans in two large independent samples. NeuroImage. (2012) 61:606–12. doi: 10.1016/j.neuroimage.2012.03.079

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Schnack HG, Nieuwenhuis M, van Haren NEM, Abramovic L, Scheewe TW, Brouwer RM, et al. Can structural MRI aid in clinical classification? A machine learning study in two independent samples of patients with schizophrenia, bipolar disorder and healthy subjects. NeuroImage. (2014) 84:299–306. doi: 10.1016/j.neuroimage.2013.08.053

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Winterburn JL, Voineskos AN, Devenyi GA, Plitman E, de la Fuente-Sandoval C, Bhagwat N, et al. Can we accurately classify schizophrenia patients from healthy controls using magnetic resonance imaging and machine learning? A multi-method and multi-dataset study. Schizophr Res. (2019) 214:3–10. doi: 10.1016/j.schres.2017.11.038

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Iwabuchi SJ, Liddle PF, Palaniyappan L. Clinical utility of machine-learning approaches in schizophrenia: improving diagnostic confidence for translational neuroimaging. Front Psychiatry. (2013) 4:95. doi: 10.3389/fpsyt.2013.00095

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Zarogianni E, Moorhead TWJ, Lawrie SM. Towards the identification of imaging biomarkers in schizophrenia, using multivariate pattern classification at a single-subject level. Neuroimage Clin. (2013) 3:279–89. doi: 10.1016/j.nicl.2013.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Chin R, You AX, Meng F, Zhou J, Sim K. Recognition of schizophrenia with regularized support vector machine and sequential region of interest selection using structural magnetic resonance imaging. Sci Rep. (2018) 8:13858. doi: 10.1038/s41598-018-32290-9

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Rozycki M, Satterthwaite TD, Koutsouleris N, Erus G, Doshi J, Wolf DH, et al. Multisite machine learning analysis provides a robust structural imaging signature of schizophrenia detectable across diverse patient populations and within individuals. Schizophr Bull. (2018) 44:1035–44. doi: 10.1093/schbul/sbx137

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Zeng L-L, Wang H, Hu P, Yang B, Pu W, Shen H, et al. Multi-site diagnostic classification of schizophrenia using discriminant deep learning with functional connectivity MRI. EBioMedicine. (2018) 30:74–85. doi: 10.1016/j.ebiom.2018.03.017

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Oh K, Kim W, Shen G, Piao Y, Kang N-I, Oh I-S, et al. Classification of schizophrenia and normal controls using 3D convolutional neural network and outcome visualization. Schizophr Res. (2019) 212:186–95. doi: 10.1016/j.schres.2019.07.034

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Qureshi MNI, Oh J, Lee B. 3D-CNN based discrimination of schizophrenia using resting-state fMRI. Artif Intell Med. (2019) 98:10–7. doi: 10.1016/j.artmed.2019.06.003

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Sherif T, Kassis N, Rousseau M-Ã, Adalat R, Evans AC. BrainBrowser: distributed, web-based neurological data visualization. Front Neuroinform. (2015) 8:89. doi: 10.3389/fninf.2014.00089

CrossRef Full Text | Google Scholar

18. Pinaya WHL, Mechelli A, Sato JR. Using deep autoencoders to identify abnormal brain structural patterns in neuropsychiatric disorders: a large-scale multi-sample study. Hum Brain Mapp. (2019) 40:944–54. doi: 10.1002/hbm.24423

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Pinaya WHL, Gadelha A, Doyle OM, Noto C, Zugman A, Cordeiro Q, et al. Using deep belief network modelling to characterize differences in brain morphometry in schizophrenia. Sci Rep. (2016) 6:1–9. doi: 10.1038/srep38897

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Fu CHY, Costafreda SG. Neuroimaging-based biomarkers in psychiatry: clinical opportunities of a paradigm shift. Can J Psychiatry. (2013) 58:499–508. doi: 10.1177/070674371305800904

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Yang A, Tsai S-J. T9. Explainable deep learning of neuroimaging reveals key structural deficits in schizophrenia. Schizophr Bull. (2019) 45:S206. doi: 10.1093/schbul/sbz019.289

CrossRef Full Text | Google Scholar

22. Yang AC, Hong C-J, Liou Y-J, Huang K-L, Huang C-C, Liu M-E, et al. Decreased resting-state brain activity complexity in schizophrenia characterized by both increased regularity and randomness. Hum Brain Mapp. (2015) 36:2174–86. doi: 10.1002/hbm.22763

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Yang AC, Wang S-J, Lai K-L, Tsai C-F, Yang C-H, Hwang J-P, et al. Cognitive and neuropsychiatric correlates of EEG dynamic complexity in patients with Alzheimer's disease. Prog Neuropsychopharmacol Biol Psychiatry. (2013) 47:52–61. doi: 10.1016/j.pnpbp.2013.07.022

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Yang AC, Huang C-C, Liu M-E, Liou Y-J, Hong C-J, Lo M-T, et al. The APOE ε4 allele affects complexity and functional connectivity of resting brain activity in healthy adults. Hum Brain Mapp. (2014) 35:3238–48. doi: 10.1002/hbm.22398

CrossRef Full Text | Google Scholar

25. Lei K, Ma Y, Tan Z. Performance comparison and evaluation of web development technologies in PHP, Python, and Node.js. IEEE Comput Soc. (2014) 661–8. doi: 10.1109/CSE.2014.142

CrossRef Full Text | Google Scholar

26. Fan Y, Shen D, Gur RC, Gur RE, Davatzikos C. COMPARE: classification of morphological patterns using adaptive regional elements. IEEE Trans Med Imaging. (2007) 26:93–105. doi: 10.1109/TMI.2006.886812

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Kawasaki Y, Suzuki M, Kherif F, Takahashi T, Zhou S-Y, Nakamura K, et al. Multivariate voxel-based morphometry successfully differentiates schizophrenia patients from healthy controls. Neuroimage. (2007) 34:235–42. doi: 10.1016/j.neuroimage.2006.08.018

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Di Martino A, Yan C-G, Li Q, Denio E, Castellanos FX, Alaerts K, et al. The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol Psychiatry. (2014) 19:659–67. doi: 10.1038/mp.2013.78

CrossRef Full Text | Google Scholar

29. Mueller SG, Weiner MW, Thal LJ, Petersen RC, Jack CR, Jagust W, et al. Ways toward an early diagnosis in Alzheimer's disease: the Alzheimer's Disease Neuroimaging Initiative (ADNI). Alzheimers Demen. (2005) 1:55–66. doi: 10.1016/j.jalz.2005.06.003

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Tamminga CA, Pearlson G, Keshavan M, Sweeney J, Clementz B, Thaker G. Bipolar and schizophrenia network for intermediate phenotypes: outcomes across the psychosis continuum. Schizophr Bull. (2014) 40(Suppl 2):S131–7. doi: 10.1093/schbul/sbt179

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Dubey R, Zhou J, Wang Y, Thompson PM, Ye J. Analysis of sampling techniques for imbalanced data: an N=648 Adni Study. Neuroimage. (2014) 87:220–41. doi: 10.1016/j.neuroimage.2013.10.005

PubMed Abstract | CrossRef Full Text | Google Scholar