Liming Shu1,2†Kaiyi Zhong3†Nanya Chen4
Wenxin Gu4Wenjing Shang3
Jiahui Liang5,6
Jiangtao Ren4*
Hua Hong3*- 1Department of Neurology, Seventh Affiliated Hospital, Sun Yat-sen University, Shenzhen, China
- 2Department of Neurology, Second Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
- 3Department of Neurology, First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
- 4School of Data and Computer Science, Sun Yat-sen University, Guangzhou, China
- 5Department of Medical Imaging, Sun Yat-sen University Cancer Center, Guangzhou, China
- 6Guangdong Key Laboratory of Non-human Primate Research, Guangdong-Hongkong-Macau Institute of CNS Regeneration, Jinan University, Guangzhou, China
Background and purpose: As one common feature of cerebral small vascular disease (cSVD), white matter lesions (WMLs) could lead to reduction in brain function. Using a convenient, cheap, and non-intrusive method to detect WMLs could substantially benefit to patient management in the community screening, especially in the settings of availability or contraindication of magnetic resonance imaging (MRI). Therefore, this study aimed to develop a useful model to incorporate clinical laboratory data and retinal images using deep learning models to predict the severity of WMLs.
Methods: Two hundred fifty-nine patients with any kind of neurological diseases were enrolled in our study. Demographic data, retinal images, MRI, and laboratory data were collected for the patients. The patients were assigned to the absent/mild and moderate–severe WMLs groups according to Fazekas scoring system. Retinal images were acquired by fundus photography. A ResNet deep learning framework was used to analyze the retinal images. A clinical-laboratory signature was generated from laboratory data. Two prediction models, a combined model including demographic data, the clinical-laboratory signature, and the retinal images and a clinical model including only demographic data and the clinical-laboratory signature, were developed to predict the severity of WMLs.
Results: Approximately one-quarter of the patients (25.6%) had moderate–severe WMLs. The left and right retinal images predicted moderate–severe WMLs with area under the curves (AUCs) of 0.73 and 0.94. The clinical-laboratory signature predicted moderate–severe WMLs with an AUC of 0.73. The combined model showed good performance in predicting moderate–severe WMLs with an AUC of 0.95, while the clinical model predicted moderate–severe WMLs with an AUC of 0.78.
Conclusion: Combined with retinal images from conventional fundus photography and clinical laboratory data are reliable and convenient approach to predict the severity of WMLs and are helpful for the management and follow-up of WMLs patients.
1. Introduction
Stroke is a leading cause of mortality and long-term disability, especially in low- and middle-income countries (1). In China, among 28.76 million prevalent cases of stroke in 2019, and most of the cases were ischemic stroke (2). Cerebral small vascular disease (cSVD), including white matter lesions (WMLs), is a major challenge to brain health, accounting for approximately 30% of the cause of ischemic stroke (3). It was estimated that 36% of cSVD cases were due to WMLs (4).
The severity of WMLs is negatively associated with cognitive function, especially in the patients of Fazekas 3 and 4 WMLs (5, 6). In addition, WMLs was also association with new-onset depression in older people in the community and the risk of post-stroke depression and dysfunction of bowel and bladder (7). WMLs are associated with poorer gait and balance (8). And WMLs are a strong predictor of the incident stroke (9). The severity of WMLs progressed over time. The Leukoaraiosis and Disability Study showed that about 73.6% of the participants had WMLs progression during 3 years follow-up and among the patients with severe WMLs, more than 80% of them had WMLs progression (10). Thus, regular screening is crucial for the early identification of WMLs and the prevention of its progression. Magnetic resonance imaging (MRI) has been widely used to detect WMLs in vivo (11). However, MRI is not available for all the candidates who needed WMLs screening or diagnosis due to insufficient economic resource, unavailability of MRI equipment, contraindication of MRI and so on. Convenient substitution of MRI to assess WMLs is helpful for patient management and economic efficiency.
It’s worth noting that the brain and retinal vasculature are homology (12). More specifically, the blood–retinal barrier mirrors the blood–brain barrier. In addition, the microcirculation of the brain and retina share a similar autoregulation function to maintain blood flow (13). The retinal microcirculation has a similar embryonic, anatomical, and physiological basis to that of small intracranial vessels (14, 15). Thus, the retina is a window through which vasculature and neural tissues can be dynamically and non-invasively observed. Retinal blood vessel lesions have been found to be independently associated with WMLs (16–18).
Significant progress in deep learning classifiers has been achieved in medical imaging fields, such as radiology, dermatology, pathology and ophthalmology (19). Among various deep learning techniques, convolutional neural networks make automatic, efficient and accurate image-based diagnosis possible (20, 21). Specifically, deep learning models based on retinal images rival ophthalmologists when screening diabetic retinopathy (22). Moreover, there is early evidence that identification of central nervous system diseases based on retinal images is possible. Recently, texture characteristics of retinal images discovered by automated retinal image analysis have been found to predict WMLs in community participants (23). A deep learning algorithm has been applied to raw retinal images was put forward to predict WMLs (24). However, it is worth noticing that the capability to predict WMLs of these retinal images-based models needs further improvement. Thus, a comprehensive model that not only includes retinal images but also these clinical data need to be developed. A recent study developed a comprehensive model combined a deep learning model based on retinal images and clinical information, which accurately predicted chronic kidney disease and type 2 diabetes (25). Concerning that hypertension (26), hyperlipidemia (27), impaired glucose metabolism (28), kidney failure (29) and systemic inflammation (30) are risk factors of WMLs, incorporating the clinical information of these risk factors into the retinal images-based prediction model may improve the capability of prediction and enhance the clinical practicability of the model. To the best of our knowledge, no studies have been conducted to develop a deep learning model using retinal images combined with the clinical laboratory data to predict the severity of WMLs.
Therefore, based on the previous findings that retina and its vasculature share similarity with cerebral small vessels, we aimed to develop a comprehensive model including retinal images and clinical laboratory data to predict WMLs severity, which may facilitate screening WMLs patients and follow-up.
We first built up a ResNet deep learning neural network to analyze correlation between the retinal images and the severity of WMLs. Then, we fused a clinical-laboratory signature based on the clinical laboratory data. A model combining demographic-clinical characteristics, the clinical-laboratory signature, and the outputs of the retina deep learning neural network was further developed to predict the severity of WMLs. To test the significance of the retinal images on predicting the severity of WMLs, the clinical model only included demographic-clinical characteristics and the clinical-laboratory signature. The performance of predicting WMLs severity of the two models was compared.
2. Methods
2.1. Patient selection
The clinical documentations of patients admitted to the Department of Neurology, the First Affiliated Hospital of Sun Yat-sen University, from January 2018 to January 2020 were retrospectively reviewed. The inclusion criteria were as follows: (1) patients who underwent nonmydriatic fundus photography; (2) patients who underwent brain MRI; and (3) over 18 years old. The exclusion criteria were as follows: (1) poor quality nonmydriatic fundus photography or MRI images; (2) missing clinical data; (3) WMLs caused by hereditary diseases, immune disorder, or infection (e.g., Fabry disease, cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy, cerebral autosomal recessive arteriopathy with subcortical infarcts and leukoencephalopathy, multiple sclerosis, neuromyelitis optica spectrum disorder, encephalitis); (4) intracranial hypertension or hydrocephalus; (5) major brain edema or intracerebral hemorrhage; (6) primary ophthalmological diseases impairing the retina or hindering observation of the retina by fundus photography; (7) recent (<6 months) ophthalmological surgery; and (8) concomitant significant systemic infection. Demographic-clinical data, including age, sex, systolic blood pressure (SBP) and diastolic blood pressure (DBP) on admission, were collected. Comorbidities, including hypertension, diabetes mellitus, statin usage, atrial fibrillation, coronary heart disease, previous stroke or transient ischemic attack (TIA), were documented. Hypertension was defined as systolic blood pressure of ≥140 mmHg or diastolic blood pressure of ≥90 mmHg or current treatment with anti-hypertensive medications. Diabetes mellitus was defined as fasting plasma glucose ≥7.0 mmol/L or hemoglobin A1c (HbA1c) ≥ 6.5% or current treatment with blood glucose lowering medication. This study met the criteria of the Declaration of Helsinki, and it was approved by the Ethics Committee of the First Affiliated Hospital of Sun Yat-sen University, which waived the requirement for written informed consent due to this study was retrospectively designed [No. (2022) 202].
2.2. Magnetic resonance imaging acquisition and analysis
All patients underwent brain MRI at 3 Tesla scanners (Siemens Trio Tim or Siemens Verio). In the present study, T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), T2-fluid attenuation inversion recovery (T2-FLAIR), and diffusion-weighted imaging (DWI) were collected for all patients. The parameters were as follows: (1) T1WI, repetition time (TR)/echo time (TE) 500/8.9 ms, and slice thickness (SL) 6.0 mm; (2) T2WI, TR/TE = 4000/100 ms, and SL 6.0 mm; (3) T2-FLAIR, TR/TE = 9000/111 ms, inversion recovery 2,500 ms, and SL 6.0 mm; and (4) TR/TE = 5800/100 ms, b = 0/1000 s/mm2, and SL 5.0 mm.
WMLs were analyzed on T2WI and T2-FLAIR images and semi-quantitatively graded based on the Fazekas grading system. Periventricular hyperintensity (PVH) was scored as 0 for the absence of any lesions, 1 for cap or pencil-like lesions, 2 for halo-like lesions with diameters of 6 mm ~ 10 mm, and 3 for irregular lesions with diameters of more than 10 mm. Deep white matter hyperintensity (DWMH) was scored as 0 for the absence of any lesion, 1 for punctate-like lesions, 2 for small confluent lesions, and 3 for large confluent lesions (Figure 1). For patients with significant brain edema induced by ischemic stroke or intracerebral hemorrhage, the Fazekas score was defined by the presence of a WML in the contralateral hemisphere. The scores of PVH and DWMH were further combined to represent the overall WMLs severity (31). The summation of the PVH score and DWMH score is the final Fazekas score that ranged from 0 to 6. The final Fazekas score was determined by two vascular neurologists (LS and KZ), and any discordant results between the two observers were resolved by consensus. Thus, the numbers of the patients of each final Fazekas score were as follow: final Fazekas score 0 (n = 81), final Fazekas score 1 (n = 62), final Fazekas score 2 (n = 51), final Fazekas score 3 (n = 29), final Fazekas score 4 (n = 16), final Fazekas score 5 (n = 8), and final Fazekas score 6 (n = 12). To minimize the redundancy and maximize the clinical practicability of our model, we assigned the patients with final Fazekas score 0 to 2 in the absent/mild WMLs group and the patients with final Fazekas score 3 to 6 in the moderate–severe WMLs group.
Figure 1. The transverse (A) and coronal (B) sections of MRI of Fazekas scale to semi-quantitatively quantify the WMLs severity. For deep white matter hyperintensity (DWMH): punctate foci, score 1; small confluences, score 2; large confluences, score 3. For periventricular hyperintensity (PVH): caps or pencil-thin linings, score 1; smooth halos with diameters of 6 mm ~ 10 mm, score 2; irregular lesions with diameters of more than 10 mm, score 3. DWMH, deep white matter hyperintensity; PVH, periventricular hyperintensity.
2.3. Retinal images acquisition
Fundus photography was performed on both eyes for all patients without pupil dilatation (at a KOWA nonmyd7, Japan). In brief, patients sat in front of the fundus photography system and were instructed to focus on the center of the viewing system while the fundus was examined. The field of view was set to 45° to enable visualization of the posterior portion of the examined eye. The position of the camera and focal length were finely adjusted so that the optic disk and macula were clearly viewed. The images of both eyes were stored in a workstation for further analysis (Figure 2).
Figure 2. Fundus images of two patients with mild WMLs (A) and severe WMLs (B). The white hollow arrow represents rigid exudation, the black hollow arrow represents arteriovenous pressure, the white arrow represents microhemorrhage, and the black arrow represents retinal artery stenosis. WMLs, white matter lesions.
2.4. Deep learning neural network for retinal images
We used ResNet-18 model as the training model to predict the present moderate–severe WMLs in brain MRI scan by using retinal images. ResNet-18 is a residual network pretrained using the ImageNet dataset, but the associated fully connected softmax layer has been replaced by a supervised classifier. The pre-trained model used in our study is available on the code repository.1 We further modified this pre-trained model for our study. The convolution kernel of the convolutional layer is 7 × 7. The max pooling layer was directly performed by a convolutional layer with stride 2, followed by batch normalization after each convolution operation and before the ReLU activation function. The image was resized to 224 × 224 to match the input image size required by ResNet-18. Then we used a linear classifier to perform feature scaling on the 2048-dimensional output feature vector of the last convolutional layer for dimensionality reduction, and set the final output layer dimension as 2. Each network includes three main modules: the input module, the output module, and the intermediate convolution module. We used the cross-entropy loss function as objective function. The optimizer was Adam, learning rate was 0.001. Finally, the deep-learning model produced a single value ranged 0 to 1 for each retinal image. This value indicates the probability of moderate/severe WMLs that was predicted by this deep-learning model and we took this value for further analysis. The class activation maps for each retina were generated to visualize the weight of predicting WMLs in the sub-regions of retina. The class activation maps for each retinal images were generated by the Grad-CAM algorithm.
2.5. Clinical-laboratory signature
Clinical-laboratory data were examined for all patients. The clinical-laboratory data can be concisely divided into four regiments, including lipid-glucose metabolism, systemic inflammation, and liver and renal function. In detail, the lipid-glucose metabolism regimen included total cholesterol (CHOL), total triglyceride (TG), low density lipoprotein cholesterol (LDL-c), high density lipoprotein cholesterol (HDL-c), apolipoprotein-E (Apo-E), apolipoprotein-A1 (Apo-A1), apolipoprotein-B (Apo-B), Apo-A1/Apo-B ratio, lipoprotein-α (Lp-α), fasting blood glucose (FBG), HbA1c, free fat acid (FFA), fructosamine, and homocysteine. The systemic inflammation regimen included C-reactive protein (CRP), high-sensitivity C-reactive protein (Hs-CRP), and the Hs-CRP/CRP ratio. The liver function regimen included aspartate aminotransferase (AST), alanine aminotransferase (ALT), total protein (TP), total bilirubin (TBIL), albumin, globulin, and albumin/globulin ratio (A/G ratio). The renal function regimen, including creatine and uric acid (UA).
We developed a clinical-laboratory signature representing the metabolism, systemic inflammation, liver and renal function. A least absolute shrinkage and selection operator (LASSO) logistic regression model was implemented to eliminate collinearity, reduce overfitting, filter and select the clinical laboratory data that correlate with moderate–severe WMLs by tuning the hyperparameters that minimize Akaike information criteria (AIC).
2.6. Development of the WMLs severity prediction model
We developed a combined model including demographic-clinical characteristics, the clinical-laboratory signature, and the outputs of the retina deep learning neural network to predict WMLs severity. The demographic-clinical characteristics used in the predictive model were age, sex, SBP and DBP on admission. Age was included in the model as a continuous variable. Male was coded as 1 and female was coded as 0. SBP was further transformed into a hierarchical variable by defining values <140 mmHg as 0, values of 140 ~ 159 mmHg as 1, values of 160 ~ 179 mmHg as 2, and values of ≥180 mmHg as 3. Similarly, DBP was transformed by defining values <90 mmHg as 0, values of 90 ~ 99 mmHg as 1, values of 100 ~ 109 mmHg as 2, and values ≥110 mmHg as 3. The combined model was formulated based on the results of multivariate logistic regression that minimized the penalty parameter conducted by 10-fold cross-validation. Furthermore, a clinical model including demographic-clinical characteristics and clinical-laboratory signatures was generated based on the results of multivariate logistic regression that minimized the penalty parameter conducted by 10-fold cross-validation without the outputs of retina deep learning neural network.
2.7. Statistics
Data for continuous variables are summarized as the mean and standard deviation. Categorical variables were summarized by number and percentage. Univariate logistic regression was carried out in R (v4.0.5) to explore the relationship among demographic-clinical characteristics, individual clinical laboratory data and WMLs severity. The continuous variables were compared by independent-sample t test between patients in the absent/mild WMLs group and patients in the moderate–severe WMLs group. The categorical variables were compared by the chi-square test or Fisher’s exact test. Univariate logistic regression was used to explore the association between clinical-laboratory indices and dichotomized WMLs severity. Odds ratios (ORs) and 95% confident intervals (95% CIs) were used to predict the association between clinical-laboratory data and WMLs severity. The predictive model development and compilation of the clinical-laboratory signature were conducted in Python (v 3.7) and the scikit-learn package.2 The performance of the clinical-laboratory signature, the retinal images, the combined model, and the clinical model was comprehensively predicted by the receiver operating characteristic curve. The area under curve (AUC), accuracy, precision, recall, F1 score, sensitivity, and specificity were calculated. The coefficient of determination (R2) and recall rate were further utilized to predict the performance of the clinical-laboratory signature, and the clinical mode.
3. Results
3.1. Patient clinical characteristics
Two hundred fifty-nine patients were enrolled in our study. The mean age of all patients was 52.3 ± 13.5 years. More than half of the included patients were male. The mean SBP of all patients was 140.4 ± 24.8 mmHg. The mean DBP of all patients was 87.7 ± 15.4 mmHg. The most common comorbidity was hypertension, which was present in 56.8% of all patients. Current smoking (35.1%), statin usage (21.6%), diabetes mellitus (17.4%), alcohol abuse (14.7%) and previous ischemic stroke/TIA (18.5%) were also common among all patients. A small proportion of the patients had atrial fibrillation (2.3%) and coronary heart disease (3.5%). Furthermore, the clinical characteristics of the patients with absent/mild and moderate–severe WMLs were compared. The patients with moderate–severe WMLs were significantly older than those with absent/mild WMLs (p < 0.001). The SBP and DBP on admission were higher in the patients with moderate–severe WMLs than in those with absent/mild WMLs (p < 0.001). The proportions of hypertension, previous ischemic stroke/TIA, and atrial fibrillation were higher in the patients with moderate–severe WMLs than in those with absent/mild WMLs, while the frequency of diabetes mellitus was higher in the patients with absent/mild WMLs. The clinical characteristics of all patients are summarized in Table 1. The diagnoses of our patients of each WMLs group were demonstrated in the Supplementary Table S2. The majority of our patients were diagnosed with cerebral infarction (65.8% for the mild/absent WMLs group, 75.8% for the moderate–severe WMLs group). The proportions of cerebral infarction were not significantly different between the mild/absent and moderate–severe WMLs groups (p = 0.127; Supplementary Table S2). The number of other diagnoses was very small. Hence, the results of the chi-square test for these diagnoses were not reliable. To avoid mistranslating the results, we did not present the p values of these diagnoses.
Table 1. Clinical characteristics of all patients.
3.2. The association between clinical-laboratory data and WMLs severity
The association between clinical-laboratory Data and WMLs severity was summarized by univariate logistic regression (Table 2). Homocysteine (OR: 1.05, 95%CI: 1.01–1.08), globulin (OR: 1.11, 95%CI: 1.04–1.17), UA (OR: 1.00, 95%CI: 1.00–1.01), and creatine (OR: 1.03, 95%CI: 1.02–1.04) were positively associated with moderate–severe WMLs. Meanwhile, a decreased A/G ratio was associated with moderate–severe WMLs. Furthermore, LASSO regression was used to select the clinical-laboratory indices. Globulin (β = 0.01157, p < 0.05), creatine (β = 0.00407; p < 0.05), ApoA1 (β = 0.08155; p < 0.05), and TG (β = −0.00494; p < 0.05) were included in the regression model to fuse a clinical-laboratory signature for each patient.
Table 2. Univariate logistics regression of clinical-laboratory indices and WMLs severity.
3.3. Prediction of severity of WMLs
3.3.1. Performance of retinal deep learning neural network
In this study, 204 images of retinas from the left eyes and 240 images from right eyes from total 259 participants were supplied to two ResNet-18 deep learning neural network frameworks to predict the dichotomized WML severity. The class activation maps and architecture was presented in the Supplementary Figure S1, S2. The models based on both sides of the retina could predict moderate–severe WMLs. The AUC, accuracy, precision, recall, F1 score, sensitivity, and specificity of left retina model predicting moderate–severe WMLs on the test set were 0.73, 0.75, 0.38, 0.13, 0.19, 0.13, and 0.94, respectively (Figure 3; Supplementary Table S1). The model based on the right retina was more powerful to predict WMLs severity. The model based on the right retina predicted moderate–severe WMLs with an AUC of 0.94, accuracy of 0.93, precision of 0.87, recall rate of 0.83, F1 score of 0.85, sensitivity of 0.83, specificity of 0.96 (Figure 3; Supplementary Table S1). The class activation maps indicated high weight of predicting moderate–severe WMLs in the regions containing retinal vasculature (Supplementary Figure S1).
Figure 3. The ROC curves of the combined model (model 1, AUC = 0.95), the clinical model (model 2, AUC = 0.78), left retina model (AUC = 0.73), right retina model (AUC = 0.94) and laboratory signature model (AUC = 0.73). ROC, receiver operating characteristic; AUC, area under the ROC curve.
3.3.2. The laboratory signature and the clinical model
This clinical-laboratory signature included globulin, creatine, Apo-A1, and TG to predict moderate–severe WMLs with a determination R2 of 0.14. This signature showed an AUC of 0.73, accuracy of 0.77, precision of 0.85, recall rate of 0.18, F1 score of 0.29, sensitivity of 0.18, specificity of 0.95 (Figure 3; Supplementary Table S1).
The clinical model combined with the clinical-laboratory signature and the demographic data including age, sex, ranked SBP, and ranked DBP. This model predicted moderate–severe WMLs with a determination R2 of 0.17, an AUC of 0.78, accuracy of 0.76, precision of 0.58, recall rate of 0.30, F1 score of 0.40, sensitivity of 0.30, specificity of 0.93 (Figure 3; Supplementary Table S1).
3.3.3. The combined model for predicting the severity of WMLs
The combined model included age, sex, ranked SBP, ranked DBP, clinical-laboratory signature, the outputs of retina deep learning neural network based on the left and right retinal image (Table 3). The combined model could well recognize moderate–severe WMLs with an AUC of 0.95, accuracy of 0.90, precision of 0.85, recall rate of 0.79, F1 score of 0.82, sensitivity of 0.79, specificity of 0.95 (Figure 3; Supplementary Table S1).
Table 3. Multivariate logistics regression of combined and clinical models.
4. Discussion
To the best of our knowledge, this is the first study to use the combination of retina image, clinical features and laboratory data to develop a comprehensive model for predicting the severity of WMLs. The WMLs is a major threat to public health and early detection and routine follow-up are critical for controlling advance of WMLs and preventing deterioration of cognitive function. WMLs can be induced by various pathological mechanisms. Among all subtypes of WMLs, arteriosclerotic WMLs (namely, age- and vascular risk factor-related WMLs) are the most common in clinical practice (32). We excluded patients with autoimmune diseases, intoxication, metabolic brain disease, and hereditary WMLs. The patients with moderate–severe WMLs were significantly older and had higher SBP and DBP on admission. Previous studies have shown that advanced age and elevated blood pressure are significant causes of arteriosclerosis WMLs (11). Furthermore, elevated homocysteine, a well-known risk factor for arteriosclerosis (11), was found to be higher in the patients with moderate–severe WMLs, which is consistent with previous findings that homocysteine levels are positively correlated with the volume of arteriosclerotic WMLs (33–35). Hence, the WMLs investigated in the current study was further indicated to be of the arteriosclerotic subtype.
The WMLs significantly impair cognitive function (36) and are risk factors of ischemic stroke (37) and gait dysfunction (38). A convenient approach with high accuracy is helpful for screening and follow-up. In this study, we developed a comprehensive model that combined deep learning in retinal images and clinical laboratory data that can reliably predict the severity of WMLs.
ResNet has been applied to predict Alzheimer’s disease (39), diagnose myocardial infarction (40), analyze blood cells (41), diagnose COVID-19 (42), and diagnose malignant tumor (43) and tumor metastasis (44). In addition to the aforementioned research areas, ResNet deep learning models can also be used to detect retinal exudative lesions (45). The ResNet-18 is a robust deep learning network, where the convolutional layers are pre-trained and can start training the network without a large dataset. Therefore, this residual network overcomes the limitation of using deep learning for medical image studies with small datasets. Among all the models, the right retinal images and the combined model well predicted moderate–severe WMLs and outperformed the clinical model and clinical-laboratory signature. Furthermore, the deep learning network based on the right retinal images alone shared similar performance with the combined model in which the retina image was combined with clinical information. Our results were in line with the previous studies that reflect the pathophysiological basis that the retina shares similar vasculature and vascular risk factors (46–50). Previous studies have shown that chronic kidney disease (51) and dyslipidemia (11) were associated with WMLs. Similarly, in our study, the clinical-laboratory signature including certain laboratory indices of lipid metabolism (Apo-A1, TG) and kidney function (creatine) can predict the severity of WMLs. Thus, in our study, the abovementioned pathophysiological basis may explain the good performance of the comprehensive model combining deep learning neural network in retinal images with clinical laboratory data.
Traditionally, the retinal vascular lesion was manually evaluated according the presentations of exudates, vessel tortuosity, arteriovenous nicking and so on. However, this evaluation method was affected by raters’ bias and not quantitative. In our study, we focused on the entire retina image instead of the traditional evaluation method. We developed the deep learning model to analyze the entire image of the retina and show good AUC to predict WMLs. Our result further confirmed the findings of previous studies that the retinal lesions were associated with WMLs. In addition, the regression coefficient of retina deep learning model was higher than the traditional risk factors such as age and blood pressure in the combined model. This result indicated that the retinal images can predict the severity of WMLs.
Several limitations of this study should be mentioned. First, the WMLs severity was dichotomized as absent/mild or moderate–severe according to the Fazekas score in our study. Consequently, the models developed in our study could not quantitatively predict WMLs severity. Recent studies showed that severe WMLs were association with multiple neurological diseases such as cognitive deterioration (6, 52, 53) and ischemic stroke (37), while few study reported the association of mild WMLs and other neurological diseases. Thus, our model showed clinical significance in predicting moderate–severe WMLs. Second, our study was hospital-based and retrospectively designed. Some of retinal images were excluded due to the poor quality. The number of the eligible left retinal images included in our analysis was less than the right, which may explain that the inconformity of the performance of the deep learning neural network between left and right retinal images. Further studies may be needed to explore the potential difference of the predicting performance between left and right retinal images. On the other hand, the AUC of right retina image was high in our study, which indicates that the retinal image and deep learning network are capable of predicting the severity of WMLs.
5. Conclusion
In conclusion, we integrated demographic-clinical characteristics, clinical laboratory data and deep learning on retinal images to develop a comprehensive model. This comprehensive model accurately predicted the severity of WMLs and offered non-invasive, high-throughput and low-cost screening tool for early detection of moderate–severe WMLs in the population in community, resource-poor area or with contraindication of MRI.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by Ethics Committee of the First Affiliated Hospital of Sun Yat-sen University. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.
Author contributions
LS and KZ carried out the study and wrote the manuscript with support from HH, WS, and JL. NC and WG develop the code for ResNet and statistics. HH conceived the original idea. HH and JR supervised the project. All authors contributed to the article and approved the submitted version.
Funding
This work was supported by the Guangdong Key Laboratory of Non-human Primate Research (grant number 202OB121201006) and the Liuzhou Science and Technology Plan Project (grant number 2021YB0104B056).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2023.1168836/full#supplementary-material
Footnotes
References
1. Lin, H, Guo, Y, Di, Q, Zheng, Y, Kowal, P, Xiao, J, et al. Ambient PM(2.5) and stroke: effect modifiers and population attributable risk in six Low- and middle-income countries. Stroke. (2017) 48:1191–7. doi: 10.1161/STROKEAHA.116.015739
2. Ma, Q, Li, R, Wang, L, Yin, P, Wang, Y, Yan, C, et al. Temporal trend and attributable risk factors of stroke burden in China, 1990-2019: an analysis for the global burden of disease study 2019. Lancet Public Health. (2021) 6:e897–906. doi: 10.1016/S2468-2667(21)00228-0
3. Wu, S, Wu, B, Liu, M, Chen, Z, Wang, W, Anderson, CS, et al. Stroke in China: advances and challenges in epidemiology, prevention, and management. Lancet Neurol. (2019) 18:394–405. doi: 10.1016/S1474-4422(18)30500-3
4. Hilal, S, Mok, V, Youn, YC, Wong, A, Ikram, MK, and Chen, CL. Prevalence, risk factors and consequences of cerebral small vessel diseases: data from three Asian countries. J Neurol Neurosurg Psychiatry. (2017) 88:669–74. doi: 10.1136/jnnp-2016-315324
5. Hu, HY, Ou, YN, Shen, XN, Qu, Y, Ma, YH, Wang, ZT, et al. White matter hyperintensities and risks of cognitive impairment and dementia: a systematic review and meta-analysis of 36 prospective studies. Neurosci Biobehav Rev. (2021) 120:16–27. doi: 10.1016/j.neubiorev.2020.11.007
6. Zeng, W, Chen, Y, Zhu, Z, Gao, S, Xia, J, Chen, X, et al. Severity of white matter hyperintensities: lesion patterns, cognition, and microstructural changes. J Cereb Blood Flow Metab. (2020) 40:2454–63. doi: 10.1177/0271678X19893600
7. Group LS. 2001-2011: a decade of the LADIS (Leukoaraiosis and DISability) study: what have we learned about white matter changes and small-vessel disease? Cerebrovasc Dis. (2011) 32:577–88. doi: 10.1159/000334498
8. Soumare, A, Elbaz, A, Zhu, Y, Maillard, P, Crivello, F, Tavernier, B, et al. White matter lesions volume and motor performances in the elderly. Ann Neurol. (2009) 65:706–15. doi: 10.1002/ana.21674
9. Windham, BG, Deere, B, Griswold, ME, Wang, W, Bezerra, DC, Shibata, D, et al. Small brain lesions and incident stroke and mortality: a cohort study. Ann Intern Med. (2015) 163:22–31. doi: 10.7326/M14-2057
10. Jokinen, H, Melkas, S, Madureira, S, Verdelho, A, Ferro, JM, Fazekas, F, et al. Cognitive reserve moderates long-term cognitive and functional outcome in cerebral small vessel disease. J Neurol Neurosurg Psychiatry. (2016) 87:1296–302. doi: 10.1136/jnnp-2016-313914
11. Cannistraro, RJ, Badi, M, Eidelman, BH, Dickson, DW, Middlebrooks, EH, and Meschia, JF. CNS small vessel disease: a clinical review. Neurology. (2019) 92:1146–56. doi: 10.1212/WNL.0000000000007654
12. Cabrera DeBuc, D, Somfai, GM, and Koller, A. Retinal microvascular network alterations: potential biomarkers of cerebrovascular and neural diseases. Am J Physiol Heart Circ Physiol. (2017) 312:H201–12. doi: 10.1152/ajpheart.00201.2016
13. Koller, A, and Toth, P. Contribution of flow-dependent vasomotor mechanisms to the autoregulation of cerebral blood flow. J Vasc Res. (2012) 49:375–89. doi: 10.1159/000338747
14. Patton, N, Aslam, T, Macgillivray, T, Pattie, A, Deary, IJ, and Dhillon, B. Retinal vascular image analysis as a potential screening tool for cerebrovascular disease: a rationale based on homology between cerebral and retinal microvasculatures. J Anat. (2005) 206:319–48. doi: 10.1111/j.1469-7580.2005.00395.x
15. Tso, MO, and Jampol, LM. Pathophysiology of hypertensive retinopathy. Ophthalmology. (1982) 89:1132–45. doi: 10.1016/S0161-6420(82)34663-1
16. Qiu, C, Cotch, MF, Sigurdsson, S, Klein, R, Jonasson, F, Klein, BE, et al. Microvascular lesions in the brain and retina: the age, gene/environment susceptibility-Reykjavik study. Ann Neurol. (2009) 65:569–76. doi: 10.1002/ana.21614
17. Mutlu, U, Cremers, LG, De Groot, M, Hofman, A, Niessen, WJ, Van der Lugt, A, et al. Retinal microvasculature and white matter microstructure: the Rotterdam study. Neurology. (2016) 87:1003–10. doi: 10.1212/WNL.0000000000003080
18. Hughes, AD, Falaschetti, E, Witt, N, Wijetunge, S, Thom, SA, Tillin, T, et al. Association of Retinopathy and Retinal Microvascular Abnormalities with Stroke and cerebrovascular disease. Stroke. (2016) 47:2862–4. doi: 10.1161/STROKEAHA.116.014998
19. Ting, DSW, Pasquale, LR, Peng, L, Campbell, JP, Lee, AY, Raman, R, et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol. (2019) 103:167–75. doi: 10.1136/bjophthalmol-2018-313173
20. Kermany, DS, Goldbaum, M, Cai, W, Valentim, CCS, Liang, H, Baxter, SL, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cells. (2018) 172:1122–31.e9. doi: 10.1016/j.cell.2018.02.010
21. Schmidt-Erfurth, U, Sadeghipour, A, Gerendas, BS, Waldstein, SM, and Bogunović, H. Artificial intelligence in retina. Prog Retin Eye Res. (2018) 67:1–29. doi: 10.1016/j.preteyeres.2018.07.004
22. Gulshan, V, Peng, L, Coram, M, Stumpe, MC, Wu, D, Narayanaswamy, A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. (2016) 316:2402–10. doi: 10.1001/jama.2016.17216
23. Lau, AY, Mok, V, Lee, J, Fan, Y, Zeng, J, Lam, B, et al. Retinal image analytics detects white matter hyperintensities in healthy adults. Ann Clin Transl Neurol. (2019) 6:98–105. doi: 10.1002/acn3.688
24. Cho, BJ, Lee, M, Han, J, Kwon, S, Oh, MS, Yu, KH, et al. Prediction of white matter Hyperintensity in brain MRI using fundus photographs via deep learning. J Clin Med. (2022) 11:3309. doi: 10.3390/jcm11123309
25. Zhang, K, Liu, X, Xu, J, Yuan, J, Cai, W, Chen, T, et al. Deep-learning models for the detection and incidence prediction of chronic kidney disease and type 2 diabetes from retinal fundus images. Nat Biomed Eng. (2021) 5:533–45. doi: 10.1038/s41551-021-00745-6
26. Nasrallah, IM, Pajewski, NM, Auchus, AP, Chelune, G, Cheung, AK, Cleveland, ML, et al. Association of Intensive vs standard blood pressure control with cerebral white matter lesions. JAMA. (2019) 322:524–34. doi: 10.1001/jama.2019.10551
27. Smit, RA, Trompet, S, Sabayan, B, Le Cessie, S, Van der Grond, J, Van Buchem, MA, et al. Higher visit-to-visit Low-density lipoprotein cholesterol variability is associated with lower cognitive performance, lower cerebral blood flow, and greater white matter Hyperintensity load in older subjects. Circulation. (2016) 134:212–21. doi: 10.1161/CIRCULATIONAHA.115.020627
28. Grosu, S, Lorbeer, R, Hartmann, F, Rospleszcz, S, Bamberg, F, Schlett, CL, et al. White matter hyperintensity volume in pre-diabetes, diabetes and normoglycemia. BMJ Open Diabetes Res Care. (2021) 9:e002050. doi: 10.1136/bmjdrc-2020-002050
29. Vemuri, P, Davey, C, Johansen, KL, Zuk, SM, Reid, RI, Thostenson, KB, et al. Chronic kidney disease associated with worsening white matter disease and ventricular enlargement. J Alzheimers Dis. (2021) 83:1729–40. doi: 10.3233/JAD-210604
30. Low, A, Mak, E, Rowe, JB, Markus, HS, and O'Brien, JT. Inflammation and cerebral small vessel disease: a systematic review. Ageing Res Rev. (2019) 53:100916. doi: 10.1016/j.arr.2019.100916
31. Fazekas, F, Chawluk, JB, Alavi, A, Hurtig, HI, and Zimmerman, RA. MR signal abnormalities at 1.5 T in Alzheimer's dementia and normal aging. AJR Am J Roentgenol. (1987) 149:351–6. doi: 10.2214/ajr.149.2.351
32. Wardlaw, JM, Smith, C, and Dichgans, M. Mechanisms of sporadic cerebral small vessel disease: insights from neuroimaging. Lancet Neurol. (2013) 12:483–97. doi: 10.1016/S1474-4422(13)70060-7
33. Li, M, Fu, B, and Dong, W. Correlations between plasma homocysteine and MTHFR gene polymorphism and white matter lesions. Folia Neuropathol. (2018) 56:301–7. doi: 10.5114/fn.2018.80863
34. Wang, X, Yin, H, Ji, X, Sang, S, Shao, S, Wang, G, et al. Association between homocysteine and white matter hyperintensities in rural-dwelling Chinese people with asymptomatic intracranial arterial stenosis: a population-based study. Brain Behav. (2021) 11:e02205. doi: 10.1002/brb3.2205
35. Lee, KO, Woo, MH, Chung, D, Choi, JW, Kim, NK, Kim, OJ, et al. Differential impact of plasma homocysteine levels on the periventricular and subcortical white matter Hyperintensities on the brain. Front Neurol. (2019) 10:1174. doi: 10.3389/fneur.2019.01174
36. Prabhakaran, S. Blood pressure, brain volume and white matter Hyperintensities, and dementia risk. JAMA. (2019) 322:512–3. doi: 10.1001/jama.2019.10849
37. Derraz, I, Abdelrady, M, Gaillard, N, Ahmed, R, Cagnazzo, F, Dargazanli, C, et al. White matter Hyperintensity burden and collateral circulation in large vessel occlusion stroke. Stroke. (2021) 52:3848–54. doi: 10.1161/STROKEAHA.120.031736
38. Oveisgharan, S, Yu, L, Poole, VN, Evia, AM, Barnes, LL, Schneider, JA, et al. Association of White Matter Hyperintensities with Pathology and Progression of parkinsonism in aging. JAMA Neurol. (2021) 78:1494–502. doi: 10.1001/jamaneurol.2021.3996
39. Fulton, LV, Dolezel, D, Harrop, J, Yan, Y, and Fulton, CP. Classification of Alzheimer's disease with and without imagery using gradient boosted machines and ResNet-50. Brain Sci. (2019) 9:212. doi: 10.3390/brainsci9090212
40. Han, C, and Shi, L. ML-ResNet: a novel network to detect and locate myocardial infarction using 12 leads ECG. Comput Methods Prog Biomed. (2020) 185:105138. doi: 10.1016/j.cmpb.2019.105138
41. Ma, L, Shuai, R, Ran, X, Liu, W, and Ye, C. Combining DC-GAN with ResNet for blood cell image classification. Med Biol Eng Comput. (2020) 58:1251–64. doi: 10.1007/s11517-020-02163-3
42. Ko, H, Chung, H, Kang, WS, Kim, KW, Shin, Y, Kang, SJ, et al. COVID-19 pneumonia diagnosis using a simple 2D deep learning framework with a single chest CT image: model development and validation. J Med Internet Res. (2020) 22:e19569. doi: 10.2196/19569
43. Xi, IL, Zhao, Y, Wang, R, Chang, M, Purkayastha, S, Chang, K, et al. Deep learning to distinguish benign from malignant renal lesions based on routine MR imaging. Clin Cancer Res. (2020) 26:1944–52. doi: 10.1158/1078-0432.CCR-19-0374
44. Yuan, Z, Xu, T, Cai, J, Zhao, Y, Cao, W, Fichera, A, et al. Development and validation of an image-based deep learning algorithm for detection of synchronous peritoneal Carcinomatosis in colorectal Cancer. Ann Surg. (2022) 275:e645–51. doi: 10.1097/SLA.0000000000004229
45. Khojasteh, P, Passos Júnior, LA, Carvalho, T, Rezende, E, Aliahmad, B, Papa, JP, et al. Exudate detection in fundus images using deeply-learnable features. Comput Biol Med. (2019) 104:62–9. doi: 10.1016/j.compbiomed.2018.10.031
46. Levy, BI, Ambrosio, G, Pries, AR, and Struijker-Boudier, HA. Microcirculation in hypertension: a new target for treatment? Circulation. (2001) 104:735–40. doi: 10.1161/hc3101.091158
47. London, A, Benhar, I, and Schwartz, M. The retina as a window to the brain-from eye research to CNS disorders. Nat Rev Neurol. (2013) 9:44–53. doi: 10.1038/nrneurol.2012.227
48. Wong, KH, Hu, K, Peterson, C, Sheibani, N, Tsivgoulis, G, Majersik, JJ, et al. Diabetic retinopathy and risk of stroke: a secondary analysis of the ACCORD eye study. Stroke. (2020) 51:3733–6. doi: 10.1161/STROKEAHA.120.030350
49. Scheifele, HM, Ulbrich, P, Garz, C, Carare, RO, Heinze, HJ, Schreiber, S, et al. Retinal vascular pathology in a rat model of cerebral small vessel disease. Front Neurol. (2020) 11:533. doi: 10.3389/fneur.2020.00533
50. Mutlu, U, Bonnemaijer, PWM, Ikram, MA, Colijn, JM, Cremers, LGM, Buitendijk, GHS, et al. Retinal neurodegeneration and brain MRI markers: the Rotterdam study. Neurobiol Aging. (2017) 60:183–91. doi: 10.1016/j.neurobiolaging.2017.09.003
51. Wei, CS, Yan, CY, Yu, XR, Wang, L, Zhang, R, Jiang, JY, et al. Association between white matter Hyperintensities and chronic kidney disease: a systematic review and Meta-analysis. Front Med (Lausanne). (2022) 9:770184. doi: 10.3389/fmed.2022.770184
52. Wang, J, Liang, Y, Chen, H, Wang, W, Wang, Y, Liang, Y, et al. Structural changes in white matter lesion patients and their correlation with cognitive impairment. Neuropsychiatr Dis Treat. (2019) 15:1355–63. doi: 10.2147/NDT.S194803
Keywords: cerebrovascular disease, white matter lesions, retinal images, deep learning, prediction model
Citation: Shu L, Zhong K, Chen N, Gu W, Shang W, Liang J, Ren J and Hong H (2023) Predicting the severity of white matter lesions among patients with cerebrovascular risk factors based on retinal images and clinical laboratory data: a deep learning study. Front. Neurol. 14:1168836. doi: 10.3389/fneur.2023.1168836
Edited by:
Hanna Gwendolyn Zimmermann, Charité University Medicine Berlin, GermanyReviewed by:
Seyedamirhosein Motamedi, Charité University Medicine Berlin, GermanyMarine Tanashyan, Research Center of Neurology, Russia
Copyright © 2023 Shu, Zhong, Chen, Gu, Shang, Liang, Ren and Hong. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Hua Hong, aG9uZ2h1YUBtYWlsLnN5c3UuZWR1LmNu; Jiangtao Ren, aXNzcmp0QG1haWwuc3lzdS5lZHUuY24=
†These authors have contributed equally to this work and share first authorship