Identifying depression with mixed features: the potential value of eye-tracking features

Objectives: To investigate the utility of eye-tracking features as a neurobiological marker for identifying depression with mixed features (DMF), a psychiatric disorder characterized by the presence of depressive symptoms alongside subsyndromal manic features, thereby complicating both diagnosis and therapeutic intervention.

Methods: A total of 93 participants were included, comprising 41 patients with major depressive disorder (MDD), of whom 20 were classified as DMF, and 52 healthy controls (HC). Eye-tracking features were collected using an infrared-based device, and participants were evaluated using clinical scales including the Montgomery-Åsberg Depression Rating Scale (MADRS), Young Mania Rating Scale (YMRS), and Brief Psychiatric Rating Scale (BPRS). Performance of extreme gradient boosting (XGBoost) model based on demographic and clinical characteristics was compared with that of the model created after adding ocular movement data.

Results: Significant differences were observed in certain eye-tracking features between DMF, MDD, and HC, particularly in orienting saccades and overlapping saccades. Incorporating eye-tracking features into the XGBoost model enhanced the predictive accuracy for DMF, as evidenced by an increase in the area under the curve (AUC) from 0.571 to 0.679 (p < 0.05), representing an 18.9% improvement. This suggests a notable enhancement in the model’s ability to distinguish DMF from other groups. The velocity of overlapping saccades and task completion time during free viewing were identified as significant predictive factors.

Conclusion: Eye-tracking features, especially the velocity of overlapping saccades and free viewing task completion time, hold potential as non-invasive biomarkers for the identification of DMF. The integration of these parameters into the XGBoost machine learning model significantly improved the accuracy of DMF diagnosis, offering a promising approach for enhancing clinical decision-making in psychiatric settings.

1 Introduction

Depression ranks among the most prevalent mental disorders, characterized by high incidence, recurrence, and disability rates, which impose a significant burden on society, families, and individuals (1). Approximately 11.6% of patients with depression exhibit mixed features (2), defined by the presence of typical depressive symptoms alongside some manic or hypomanic characteristics, without meeting the diagnostic criteria for mania or hypomania (3, 4). Mixed features complicate the diagnosis and treatment of depression, as conventional single antidepressant therapies may be insufficient to address the complexities of mixed presentations and potentially exacerbate manic symptoms (5, 6). Furthermore, the presence of mixed features not only increases the risk of suicide among patients (2), but also contribute to treatment resistance and a poor prognosis (7).

Diagnosing depression with mixed features (DMF) presents significant challenges in clinical practice due to the diverse and complex nature of its symptoms, which can mimic those of mania or hypomania (5). Clinicians must remain highly vigilant and conduct comprehensive assessments to accurately differentiate between the various manifestations of depression and bipolar disorder. In the absence of specific biomarkers, the diagnosis of DMF relies heavily on patients’ subjective reports and clinical observations, increasing the risk of both misdiagnosis and missed diagnosis (2). Consequently, the development of novel diagnostic strategies and the improvement of accuracy in identifying mixed features have become key research priorities in the field of mental health.

Eye-tracking technology, a non-invasive physiological measurement technique, has increasingly gained prominence in recent years within the fields of neurology, psychiatry and psychology research. This technology offers direct insights into cognitive processes, attention allocation, and emotional states by monitoring parameters such as gaze point, eye movement velocity, and eye movement trajectory (8). In the field of mental health, eye-tracking technology has been employed to investigate a variety of mental illnesses (9–11), due to its potential to elucidate the neurobiological mechanisms underlying these conditions.

Compared with healthy individuals, patients with depression exhibit distinct eye-tracking features, including a preference for negative emotional stimuli, a reduced fixation duration and number on positive stimuli, and an increased fixation duration and number on negative stimuli (12, 13). These features may serve as non-invasive biomarkers for diagnosing depression. Combining eye movement tracking with facial movement tracking can achieve up to 80% accuracy in predicting the worsening of depressive symptoms (14). Furthermore, distinct eye movement differences between mania and depression have been identified. Adolescents with depression exhibit a reduced response during the initial cognitive processing of emotional stimuli, while adolescents with mania do not significantly differ from healthy individuals (15). Eye-tracking features also differ among patients with bipolar disorder presenting different clinical manifestations. Patients with psychotic bipolar disorder exhibit more pronounced smooth pursuit dysfunction compared to those with non-psychotic bipolar disorder, which can be used to distinguish between the two (16).

While eye-tracking technology has been utilized to differentiate between patients with depression and healthy controls, as well as between individuals with bipolar disorder and those with unipolar depression (17), no studies to date have investigated the potential of eye-tracking technology in identifying MDF in patients with depression. Given the significant overlap between mixed features and manic symptoms, we hypothesize that patients with DMF will exhibit distinct eye-tracking features compared to those with pure depression. This study aims to collect and analyze eye-tracking features in patients with DMF and those without mixed features. Additionally, this study will develop a machine learning prediction model incorporating demographic data, clinical scales, and eye-tracking features to assess the potential of eye movement metrics in identifying mixed features. We anticipate that this research will provide a novel perspective for accurately identifying DMF.

2 Methods

2.1 Participants

Our study included a total of 93 participants, consisting of 41 patients with Major Depressive Disorder (MDD) and 52 healthy controls (HC). The MDD patients underwent clinical assessments conducted by two experienced clinical psychiatrists (each holding a doctoral degree and possessing over 5 years of experience in psychiatry) and were subsequently categorized into the DMF group (N = 20) and the non-DMF group (labeled as MDD, N = 21) based on the diagnostic criteria of DSM-5. The study flowchart is presented in Figure 1.

Figure 1

Figure 1. Flow chart of the study. MDD, Major depressive disorder; DMF, Depression with mixed features; HC, Healthy controls.

The patient group was recruited from psychiatric inpatients at Guangdong Provincial People’s Hospital in Guangzhou, China. The inclusion criteria were as follows: (a) meeting the diagnostic criteria for MDD as specified in the DSM-5; (b) no prior physical therapy, including modified electroconvulsive therapy or repetitive transcranial magnetic stimulation; (c) aged 10–25 years; (d) of Han ethnicity; (e) right-handed; (f) able to cooperate with the study; and (g) capable of understanding the study’s content with family assistance, willing to participate, and able to provide informed consent. The exclusion criteria included: (a) comorbidity with other mental or neurological disorders; (b) MDD in clinical remission or resolved; and (c) severe physical illness.

Healthy participants were recruited via community advertisements in Guangzhou, China. The inclusion criteria for healthy participants were: (a) no mental illness as per the DSM-5; (b) no family history of mental disorders; with the remaining criteria matching those of the patient group from (c) to (g). The exclusion criteria for healthy participants were: (a) neurological diseases; and (b) severe physical illnesses.

It is important to highlight that DMF is most prevalent in children and adolescents (7), whereas the peak onset of depression typically occurs later in life. To ensure comparability of the results, we limited the age range of all study participants to 10–25 years.

2.2 Demographic and clinical scale assessment

This study collected demographic information, including gender, age, educational attainment, and history of leave of absence, from all participants. Each participant was interviewed by two experienced clinical psychiatrists, both holding doctorate degrees and possessing over 5 years of psychiatric experience. Interviews and standardized scales were utilized to evaluate suicide history, depression severity, mania levels, and the severity of psychotic symptoms.

The Montgomery-Asberg Depression Rating Scale (MADRS) is utilized to assess the severity of depressive symptoms (18). The Young Mania Rating Scale (YMRS) is employed to evaluate the level of mania (19). The Brief Psychiatric Rating Scale (BPRS) is utilized to assess the severity of psychotic symptoms, comprising five subscales: anxiety/depression, lack of energy, cognitive disturbance, activation, and hostility/suspiciousness (20). The Chinese version of the BPRS includes two additional items: impaired insight and work incapacity (21).

2.3 Eye-tracking

The eye-tracking features were recorded using a pupil center corneal reflections method that sampled at a 90-Hz rate. Eye-tracking system is name EyeKnow (Beijing CAS-Ruiyi Information Technology Co., Ltd). Prior to the experiment, a five-point calibration procedure was performed to calibrate the eye movement data with a maximum calibration error of 2° in radius. The eye-tracking features were analyzed and calculated by the eye-tracking system’s embedded data processing module. Participants received standardized and comprehensive instructions before testing to minimize potential variability. A detailed protocol was followed, outlining the sequence of tests to further reduce variability. The total time required to complete the entire task was approximately 10 min, including time for calibration, an audio-visual demonstration, and task execution.

The eye-tracking paradigms include: (1) Smooth Pursuit Tracking: A green dot moved along a sinusoidal trajectory with a horizontal/vertical/circular amplitude of 20° and a frequency of 0.2 Hz. Participants were asked to continuously track the sinusoidal movement of the target dot. Participants complete two trials. The first trial involves a single task, while the second trial requires participants to manage two tasks simultaneously, such as tracking a moving target while simultaneously remembering or identifying its characteristics. The single task assesses participants’ fundamental visuomotor processing abilities, whereas the dual-task assesses these abilities in a distracting multitasking environment. The recorded metrics for this component include latency, tracking speed, tracking acceleration, deviation count, total deviation, and mean deviation. Additionally, the single task also requires recording overshoot count, undershoot count, tracking gain, and reaction time. (2) Orienting Saccades (OS): A green target dot was displayed, and participants were instructed to rapidly shift their gaze from one point to another. This task evaluated participants’ saccadic control and the speed of their visual attention shifts. The recorded metrics included accuracy, latency, fastest completion time, average completion time, average saccade speed, and maximum saccade speed. (3) Overlapping Saccades (OLS): Participants were instructed to scan a series of overlapping targets. This task evaluated participants’ saccadic control and attention allocation abilities in processing continuous visual stimuli. The recorded metrics included accuracy, latency, fastest completion time, average completion time, average saccade speed, maximum saccade speed, overshoot count, undershoot count, multi-step saccade count, saccade gain, and maximum vertical eye movement. (4) Anti-saccades (AS): Similar to orienting saccades, participants were instructed to execute a saccade to the opposite position of the target dot as quickly as possible. This task evaluated participants’ inhibitory control and proactive attention shifting abilities. The recorded metrics included accuracy, latency, fastest completion time, average completion time, uncorrected error rate, average saccade speed, maximum saccade speed, and maximum vertical eye movement. (5) Free Viewing (FV): The task allowed participants to freely observe a scene without specific instructions. This task evaluated participants’ natural visual exploration behavior and attention allocation. The recorded metrics included saccade count, fixation count, and task average completion time. (6) Go/No Go Task; Participants were required to respond to a series of stimuli, where some stimuli (Go) required a response, while others (No Go) required inhibiting a response. This task evaluated participants’ selective attention and inhibitory control abilities. The recorded metrics included accuracy and latency. For data collection methods across different paradigms, refer to relevant research papers (12, 22, 23).

2.4 Machine learning (ML)

We constructed a dataset using baseline demographic characteristics such as gender, age, educational attainment, and school absenteeism, along with clinical questionnaires (BPRS, MADRS, YMRS) and eye movement data (OS, OLS, AS, FV). These features were used to train a model to predict whether each patient with depression exhibits mixed features. The data were normalized prior to analysis. The dataset was then randomly divided into a training set and a test set in a 7:3 ratio.

The machine learning algorithm employing extreme gradient boosting (XGBoost), implemented using the XGBoost library in Python 3.7, was used to develop predictive models that effectively perform classification tasks. To investigate the correlation between eye movement parameters and mixed features, two distinct feature sets were developed. XGBoost-1 was trained using demographic characteristics (gender, age, educational attainment, and school absenteeism) and clinical questionnaires (BPRS, MADRS, YMRS), without including any eye movement parameters. In contrast, XGBoost-2 includes the aforementioned characteristics and eye movement parameters (OS, OLS, AS, FV). To optimize the performance of the XGBoost model, we systematically tuned the key hyperparameters. Supplementary Table 1 shows the ranges of the hyperparameters that were tuned and the final optimal parameter values that were determined. Supplementary Figure 1 illustrates the changes in cross—validation AUC scores during the Bayesian optimization process. The model’s predictive discrimination ability was evaluated using accuracy, precision, recall, F1 score, and area under the curve (AUC), calculated using R-Studio 4.2.2. Model performance was compared using AUC, decision curve analysis (DCA), and net reclassification improvement (NRI). The AUC of the two models was compared using the DeLong test in MedCalc 22.021. Additionally, the local explanation technique SHAP (Shapley Additive Explanations) was used to identify which features significantly influence the model’s output and to elucidate the model’s decision-making process.

2.5 Statistical analysis

Descriptive analyses were conducted on the demographic data and clinical symptom assessment data from the three subject groups in this study, including demographic characteristics, clinical questionnaires, and eye movement data. The analysis of variance (ANOVA) F test and the Pearson chi-square test were used to compare continuous and categorical data among the three groups: DMF, MDD, and HC. Groups that exhibited statistically significant differences after the analysis of variance underwent further post hoc comparisons. Spearman correlation analysis was used to assess the relationship between disease group scale scores and eye movement parameters. The significance level was set at p < 0.05. All statistical analyses were performed using SPSS version 26.0.

3 Results

3.1 Demographic and clinical characteristics

This study included 93 participants, comprising 20 individuals with DMF, 21 with MDD, and 52 healthy controls (HC). Table 1 presents the demographic and clinical characteristics across the three groups. A higher proportion of women was observed in both the DMF and MDD groups, but the statistical difference between these two groups was not significant (Female: DMF, 85%; MDD, 76.2%; χ² = 0.506, p = 0.477; HC, 50%). There were no dropouts in the HC group, while a proportion of participants in both the DMF and MDD groups did drop out of school. The difference in dropout rates between these two groups was also not statistically significant (School Dropout: DMF, 35%; MDD, 19%; χ² = 1.328, p = 0.249). Both the DMF and MDD groups exhibited suicidal ideation or behavior, with no significant statistical difference between them (χ² = 1.205, p = 0.272). In terms of clinical characteristics, with the exception of the Young Mania Rating Scale (YMRS) results, no statistically significant differences were found between the DMF group and the MDD group (all p > 0.05). The Impaired Insight score was significantly higher in the DMF group than in the HC group (t = −2.878, p = 0.005), although the difference from the MDD group was not statistically significant. Additionally, the difference in Impaired Insight scores between the HC group and the MDD group was not statistically significant. Regarding YMRS scores, the DMF group exhibited significantly higher scores than both the MDD group and the HC group (DMF vs. MDD: t = 3.041, p = 0.006; DMF vs. HC: t = −3.652, p = 0.002), while the statistical difference between the MDD and HC groups was not significant (MDD vs. HC: t = −1.594, p = 0.126).

Table 1

Table 1. Demographic and clinical features.

3.2 Eye-tracking features

Table 2 presents the eye-tracking features for each group. The metrics for smooth pursuit tracking (both dual task and single task), eye fixations, and responsive search score did not show statistically significant differences among the three groups. However, several indicators related to OS, OLS, AS, FV, and the Go/NoGo Task showed statistically significant differences among the groups (p < 0.05). The specific indicators are as follows: OS-fastest completion time (F = 3.385, p = 0.039), OS-maximum saccade speed (F = 3.577, p = 0.033), OLS-fastest completion time (F = 3.128, p = 0.049), OLS-average saccade speed (F = 4.233, p = 0.01), OLS-maximum saccade speed (F = 3.403, p = 0.038), OLS-undershoot count (F = 3.264, p = 0.043), OLS-saccade gain (F = 4.245, p = 0.018), AS-average completion Time (F = 3.181, p = 0.047), FV-saccade count (F = 8.028, p = 0.001), FV-fixation count (F = 7.583, p = 0.001), FV-task average completion time (F = 3.378, p = 0.039), Go/NoGo-accuracy (F = 3.922, p = 0.024).

Table 2

Table 2. Oculometric parameters.

Figure 2 displays the results of post hoc comparisons among the three groups. The OS-maximum saccade speed, OLS-average saccade speed, and OLS-maximum saccade speed were slower in the DMF than in the MDD, while the OLS-fastest completion time was longer in the DMF than in the MDD. No statistically significant differences were observed for the remaining indicators between the DMF and MDD groups. The indicators that showed differences between the DMF and HC include OS-fastest completion time, OLS-average saccade speed, OLS-saccade gain, and FV-task average completion time. In contrast, the indicators that showed differences between the MDD and HC are FV-saccade count, FV-fixation count, and Go/NoGo accuracy.

Figure 2

Figure 2. Inter-group differences in eye-tracking features. (a) Orienting saccades: fastest completion time; (b) Orienting saccades: maximum saccade speed; (c) Overlapping saccades: fastest completion time; (d) Overlapping saccades: average saccade speed; (e) Overlapping saccades: maximum saccade speed; (f) Overlapping saccades: undershoot count; (g) Overlapping saccades: saccade sain; (h) Anti-saccades: average completion time; (i) Free viewing: saccade count; (j) Free viewing: fixation count; (k) Free viewing: task average completion time; (l) Go/NoGo task: accuracy.

3.3 Correlation analysis between eye-tracking features and clinical scales

Figure 3 presents the correlation analysis between eye-tracking features and clinical scales in the DMF and MDD. Notably, only the OS-maximum saccade speed showed a negative correlation with BPRS-impaired insight (p < 0.05), while no significant correlations were observed between other eye-tracking features and clinical scales. These findings suggest that eye-tracking features represent a distinct set of characteristic data for patients with depression, independent of clinical scale scores.

Figure 3

Figure 3. Heatmap of correlations between scale results and eye-tracking features. OLS, overlapping saccades; OS, Orienting saccades; FV, Free viewing; AS, Anti-saccades; BPRS, Brief Psychiatric Rating Scale; MADRS, Montgomery-Asberg Depression Rating Scale; YMRS, Young Mania Rating Scale.

3.4 Predictive value of eye-tracking features for DMF

The dataset was randomly divided into two groups: a training set comprising 70% of the 34 patients and a test set consisting of the remaining 30%. XGBoost-1 was trained using demographic and clinical features, achieving an area under the curve (AUC) of 0.571 (95% confidence interval [CI]: 0.143, 0.964) in the test set. When additional eye movement parameters were incorporated, including OS-fastest completion time, OS-maximum saccade speed, OLS-fastest completion time, OLS-average saccade speed, OLS-maximum saccade speed, OLS-undershoot count, OLS-saccade gain, AS-average completion time, FV-saccade count, FV-fixation count, FV-task average completion time, and Go/NoGo-accuracy, the AUC of XGBoost-2 improved to 0.679 (95% CI: 0.321, 0.964) (Figure 4). The DeLong test indicates that the AUC of the XGBoost algorithm improved by 0.108 (p = 0.039), reflecting an 18% enhancement in performance. Additionally, the metrics of accuracy, precision, recall, and F1 score increased by 33.2, 12.4, 50.1, and 18.9%, respectively (Table 3). DCA shows that when the risk threshold is set between 0.69 and 0.78, XGBoost-2 exhibits superior diagnostic efficiency and net benefit compared to XGBoost-1.

Figure 4

Figure 4. The performance of the mixed features pre-prediction model, as assessed by the receiver operating curve and decision curve analysis, for XGBoost-1 and XGBoost-2 on the testing dataset. (a) The ROC curve comparison; (b) Decision curve analysis. AUC, area under the curve; ROC, receiver operating characteristic.

Table 3

Table 3. Prediction results of XGBoost.

3.5 Eye-tracking features prediction the optimal feature set of DMF

Figure 5 illustrates the optimal feature set for the XGBoost-2 model. The YMRS emerges as the most significant predictor of XGBoost-2 classification, followed closely by OLS-maximum saccade speed and BPRS-hostility/suspiciousness. Among the eye-tracking features, the top three predictive factors are OLS-maximum saccade speed, FV-task average completion time, and OLS-average saccade speed. The SHAP analysis indicates that a longer AS-average completion time, a slower OLS-maximum saccade speed, and lower MADRS scores are closely associated with the identification of mixed features. This evidence supports the classification of these features as DMF by the XGBoost-2 model. By integrating results from eye-tracking features (OLS, AS, FV) and clinical scales (YMRS, MADRS), the predictive capacity of the XGBoost model for DMF is enhanced.

Figure 5

Figure 5. The feature sets of the top-performing model were evaluated using SHAP analysis. (a) Features are ordered from the most influential to the least, based on their contribution to the predictive accuracy of XGBoost-2 in forecasting mixed features. (b) Feature Decision Analysis illustrates how the model utilizes these features to make decisions. The lines in the figure depict the trajectory of changes in model output values as feature values vary. It can be observed that changes in AS—Average Completion Time lead to significant fluctuations in model output values (within a range of approximately −2 to 2), while features at the bottom, such as FV—Fixation Count, have relatively smaller impacts on the model output. (c) The SHAP Value Distribution shows the contribution distribution of each feature to the model’s prediction results. The influence of AS—Average Completion Time is the most significant. When its value is high (red points), it tends to have a positive contribution to the prediction results, while a low value (blue points) has a negative contribution. OLS—Maximum Saccade Speed and MADRS also show strong predictive power. OLS, overlapping saccades; OS, Orienting saccades; FV, Free viewing; AS, Anti-saccades; BPRS, Brief Psychiatric Rating Scale; MADRS, Montgomery-Asberg Depression Rating Scale; YMRS, Young Mania Rating Scale.

4 Discussion

This study investigated the relationship between eye-tracking features and depression, aiming to develop a prediction model for mixed features using machine learning methods. A key finding was the identification of significant differences in eye-tracking features among DMF, MDD, and HC. Notably, in certain indicators of OS and OLS, the DMF exhibited slower maximum and average saccade speeds, as well as longer fastest completion times compared to the MDD. Furthermore, the DMF demonstrated significantly more saccades and fixations during the FV than the HC, suggesting that DMF may experience specific cognitive impairments related to visual attention and processing speed. Another significant finding was that incorporating eye-tracking features into the machine learning prediction model markedly enhanced its ability to predict DMF. This indicates that eye-tracking features provide valuable additional information for identifying DMF, underscoring their potential role in diagnosing mental health disorders.

Patients with depression exhibit abnormalities in eye-tracking features, potentially linked to the neural mechanisms underlying emotion regulation and cognitive control (13). We found that eye-tracking features are also correlated with DMF. Notably, the reduced performance of DMF patients in the maximum saccade speed of OS and the average saccade speed of OLS may indicate distinct neurocognitive substrates compared to patients with non-mixed depression. These abnormalities may reflect a reduced ability to process emotional stimuli, which is consistent with the attentional bias and emotional regulation deficits often observed in patients with depression (5). A decreased ability to process emotional stimuli often indicates a poorer prognosis (24). Moreover, the observed slowing of eye-tracking features may be associated with functional abnormalities in the prefrontal cortex and limbic system, which are crucial for regulating attention and emotional responses (25, 26). Some studies suggest that patients with bipolar disorder exhibit varying attention preferences for emotion-related images at different clinical stages, potentially linked to alterations in eye movement parameters (27). Our findings support this notion and specifically focus on DMF, a subgroup believed to share certain characteristics with patients diagnosed with bipolar disorder, yet have received comparatively less attention in previous studies (6).

Eye-tracking technology has garnered significant attention for its applications in mental disorder-related conditions, due to its non-invasive nature and its ability to directly measure visual attention allocation. This technology can reveal patients’ attention biases when processing emotional stimuli, thereby aiding in understanding the pathological mechanisms underlying depression (28). Researchers have utilized eye tracking within the emotion recognition paradigm to explore the connections between MDD, childhood trauma, and emotional processing (29). In terms of disease diagnosis, the parameters derived from eye tracking provide an objective tool for diagnosis, thereby reducing reliance on patients’ subjective reports (14). Our study also found that there were certain correlations among the BPRS, YMRS, and MADRS scores in MDD patients, while there were almost no correlations between eye-tracking features and the above—mentioned scale results. This confirms that eye-tracking features are a set of independent parameters that can bypass the ambiguity of scales and be included in the objective diagnostic markers for MDD. Moreover, the application of eye-tracking technology can go beyond diagnosis; it can also be used to monitor disease progression and evaluate treatment effectiveness. By identifying patients’ responses to treatment over an extended period, clinicians can adjust treatment plans in a timely manner for more precise interventions. Currently, some researchers are investigating the use of eye-tracking technology for long-term assessment of the severity of depression in order to better adjust treatment strategies (30).

Identifying mixed features in patients with depression presents a significant challenge in psychiatry. Table 4 summarizes the advantages and disadvantages of the existing diagnostic schemes for MDF. Current diagnostic standards primarily rely on symptomatology, leading to a high rate of missed diagnoses and misdiagnoses. Some studies have used the YMRS to identify DMF, employing criteria such as a Depressive Symptomatology-Clinician-Rated Version score ≥ 15 and a YMRS score between 2 and 12. However, researchers have noted that existing scales are insufficient for detecting primary DMF (31, 32). In this study, the XGBoost-1 model, incorporating demographic and clinical scales (including YMRS), yielded an AUC of only 0.571, indicating that common clinical scales may lack reliability in identifying DMF, consistent with previous findings. In contrast, the XGBoost-2 model, integrating eye-tracking features, achieved an AUC of 0.679, representing an 18.9% improvement, along with enhanced accuracy and precision. SHAP analysis identified that certain indicators within OLS, AS, and FV significantly influence the prediction model. These results provide evidence that eye-tracking, an emerging auxiliary technology, can be applied in clinical settings. As an objective indicator, clinicians may consider incorporating eye-tracking features into early screening protocols for DMF to achieve more accurate diagnoses.

Table 4

Table 4. Advantages and disadvantages of MDF diagnostic schemes.

Additionally, the application of ML in neurological and psychiatric disorders is becoming an increasingly important trend. By analyzing extensive clinical, psychological, and biological data, machine learning models can identify patterns and trends related to neurological and psychiatric disorders, thereby supporting the prediction, diagnosis, and treatment of these conditions. For instance, 14 types of ML algorithms are effective for early detection of cognitive impairment (33). In depression research, XGBoost and network analysis help identify depression-related factors and their relationships, and can be applied to epidemiological studies using large survey data (34, 35). By incorporating multimodal data and combining it with ML, depression diagnosis technology may further develop, particularly in improving the accuracy of identifying depression subtypes.

When using ML in research, it is crucial to be cautious about over-fitting, as it can lead to a model performing well on the training data but poorly on unseen data, thereby severely affecting the model’s generalization ability. In this study, we used the Bayesian optimization algorithm to automatically search for the optimal hyperparameter combination for the XGBoost model: limiting the maximum depth of trees (max depth: 3–10) to prevent the trees from becoming too complex, using gamma (0–1) and min child weight (1–10) parameters to control the threshold for tree node splitting to reduce model complexity, adopting a small learning rate range (0.01–0.3) to stabilize model training, and combining 5-fold cross-validation (cv. = 5) to evaluate the model’s performance on different data subsets. This optimization strategy can effectively prevent over-fitting while maintaining the model’s predictive performance.

This study does have certain limitations. Firstly, the relatively small sample size may have limited the statistical power and generalizability of the results. Secondly, participants were recruited from specific geographical locations and cultural backgrounds, which may affect the representativeness of the sample and the applicability of the findings. In visual search tasks, Saudi participants were less efficient in their search behavior than British participants (36); in fixation tasks, Czech participants focused more on the focal objects measured by fixation counts, while participants from Taiwan spent more time fixating on the background (37); hence, studies involving a more diverse population are necessary to validate our results. Additionally, the measurement of eye-tracking features could be influenced by individual differences, such as medication use or cognitive ability (38, 39), and these factors may not have been adequately controlled in this study. Lastly, while we utilized common machine learning models for data analysis, the predictive accuracy of these models may be influenced by data quality and feature selection. Model performance may vary across different datasets and clinical settings, indicating that further validation studies are essential to assess the model’s robustness.

Based on the findings and limitations of this study, we suggest several avenues for future research. Firstly, expanding the sample size and enhancing the diversity of research participants is crucial. Secondly, conducting long-term follow-up studies will help clarify changes in eye movement parameters over time and their correlation with the progression and treatment efficacy of depression. Additionally, standardized procedures for the clinical application of eye-tracking should be gradually established to reduce information bias, which relies on long-term clinical research and practice. Finally, it is important to investigate other potential biomarkers beyond eye-tracking features, such as neuroimaging markers, genetic markers, and behavioral characteristics (40–43). Developing a more comprehensive hybrid feature recognition framework by integrating multi-modal data could significantly enhance diagnostic accuracy.

5 Conclusion

This study underscores the potential value of eye-tracking technology in identifying DMF. Eye-tracking features may serve as non-invasive biomarkers that enhance the objectivity and accuracy of depression diagnoses. By integrating data on eye-tracking features, the application of ML models reveals significant potential for identifying DMF. This data-driven approach promises to yield more precise predictions for clinical decision-making. Future research and clinical practice should further explore and validate these findings to achieve accurate diagnosis and treatment of depression.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors without undue reservation.

Ethics statement

The studies involving humans were approved by Guangdong Provincial People’s Hospital in Guangzhou, China. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation in this study was provided by the participants’ legal guardians/next of kin.

Author contributions

X-CL: Methodology, Visualization, Writing – original draft. MC: Methodology, Validation, Writing – review & editing. Y-JJ: Formal analysis, Visualization, Writing – review & editing. H-BC: Writing – review & editing. Y-QL: Writing – review & editing. ZX: Investigation, Writing – review & editing. Q-YG: Investigation, Writing – review & editing. W-QO: Investigation, Writing – review & editing. Y-YW: Investigation, Writing – review & editing. Q-LX: Writing – review & editing. X-C-CH: Investigation, Writing – review & editing. J-FZ: Investigation, Writing – review & editing. Y-KH: Writing – review & editing. Q-TY: Conceptualization, Project administration, Writing – original draft. M-JJ: Conceptualization, Supervision, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This study was supported by the Science and Technology Program of Guangzhou, China. Grant/Award Number: 202102080299.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The authors declare that no Gen AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2025.1555630/full#supplementary-material

SUPPLEMENTARY FIGURE 1 | The changes in cross-validation AUC scores during the Bayesian optimization process. As the number of iterations increases, the AUC scores gradually improve and stabilize after a certain iteration. The red dot in the figure indicates the best AUC score and the corresponding iteration number obtained throughout the optimization process. Through this optimization method, the model can iteratively adjust the hyperparameters to achieve optimal performance.

References

1. Miret, M, Ayuso-Mateos, JL, Sanchez-Moreno, J, and Vieta, E. Depressive disorders and suicide: epidemiology, risk factors, and burden. Neurosci Biobehav Rev. (2013) 37:2372–4. doi: 10.1016/j.neubiorev.2013.01.008

PubMed Abstract | Crossref Full Text | Google Scholar

2. Na, K, Kang, JM, and Cho, S. Prevalence of DSM-5 mixed features: a meta-analysis and systematic review. J Affect Disord. (2021) 282:203–10. doi: 10.1016/j.jad.2020.12.149

PubMed Abstract | Crossref Full Text | Google Scholar

3. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th ed. Arlington, VA: American Psychiatric Publishing (2013).

Google Scholar

4. Ostacher, MJ, and Suppes, T. Depression with mixed features in major depressive disorder: a new diagnosis or there all along? J Clin Psychiatry. (2018) 79:94–5. doi: 10.4088/JCP.17ac11974

PubMed Abstract | Crossref Full Text | Google Scholar

5. Stahl, SM. Mixed-up about how to diagnose and treat mixed features in major depressive episodes. CNS Spectr. (2017) 22:111–5. doi: 10.1017/S1092852917000207

PubMed Abstract | Crossref Full Text | Google Scholar

6. Vieta, E, and Valentí, M. Mixed states in DSM-5: implications for clinical care, education, and research. J Affect Disord. (2013) 148:28–36. doi: 10.1016/j.jad.2013.03.007

PubMed Abstract | Crossref Full Text | Google Scholar

7. Stahl, SM, Morrissette, DA, Faedda, G, Fava, M, Goldberg, JF, Keck, PE, et al. Guidelines for the recognition and management of mixed depression. CNS Spectr. (2017) 22:203–19. doi: 10.1017/S1092852917000165

Crossref Full Text | Google Scholar

8. Skaramagkas, V, Giannakakis, G, Ktistakis, E, Manousos, D, Karatzanis, I, Tachos, N, et al. Review of eye tracking metrics involved in emotional and cognitive processes. IEEE Rev Biomed Eng. (2023) 16:260–77. doi: 10.1109/RBME.2021.3066072

PubMed Abstract | Crossref Full Text | Google Scholar

9. de Belen, RA, Pincham, H, Hodge, A, Silove, N, Sowmya, A, Bednarz, T, et al. Eye-tracking correlates of response to joint attention in preschool children with autism spectrum disorder. BMC Psychiatry. (2023) 23:211. doi: 10.1186/s12888-023-04585-3

PubMed Abstract | Crossref Full Text | Google Scholar

10. Levantini, V, Muratori, P, Inguaggiato, E, Masi, G, Milone, A, Valente, E, et al. EYES are the window to the mind: eye-tracking technology as a novel approach to study clinical characteristics of ADHD. Psychiatry Res. (2020) 290:113135. doi: 10.1016/j.psychres.2020.113135

PubMed Abstract | Crossref Full Text | Google Scholar

11. Zhu, Y, Xu, L, Guo, Q, Zhang, T, Hu, X, Enck, P, et al. Altered attentive bias towards interpersonal communication information across phases of schizophrenia: an eye-tracking study. Gen Psychiatry. (2022) 35:e100699. doi: 10.1136/gpsych-2021-100699

PubMed Abstract | Crossref Full Text | Google Scholar

12. Gao, M, Xin, R, Wang, Q, Gao, D, Wang, J, and Yu, Y. Abnormal eye movement features in patients with depression: preliminary findings based on eye tracking technology. Gen Hosp Psychiatry. (2023) 84:25–30. doi: 10.1016/j.genhosppsych.2023.04.010

PubMed Abstract | Crossref Full Text | Google Scholar

13. Huang, G, Li, Y, Zhu, H, Feng, H, Shen, X, and Chen, Z. Emotional stimulation processing characteristics in depression: meta-analysis of eye tracking findings. Front Psychol. (2023) 13:1089654. doi: 10.3389/fpsyg.2022.1089654

PubMed Abstract | Crossref Full Text | Google Scholar

14. Stolicyn, A, Steele, JD, and Seriès, P. Prediction of depression symptoms in individual subjects with face and eye movement tracking. Psychol Med. (2022) 52:1784–92. doi: 10.1017/S0033291720003608

PubMed Abstract | Crossref Full Text | Google Scholar

15. Wang, Q, Wei, X, Dang, R, Zhu, F, Yin, S, and Hu, B. An eye tracking and event-related potentials study with visual stimuli for adolescents emotional issues. Front Psych. (2022) 13:933793. doi: 10.3389/fpsyt.2022.933793

PubMed Abstract | Crossref Full Text | Google Scholar

16. Meyhoefer, I, Sprenger, A, Derad, D, Grotegerd, D, Leenings, R, Leehr, EJ, et al. Evidence from comprehensive independent validation studies for smooth pursuit dysfunction as a sensorimotor biomarker for psychosis. Sci Rep. (2024) 14:13859. doi: 10.1038/s41598-024-64487-6

PubMed Abstract | Crossref Full Text | Google Scholar

17. Carvalho, N, Laurent, E, Noiret, N, Chopard, G, Haffen, E, Bennabi, D, et al. Eye movement in unipolar and bipolar depression: a systematic review of the literature. Front Psychol. (2015) 6:1809. doi: 10.3389/fpsyg.2015.01809

PubMed Abstract | Crossref Full Text | Google Scholar

18. Liu, J, Xiang, Y, Lei, H, Wang, Q, Wang, G, Ungvari, GS, et al. Guidance on the conversion of the Chinese versions of the quick inventory of depressive symptomatology-self-report (C-QIDS-SR) and the Montgomery–Asberg scale (C-MADRS) in Chinese patients with major depression. J Affect Disord. (2014) 152-154:530–3. doi: 10.1016/j.jad.2013.09.023

PubMed Abstract | Crossref Full Text | Google Scholar

19. Young, RC, Biggs, JT, Ziegler, VE, and Meyer, DA. A rating scale for mania: reliability, validity and sensitivity. Br J Psychiatry. (1978) 133:429–35. doi: 10.1192/bjp.133.5.429

PubMed Abstract | Crossref Full Text | Google Scholar

20. Chan, DW, and Lai, B. Assessing psychopathology in Chinese psychiatric patients in Hong Kong using the brief psychiatric rating scale. Acta Psychiatr Scand. (1993) 87:37–44. doi: 10.1111/j.1600-0447.1993.tb03327.x

PubMed Abstract | Crossref Full Text | Google Scholar

21. Chen, Y. Introduction to psychiatric diagnostic scales. Shanghai Arch Psychiatry. (1993) 2:127–9. (in Chinese)

Google Scholar

22. Tang, HY, Liao, X, Li, P, Zhang, P, Yao, J, Xing, Y, et al. Efficacy of high-frequency rTMS in the treatment of gait disorder and cognition in patients with Parkinson’s disease based on wearable devices and eye-movement assessments. Front Aging Neurosci. (2024) 16:1480171. doi: 10.3389/fnagi.2024.1480171

PubMed Abstract | Crossref Full Text | Google Scholar

23. Hao, C, Zhang, X, An, J, Bao, W, Yang, F, Chen, J, et al. An effective screening model for subjective cognitive decline in community-dwelling older adults based on gait analysis and eye tracking. Front Aging Neurosci. (2024) 16:1444375. doi: 10.3389/fnagi.2024.1444375

PubMed Abstract | Crossref Full Text | Google Scholar

24. Joormann, J, and Stanton, CH. Examining emotion regulation in depression: a review and future directions. Behav Res Ther. (2016) 86:35–49. doi: 10.1016/j.brat.2016.07.007

PubMed Abstract | Crossref Full Text | Google Scholar

25. Funahashi, S. Saccade-related activity in the prefrontal cortex: its role in eye movement control and cognitive functions. Front Integr Neurosci. (2014) 8:54. doi: 10.3389/fnint.2014.00054

PubMed Abstract | Crossref Full Text | Google Scholar

26. Liversedge, SP, and Findlay, JM. Saccadic eye movements and cognition. Trends Cogn Sci. (2000) 4:6–14. doi: 10.1016/S1364-6613(99)01418-7

PubMed Abstract | Crossref Full Text | Google Scholar

27. García-Blanco, A, Salmerón, L, Perea, M, and Livianos, L. Attentional biases toward emotional images in the different episodes of bipolar disorder: an eye-tracking study. Psychiatry Res. (2014) 215:628–33. doi: 10.1016/j.psychres.2013.12.039

PubMed Abstract | Crossref Full Text | Google Scholar

28. Armstrong, T, and Olatunji, BO. Eye tracking of attention in the affective disorders: a meta-analytic review and synthesis. Clin Psychol Rev. (2012) 32:704–23. doi: 10.1016/j.cpr.2012.09.004

PubMed Abstract | Crossref Full Text | Google Scholar

29. Flechsenhar, A, Seitz, KI, Bertsch, K, and Herpertz, SC. The association between psychopathology, childhood trauma, and emotion processing. Psychol Trauma Theory Res Pract Policy. (2024) 16:S190–203. doi: 10.1037/tra0001261

PubMed Abstract | Crossref Full Text | Google Scholar

30. Imbert, L, Neige, C, Moirand, R, Piva, G, Bediou, B, Vallet, W, et al. Eye-tracking evidence of a relationship between attentional bias for emotional faces and depression severity in patients with treatment-resistant depression. Sci Rep. (2024) 14:12000. doi: 10.1038/s41598-024-62251-4

PubMed Abstract | Crossref Full Text | Google Scholar

31. Miller, S, Suppes, T, Mintz, J, Hellemann, G, Frye, MA, McElroy, SL, et al. Mixed depression in bipolar disorder: prevalence rate and clinical correlates during naturalistic follow-up in the Stanley bipolar network. Am J Psychiatry. (2016) 173:1015–23. doi: 10.1176/appi.ajp.2016.15091119

PubMed Abstract | Crossref Full Text | Google Scholar

32. Zimmerman, M, and Mackin, D. Identifying the DSM-5 mixed features specifier in depressed patients: a comparison of measures. J Affect Disord. (2023) 339:854–9. doi: 10.1016/j.jad.2023.07.102

PubMed Abstract | Crossref Full Text | Google Scholar

33. Yousefi, M, Akhbari, M, Mohamadi, Z, Karami, S, Dasoomi, H, Atabi, A, et al. Machine learning based algorithms for virtual early detection and screening of neurodegenerative and neurocognitive disorders: a systematic-review. Front Neurol. (2024) 15:1413071. doi: 10.3389/fneur.2024.1413071

PubMed Abstract | Crossref Full Text | Google Scholar

34. Nam, SM, Peterson, TA, Seo, KY, Han, HW, and Kang, JI. Discovery of depression-associated factors from a nationwide population-based survey: epidemiological study using machine learning and network analysis. J Med Internet Res. (2021) 23:e27344. doi: 10.2196/27344

PubMed Abstract | Crossref Full Text | Google Scholar

35. Tian, Z, Qu, W, Zhao, Y, Zhu, X, Wang, Z, Tan, Y, et al. Predicting depression and anxiety of Chinese population during COVID-19 in psychological evaluation data by XGBoost. J Affect Disord. (2023) 323:417–25. doi: 10.1016/j.jad.2022.11.044

PubMed Abstract | Crossref Full Text | Google Scholar

36. Alotaibi, A, Underwood, G, and Smith, AD. Cultural differences in attention: eye movement evidence from a comparative visual search task. Conscious Cogn. (2017) 55:254–65. doi: 10.1016/j.concog.2017.09.002

PubMed Abstract | Crossref Full Text | Google Scholar

37. Čeněk, J, Tsai, J, and Šašinka, Č. Cultural variations in global and local attention and eye-movement patterns during the perception of complex visual scenes: comparison of Czech and Taiwanese university students. PLoS One. (2020) 15:e0242501. doi: 10.1371/journal.pone.0242501

PubMed Abstract | Crossref Full Text | Google Scholar

38. Reilly, JL, Lencer, R, Bishop, JR, Keedy, S, and Sweeney, JA. Pharmacological treatment effects on eye movement control. Brain Cogn. (2008) 68:415–35. doi: 10.1016/j.bandc.2008.08.026

PubMed Abstract | Crossref Full Text | Google Scholar

39. Lin, Y, Intoy, J, Clark, AM, Rucci, M, and Victor, JD. Cognitive influences on fixational eye movements. Curr Biol. (2023) 33:1606–1612.e4. doi: 10.1016/j.cub.2023.03.026

PubMed Abstract | Crossref Full Text | Google Scholar

40. Yakemow, G, Kolesar, TA, Wright, N, Beheshti, I, Choi, EH, Ryner, L, et al. Investigating neural markers of Alzheimer's disease in posttraumatic stress disorder using machine learning algorithms and magnetic resonance imaging. Front Neurol. (2024) 15:1470727. doi: 10.3389/fneur.2024.1470727

PubMed Abstract | Crossref Full Text | Google Scholar

41. Chen, B, Sun, X, Huang, H, Feng, C, Chen, W, and Wu, D. An integrated machine learning framework for developing and validating a diagnostic model of major depressive disorder based on interstitial cystitis-related genes. J Affect Disord. (2024) 359:22–32. doi: 10.1016/j.jad.2024.05.061

PubMed Abstract | Crossref Full Text | Google Scholar

42. Sokolov, AV, and Schiöth, HB. Decoding depression: a comprehensive multi-cohort exploration of blood DNA methylation using machine learning and deep learning approaches. Transl Psychiatry. (2024) 14:287. doi: 10.1038/s41398-024-02992-y

PubMed Abstract | Crossref Full Text | Google Scholar

43. Ji, J, Dong, W, Li, J, Peng, J, Feng, C, Liu, R, et al. Depressive and mania mood state detection through voice as a biomarker using machine learning. Front Neurol. (2024) 15:1394210. doi: 10.3389/fneur.2024.1394210

PubMed Abstract | Crossref Full Text | Google Scholar