Measuring depression in Primary Health Care in Spain: Psychometric properties and diagnostic accuracy of HSCL-5 and HSCL-10

Background: Depression has a high prevalence among European countries. Several instruments have been designed to assess its symptoms in different populations. The Hopkins Symptom Checklist 25 (HSCL-25) scale has been identified as valid, reproducible, effective, and easy to use. There are short versions of this scale that could be useful in Primary Care (PC) settings, but their psychometric properties are unknown.

Aim: To assess in PC patients the psychometric properties and diagnostic accuracy of the Spanish version of the HSCL-10 and the HSCL-5 consisting of 10 and 5 items, respectively.

Methods: A multicenter, cross-sectional study was carried out at six PC centers in Spain. The HSCL-25 was administered to outpatients aged 45–75 who also participated in the structured Composite International Diagnostic Interview (CIDI). HSCL-10 and HSCL-5 were assessed and compared to HSCL-25 regarding total score correlation, internal consistency, and criterion validity against the gold-standard CIDI. This is a methodological study from a secondary data analysis and the primary data has been previously published.

Results: Out of 790 patients, 767 completed the HSCL-25 and 736 the CIDI interview (96.0%). Cronbach’s Alpha was 0.84 for HSCL-10 and 0.77 for HSCL-5. The known-group method and confirmatory factor analysis were acceptable for the establishment of construct validity. Sensitivity was 79.7% (CI95%, 67.7–88.0%) for HSCL-10, and 78.0% (CI95%, 65.9–86.6%) for HSCL-5, whereas specificity was 83% (CI95%, 80.0–85.7%) for HSCL-10, and 72.8% (CI95%, 69.3–76.0%) for HSCL-5. Area under the curve against CIDI was 0.88 (CI95%, 0.84–0.92%) for HSCL-10, and 0.85 (CI95%, 0.81–0.89%) for HSCL-5. Optimum cutoff point calculated with Youden Index was 1.90 for the HSCL-10 and 1.80 for the HSCL-5.

Conclusion: HSCL-10 and HSCL-5 are reliable and valid tools to detect depression symptoms and can be used in PC settings.

Introduction

Mental health is an issue of increasing concern in Spain and other European countries and represents a considerable percentage of Primary Care (PC) consultations. The clinical diagnosis of mental disorders is based on the symptoms of the Diagnostic and Statistical Manual of Mental Disorders (DSM). In addition, there are many questionnaires available which assess mental health diseases in populations and are frequently employed in PC (1). Such instruments are used for the purposes of screening and symptom detection to aid diagnosis.

The United States Preventive Services Task Force (USPSTF) recommends universal screening for depression in the general adult population (2) as it has been shown to reduce symptoms and improve quality of life and functional status (3). Screening should, however, be combined with adequate systems to ensure accurate diagnosis, effective treatment, and appropriate follow-up.

Up to 75% of patients with depression are treated exclusively in PC (4). Nevertheless, Family Doctors (FD) report difficulties in the identification of mental health disorders (5) and consider that evaluative tools permitting rapid diagnosis with a limited number of items are required (6). The issue of time is particularly relevant within the context of PC given its complexity and limited consultation schedules. Moreover, FD have been found to feel more confident in detecting and managing depression when employing questionnaires (7).

The ideal questionnaire should be reliable, valid, ergonomic, and easy to use for both patients and healthcare professionals. It should be validated according to the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) (8) for the language and population they are directed at.

The Hopkins Symptom Checklist 25 (HSCL-25) (9) is a widely used, self-administered screening tool for depression and psychological distress; it is also employed for research purposes. It has been shown to be valid and reliable for different populations (10–14), useful in PC (15), and it is available in several languages (16) including Spanish. The Spanish version of HSCL-25 was translated, culturally adapted, and validated following the COSMIN (17, 18).

The HSCL-25 is a short version of the Symptom Checklist 90-Revised (SCL-90-R) (19). There are, however, other questionnaires with fewer items (20) which are of interest due to their brevity and reduced completion time. Shorter versions are useful as they can save time in complex settings such us the PC context. The HSCL-10 and HSCL-5 were developed by selecting 10 and 5 items, respectively, due to their strong correlation with the HSCL-25 mean score (19, 21). Items of the HSCL-10 and HSCL-5 are included in the longer HSCL-25. Both the HSCL-10 and the HSCL-5 have shown a strong mean score correlation with respect to the HSCL-25 and high reliability (Cronbach’s Alpha Coefficient >0.80) (22). Both short versions, especially the HSCL-10, have been used in different populations such as adolescents (23–25), industry workers (26), population surveys (22, 27), patients with alcohol use disorder (28), refugees (29), and also to measure symptoms of depression and anxiety due to the COVID pandemic (30–32). Their psychometric properties recommend them for clinical use as screening and symptom assessment tools and for research purposes (25, 27, 33).

The aim of this article is to report the psychometric properties and diagnostic accuracy of the Spanish versions of the HSCL-5 and HSCL-10 for their use as rapid and accessible depression screening instruments in PC.

Materials and methods

Data collection, study population, and variables

The present study was based on data from a cross-sectional multicenter study designed to validate the HSCL-25 in a Spanish PC population. This is a methodological study from a secondary data analysis and the primary data has been previously published (18). Participants were patients attending six Spanish Primary Health Centers (PHC) taking part in the EIRA study (34, 35). Inclusion criteria were to be aged 45–75 years and presenting two or more of the following: Smoking, low adherence to the Mediterranean dietary pattern, and insufficient physical activity. Exclusion criteria were: Advanced serious illness, cognitive impairment, dependence in basic everyday activities, severe mental illness, unable to attend the PHC, under treatment for cancer or in end-of-life care, or planning to travel during the intervention period.

Participants were recruited by consecutive sampling of patients attending the PHC for any reason during a 6-month-period in 2017. They were asked to complete sociodemographic data (gender, age, nationality, marital status, current employment, and education level) and the self-administered HSCL-25 questionnaire (and other questionnaires/forms related to the EIRA study). Afterward, trained professionals, blinded to the HSCL-25 score, conducted the gold-standard CIDI interview with all participants.

Hopkins symptom checklist-25 (HSCL-25)

The HSCL-25 is a widely used, self-administered questionnaire designed to measure anxiety and depression symptoms (9, 11) and takes 5–10 min to complete (13). It consists of 10 and 15 items belonging to the anxiety and depression dimensions, respectively. The items are answered on a four-point Likert-like scale: 1 = “Not at all;” 2 = “A little;” 3 = “Quite a bit;” 4 = “Extremely.” The average score, ranging from 1 to 4, is calculated by dividing the total score by the number of the items. A cutoff value of 1.75 is generally used for major depression diagnosis, as it is considered a valid predictor of mental disorder (10, 13, 36).

Items belonging to HSCL-25, HSCL-10, and HSCL-5 are shown in Table 1. The corresponding cutoffs points are 1.85 for the HSCL-10 and 2.00 for the HSCL-5 (22). The Spanish version of the HSCL-25 was used in this study (17).

TABLE 1

Table 1. Items belonging to HSCL-25, HSCL-10, and HSCL-5 scale.

Composite international diagnostic interview (CIDI)

The CIDI is a well-known, standardized interview designed by the World Health Organization (WHO) based on the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) and the International Classification of Diseases-10 (ICD-10) criteria (37). It is administered by trained interviewers and available in different languages (38). For this study, section E (questions referring to depression) of the Spanish version was used. The CIDI was conducted by trained psychologists.

As the diagnose of depression is a clinical interview performed by a trained professional and conducted using the DSM criteria, the CIDI is considered the gold-standard in the present study.

Ethical considerations

The study was developed according to national and international legislation (the Declaration of Helsinki and latest versions). The protocol was evaluated by the IDIAP Jordi Gol Ethical Research Committee (approval number: P16/025) and by the corresponding regional governments. Written consent was obtained from the participants, and the questionnaires were codified with an identification number to protect anonymity and confidentiality.

Statistical analysis

Analysis was conducted with STATA version 15. Missing values for the HSCL-25 were replaced with the individual mean for the rest of the items. Subjects with ≥50% missing items were excluded.

Total score was calculated for HSCL-25, HSCL-10, and HSCL-5 for the total population and in relation to gender and age categories, and by the following sociodemographic groups: marital status, education level, and current employment.

Reliability of the HSCL-10 and HSCL-5 was analyzed by calculating the Cronbach’s Alpha Coefficient and for each of the two depression and anxiety subscales. A value of ≥0.7 was considered adequate (39). Cronbach’s Alpha without the item was also calculated to assess the contribution of each item to the internal consistency of two versions.

Construct validity was measured with the known-groups method by comparing the total score of the HSCL-10 and the HSCL-5 by gender. The total score was expected to be significantly higher among women (40, 41). Independent sample t-test was performed, a significant result (p < 0.05) was considered satisfactory (42). Confirmatory factor analysis (CFA) was conducted to assess the structural validity. To evaluate the estimated model fit, the absolute fit index was calculated with chi-squared distribution. Given that this value may be affected by the sample size, complementary indices were employed: The root mean square error of approximation (RMSEA), the standardized root mean square residual (SRMR), the comparative fit index (CFI), and the Tucker–Lewis fit index (TLI) (43, 44). Cutoff values considered adequate were: SRMR < 0.05, RMSEA ≤ 0.08, CFI > 0.90, and TLI > 0.90 (45).

Criterion validity and diagnostic accuracy were measured by calculating the ROC curve for the HSCL-10 and HSCL-5 scale in comparison with the gold-standard CIDI (8). The area under the curve (AUC) was estimated with 95% confidence interval (CI95%). Best cutoff points for the study population and by gender were calculated with the Youden Index for both HSCL-10 and HSCL-5. Youden Index is defined as “Sensitivity + Specificity −1,” it is a value that indicates the validity of the instrument for a specific cutoff point (46). Sensitivity and specificity were assessed as measures of internal validity; positive and negative predictive values were also calculated. Both the sensitivity and specificity of a screening test should be greater than 0.70 (42). For these calculations, cutoffs those proposed by Strand et al. were followed: 1.85 for the HSCL-10 and 2.00 for the HSCL-5 (22). Other authors have employed the same cutoff points (47).

Results

Participants

From a total of 790 patients, 767 completed the HSCL-25 (97.1% response rate). Participants’ mean age was 58.4 years (± 8.2), 54.4% were women, and there were no significant gender differences among age categories. Table 2 shows the mean score and standard deviation (SD) of the three HSCL versions in relation to the sociodemographic characteristics of the sample.

TABLE 2

Table 2. HSCL-25, HSCL-10, and HSCL-5 scale mean scores and standard deviation (SD) in relation to age, gender, marital status, education, and current employment.

There were statistically significant differences in total scores for the three versions regarding gender. Women scored higher with a minimum difference >0.25 points. There were also statistically significant differences in the total scores of the HSCL-25 and HSCL-10, but not the HSCL-5, with respect to age and marital status. No differences were observed regarding education level or current occupation.

Reliability: internal consistency

Cronbach’s Alpha was 0.8417 and 0.7712 for the HSCL-10 and HSCL-5, respectively. When analyzing the two dimensions separately, this value was higher for the depression dimension than the anxiety one. These values, and the value of the coefficient without the item, are depicted in Table 3.

TABLE 3

Table 3. Cronbach’s alpha coefficient without the item and for total values.

The most consistent item for the HSCL-10 was 17 “Feeling blue” followed by 22 “Feeling that everything is an effort.” The least consistent was 25 “Sleep disturbance” followed by 1 “Being scared for no reason.” Item 25 “Sleep disturbance” had a Cronbach’s Alpha without the item of 0.8424, as a result, this item worsened the reliability of this version, as this value was above 0.8417. In the HSCL-5, item 17 “Feeling blue” was also the most consistent whilst two “Feeling fearful” was the least consistent although without affecting reliability.

Construct validity: Known-groups method and confirmatory factor analysis (CFA)

The known-group method analysis showed that the total score of the HSCL-10 indicated that women had significantly higher scores (mean = 1.68; SD 0.03) than men (mean = 1.41; SD 0.02; t = 7.76; p < 0.001). Results were in the same direction with the HSCL-5, total score was significantly higher in women (mean = 1.88; SD 0.03) than in men (mean = 1.53; SD 0.02; t = 8.51; p < 0.001).

Table 4 shows the results of the CFA: the factor loading for each model and correlation in the two-factor models. All the factor loadings were positive, statistically significant (p < 0.001), and above 0.30. In fact, all factor loadings in the different versions were above 0.45. The range of loadings was 0.45–0.81 for the one factor HSCL-10, 0.50–0.83 for the one factor HSCL-5, 0.47–0.82 for the two correlated factor HSCL-10, and 0.51–0.85 for the two correlated factor HSCL-5. Item 17 “Feeling blue” was the item with the highest factor loadings in all the models analyzed. When analyzing the models with two correlated factors, a strong correlation between the two-factors of depression and anxiety was observed for both the HSCL-10 and the HSCL-5.

TABLE 4

Table 4. Confirmatory factorial analysis: Factor loading values and correlation between two anxiety and depression factors.

The goodness-of-fit indices in the studied factor models can be consulted in Supplementary Table 1 of the Supplementary materials. Globally, the indices showed that the HSCL-10 and the HSCL-5 do not have a stable factor structure.

Criterion validity and diagnostic accuracy: Relation of HSCL-10 and HSCL-5 with gold-standard CIDI

Of the 767 participants who completed the HSCL-25, 736 also took part in the CIDI interview (96.0%). Depression prevalence varied depending on the questionnaire employed (Table 5). Prevalence measured with the HSCL-10 was similar to that obtained with the full HSCL-25 version and higher than the value obtained with the HSCL-5.

TABLE 5

Table 5. Prevalence of depression according to CIDI, HSCL-25, HSCL-10, and HSCL-5, total and by gender.

Table 6 shows sensitivity, specificity, positive, and negative predictive values for the total number of participants and by gender. Sensitivity was similar for both genders, especially in the HSCL-10, whilst specificity was better in men. Negative predictive values were >95% for both versions, these values were for the total population and by gender. All values were higher for the HSCL-10 than for the HSCL-5.

TABLE 6

Table 6. Sensitivity, specificity, positive and negative predictive values, total and by gender in the HSCL-10, and the HSCL-5.

HSCL10 vs. CIDI

The AUC between the HSCL-10 and the CIDI was 0.877 (CI95% 0.836–0.919). In the gender analysis it was greater in men with an AUC of 0.943 (CI95% 0.897–0.989) compared to women who had an AUC of 0.825 (CI95% 0.765–0.886). The ROC curve is depicted in Figure 1 and by gender in the Supplementary Figure 1.

FIGURE 1

Figure 1. ROC curve and AUC HSCL-10 vs. CIDI.

HSCL5 vs CIDI

The AUC between the HSCL-5 and the CIDI was 0.853 (CI95% 0.812–0.894). In the gender analysis it was greater in men with an AUC of 0.918 (CI95% 0.859–0.977) than in women who had an AUC of 0.795 (CI95% 0.734–0.855). The ROC curve is depicted in Figure 2 and by gender in the Supplementary Figure 2.

FIGURE 2

Figure 2. ROC curve and AUC HSCL-5 vs. CIDI.

The optimum cutoff points for the study population were calculated with the Youden Index. For the HSCL-10 it was 1.90 for the total population and for both genders. For the HSCL-5 the optimum cutoff point was 1.80 for the total population and for women, and 2.00 for men. A table including sensitivity, specificity, positive, and negative predictive values using optimal cutoff points is available in the Supplementary Table 2.

Discussion

This study is the first to analyze the psychometric properties and diagnostic accuracy of the Spanish versions of the HSCL-10 and HSCL-5, the results obtained now allow the use of both scales as screening tools for depression in the PC setting in Spain. The results demonstrated that in the Spanish PC population, the HSCL-10 presents high reliability and validity. The HSCL-5 also showed acceptable psychometric properties although slightly worse than the HSCL-10. Both scales showed adequate sensitivity and specificity when compared to the semistructured clinical interview CIDI conducted by trained professionals. The optimal cutoff points obtained were very close to those proposed by other authors.

The PC setting is ideal for the detection, diagnosis, and investigation of chronic, highly prevalent disorders as it allows an early study of such pathologies (48). With respect to depression, it is widespread in Spanish and European countries (40) yet patients frequently consult their FD for other motives than their mood disorders (6).

There are many questionnaires that can be employed as screening tools to detect depression at all its stages (49–52), some of which have been validated within the PC setting (1). Moreover, shortened versions have been proposed in order to provide a similar diagnostic value that saves time for both patient and healthcare professional (20, 53, 54). Recently, the HSCL-10 and HSCL-5 have shown acceptable reliability and validity (22, 27), particularly the HSCL-10 (25, 33, 47).

With respect to reliability, Cronbach’s Alpha results of 0.84 and 0.77 were obtained for the HSCL-10 and HSCL-5, respectively. As both were above 0.7, they are considered acceptable (55). Such findings are similar to those obtained by other authors (33) and, as occurred in other studies (19, 22, 27), the reliability of the two short versions was lower than the HSCL-25.

In the full 25-item version, 17 “Feeling blue” was the most consistent (18). This item which asks about sadness, a basic characteristic in patients with depression, is included in the HSCL-10 and the HSCL-5 and was also the most consistent in the two short versions. Once removed, reliability diminished to the minimum as can be seen in Table 3. The next most consistent items in the HSCL-25 were four “Nervousness” and seven “Feeling tense.” The former was included in the 5-item version and the latter in the 10-item one. Item 24 was the least consistent in the HSCL-25 and is not present in either of the short versions. The following least consistent items in the HSCL-25 were 8 “Headache” and 18 “Thinking of ending one’s life,” neither of which is included in the two short versions.

When analyzing Cronbach’s Alpha without the item in the short versions, it was observed that 25 “Sleep disturbance” worsened HSCL-10 reliability, that is to say, by eliminating this item reliability improved. Such a finding concurs with that reported by Kleppang et al. who employed the 10-item version with adolescents in Norway (25). The other items contributed to good reliability in both the HSCL-10 and HSCL-5.

Regarding analysis of the scale’s factorial structure, this was performed with the CFA as the HSCL-25 has been widely studied with one single factor or two correlated ones even though other models have been proposed (10). The HSCL-10 and the HSCL-5 maintain the same factor structure (27). The goodness-of-fit indices for the studied models were not optimal, showing that the data did not fit the hypothesized factor structure of one factor and two correlated factors. The factor structure was unstable for both scales, this is a limitation of our study. Further validation studies should be done to assess other alternative models of the factor structure of the HSCL-10 and the HSCL-5. By examining the factor loading of 10 and 5 items, respectively, all the items were significantly loaded to the hypothesized construct, and all factor loadings were statistically significant, positive, and above 0.45. Item 17 “Feeling blue” had the highest factor loadings in all models tested and for both the HSCL-10 and the HSCL-5. In the study of the two-factor models, there was a factorial correlation of 0.90 between the dimensions in the HSCL-10 and of 0.87 in the HSCL-5, these findings indicated that the depression and anxiety dimensions strongly correlated in a positive manner. The correlation is expectable as anxiety and depression are frequently found to be associated comorbidities (56).

The calculation of the total score is done in the same way in the different versions of the HSCL, by dividing the total score by the corresponding number of ítems answered. With respect to the total score, in comparison with the 25-item long version, the mean score of the HSCL-25 was 1.57, very similar to the 1.55 obtained with the HSCL-10. The mean score increased to 1.72 with the five-item version. The means were significantly greater in women for all three versions, this is unsurprising considering that depression is more prevalent in the female gender (40, 41). Significant differences reported in mean scores according to age and marital status for the 25 and 10-item versions were lost in the 5-item one.

With respect to prevalence, it was higher in the HSCL-5 followed by the HSCL-10 and then the HSCL-25. All three versions and the gold-standard CIDI showed greater prevalence in women than men. Employing the cutoffs corresponding to each version (22), sensitivity was similar for both genders whilst specificity was greater for men in the two short versions. Findings that concur with those reported for the HSCL-25 (18). In a study comparing the HSCL-10 with the CIDI (33), slightly higher results were reported with respect to sensitivity, although a different cutoff was employed. Another multicenter study conducted in General Practice in Norway and Denmark using the HSCL-10 and the CIDI obtained similar results to ours in terms of sensitivity and specificity (23).

Cutoffs can be based on previous studies, cutoffs used in clinical practice, cutoffs recommended by clinical practice guidelines, or cutoffs recommended by the original authors (57). There is very little literature on the appropriate cutoff point for the HSCL-10 and the HSCL-5, Strand et al. recommend 1.85 for the HSCL-10 and 2.00 for the HSCL-5 (22). By interpreting the results from ROC curves, the accuracy for various cutoffs was explored, and optimal cutoff values were obtained considering the maximum value of Youden Index. The optimum cutoff that we calculated for the study population is very close to that in the literature (22). Therefore, we considered that those of 1.85 and 2.00 for the HSCL-10 and HSCL-5, respectively, were appropriate to use showing adequate sensitivity, specificity, and AUC. The negative predictive values were >90% for the two short versions and both genders whilst the positive predictive values were low. Such a finding reinforces the need to complete the diagnosis of depression through a clinical interview.

Other authors have analyzed validity between the HSCL-25 and the two short versions with ROC curves (22). As we had the semi-structured CIDI interview for all our participants, we contrasted it against the two short versions with ROC curves. This is one of the strengths of the present study as sometimes there is a lack of gold-standard and the full version is used as a reference to assess validity. In other studies, only those participants who have a positive result on the scale and a small sample of those with a negative one, undergo the clinical interview of reference. The AUC was 0.88 for the HSCL-10 and 0.85 for the HSCL-5, both above the 0.75 cutoff considered to be of clinical utility, and greater than the 0.80 which confers a “good” classification (≥0.90 is considered “excellent”) in terms of discriminative properties of the diagnostic accuracy (58). A study carried out by Haavet et al. (33) also obtained an AUC of 0.88 when comparing the HSCL-10 with the CIDI, other studies with similar methodology have obtained a lower AUC (23). For both short versions the AUC was greater in men, thus the probability of accuracy in diagnosis in males is greater. Such a gender difference has also been reported for the longer HSCL-25 (18).

The main limitation of our study is that the psychometric properties of the two short versions were evaluated based on the responses to the 25-item scale as performed by other authors (22, 27). The study population came from the EIRA study (34, 35) and were patients aged 45–75 years presenting an unhealthy behavior. Adults often engage in two or more unhealthy behaviors simultaneously, the co-occurrence of unhealthy diet with insufficient physical activity ranges between 47 and 54%, unhealthy diet with smoking between 23 and 28%, and insufficient physical activity with smoking between 8 and 20% (59). Unhealthy diet can be associated to depression (60). Despite the limited age range and the selection criteria in the present study, the authors consider that the results are transferable given the large number of participants, moreover, they had attended the PHC for any motive, consequently, the sample is sufficiently representative.

When deciding which questionnaire to use, there are a few relevant factors to be taken into account, including the population involved and the setting. A balance must be achieved between psychometric properties and such pragmatic characteristics as self-administration, number of items, simplicity/interpretability of scores, and accessibility (61). Scales with a reduced number of items may be the best option as they are practical and feasible within the PC setting without excessively losing reliability and validity.

These findings indicate that the HSCL-10 and HSCL-5 questionnaires show adequate reliability and validity in order to be employed in PC to detect and evaluate depressive symptoms. With such a short number of items they are timesaving and facilitate the detection of cases of depression that could otherwise go unnoticed.

Conclusion

The Spanish versions of the HSCL-10 and HSCL-5, especially the HSCL-10, are reliable and valid tools to detect depressive symptoms and can be used in Primary Care settings.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the Clinical Research Ethics Committee of the University Institute Foundation for Research into Primary Health Care Jordi Gol from Barcelona (reference: P16/025). All the participants gave their written informed consent, and the trial was conducted in accordance with the ethical standards set out in the Declaration of Helsinki. Data were anonymously processed and were used only for the study objectives. Personal data confidentiality was ensured as set out in applicable regulations. Access to medical records was gained as indicated in the legislation currently in force. The patients/participants provided their written informed consent to participate in this study.

Author contributions

MR-B, MF-S-M, AC, JL, and PN: conceptualization. EM and IG-G: data collection. MR-B, MF-S-M, and AC: formal analysis. MR-B, MF-S-M, and EP-R: writing—original draft preparation. All authors have reviewed and agreed to the published version of the manuscript.

Funding

This study was supported by the Carlos III Health Institute, the Spanish Ministry of Economy and Competitiveness via a health research grant (PI15/00114, PI15/00565, PI15/00762, PI15/01072, PI15/00896, PI15/01412, PI15/01151, PI15/00519, and PI15/01133) through the Research Network in Preventive Activities and Health Promotion in Primary Care (redIAPP), (RD12/0005/0001, RD16/0007/0001, RD16/0007/0002, RD16/0007/0003, RD16/0007/0004, RD16/00 07/0005, RD16/0007/0006, RD16/0007/0008, RD16/0007/0009, RD16/0007/0010, RD16/0007/0012, RD16/0007/0013, and RD16/0007/0015), the European Union ERDF funds, (European Regional DevelopmentFund) and the Health Department (SLT002/16/00112) of the Generalitat de Catalunya. This study received a predoctoral grant from the University Institute of Research in Primary Care IDIAP Jordi Gol (Spain) and another grant from the Autonomous University of Barcelona (UAB)–Novartis for research in Family and Community Medicine (reference: 7Z19/019), both in 2019. The study obtained funding from the Spanish Society of Family Medicine (semFYC) as it was awarded a predoctoral grant, “Isabel Fernández.” The project also received a research grant from the Carlos III Institute of Health, Ministry of Science and Innovation (Spain) co-funded with European Union—NextGenerationEU funds, through the Network for Research on Chronicity, Primary Care, and Health Promotion (RICAPPS), with reference RD21/0022. This study received funding from Novartis. The funder was not involved in the study design, collection, analysis, interpretation of data, and the writing of this article or the decision to submit it for publication.

Acknowledgments

We thank to the Family Practice Depression and Multimorbidity group (FPDM) of the European General Practice Research Network (EGPRN). To the REDIAPP network, family physicians, other professionals, and the patients who participated in the study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2022.1014340/full#supplementary-material

References

1. Nabbe P, Le Reste J, Guillou-Landreat M, Munoz Perez M, Argyriadou S, Claveria A, et al. Which DSM validated tools for diagnosing depression are usable in primary care research? A systematic literature review. Eur Psychiatry. (2017) 39:99–105. doi: 10.1016/j.eurpsy.2016.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Siu A, Bibbins-Domingo K, Grossman D, Baumann L, Davidson K, Ebell M, et al. Screening for depression in adults US preventive services task force recommendation statement. JAMA J Am Med Assoc. (2016) 315:380–7. doi: 10.1001/jama.2015.18392

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Smithson S, Pignone M. Screening adults for depression in primary care. Med Clin North Am. (2017) 101:807–21. doi: 10.1016/j.mcna.2017.03.010

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Kroenke K. Depression screening and management in primary care. Fam Pract. (2018) 35:1–3.

Google Scholar

5. Garrido-Elustondo S, Reneses B, Navalón A, Martín O, Ramos I, Fuentes M. Capacidad de detección de patología psiquiátrica por el médico de familia. Aten Primaria. (2016) 48:449–57. doi: 10.1016/j.aprim.2015.09.009

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Latorre Postigo J, Navarro Bravo B, Parra Delgado M, Salguero J, Mae Wood C, Cano Vindel A. Evaluación e intervención de los problemas de ansiedad y depresión en atención primaria: un problema sin resolver [Assessment and intervention of Anxiety and Depression disorders in Primary Care: an unsolved problem]. Rev Clín Med Fam. (2012) 5:37–45. doi: 10.4321/s1699-695x2012000100007

CrossRef Full Text | Google Scholar

7. Tavabie J, Tavabie O. Improving care in depression: qualitative study investigating the effects of using a mental health questionnaire. Qual Prim Care. (2009) 17:251–61.

Google Scholar

8. Mokkink L, Prinsen C, Patrick D, Alonso J, Bouter LM, de Vet H, et al. COSMIN Study Design Checklist for Patient-Reported Outcome Measurement Instruments. La Maddalena: COSMIN (2019). p. 1–32.

Google Scholar

9. Derogatis L, Lipman R, Rickels K, Uhlenhuth E, Covi L. The Hopkins symptom checklist (HSCL): a self-report symptom inventory. Behav Sci. (1974) 19:1–15.

Google Scholar

10. Glaesmer H, Braehler E, Grande G, Hinz A, Petermann F, Romppel M. The German version of the Hopkins symptoms checklist-25 (HSCL-25) - factorial structure, psychometric properties, and population-based norms. Compr Psychiatry. (2014) 55:396–403. doi: 10.1016/j.comppsych.2013.08.020

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Nettelbladt P, Hansson L, Stefansson C, Borgquist L, Nordström G. Test characteristics of the Hopkins symptom check list-25 (HSCL-25) in Sweden, using the present state examination (PSE-9) as a caseness criterion. Soc Psychiatry Psychiatr Epidemiol. (1993) 28:130–3. doi: 10.1007/BF00801743

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Lundin A, Hallgren M, Forsell Y. The validity of the symptom checklist depression and anxiety subscales: a general population study in Sweden. J Affect Disord. (2015) 183:247–52. doi: 10.1016/j.jad.2015.05.024

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Nabbe P, Le Reste J, Guillou-Landreat M, Gatineau F, Le Floch B, Montier T, et al. The French version of the HSCL-25 has now been validated for use in primary care. PLoS One. (2019) 14:e0214804. doi: 10.1371/journal.pone.0214804

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Kamacooko O, Bagiire D, Kasujja F, Mirembe M, Seeley J, King R. Prevalence of probable depression and factors associated with mean Hopkins symptom checklist (HSCL) depression score among young women at high risk aged 15–24 years in Kampala, Uganda. PLoS One. (2022) 17:e0270544. doi: 10.1371/JOURNAL.PONE.0270544

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Nabbe P, Le Reste J, Guillou-Landreat M, Beck-Robert E, Assenova R, Lazic D, et al. One consensual depression diagnosis tool to serve many countries: a challenge! A RAND/UCLA methodology. BMC Res Notes. (2018) 11:4. doi: 10.1186/s13104-017-3111-x

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Roncone R, Gone J, Barlati S, Nabbe P, Reste L, Lazic K, et al. Nine forward-backward translations of the Hopkins symptom checklist-25 with cultural checks. Front Psychiatry. (2021) 1:688154. doi: 10.3389/fpsyt.2021.688154

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Clavería A, Rodríguez-Barragán M, Fernández-San-Martín M, Nabbe P, Le Reste J, Miguéns-Blanco I, et al. Translation and cross-cultural adaptation into Spanish, Catalan and Galician of the Hopkins symptom checklist-25 to identify depression in primary care. Aten Primaria. (2020) 52:539–47. doi: 10.1016/j.aprim.2020.05.017

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Rodríguez-Barragán M, Fernández-San-martín M, Clavería-Fontán A, Aldecoa-Landesa S, Casajuana-Closas M, Llobera J, et al. Validation and psychometric properties of the Spanish version of the Hopkins symptom checklist-25 scale for depression detection in primary care. Int J Environ Res Public Health. (2021) 18:7843. doi: 10.3390/ijerph18157843

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Müller J, Postert C, Beyer T, Furniss T, Achtergarde S. Comparison of eleven short versions of the symptom checklist 90-revised (SCL-90-R) for use in the assessment of general psychopathology. J Psychopathol Behav Assess. (2009) 32:246–54. doi: 10.1007/s10862-009-9141-5

CrossRef Full Text | Google Scholar

20. Tambs K, Røysamb E. Selection of questions to short-form versions of original psychometric instruments in MoBa *. Nor Epidemiol. (2014) 24:195–201.

Google Scholar

21. Tambs K, Moum T. How well can a few questionnaire items indicate anxiety and depression? Acta Psychiatr Scand. (1993) 87:364–7.

Google Scholar

22. Strand B, Dalgard O, Tambs K, Rognerud M. Measuring the mental health status of the Norwegian population: a comparison of the instruments SCL-25, SCL-10, SCL-5 and MHI-5 (SF-36). Nord J Psychiatry. (2003) 57:113–8. doi: 10.1080/08039480310000932

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Sirpal M, Haugen W, Sparle K, Haavet O. Validation study of HSCL-10, HSCL-6, WHO-5 and 3-key questions in 14-16 year ethnic minority adolescents. BMC Fam Pract. (2016) 17:7. doi: 10.1186/s12875-016-0405-3

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Søberg Finbråten H, Kleppang A, Steigen A. Using the Rasch measurement theory to assess the psychometric properties of the Hopkins symptom checklist-10 in adolescents. Health Qual Life Outcomes. (2021) 19:248. doi: 10.1186/s12955-021-01884-9

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Kleppang A, Hagquist C. The psychometric properties of the Hopkins symptom checklist-10: a Rasch analysis based on adolescent data from Norway. Fam Pract. (2016) 33:740–5. doi: 10.1093/FAMPRA/CMW091

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Ljoså C, Tyssen R, Lau B. Mental distress among shift workers in Norwegian offshore petroleum industry - relative influence of individual and psychosocial work factors. Scand J Work Environ Heal. (2011) 37:551–5. doi: 10.5271/sjweh.3191

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Schmalbach B, Zenger M, Tibubos A, Kliem S, Petrowski K, Brähler E. Psychometric properties of two brief versions of the Hopkins symptom checklist: HSCL-5 and HSCL-10. Assessment. (2019) 28:617–31. doi: 10.1177/1073191119860910

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Lien I, Bolstad I, Lien L, Bramness J. Screening for depression in patients in treatment for alcohol use disorder using the beck depression inventory-II and the Hopkins symptom checklist-10. Psychiatry Res. (2022) 308:114363. doi: 10.1016/J.PSYCHRES.2021.114363

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Nesterko Y, Jäckle D, Friedrich M, Holzapfel L, Glaesmer H. Factors predicting symptoms of somatization, depression, anxiety, post-traumatic stress disorder, self-rated mental and physical health among recently arrived refugees in Germany. Confl Health. (2020) 14:44. doi: 10.1186/s13031-020-00291-z

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Harris S, Sandal G. COVID-19 and psychological distress in Norway: the role of trust in the healthcare system. Scand J Public Health. (2021) 49:96–103. doi: 10.1177/1403494820971512

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Karlsson Stafseth S, Skogstad L, Raeder J, Strand Hovland I, Hovde H, Ekeberg Ø, et al. Symptoms of anxiety, depression, and post-traumatic stress disorder in health care personnel in Norwegian ICUs during the first wave of the COVID-19 pandemic, a prospective, observational cross-sectional study. Int J Environ Res Public Heal. (2022) 19:7010. doi: 10.3390/ijerph19127010

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Frounfelker R, Li Z, Santavicca T, Miconi D, Rousseau C. Latent class analysis of COVID-19 experiences, social distancing, and mental health. Am J Orthopsychiatry. (2022) 92:121–32. doi: 10.1037/ORT0000593

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Haavet O, Sirpal M, Haugen W, Christensen K. Diagnosis of depressed young people in primary health care-a validation of HSCL-10. Fam Pract. (2011) 28:233–7. doi: 10.1093/fampra/cmq078

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Zabaleta-Del-Olmo E, Pombo H, Pons-Vigués M, Casajuana-Closas M, Pujol-Ribera E, López-Jiménez T, et al. Complex multiple risk intervention to promote healthy behaviours in people between 45 to 75 years attended in primary health care (EIRA study): study protocol for a hybrid trial. BMC Public Health. (2018) 18:874. doi: 10.1186/s12889-018-5805-y

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Represas-Carrera F, Couso-Viana S, Méndez-López F, Masluk B, Magallón-Botaya R, Recio-Rodríguez J, et al. Effectiveness of a multicomponent intervention in primary care that addresses patients with diabetes mellitus with two or more unhealthy habits, such as diet, physical activity or smoking: multicenter randomized cluster trial (EIRA Study). Int J Environ Res Public Heal Artic Public Heal. (2021) 18:5788. doi: 10.3390/ijerph18115788

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Veijola J, Jokelainen J, Läksy K, Kantojärvi L, Kokkonen P, Järvelin M, et al. The Hopkins symptom checklist-25 in screening DSM-III-R axis-I disorders. Nord J Psychiatry. (2003) 57:119–23. doi: 10.1080/08039480310000941

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Kessler R, Bedirhan Üstün T. The World mental health (WMH) survey initiative version of the World Health Organization (WHO) composite international diagnostic interview (CIDI). Int J Methods Psychiatr Res. (2004) 13:93–121.

Google Scholar

38. Harkness J, Pennell B, Villar A, Gebler N, Aguilar-Gaxiola S, Bilgen I. Translation procedures and translation assessment in the World Mental Health survey initiative. In: Kessler RC, Bedirhan T editors. The WHO World Mental Health Surveys: Global Perspectives on the Epidemiology of Mental Disorders. New York, NY: Cambridge University Press, World Health Organization (2008). p. 91–113.

Google Scholar

39. Terwee C, Bot S, de Boer M, van der Windt D, Knol D, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. (2007) 60:34–42. doi: 10.1016/j.jclinepi.2006.03.012

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Ferrari A. Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990-2019: a systematic analysis for the Global burden of disease study 2019. Lancet Psychiatry. (2022) 9:137–50. doi: 10.1016/S2215-0366(21)00395-3

CrossRef Full Text | Google Scholar

41. Portellano-Ortiz C, Garre-Olmo J, Calvó-Perxas L, Conde-Sala J. Depresión y variables asociadas en personas mayores de 50 años en España. Rev Psiquiatr y Salud Ment. (2018) 11:216–26. doi: 10.1016/j.rpsm.2016.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Huang E, Cheung J, Liu J, Kwan R, Lam S. Groningen frailty indicator–Chinese (GFI-C) for pre-frailty and frailty assessment among older people living in communities: psychometric properties and diagnostic accuracy. BMC Geriatr. (2022) 22:788. doi: 10.1186/s12877-022-03437-1

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Jackson D, Gillaspy J, Purc-Stephenson R. Reporting practices in confirmatory factor analysis: an overview and some recommendations. Psychol Methods. (2009) 14:6–23. doi: 10.1037/a0014694

PubMed Abstract | CrossRef Full Text | Google Scholar

44. de Vet H, Terwee C, Mokkink L, Knol D. Measurement in Medicine. Practical Guides to Biostatistics and Epidemiology. Cambridge: Cambridge University Press, World Health Organization (2018). 338 p.

Google Scholar

45. Viladrich C, Doval E, Penelo E. Fiabilidad y Validez. 12th ed. Bellaterra: Universitat Autònoma de Barcelona Laboratori d’Estadística Aplicada i de Modelització (2019).

Google Scholar

46. Brehaut E, Neupane D, Levis B, Wu Y, Sun Y, Ioannidis J, et al. Optimal” cutoff selection in studies of depression screening tool accuracy using the PHQ-9, EPDS, or HADS-D: a meta-research study. Int J Methods Psychiatr Res. (2022):e1956. doi: 10.1002/MPR.1956

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Søgaard A, Bjelland I, Tell G, Røysamb E. A comparison of the CONOR mental health index to the HSCL-10 and HADS. Nor Epidemiol. (2003) 13:279–84.

Google Scholar

48. Violán Fors C, Odriozola G, Zabaleta-del-Olmo E, Moral E. La investigación en atención primaria como área de conocimiento. Informe SESPAS 2012. Gac Sanit. (2012) 26:76–81. doi: 10.1016/j.gaceta.2011.05.018

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Reuland D, Cherrington A, Watkins G, Bradford D, Blanco R, Gaynes B. Diagnostic accuracy of Spanish language depression-screening instruments. Ann Fam Med. (2009) 7:455–62. doi: 10.1370/afm.981

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Sanz J, Garcia-Vera M, Espinosa R, Fortún M. Adaptación española del inventario para la depresión de beck-II (BDI-II): 3. Propiedades psicométricas en pacientes con trastornos psicológicos. Clín Salud. (2005) 16:121–42.

Google Scholar

51. Fernández-San Martín M, Andrade C, Molina J, Muñoz P, Carretero B, Rodríguez M, et al. Validation of the Spanish version of the geriatric depression scale (GDS) in primary care. Int J Geriatr Psychiatry. (2002) 17:279–87. doi: 10.1002/gps.588

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Kakemam E, Navvabi E, Albelbeisi A, Saeedikia F, Rouhi A, Majidi S. Psychometric properties of the Persian version of depression anxiety stress scale-21 items (DASS-21) in a sample of health professionals: a cross-sectional study. BMC Health Serv Res. (2021) 22:111. doi: 10.1186/s12913-022-07514-4

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Kroenke K, Spitzer R, Williams J, Löwe B. An ultra-brief screening scale for anxiety and depression: the PHQ–4. Psychosomatics. (2009) 50:613–21. doi: 10.1016/S0033-3182(09)70864-3

CrossRef Full Text | Google Scholar

54. Petrowski K, Zenger M, Schmalbach B, Bastianon C, Strauss B. Psychometric properties and validation of the English version Giessen subjective complaints list (GBB-8). BMC Psychol. (2022) 10:60. doi: 10.1186/s40359-022-00741-8

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Nunnally J. Psychometric Theory. 2nd ed. New York, NY: McGraw-Hill (1978).

Google Scholar

56. Martinussen L, Šaltytë Benth J, Almdahl I, Borza T, Selbæk G, Mcpherson B, et al. The effect of anxiety on cognition in older adult inpatients with depression: results from a multicenter observational study. Heliyon. (2019) 5:e02235. doi: 10.1016/j.heliyon.2019.e02235

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Cohen J, Korevaar D, Altman D, Bruns D, Gatsonis C, Hooft L, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. (2016) 6:e012799. doi: 10.1136/bmjopen-2016

CrossRef Full Text | Google Scholar

58. Polo T, Miot H. Aplicações da curva ROC em estudos clínicos e experimentais. J Vasc Bras. (2020) 19:e20200186. doi: 10.1590/1677-5449.200186

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Meader N, King K, Moe-Byrne T, Wright K, Graham H, Petticrew M, et al. A systematic review on the clustering and co-occurrence of multiple risk behaviours. BMC Public Health. (2016) 16:657. doi: 10.1186/s12889-016-3373-6

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Oliván-Blázquez B, Aguilar-Latorre A, Motrico E, Gómez-Gómez I, Sarasa-Bosque C, Sanchez-Perez Á, et al. The Relationship between adherence to the mediterranean diet, intake of specific foods and depression in an adult population (45–75 Years) in primary health care. A cross-sectional descriptive study. Nutrients. (2021) 13:2724. doi: 10.3390/nu13082724

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Kroenke K, Monahan P, Kean J. Pragmatic characteristics of patient-reported outcome measures are important for use in clinical practice. J Clin Epidemiol. (2016) 68:1085–92. doi: 10.1016/j.jclinepi.2015.03.023

PubMed Abstract | CrossRef Full Text | Google Scholar