Factor structure of the patient health questionnaire-4 in adults with attention-deficit/hyperactivity disorder

Background: Persons with attention-deficit/hyperactivity disorder (ADHD) frequently experience symptoms of anxiety and depression. In this population, there is a need for validated brief self-report screening questionnaires to assess the severity of comorbid mental health problems. The Patient Health Questionnaire 4 (PHQ-4) is a self-report questionnaire that may contribute to this purpose as it can screen for both disorders efficiently. However, this will be the first study examining the factor structure of the PHQ-4 in samples of adults with ADHD, and also evaluating the validity of the Norwegian version of the PHQ-4.

Objectives: The aim of the current cross-sectional study was to examine the factor structure and validity of the Norwegian version of the PHQ-4 in a sample of adults who reported having been diagnosed with ADHD.

Methods: Of 496 invited, a total of 326 participants (66%) completed the PHQ-4, The World Health Organization Five Well-Being Index, the Oslo Social Support Scale and the 4-item Perceived Stress Scale electronically in a web-portal between the 9th and 30th of June 2020.

Results: Confirmatory factor analysis of the PHQ-4 supported a two-factor structure [RMSEA = 0.038 (90% CI 0.000–0.159), CFI = 1.00, TLI = 0.999, SRMR = 0.004], consisting of a depression factor and an anxiety factor. Standardized factor loadings were 0.79 to 0.97. The PHQ-4 was negatively correlated with well-being and social support and positively correlated with perceived level of stress.

Conclusion: This study indicates promising psychometric properties of the PHQ-4 as a measure of anxiety and depressive symptoms in adults with self-reported ADHD who are fluent in Norwegian. The questionnaire’s brevity makes it a valuable resource in research and clinical settings. However, more studies are needed to test the instrument in a clinical sample.

1. Introduction

Attention-deficit/hyperactivity disorder (1) has a prevalence of 2.8% in adults internationally (2). Persons with ADHD often struggle with psychiatric comorbidity (3–7), and adult patients frequently experience comorbid symptoms of anxiety and depression (8), to a larger degree than adults without ADHD (9). Although psychological (10) and pharmacological treatment (11) may be effective, still many patients are left untreated and better recognition and screening of adults with ADHD is needed. Even though general screening questionnaires and self-report measures of anxiety and depression have been developed, there is a need for validated questionnaires to measure comorbid psychiatric symptoms in adults with ADHD (12). Due to the core difficulties related to inattention, impulsivity and hyperactivity in ADHD (13), brief self-report measures are preferable. The development of reliable and validated brief screening questionnaires to assess the level of comorbid common mental health problems in persons diagnosed with ADHD is important both for improving future research as well as for use in clinical settings.

The 4-item Patient Health Questionnaire (PHQ-4) is a brief questionnaire (14, 15) which combines two items from the Patient Health Questionnaire 9 (16) and two items from the Generalized Anxiety Disorder Scale 7 (GAD-7) (17). The two items assessing depression are commonly referred to as PHQ-2 and the two items assessing anxiety are called GAD-2. Both of these two-item scales have been supported as valid measures of depression (18–22) and anxiety (23–25), respectively. Initially developed in the United States (15), the PHQ-4 has been translated and validated in multiple languages, including German (14), Spanish (26, 27), Greek (28), Korean (29) and Arabic (30). However, to date, a validated version of the PHQ-4 in Norwegian has not been available.

The PHQ-4 has been suggested as valuable for treatment management, due to the ability to provide rapid monitoring of treatment response and adherence (15). The psychometric properties of the PHQ-4 have been supported in mental health patients (15) and in the general population (14, 28, 31) and in a mounting number of recent studies with different populations [(e.g., 32, 33)]. However, the PHQ-4 has not previously been validated in patients with ADHD. In studies measuring the PHQ-4’s construct validity, the PHQ-4 showed negative correlations with the World Health Organization-Five Well-Being Index (WHO-5) (34) and social support (35), and positive correlations with depression scales (32) and perceived stress (27) supporting divergent and convergent validity. It should be noted, however, that a study validating the Korean version of the PHQ-4 reported questionable discriminant validity (29) in a sample of psychiatric outpatients, which indicates the need to investigate the psychometric properties of the PHQ-4 in different populations. The reported internal consistency of the PHQ-4 has been acceptable to good in previous studies, with Cronbach’s coefficients ranging from 0.77 (33) to 0.86 (27). Nonetheless, the PHQ-4 has not yet been validated in adults with ADHD.

The PHQ-4 was originally conceptualized as a bidimensional measure of depression and anxiety (15). Confirmatory factor analysis (CFA) supports the proposed two-factor solution in primary care patients (15), in the general population (14, 32, 36), and in both infertile and pregnant individuals (34, 37). Although it has been questioned whether the PHQ-4 is suitable for assessing the severity of comorbid depression and anxiety in psychiatric clinic settings (29), the questionnaire is validated in primary care among patients with emotional disorders (15, 26).

In summary, brief self-report measures are preferable for adults with ADHD. The PHQ-4 may thus be a useful self-report measure to assess comorbid symptoms of depression and anxiety which are frequently reported in this population. However, the factor structure of the PHQ-4 has not been investigated in individuals with ADHD, and the validity of the Norwegian version of the PHQ-4 has not yet been explored. This study aims to examine, for the first time, the construct validity of the Norwegian version of the PHQ-4, including its factorial structure and internal consistency, in an adult ADHD sample.

2. Materials and methods

2.1. Study design and population

This anonymous cross-sectional survey was conducted between 9 June and 30 June 2020. Inclusion criteria were as follows: (1) to be 18 years and older; (2) to have proficiency in the Norwegian language; (3) to provide informed consent; and (4) to report to having been diagnosed with ADHD. We adhered to the taxonomy and methodology proposed by the Consensus-based Standards for the Selection of Health Measurements Instruments (COSMIN) (38) and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) (39) when planning and conducting the study.

2.2. Recruitment and procedures

A total of 496 potential participants of an ADHD organization were contacted. All participants were invited via email by the study collaborators. In addition to describing the study’s aim and the use of the data, the emails contained an electronic link via Questback software to the questionnaire. Participants were presented with information about the study and were informed they could skip items and discontinue the survey at any time. By clicking “I agree,” the participants indicated that they had read and understood the information in the consent form, and that they agreed to participate in the research study. All participants reported that they had received a diagnosis of ADHD in the specialist mental health service, which currently uses the International statistical classification of diseases and related health problems (ICD-10) (40), but this was not confirmed through structural clinical interviews, and the time for the participants’ diagnosis was not recorded. The maximum time spent completing the scales was 12 min.

2.3. Measurements

2.3.1. Patient health questionnaire 4

The Patient Health Questionnaire 4 (PHQ-4) (15) is a self-report measure of anxiety and depressive symptoms. The PHQ-4 has four items; two items from the PHQ-2 depression screener that assess core symptoms of major depressive disorder according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) (41) (“Over the last 2 weeks, how often have you been bothered by the following problems?”; ‘Feeling down, depressed, or hopeless’ and ‘Little interest or pleasure in doing things’). The two remaining items are from the GAD-2 anxiety screener and assess core symptoms of anxiety disorders (“Over the last 2 weeks, how often have you been bothered by the following problems?”; ‘Feeling nervous or anxious or on edge’ and ‘Not being able to stop or control worrying’).

The items are scored on a 0–3 Likert scale ranging from “not at all” (0) to “nearly every day” (3), with higher total score indicating higher symptom severity. The scores of the PHQ-2 and the GAD-2 items are added to calculate the composite PHQ-4 score, with a total score range between 0 and 12. In addition, in the present study participants also rated the degree of impairment due to anxiety and depressive symptoms by rating the question ‘If you checked off any problems, how difficult have these problems made it for you to do your work, take care of things at home, or get along with other people?” which was also scored on a 0–3 Likert scale.

2.3.2. The World Health Organization five well-being index

The World Health Organization Five Well-Being Index (WHO-5) is a generic, self-reported scale that assesses perceptions of well-being. The scale includes five items, and higher scores represent higher perceived levels of well-being. The WHO-5 is a reliable and valid self-reporting questionnaire (42). Patients were asked to rate their agreement over the previous 2 weeks on each of the items rated on a 6-point scale. The five items are scored on a Likert scale ranging from “all of the time” (0) to “at no time” (5), with total score range between 0 and 25 (higher scoring indicates better well-being). The WHO-5 has previously been translated to Norwegian (43, 44).

Well-being is a key protective factor against the negative impact of depression and anxiety. In addition to measuring the construct of well-being, the WHO-5 also measures depressive symptoms, as it was originally developed to measure self-reported depression (42, 45). As such we expected that it would be a significant association between PHQ-4 and the WHO-5.

2.3.3. The perceived stress scale 4

The Perceived Stress Scale 4 (PSS-4) (46) is a short questionnaire with four items that assesses the level of perceived stress. Each item is scored on a 0–4 Likert scale, with total score ranging from 0 to 16. Higher total score indicates higher degree of perceived stress. The PSS-4 is a reliable and valid questionnaire with good psychometric properties (46, 47). The scale has previously been translated to Norwegian (48).

Perceived stress is a common characteristic of the construct of anxiety. Therefore, consistent with previous studies (9, 27, 49) and based on the assumption that there are close relationships between anxiety and stress, we assumed there would be a significant association between PHQ-4 and PSS-4.

2.3.4. The Oslo social support scale

The Oslo Social Support Scale (OSSS-3) (50, 51) is a 3-item questionnaire that measures social support. The total score ranges from 3 to 14, where higher scores equal stronger support. The total score may be used to indicate level of support: 3 to 8 indicates poor support; 9–11 indicates moderate support and 12–14 indicates strong support. Studies show that the OSSS-3 is a reliable and valid measure with sound psychometric properties (50, 52, 53). The Norwegian Directorate of Health has recommended to use this scale to measure the quality of life in Norwegian contexts (54).

Social support is a protective factor against psychological distress. Studies have shown that adequate social support is associated with lower symptom levels of anxiety and depression (35, 55, 56). Therefore, we expected PHQ-4 and OSSS-3 to be significantly associated.

2.4. Sample size

The target sample size (n = 120) was based on assuming a 10–12% dropout rate, with a significance level of 0.05 and power of 80%, based on general for recommendations for Cronbach’s α studies, and with an absolute minimum of 100 participants (14, 29).

2.5. Statistical analyses

A total of 496 adults who self-reported being diagnosed with ADHD were initially invited to participate in the study. Out of these, 392 individuals provided their consent to participate. However, due to missing information, 42 participants were excluded from the dataset. Additionally, 24 participants were removed due to incorrectly completing their submissions. Consequently, the final sample consisted of 326 participants, corresponding to 66% of the initially invited individuals (see flow chart in Figure 1). Cronbach’s α was calculated to assess internal consistency. Concurrent validity was investigated by calculating Spearman correlations between PHQ-4 and relevant measures, due to non-normally distributed data. These analyses were conducted using SPSS version 28.

FIGURE 1

Figure 1. Flow chart of study participants.

Confirmatory Factor Analysis (CFA) was conducted to examine the fit of a one and two factor structure of the PHQ-4. Due to the ordinal nature of the Likert scale, the Weighted Least Squares Means and Variance adjusted (WLSMV) estimator was used. The model fit was evaluated according to multiple fit indices. Standardized Root Mean Square Residual (SRMR) (57) values below 0.08 indicate good fit. Root Mean Square Error of Approximation (RMSEA) (58) values from 0.00 to 0.05 are indicative of close fit, values between 0.05 and 0.08 indicate fair fit, values between 0.08 and 0.10 suggest mediocre fit, and values above 0.10 indicate poor fit. For the Comparative Fit Index (CFI) and the non-Normed Fit index (NNFI; aka Tucker–Lewis index; TLI) values greater than 0.95 indicate acceptable model fit (58). Multiple indicators, multiple causes (MIMIC) (59) modeling was used to test the effect of covariates in the model. Significant direct effects of a covariate on the factor indicate that the factor means are different at different levels of the covariate (59). In the MIMIC model, sex (women and men) and symptom severity (0–3 Likert scale) were included as covariates in the CFA model. CFA was conducted in Mplus version 8.8 (60).

3. Results

3.1. Participants’ characteristics

The sample consisted of 326 persons (226 women, 98 men, and two unknown). Age was reported in intervals. Lower education (primary or secondary school) was reported by 158 individuals, and 168 individuals reported higher education (college or university degree). Almost half the sample (n = 162) reported to be working full time, 30 were students, 41 were on sick leave, seven were on temporary lay-off, and 86 reported “other.” The majority (n = 198) reported to be married or living with a partner (Table 1).

TABLE 1

Table 1. Demographic characteristics of the participants.

3.2. Mean differences and descriptive statistics

The frequency distributions of item responses are shown in Table 2. Participants tended to choose a value of 1 more frequently than other scores. GAD-2 item 2 (“Not being able to stop or control worrying”), and both PHQ-2 items showed floor effects. Women reported higher scores on PHQ-4 than men (Mean women M_w = 5.3, SD = 3.2 versus Mean men M_m = 4.4, SD = 3.0, p = 0.02).

TABLE 2

Table 2. Internal reliability and item distribution.

3.3. Internal reliability

Internal reliability was evaluated using Cronbach’s α. Its value was 0.88, which indicates a high level of internal consistency (Table 2).

3.4. Validity

The PHQ-4 correlated negatively with well-being measured with the WHO-5 (r = −0.75, p < 0.001) and social support measured with the OSSS-3 (r = −0.38, p < 0.001), and positively with perceived level of stress measured with the PSS-4 (r = 0.31, p < 0.001). A summary of correlations between the measures is presented in Table 3.

TABLE 3

Table 3. Correlations between PHQ-4 and other measures.

3.5. Confirmatory factor analysis

The CFA supported a two-factor solution for the PHQ-4. Fit indices supported a good model fit [RMSEA = 0.038 (90% CI 0.000–0.159), CFI = 1.00, TLI = 0.999, SRMR = 0.004] with 1 degree of freedom. Standardized factor loadings ranged from 0.79 to 0.97 (Figure 2). A one-factor structure was also tested, which showed less favorable fit indices than the two-factor structure [RMSEA = 0.231 (90% CI 0.169–0.299), CFI = 0.988, TLI = 0.963, SRMR = 0.026]. The fit indices of the one-and two-factor solutions are presented in Table 4.

FIGURE 2

Figure 2. Confirmatory factor analysis of a two-factor model of PHQ-4. Standardized coefficients displayed. Arrow between factors represents correlation.

TABLE 4

Table 4. Goodness of fit for the two-factor and one-factor model of PHQ-4.

Sex and severity were included as covariates in the two-factor PHQ-4 model to test a MIMIC model, with the two factors anxiety and depression regressed on the covariates. The MIMIC model indicated a good model fit [RMSEA 0.059 (0.000–0.109), CFI = 0.996, TLI = 0.988, SRMR = 0.032]. The unstandardized estimate was significant for sex and anxiety (−0.270 p = 0.027) but not for sex and depression (0.115, p = 0.292). The negative estimate between sex and anxiety indicates that the mean score of men was 0.270 points lower than women on the anxiety factor. The unstandardized estimates between severity and anxiety (1.009, p < 0.001) and depression (0.964, p < 0.001) were significant, indicating that those who reported a higher level of impairment also scored higher on anxiety and depression, as expected.

4. Discussion

The aim of the current study was to investigate the factor structure and validity of the PHQ-4 in an adult sample of persons diagnosed with ADHD. A two-factor solution was found to be superior to a one-factor model. The psychometric properties of the PHQ-4 are promising, and our results supported the scale as a reliable, valid and brief tool for screening symptoms of anxiety and depression in adults with ADHD.

The CFA supported a two-factor solution consisting of an anxiety and a depression factor. The standardized factor loadings for item 2 and item 4 were high (λ = 0.968 and λ = 0.915, respectively), which suggest that these items may particularly important when screening for symptoms of depression and anxiety in adults with ADHD. The two-factor structure identified corroborates previous research which found the two-factor solution to be superior to a one-factor solution (32, 34, 36, 37), which is further in accordance with the factor structure proposed by the creators of the scale (15). One study found support for a one-factor model of PHQ-4 (61), but in contrast to our sample this study partly included both adolescents and young adults and the results are therefore not directly comparable to the present study. A study investigating the Korean version of the PHQ-4 in a sample of adult psychiatric outpatients also found a one-factor solution to be superior to a two-factor solution (29). However, the CFA in the latter study also showed poor model fit for the two-factor solution, and the sample did not include patients with ADHD, which means that these findings may not be comparable to the results of the present study. Why studies have found support both in favour of a one-and a two-factor solution is unclear, but the authors of the Korean validation study suggested that the high comorbidity between anxiety and depression in their sample may have influenced the results (29). In summary, the majority of previous studies appears to support a two-factor solution of the PHQ-4, and this is in line with the results of our study.

When testing whether men and women scored differently on the anxiety and depression subscales of the PHQ-4, results from the MIMIC model showed that the mean of the anxiety factor was higher for women than men. The sex difference in anxiety is in line with previous studies that have found women with ADHD to report higher levels of anxiety symptoms than men (62). However, the literature also typically finds women more likely to report higher levels of depression than men (63, 64), which was not found in the MIMIC model. Further, the MIMIC model showed significant estimates between severity and both anxiety and depression. This finding indicates that the means of the anxiety and depression factor were higher for those who reported a more severe degree of impairment, which was expected. This result shows that the PHQ-4 differentiates higher and lower levels of symptom severity.

The results showed high internal reliability, with a Cronbach’s alpha score of α = 0.88 which is in line with, or somewhat higher than, previous studies [(e.g., 39)]. Reliability was not improved if any of the items were deleted. These results are encouraging and suggest that the PHQ-4 has good reliability as a measure of comorbid symptoms of emotional distress in adults with ADHD. Furthermore, the sample reported a mean PHQ-4 score of 5.0 (SD = 3.1), which is lower than the mean score of 7.2 (SD = 3.7) in a sample of adults with comorbid ADHD and problematic gambling (65). This may suggest that individuals who suffer from comorbid conditions in addition to ADHD may experience a higher severity of emotional symptoms compared to persons without comorbidity. However, as we did not assess comorbidity with structured diagnostic interviews, we have no information regarding the extent of comorbidity in the current sample. Using a PHQ-4 score equal to or above 3 as a cut-off to indicate a possible clinical level of symptoms, 39 and 36% scored above the cut-off for anxiety and depression, respectively, while 28% scored above both cut-off values. These results are similar to a German study that found that about 30% of adults who reported having ADHD scored above the cut-off value for anxiety and depression (66).

We investigated how well the PHQ-4 was associated with other measures. The results showed that the PHQ-4 correlated negatively with the WHO-5 and the OSSS-3 and positively with the PSS-4, with the strongest correlation between PHQ-4 and WHO-5. The WHO-5 has been supported as a valid screening tool for depression (45), and the analyses showed stronger correlations with the PHQ-2 than the GAD-2. The results of our study are thus in line with previous research (34, 43). The negative correlation between the PHQ-4 and OSSS-3 is in line with the well-established finding that social support is an important protector against level of anxiety and depression symptoms (55, 56). Furthermore, previous research has demonstrated a strong association between perceived level of stress and in particular anxiety (9), and perceived stress is found to mediate the relationship between ADHD symptoms and depression (49). The positive correlation between the PHQ-4 and the PSS-4 found in our study is thus expected based on previous research (43). Altogether, the PHQ-4 correlated in the expected direction with other validated measures which supports the validity of the PHQ-4 in adults with ADHD.

The study was conducted during the COVID-19 pandemic in Norway. This may have influenced the results, as studies have indicated a general increase in symptoms of anxiety and depression during the pandemic in the general public (67, 68) and among adults with ADHD (69). However, the mean PHQ-4 score resembled those of non-clinical community respondents in a Southeast Asian study conducted during the COVID-19 pandemic (36), who reported a mean score of 4.98 (SD = 3.22). The results of the current study thus did not indicate that there was a heightened level of symptom severity in adults with ADHD during the COVID-19 pandemic in Norway. It should also be noted that a Norwegian study did not find evidence of increased mental illness during the COVID-19 pandemic, but, on the contrary, reported a decrease in mental disorders from the pre-pandemic period to the beginning of the pandemic (70). Nevertheless, the results of the present study should be compared to future studies in order to investigate if the mid-pandemic sampling period potentially influenced the results.

4.1. Limitations

The findings of this study are limited by the use of self-report data. Participants reported having received a diagnosis of ADHD in the specialist mental health service. However, the time for diagnosis was not recorded and, in the present study, the diagnosis was not confirmed through structured clinical interviews. Moreover, we did not include validated measures of anxiety and depression to verify construct validity. Furthermore, the high factor loadings of items 2 and 4 of the PHQ-4 indicate that these questions are especially important when assessing depressive and anxiety symptoms in adults with ADHD. Future research needs to verify the findings in a sample of newly diagnosed ADHD-patients in a clinical setting to investigate if the results are replicable. The cross-sectional design did not allow us to investigate the test–retest reliability of the PHQ-4. Future studies should include an assessment at a follow-up to investigate longitudinal invariance. The use of electronic data collection may pose a challenge to less digitally literate persons, and it is unclear if the results of the current study are generalizable to the traditional pen-and-paper utilization of the PHQ-4. Furthermore, it is important to acknowledge that our sample is limited by an imbalance in gender representation, with women comprising 66 percent of the participants. This gender disparity warrants caution in generalizing our findings to the broader population. However, it is worth noting that this distribution is consistent with previous studies conducted in Norway with different outpatient populations (71, 72).

4.2. Conclusion

The preliminary evidence for the psychometric properties of the PHQ-4 is promising. The results showed high internal reliability, and a two-factor solution consisting of a depression factor and an anxiety factor showed acceptable model fit (RMSEA = 0.038, CFI = 1.00, TLI = 0.999, SRMR = 0.004), with standardized loadings between 0.79 and 0.97. Due to the well-known difficulties regarding attention and hyperactivity experienced by persons with ADHD, the PHQ-4 is a brief instrument that may be particularly useful in assessing comorbid psychological symptoms. However, future studies should further evaluate the PHQ-4 in newly diagnosed adult ADHD samples to investigate its psychometric properties in a clinical setting.

Data availability statement

Data and materials are available on reasonable request.

Ethics statement

The studies involving human participants were reviewed and approved by the Regional Committee for Medicine and Health Research Ethics in Mid-Norway. The patients/participants provided their written informed consent to participate in this study.

Author contributions

ML-C and AM conceptualized the study. AH and SL performed the statistical analyses. AM contributed to the data collection, writing – reviewing, and editing. All authors contributed to the interpretation of the analyses and in critically reviewing – editing the manuscript, and read and approved the final manuscript.

Funding

During the data collection ML-C was supported by a postdoctoral grant from The Liaison Committee for education, research and innovation in Central Norway (Samarbeidsorganet; project no. 90327500). The funding body had no role in the design of the study or the collection, analysis or interpretation of the data or in writing the manuscript.

Acknowledgments

The authors thank Vårres Regional User-Led Center Central-Norway, Trondheim, Norway, who made possible the current project.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. World Health Organization. 6A05 attention deficit hyperactivity disorder. 11th ed (2019).

Google Scholar

2. Fayyad, J, Sampson, NA, Hwang, I, Adamowski, T, Aguilar-Gaxiola, S, Al-Hamzawi, A, et al. The descriptive epidemiology of DSM-IV adult ADHD in the world health organization world mental health surveys. Atten Defic Hyperact Disord. (2017) 9:47–65. doi: 10.1007/s12402-016-0208-3

CrossRef Full Text | Google Scholar

3. Gjervan, B, Torgersen, T, Nordahl, HM, and Rasmussen, K. Functional impairment and occupational outcome in adults with ADHD. J Atten Disord. (2012) 16:544–52. doi: 10.1177/1087054711413074

CrossRef Full Text | Google Scholar

4. Halmoy, A, Fasmer, OB, Gillberg, C, and Haavik, J. Occupational outcome in adult ADHD: impact of symptom profile, comorbid psychiatric problems, and treatment: a cross-sectional study of 414 clinically diagnosed adult ADHD patients. J Atten Disord. (2009) 13:175–87. doi: 10.1177/1087054708329777

CrossRef Full Text | Google Scholar

5. Kessler, RC, Chiu, WT, Demler, O, Merikangas, KR, and Walters, EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. (2005) 62:617–27. doi: 10.1001/archpsyc.62.6.617

CrossRef Full Text | Google Scholar

6. Rasmussen, K, and Levander, S. Untreated ADHD in adults: are there sex differences in symptoms, comorbidity, and impairment? J Atten Disord. (2009) 12:353–60. doi: 10.1177/1087054708314621

CrossRef Full Text | Google Scholar

7. Torgersen, T, Gjervan, B, and Rasmussen, K. ADHD in adults: a study of clinical characteristics, impairment and comorbidity. Nord J Psychiatry. (2006) 60:38–43. doi: 10.1080/08039480500520665

CrossRef Full Text | Google Scholar

8. Nutt, DJ, Kessler, RC, Alonso, J, Benbow, A, Lecrubier, Y, Lépine, JP, et al. Consensus statement on the benefit to the community of ESEMeD (European study of the epidemiology of mental disorders) survey data on depression and anxiety. J Clin Psychiatry. (2007) 68:42–8.

Google Scholar

9. Kessler, RC, Adler, L, Barkley, R, Biederman, J, Conners, CK, Demler, O, et al. The prevalence and correlates of adult ADHD in the United States: results from the National Comorbidity Survey Replication. Am J Psychiatry. (2006) 163:716–23. doi: 10.1176/ajp.2006.163.4.716

CrossRef Full Text | Google Scholar

10. Jensen, CM, Amdisen, BL, Jørgensen, KJ, and Arnfred, SM. Cognitive behavioural therapy for ADHD in adults: systematic review and meta-analyses. Atten Defic Hyperact Disord. (2016) 8:3–11. doi: 10.1007/s12402-016-0188-3

CrossRef Full Text | Google Scholar

11. Stuhec, M, Lukić, P, and Locatelli, I. Efficacy, acceptability, and tolerability of lisdexamfetamine, mixed amphetamine salts, methylphenidate, and modafinil in the treatment of attention-deficit hyperactivity disorder in adults: a systematic review and meta-analysis. Ann Pharmacother. (2019) 53:121–33. doi: 10.1177/1060028018795703

CrossRef Full Text | Google Scholar

12. Katzman, MA, Bilkey, TS, Chokka, PR, Fallu, A, and Klassen, LJ. Adult ADHD and comorbid disorders: clinical implications of a dimensional approach. BMC Psychiatry. (2017) 17:302. doi: 10.1186/s12888-017-1463-3

CrossRef Full Text | Google Scholar

13. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Washington, DC: American Psychiatric Association (2013).

Google Scholar

14. Lowe, B, Wahl, I, Rose, M, Spitzer, C, Glaesmer, H, Wingenfeld, K, et al. A 4-item measure of depression and anxiety: validation and standardization of the patient health Questionnaire-4 (PHQ-4) in the general population. J Affect Disord. (2010) 122:86–95. doi: 10.1016/j.jad.2009.06.019

CrossRef Full Text | Google Scholar

15. Kroenke, K, Spitzer, RL, Williams, JB, and Lowe, B. An ultra-brief screening scale for anxiety and depression: the PHQ-4. Psychosomatics. (2009) 50:613–21. doi: 10.1176/appi.psy.50.6.613

CrossRef Full Text | Google Scholar

16. Kroenke, K, Spitzer, RL, and Williams, JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. (2001) 16:606–13. doi: 10.1046/j.1525-1497.2001.016009606.x

CrossRef Full Text | Google Scholar

17. Spitzer, RL, Kroenke, K, Williams, JB, and Löwe, B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. (2006) 166:1092–7. doi: 10.1001/archinte.166.10.1092

CrossRef Full Text | Google Scholar

18. Arroll, B, Goodyear-Smith, F, Crengle, S, Gunn, J, Kerse, N, Fishman, T, et al. Validation of PHQ-2 and PHQ-9 to screen for major depression in the primary care population. Ann Fam Med. (2010) 8:348. doi: 10.1370/afm.1139

CrossRef Full Text | Google Scholar

19. Carey, M, Boyes, A, Noble, N, Waller, A, and Inder, K. Validation of the PHQ-2 against the PHQ-9 for detecting depression in a large sample of Australian general practice patients. Aust J Prim Health. (2016) 22:262–6. doi: 10.1071/PY14149

CrossRef Full Text | Google Scholar

20. Kroenke, K, Spitzer, RL, and Williams, JB. The patient health Questionnaire-2: validity of a two-item depression screener. Med Care. (2003) 41:1284–92. doi: 10.1097/01.MLR.0000093487.78664.3C

CrossRef Full Text | Google Scholar

21. Löwe, B, Kroenke, K, and Gräfe, K. Detecting and monitoring depression with a two-item questionnaire (PHQ-2). J Psychosom Res. (2005) 58:163–71. doi: 10.1016/j.jpsychores.2004.09.006

CrossRef Full Text | Google Scholar

22. Rancans, E, Trapencieris, M, Ivanovs, R, and Vrublevska, J. Validity of the PHQ-9 and PHQ-2 to screen for depression in nationwide primary care population in Latvia. Ann General Psychiatry. (2018) 17:33. doi: 10.1186/s12991-018-0203-5

CrossRef Full Text | Google Scholar

23. Wild, B, Eckl, A, Herzog, W, Niehoff, D, Lechner, S, Maatouk, I, et al. Assessing generalized anxiety disorder in elderly people using the GAD-7 and GAD-2 scales: results of a validation study. Am J Geriatr Psychiatry. (2014) 22:1029–38. doi: 10.1016/j.jagp.2013.01.076

CrossRef Full Text | Google Scholar

24. Delgadillo, J, Payne, S, Gilbody, S, Godfrey, C, Gore, S, Jessop, D, et al. Brief case finding tools for anxiety disorders: validation of GAD-7 and GAD-2 in addictions treatment. Drug Alcohol Depend. (2012) 125:37–42. doi: 10.1016/j.drugalcdep.2012.03.011

CrossRef Full Text | Google Scholar

25. Donker, T, van Straten, A, Marks, I, and Cuijpers, P. Quick and easy self-rating of generalized anxiety disorder: validity of the Dutch web-based GAD-7, GAD-2 and GAD-SI. Psychiatry Res. (2011) 188:58–64. doi: 10.1016/j.psychres.2011.01.016

CrossRef Full Text | Google Scholar

26. Cano-Vindel, A, Muñoz-Navarro, R, Medrano, LA, Ruiz-Rodríguez, P, González-Blanch, C, Gómez-Castillo, MD, et al. A computerized version of the patient health Questionnaire-4 as an ultra-brief screening tool to detect emotional disorders in primary care. J Affect Disord. (2018) 234:247–55. doi: 10.1016/j.jad.2018.01.030

CrossRef Full Text | Google Scholar

27. Mills, SD, Fox, RS, Pan, TM, Malcarne, VL, Roesch, SC, and Sadler, GR. Psychometric evaluation of the patient health questionnaire–4 in Hispanic Americans. Hisp J Behav Sci. (2015) 37:560–71. doi: 10.1177/0739986315608126

CrossRef Full Text | Google Scholar

28. Christodoulaki, A, Baralou, V, Konstantakopoulos, G, and Touloumi, G. Validation of the patient health Questionnaire-4 (PHQ-4) to screen for depression and anxiety in the Greek general population. J Psychosom Res. (2022) 160:110970. doi: 10.1016/j.jpsychores.2022.110970

CrossRef Full Text | Google Scholar

29. Kim, HW, Shin, C, Lee, SH, and Han, C. Standardization of the Korean version of the patient health Questionnaire-4 (PHQ-4). Clin Psychopharmacol Neurosci. (2021) 19:104–11. doi: 10.9758/cpn.2021.19.1.104

CrossRef Full Text | Google Scholar

30. Kliem, S, Moessle, T, Klatt, T, Fleischer, S, Kudlacek, D, Kroeger, C, et al. Psychometric evaluation of an Arabic version of the PHQ-4 based on a representative survey of Syrian refugees. Psychother Psychosom Med Psychol. (2016) 66:385–92. doi: 10.1055/s-0042-114775

CrossRef Full Text | Google Scholar

31. Wicke, FS, Krakau, L, Löwe, B, Beutel, ME, and Brähler, E. Update of the standardization of the patient health Questionnaire-4 (PHQ-4) in the general population. J Affect Disord. (2022) 312:310–4. doi: 10.1016/j.jad.2022.06.054

CrossRef Full Text | Google Scholar

32. Kocalevent, RD, Finck, C, Jimenez-Leal, W, Sautier, L, and Hinz, A. Standardization of the Colombian version of the PHQ-4 in the general population. BMC Psychiatry. (2014) 14:205. doi: 10.1186/1471-244X-14-205

CrossRef Full Text | Google Scholar

33. Rodríguez-Muñoz, MF, Ruiz-Segovia, N, Soto-Balbuena, C, Le, HN, Olivares-Crespo, ME, and Izquierdo-Méndez, N. The psychometric properties of the patient health Questionnaire-4 for pregnant women. Int J Environ Res Public Health. (2020) 17:7583. doi: 10.3390/ijerph17207583

CrossRef Full Text | Google Scholar

34. Ghaheri, A, Omani-Samani, R, Sepidarkish, M, Hosseini, M, and Maroufizadeh, S. The four-item patient health questionnaire for anxiety and depression: a validation study in infertile patients. Int J Fertil Steril. (2020) 14:234–9. doi: 10.22074/ijfs.2020.44412

CrossRef Full Text | Google Scholar

35. Wörfel, F, Gusy, B, Lohmann, K, Töpritz, K, and Kleiber, D. Mental health problems among university students and the impact of structural conditions. J Public Health. (2016) 24:125–33. doi: 10.1007/s10389-015-0703-6

CrossRef Full Text | Google Scholar

36. Mendoza, NB, Frondozo, CE, Dizon, JIWT, and Buenconsejo, JU. The factor structure and measurement invariance of the PHQ-4 and the prevalence of depression and anxiety in a southeast Asian context amid the COVID-19 pandemic. Curr Psychol. (2022). doi: 10.1007/s12144-022-02833-5

CrossRef Full Text | Google Scholar

37. Barrera, AZ, Moh, YS, Nichols, A, and Le, HN. The factor reliability and convergent validity of the patient health Questionnaire-4 among an international sample of pregnant women. J Women's Health. (2020) 29:8320. doi: 10.1089/jwh.2020.8320

CrossRef Full Text | Google Scholar

38. Mokkink, LB, Terwee, CB, Patrick, DL, Alonso, J, Stratford, PW, Knol, DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. (2010) 19:539–49. doi: 10.1007/s11136-010-9606-8

CrossRef Full Text | Google Scholar

39. Von Elm, E, Altman, DG, Egger, M, Pocock, SJ, Gøtzsche, PC, and Vandenbroucke, JP. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. (2007) 370:1453–7. doi: 10.7326/0003-4819-147-8-200710160-00010

CrossRef Full Text | Google Scholar

40. World Health Organization. ICD-10: International statistical classification of diseases and related health problems: Tenth revision. 2nd ed. Geneva: World Health Organization. (2004).

Google Scholar

41. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4th ed), Text Revision. Washington, DC: (2000) Author.

Google Scholar

42. Lara-Cabrera, ML, Mundal, IP, and De Las, CC. Patient-reported well-being: psychometric properties of the world health organization well-being index in specialised community mental health settings. Psychiatry Res. (2020) 291:113268. doi: 10.1016/j.psychres.2020.113268

CrossRef Full Text | Google Scholar

43. Nylén-Eriksen, M, Bjørnnes, AK, Hafstad, H, Lie, I, Grov, EK, and Lara-Cabrera, ML. Validating the five-item World Health Organization well-being index. Int J Environ Res Public Health. (2022) 19:11489. doi: 10.3390/ijerph191811489

CrossRef Full Text | Google Scholar

44. Kaiser, SKH. Måleegenskaper ved den norske versjonen av Verdens helseorganisasjon well-being index (WHO-5). [psychometric properties of the Norwegian version of the World Health Organization well-being index (WHO-5)]. PsykTestBarn. (2019) 1:63. doi: 10.21337/0063

CrossRef Full Text | Google Scholar

45. Topp, CW, Østergaard, SD, Søndergaard, S, and Bech, P. The WHO-5 well-being index: a systematic review of the literature. Psychother Psychosom. (2015) 84:167–76. doi: 10.1159/000376585

CrossRef Full Text | Google Scholar

46. Cohen, S, Karmack, T, and Mermelstein, R. A global measure of perceived stress. J Health Soc Behav. (1983) 24:385–96. doi: 10.2307/2136404

CrossRef Full Text | Google Scholar

47. Warttig, SL, Forshaw, MJ, South, J, and White, AK. New, normative, English-sample data for the short form perceived stress scale (PSS-4). J Health Psychol. (2013) 18:1617–28. doi: 10.1177/1359105313508346

CrossRef Full Text | Google Scholar

48. Carnegie Mellon University, Department of Psychology. Available at: [https://www.cmu.edu/dietrich/psychology/stress-immunity-disease-lab/scales/index.html] (Accessed June 6, 2023).

Google Scholar

49. Sahmurova, A, Arikan, S, Gursesli, MC, and Duradoni, M. ADHD symptoms as a stressor leading to depressive symptoms among university students: the mediating role of perceived stress between ADHD and depression. Int J Environ Res Public Health. (2022) 19:11091. doi: 10.3390/ijerph191711091

CrossRef Full Text | Google Scholar

50. Kocalevent, R-D, Berg, L, Beutel, ME, Hinz, A, Zenger, M, Härter, M, et al. Social support in the general population: standardization of the Oslo social support scale (OSSS-3). BMC Psychol. (2018) 6:31. doi: 10.1186/s40359-018-0249-9

CrossRef Full Text | Google Scholar

51. Dalgard, OS, Dowrick, C, Lehtinen, V, Vazquez-Barquero, JL, Casey, P, Wilkinson, G, et al. Negative life events, social support and gender difference in depression: a multinational community survey with data from the ODIN study. Soc Psychiatry Psychiatr Epidemiol. (2006) 41:444–51. doi: 10.1007/s00127-006-0051-5

CrossRef Full Text | Google Scholar

52. Glaesmer, H, Grande, G, Braehler, E, and Roth, M. The German version of the satisfaction with life scale (SWLS). Eur J Psychol Assess. (2011). doi: 10.1027/1015-5759/a000058

CrossRef Full Text | Google Scholar

53. Bøen, H, Dalgard, OS, and Bjertness, E. The importance of social support in the associations between psychological distress and somatic health problems and socio-economic factors among older adults living at home: a cross sectional study. BMC Geriatr. (2012) 12:1–12. doi: 10.1186/1471-2318-12-27

CrossRef Full Text | Google Scholar

54. Nes, R, Hansen, T, Barstad, A, Vittersø, J, Carlquist, E, and Røysamb, E. Anbefalinger for et bedre målesystem [recommendations for an improved measurement system]. Oslo: Helsedirektoratet [the Norwegian Directorate of Health] (2018).

Google Scholar

55. Connolly, RD, Lamont, A, and Speed, D. Perceived social support on the relationship between ADD/ADHD and both anxious and depressive symptoms among Canadian adults. J Atten Disord. (2023) 27:283–93. doi: 10.1177/10870547221136227

CrossRef Full Text | Google Scholar

56. Roohafza, HR, Afshar, H, Keshteli, AH, Mohammadi, N, Feizi, A, Taslimi, M, et al. What's the role of perceived social support and coping styles in depression and anxiety? J Res Med Sci. (2014) 19:944.

Google Scholar

57. Browne, MW, and Cudeck, R. (1993) Alternative ways of assessing model fit. In: Testing structural equation models. edn. Edited by Bollen, KA, and Long, JS: Newbury Park, CA: Sage. 136–162.

Google Scholar

58. Hu, L, and Bentler, PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model Multidiscip J. (1999) 6:1–55. doi: 10.1080/10705519909540118

CrossRef Full Text | Google Scholar

59. Brown, TA. Confirmatory factor analysis for applied research. New York: Guilford (2006).

Google Scholar

60. Muthén, KL, and Muthén, BO. Mplus: statistical analysis with latent variables (version 8.8). Los Angeles, CA, USA: (2022).

Google Scholar

61. Materu, J, Kuringe, E, Nyato, D, Galishi, A, Mwanamsangu, A, Katebalila, M, et al. The psychometric properties of PHQ-4 anxiety and depression screening scale among out of school adolescent girls and young women in Tanzania: a cross-sectional study. BMC Psychiatry. (2020) 20:321. doi: 10.1186/s12888-020-02735-5

CrossRef Full Text | Google Scholar

62. Fuller-Thomson, E, Lewis, DA, and Agbeyaka, SK. Attention-deficit/hyperactivity disorder casts a long shadow: findings from a population-based study of adult women with self-reported ADHD. Child Care Health Dev. (2016) 42:918–27. doi: 10.1111/cch.12380

CrossRef Full Text | Google Scholar

63. Babinski, DE, Neely, KA, Ba, DM, and Liu, G. Depression and suicidal behavior in young adult men and women with ADHD: evidence from claims data. J Clin Psychiatry. (2020) 81. doi: 10.4088/JCP.19m13130

CrossRef Full Text | Google Scholar

64. Rucklidge, JJ. Gender differences in attention-deficit/hyperactivity disorder. Psychiatr Clin. (2010) 33:357–73. doi: 10.1016/j.psc.2010.01.006

CrossRef Full Text | Google Scholar

65. Reid, RC, Campos, M, Selochan, N, and Fong, TW. Characteristics of treatment seeking problem gamblers with adult ADHD. Int J Ment Heal Addict. (2020) 18:875–90. doi: 10.1007/s11469-018-9986-5

CrossRef Full Text | Google Scholar

66. de Zwaan, M, Gruß, B, Müller, A, Graap, H, Martin, A, Glaesmer, H, et al. The estimated prevalence and correlates of adult ADHD in a German community sample. Eur Arch Psychiatry Clin Neurosci. (2012) 262:79–86. doi: 10.1007/s00406-011-0211-9

CrossRef Full Text | Google Scholar

67. Schafer, KM, Lieberman, A, Sever, AC, and Joiner, T. Prevalence rates of anxiety, depressive, and eating pathology symptoms between the pre-and peri-COVID-19 eras: a meta-analysis. J Affect Disord. (2022) 298:364–72. doi: 10.1016/j.jad.2021.10.115

CrossRef Full Text | Google Scholar

68. Robinson, E, Sutin, AR, Daly, M, and Jones, A. A systematic review and meta-analysis of longitudinal cohort studies comparing mental health before versus during the COVID-19 pandemic in 2020. J Affect Disord. (2022) 296:567–76. doi: 10.1016/j.jad.2021.09.098

CrossRef Full Text | Google Scholar

69. Behrmann, JT, Blaabjerg, J, and Jordansen, J. Jensen de López KM: systematic review: investigating the impact of COVID-19 on mental health outcomes of individuals with ADHD. J Atten Disord. (2022) 26:959–75. doi: 10.1177/10870547211050945

CrossRef Full Text | Google Scholar

70. Knudsen, AKS, Stene-Larsen, K, Gustavson, K, Hotopf, M, Kessler, RC, Krokstad, S, et al. Prevalence of mental disorders, suicidal ideation and suicides in the general population before and during the COVID-19 pandemic in Norway: a population-based repeated cross-sectional analysis. Lancet Reg Health Eur. (2021) 4:100071. doi: 10.1016/j.lanepe.2021.100071

CrossRef Full Text | Google Scholar

71. Moljord, IEO, Lara-Cabrera, ML, Perestelo-Pérez, L, Rivero-Santana, A, Eriksen, L, and Linaker, OM. Psychometric properties of the patient activation Measure-13 among out-patients waiting for mental health treatment: a validation study in Norway. Patient Educ Couns. (2015) 98:1410–7. doi: 10.1016/j.pec.2015.06.009

CrossRef Full Text | Google Scholar

72. Pedersen, H, Havnen, A, Brattmyr, M, Attkisson, CC, and Lara-Cabrera, ML. A digital Norwegian version of the client satisfaction questionnaire 8: factor validity and internal reliability in outpatient mental health care. BMC Psychiatry. (2022) 22:671. doi: 10.1186/s12888-022-04281-8

CrossRef Full Text | Google Scholar