Reliability and Validity of General Health Questionnaire-12 in Chinese Dental Healthcare Workers During the COVID-19 Pandemic

Background: The General Health Questionnaire-12 (GHQ-12) is a widely used instrument to assess mental health status. However, little is known about its applicability in Chinese healthcare workers. This study aimed to evaluate the reliability and validity of the GHQ-12 in Chinese dental healthcare workers.

Methods: Dental healthcare workers participated in the first occupational survey in China conducted by the Chongqing Stomatological Association from February 2021 to March 2021 by filling out GHQ-12. The reliability and validity of GHQ-12 were then tested.

Results: A total of 3,020 valid electronic questionnaires were acquired. The positive detection rate of self-reported mental health status was 23.80% (719/3,020). The Cronbach's α coefficient of the GHQ-12 was 0.892, and the Cronbach's α coefficient was 0.877–0.888 after the deletion of individual items, and the split-half reliability was 0.843. The correlation coefficient between the item-total score ranged from 0.465 to 0.762 (P<0.05). The exploratory factor analysis found 2 common factors with a factor load of 0.564–0.818. The confirmatory factor analysis showed that the factor load on the specified items was 0.480–0.790.

Conclusions: The two-factor model of GHQ-12 featured good reliability and validity, which could be used to assess the mental health status of Chinese dental healthcare workers.

Introduction

Mental health problems are widespread among healthcare workers globally, and they are not optimistic in Chinese healthcare workers. A study from China showed that 48.28% of resident physicians had moderate depressive symptoms (1). Another large-scale epidemiological survey showed that 38% of nurses in public hospitals in China had depressive symptoms (2). There are 1.8 physicians for every 1,000 people in China, which is less than half of other developed countries (3, 4). As a direct result of the shortage of health care workers, the workload is enormous. Meanwhile, they also face great academic pressure to achieve promotion in professional titles, all of which have been shown to be risk factors for mental health status (5, 6). With the outbreak of Corona Virus Disease 2019 (COVID-19), the workload of healthcare workers has dramatically increased, and the mental health status of healthcare workers is facing serious challenges.

Medical healthcare workers are high-risk groups for the infection of COVID-19. Especially for dental healthcare workers, as they regularly have face-to-face contact with patients' saliva, blood and crevicular gingival fluid (7, 8). It has been showed that fomites and airborne are the main transmission routes of COVID-19, while previous studies also showed that aerosols might be a medium of transmission. While using instruments in dental procedures, a large number of aerosols will be generated. Virus-laden aerosols, which can hang in the air for a long time (9), are considered to be potentially dangerous particles that can be inhaled into the lungs to spread disease and aggravate the severity of disease progression (10, 11). With the outbreak of COVID-19, dental healthcare workers are continuously exposed to an infectious environment. The huge workload and high-risk working environment put them under great pressure. Therefore, it is necessary to screen the mental health status of dental healthcare workers efficiently.

Currently, there are many scales for the self-measurement of mental health status, and the General Health Questionnaire (GHQ), developed by the scholar, Goldberg, is one of the most representative instruments (12). The GHQ scale includes multiple versions, such as GHQ-60, GHQ-30, GHQ-20, GHQ-18, and GHQ-12. Among these, the GHQ-12 has become one of the most important tools for screening mental health status due to its simplicity and ease of operation (12, 13). It contains 12 items (six positive items and six negative items separately) (14), and is most commonly scored by 0-1-2-3 or 0-0-1-1 (15). Preliminary studies have found that the GHQ-12 has been translated into many versions and is widely used in different regions, such as Spain, Iran, Australia, India, and Sweden (16–20). It was also adopted to assess the mental health status of Chinese civil servants, adolescents, and university students (21–23).

Nevertheless, the reliability and validity of the GHQ-12 have not been clarified in Chinese healthcare workers, especially among dental healthcare workers. To address this issue, the purpose of this study was to evaluate the reliability and validity of the GHQ-12 and provide an effective tool for rapid screening of the mental health status of Chinese dental healthcare workers after the COVID-19 outbreak.

Materials and Methods

Participants

The data of this study were derived from the first occupational survey of dental healthcare workers in China conducted by the Chongqing Stomatological Association from February 2021 to March 2021. Based on the hospitals with a stomatological department in China, the Chongqing Stomatological Association used a convenience sampling method to send questionnaires by e-mail. Specifically, the staff of Chongqing Stomatological Association contacted the director in charge of the stomatological hospitals, the stomatological departments of the general hospitals, the dental clinics and invited them to participate in this research. Once the director agreed to participate in this study, the director invited the corresponding hospital/department healthcare workers to participate.

Research Instruments

Each participant was provided with a self-administrated questionnaire consisting of two parts. Part one was the basic characteristics of dental healthcare workers, including gender, age, academic degree obtained, technician, monthly income, working years, hours worked per week, relationship status, having children or not, the number of patients treated per day, whether tube bed, hospital type, job type, major type, commuting time, and whether they undertaking teaching tasks. Part two was the GHQ-12. The scale reflects the mental health status of the respondents through 12 self-assessment items. Each item includes four options (A, B, C, and D), and the bimodal scoring method (0-0-1-1) was adopted. Specifically, If A or B was selected, the value equates to 0; if C or D was selected, the value equates to 1. The detection rate of mental health was considered to be positive (such as the feeling of depression and anxiety, insomnia, or suicidality) when the cumulative score ≥ four (24–26).

Quality Control

To ensure the accuracy of the data, we evaluated the quality of all questionnaires. Criteria one: if the same answer was chosen for the whole questionnaire items, the questionnaire was considered invalid. Criteria two: we also checked the questionnaires from the same hospital. If two or more consecutive questionnaires had completely the same answers, only one questionnaire was included, and the other identical questionnaires were defined as invalid.

Statistical Analysis

Data were entered and checked through Excel 2013, and all statistical analyses were conducted using SPSS 21.0 (IBM Corp., Armonk, NY) and AMOS 21.0. The mean and standard deviation (SD) was calculated for quantitative data, and the percentage was calculated for qualitative data. The scale reliability was assessed by Cronbach's α coefficient, Guttman Split-Half coefficient, and a coefficient >0.70 was considered acceptable (27). In addition, the Spearman correlation coefficient was used as an index to judge the reliability, and a value >0.30 was considered acceptable (28, 29). The higher the coefficient, the stronger the correlation between items. The scale construct validity was assessed by exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). In this study, three-fourths of participants were assigned to EFA, and one-fourths were assigned to CFA. Specific indicators were used to evaluate the model fit. For EFA analysis, a Kaiser-Meyer-Olkin (KMO) coefficient >0.70, engine value >1.0 (30, 31) cumulative variance contribution rate >50%, and factor load >0.30 were recommended (32, 33). For CFA analysis, the Root Mean Square Error of Approximation (RMSEA) should be <0.10 (better if below 0.05); otherwise, the model should be rejected (33, 34). Comparative Fit Index (CFI), Normative Fit Index (NFI), Incremental Fit Index (IFI), Goodness of Fit Index (GFI) should be >0.90 (better if above 0.95) to indicate a proper fit (35–37). χ²/df was extremely sensitive to the sample size and was not a suitable indicator for model fit for sample size above 200 (38, 39). Therefore, χ²/df was not selected as an evaluation indicator in this study. A P-value < 0.05 was considered to be statistically significant.

Results

Demographic Characteristics

By the end of March 2021, a total of 3,128 questionnaires from 11 provinces in China were collected (Supplementary Table 1). There were 3,020 valid questionnaires (the effective rate was 96.55%) and 108 invalid questionnaires. Among these, 99 questionnaires were excluded by the exclusion criteria one, 9 questionnaires were excluded by the exclusion criteria two. For the respondents, 76.1% were female, 54.9% held a bachelor's degree, and 65.7% were 20–35 years old. 58.4% were junior technicians, and 45.4% had a monthly income between 5,000 and 10,000 RMB. Two-thirds worked <10 years, while 71.9% worked <45 h per week. 67.0% were married, 58.6% had children. 42.5% treated 10–20 patients per day. 88.6% tubed bed. 65.4% worked in a dental specialty hospital. 61.4% were doctors, and 44.1% worked in the general department. 40.9% commuted between 15 and 30 min, while two-thirds did not undertake teaching tasks. The demography characteristics are summarized in Table 1.

TABLE 1

Table 1. Demography characteristics of the participants.

GHQ-12 Characteristics

The positive rate of mental health status was 23.80% (719/3,020). The quartile of the GHQ-12 total score was 2 (0, 5) points, which was lower than the positive detection rate threshold. Among all items, item 12 had the lowest frequency of answering 1. In contrast, more than a quarter of the participants responded positively (answered 1) among item 1 and item 2, indicating that the positive findings in mental health status were mainly caused by “Lost much sleep” and “Under stress”, as shown in Table 2.

TABLE 2

Table 2. Item respondents of GHQ-12 items (n = 3,020).

Reliability and Correlations Analysis

The Cronbach's α coefficient of the whole GHQ-12 was 0.892. After excluding individual items, the Cronbach's α coefficient of the total scale ranged from 0.877 to 0.888, all of which were lower than the overall scale's 0.892 (Table 3). The Spearman-Brown coefficient of the half-fold reliability of the scale was 0.843, indicating that the scale's reliability was acceptable.

TABLE 3

Table 3. Cronbach's α and correlations for GHQ-12 scale (n = 3,020).

Spearman correlation analysis showed that the correlation coefficient between the item score and total scores was 0.465–0.762 (P<0.05). Item 2 showed the highest correlation with the total score, and item 12 showed the lowest correlation with the total score, which was higher than the criterion of 0.30 (Table 3). The correlation ranged from 0.264 to 0.665 among the inter-item scores (Supplementary Table 2). Item 2 showed the lowest correlation with item 12, and item 8 had the highest correlation with item 10. The correlation coefficient between the items-total score was higher than the inter-items, indicating the reliability of this scale was acceptable.

Exploratory Factor Analysis

KMO and Bartlett sphericity tests were performed on 12 items of the scale, and the KMO coefficient was 0.924, χ^{2 =} 11,642.038, df =66, and P-value <0.001, indicating that it was suitable for factor analysis. The maximum variance method was used for principal component analysis, and two common factors were extracted with eigenvalues >1, as shown in Supplementary Figure 1. Common factor 1 contains 6 items, namely items 1, 2, 3, 8, 9, 10, with a variance contribution rate of 31.999%. Common factor 2 contains 6 items, namely item 4, 5, 6, 7, 11, and 12, with a variance contribution rate of 24.767%. The cumulative variance contribution rate of the two-factor model showed a total variance of 56.766%, reaching the acceptable standard of more than 50% (32, 33) (Table 4). In addition, the factor load of items ranged from 0.564 to 0.818, which was higher than the recommended value of 0.30.

TABLE 4

Table 4. Factor load of the exploratory factor analysis for GHQ-12 (n = 2,265).

Confirmatory Factor Analysis

The χ²/df value of the original model was 5.795, with an RMSEA value of 0.08. The values of NFI, CFI, IFI, TLI, GFI were 0.914, 0.928, 0.928, 0.910, 0.933, respectively (Table 5). The correlation coefficient between common factor one and common factor two was 0.83, and the factor load on the specified factors of each item of the AMOS path ranged from 0.48 to 0.79 (Figure 1). All of the indexes were greater than the recommended value, indicating the validity of the two-factor model of GHQ-12 was acceptable. Based on the previous studies that explored the factor structure of GHQ-12 (14, 39–44), we also validated the previous factor model using our CFA sample size. All of the models produced relatively similar results. The RMSEA was more than 0.090, and NFI, as well as TLI, were >0.90 in all models. CFI, IFI and GFI were less than the recommended value of 0.90 in most models. The fit index of the CFA model is displayed in Table 5.

TABLE 5

Table 5. Index of the confirmatory factor analysis for GHQ-12 (n = 755).

FIGURE 1

Figure 1. Standard factor load of two common factor model.

Discussion

GHQ-12 is one of the most widespread scales to measure the mental health status of well-being, and has been translated to many versions worldwide (16–20). Although GHQ-12 has been frequently used in China, a few studies from China also tested the reliability and validity of GHQ-12 in civil servants, university students, and adolescents (21–23). There is no study to test the factor structure of GHQ-12 in Chinese healthcare workers. As we know, healthcare workers are at high-risk developing mental health problems due to their professional attributes, such as the huge workload, caregiver burden, high levels of job stress and low levels of job satisfaction (25, 45). Furthermore, mental health problems can lead to burnout, which will increase the risk of patient safety and decrease the healthcare quality (46, 47). Particularly, for dental healthcare workers, it is worse because of the COVID-19 outbreak and the working environment is prone to aerosol. To our knowledge, this is the first study evaluating the reliability and validity of GHQ-12, with the hope of providing an instrument for the quick assessment of the mental health status of dental healthcare workers in China.

In the current study, we gathered 3,020 valid electronic questionnaires of dental healthcare workers from 11 provinces in China. We believe this is the first large-scale use of GHQ-12 to assess the mental health status among dental healthcare workers in China. The positive rate of the mental health status of GHQ-12 was 23.8%, which was close to the healthcare workers in Nigeria (23.4%) (48), lower than the Chinese neurologists (37.8%) (25), healthcare workers in Japan (65.6%) (49), doctors in emergency medicine, anesthesia and intensive care medicine in the UK and Ireland (44.2%) (50) and obstetrics healthcare providers in Italy (51.1%) (51). It illustrated that the positive rate of mental health status of dental healthcare workers was relatively low, which could be attributed to the effective containment of COVID-19. However, nearly a quarter of participants had a positive detection of mental health status, which suggested that dental healthcare workers are still at high risk of mental health status.

In this study, Cronbach's α, Spearman-Brown coefficient, Spearman correlation analysis, EFA and CFA measurement were evaluated. On the basis of the reliability, the Cronbach's α coefficient of the total scale was 0.892. After removing individual items, the Cronbach's α coefficient of the rest of the scale was lower than 0.892 but >0.70, which means that deleting any item may reduce the reliability of the total scale. Furthermore, it also illustrates that individual items were important and cannot be deleted from the total scale, confirming previous studies' results (21). The Spearman-Brown coefficient of the total scale was 0.843, which was higher than the recommended value. In addition, the Spearman correlation coefficient of the item-total score was higher than the inter-item correlation coefficient, indicating that the reliability of the scale was appropriate, which was consistent with the published study (21).

To indicate the validity, EFA and CFA were performed to extract the common factor and verify the model separately. EFA extracted two common factors with engine-value >1.0 from GHQ-12, and the factor load was relatively high, ranging from 0.564 to 0.818. The results showed that the cumulative variance contribution rate of the two-factor model to the total variation of the GHQ-12 scale was 56.766%, which was higher than that of civil servants in China (50.22%), adolescents in China (53.27%) and the recommended value (50.0%) (32, 33). Furthermore, the CFA results showed that the RSMEA of the two-factor model was less than the recommended value of 0.10, and the fitness index NFI, IFI, TLI, GFI were higher than the recommended value of 0.90. The results are similar to the previous studies (21, 52), which means our two-factor model could interpret the mental health status of dental healthcare workers in China.

Historical studies have proposed many classical dimensional models, including unidimensional, two-dimensional and three-dimensional models (14, 39–44). Andrich proposed a classical two-dimensional model with 12 items, including six positive items (items 1, 3, 4, 7, 8, 12) and six negative items (items 2, 5, 6, 9, 10, 11) (14). Graetz proposed a classical three-dimensional model with a total of 12 items, including social disorder (items 1, 3, 4, 7, 8, 12), anxiety and depression (items 2, 5, 6, 9), and lack of confidence (items 6, 9) (42). However, in our results, common factor 1 included items 1, 2, 3, 8, 9, 10, and common factor 2 included items 4, 5, 6, 7, 11, 12. The two-factor model obtained from our study was distinct from the above models, which might be caused by the regional difference and research population difference. In addition, we also conducted CFA analysis to validate the GHQ-12 based on the seven previous classical models using our sample, and these seven classical models yield relatively similar results. Compared with the above classical models, the results also showed the model performance was superior to the other seven classical models based on the CFA fit index. This demonstrates that the two-factor model we proposed has good structural validity, and is suitable for evaluating the mental health status of Chinese dental healthcare workers.

Conclusions

Nearly a quarter of dental healthcare workers had mental health problems. The two-factor model of GHQ-12 we put forward had good reliability and validity among dental healthcare workers, which could be used as an instrument to screen the mental health status of dental healthcare workers in China.

Limitations

There are also some limitations in our study: Firstly, this study adopted a convenience sampling method, so the data may be prone to selection bias. Secondly, this study did not set a gold standard for measuring the mental health status, and lacked an efficacy standard validity analysis. Thirdly, we also did not conduct the reset reliability analysis. Fourthly, the characterization of non-respondents was not available, therefore it cannot be compared with the valid respondents. Finally, the study carried out on the population of healthcare workers, so it may not applicable to the general or other professional populations. Due to the above limitations, more studies are needed to support our results.

Data Availability Statement

The original contributions generated for this study are included in the article or supplementary material, further inquiries can be directed to the corresponding author.

Ethics Statement

The front page of the questionnaire introduced the background and purpose of the survey, informed participants of their rights and potential risks, and informed them that the survey was voluntary and anonymous. So, the informed consent was presumed if the participants returned the questionnaires. The questionnaire and study protocol were reviewed and approved by the ethics committee of the Stomatological Hospital of Chongqing Medical University.

Author Contributions

XZ and XJ: proposed the concept and design. LYan, LYang, HL, and JW: collected the data. XZ and XJ: analyzed and interpreted the data, wrote the manuscript. HW, YL, and JP: drafted and edited the manuscript. XJ, PJ, and PX: provided the funding. PX and PJ: supervised the study. All authors reviewed and approved the manuscript prior to its submission, and the authors declare no competing interests.

Funding

This work was supported by the Natural Science Foundation Project of China (Grant No. 81870775, 81500855) and the National Key R&D Program of China (Grant No. 2017YFA0505700).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to thank the Chongqing Stomatological Association for help of acquisition of data and technical guidance. We also would like to thank the dental healthcare workers who took part in this study. In addition, we thank Home for Researchers (www.home-for-researchers.com), for editing the English text of a draft of this manuscript.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2021.792838/full#supplementary-material

References

1. Chang Q, Xia Y, Bai S, Zhang X, Liu YS, Yao D, et al. Association between Pittsburgh sleep quality index and depressive symptoms in Chinese resident physicians. Front Psychiatry. (2021) 12:564815. doi: 10.3389/fpsyt.2021.564815

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Gong YH, Han TG, Yin XX, Yang GA, Zhuang RS, Chen YQ, et al. Prevalence of depressive symptoms and work-related risk factors among nurses in public hospitals in southern China: A cross-sectional study. Sci Rep-Uk. (2014) 4:7109. doi: 10.1038/srep07109

PubMed Abstract | CrossRef Full Text | Google Scholar

3. World Health Organization. E. coli. Available at: https://www.who.int/data/gho/data/indicators/indicator-details/GHO/medical-doctors-(per-10-000-population) (Accessed September 6, 2021) (2018).

4. World Health Organization. E. coli. Available at: https://www.who.int/data/gho/data/indicators/indicator-details/GHO/medical-doctors-(per-10-000-population) (Accessed September 6, 2021) (2018)

Google Scholar

5. Pu JC, Zhou XY, Zhu D, Zhong XN, Yang LN, Wang HY, et al. Gender differences in psychological morbidity, burnout, job stress and job satisfaction among Chinese neurologists: a national cross-sectional study. Psychol Health Med. (2017) 22:680–92. doi: 10.1080/13548506.2016.1211717

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Grover S, Dua D, Shouan A, Nehra R, Avasthi A. Perceived stress and barriers to seeking help from mental health professionals among trainee doctors at a tertiary care centre in North India. Asian J Psychiatr. (2019) 39:143–9. doi: 10.1016/j.ajp.2018.12.020

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Natapov L, Schwartz D, Herman HD, Markovich DD, Yellon D, Jarallah M, et al. Risk of SARS-CoV-2 transmission following exposure during dental treatment - A national cohort study. J Dent. (2021) 113:103791. doi: 10.1016/j.jdent.2021.103791

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Kim D, Ko JH, Peck KR, Baek JY, Moon HW, Ki HK, et al. A COVID-19 Exposure at a Dental Clinic Where Healthcare Workers Routinely Use Particulate Filtering Respirators. Int J Env Res Pub He. (2021) 18:6481. doi: 10.3390/ijerph18126481

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Ding S, Teo ZW, Wan MP, Ng BF. Aerosols from speaking can linger in the air for up to nine hours. Build Environ. (2021) 205:108239. doi: 10.1016/j.buildenv.2021.108239

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Locke L, Dada O, Shedd JS. Aerosol transmission of infectious disease and the efficacy of personal protective equipment (PPE): a systematic review. J Occup Environ Med. (2021) 63:e783–91. doi: 10.1097/JOM.0000000000002366

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Somsen GA, van Rijn C, Kooij S, Bem RA, Bonn D. Small droplet aerosols in poorly ventilated spaces and SARS-CoV-2 transmission. Lancet Resp Med. (2020) 8:658–9. doi: 10.1016/S2213-2600(20)30245-9

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Romppel M, Braehler E, Roth M, Glaesmer H. What is the General Health Questionnaire-12 assessing? Dimensionality and psychometric properties of the General Health Questionnaire-12 in a large scale German population sample. Compr Psychiat. (2013) 54:406–13. doi: 10.1016/j.comppsych.2012.10.010

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Abubakar A, Fischer R. The Factor Structure of the 12-item General Health Questionnaire in a Literate Kenyan Population. Stress Health. (2012) 28:248–54. doi: 10.1002/smi.1420

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Andrich D, van Schoubroeck L. The General Health Questionnaire: a psychometric analysis using latent trait theory. Psychol Med. (1989) 19:469–85. doi: 10.1017/S0033291700012502

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Montazeri A, Harirchi AM, Shariati M, Garmaroudi G, Ebadi M, Fateh A. The 12-item General Health Questionnaire (GHQ-12): translation and validation study of the Iranian version. Health Qual Life Outcomes. (2003) 1:66. doi: 10.1186/1477-7525-1-66

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Padron A, Galan I, Durban M, Gandarillas A, Rodriguez-Artalejo F. Confirmatory factor analysis of the General Health Questionnaire (GHQ-12) in Spanish adolescents. Qual Life Res. (2012) 21:1291–8. doi: 10.1007/s11136-011-0038-x

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Namjoo S, Shaghaghi A, Sarbaksh P, Allahverdipour H, Pakpour AH. Psychometric properties of the General Health Questionnaire (GHQ-12) to be applied for the Iranian elder population (vol 21, pg 1047, 2016). Aging Ment Health. (2017) 21:Ii–Ii. doi: 10.1080/13607863.2016.1196337

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Campbell A, Knowles S, A. Confirmatory Factor Analysis of the GHQ12 Using a Large Australian Sample. Eur J Psychol Assess. (2007) 23:2–8. doi: 10.1027/1015-5759.23.1.2

CrossRef Full Text | Google Scholar

19. Kashyap GC, Singh SK. Reliability and validity of general health questionnaire (GHQ-12) for male tannery workers: a study carried out in Kanpur, India. Bmc Psychiatry. (2017) 17:102. doi: 10.1186/s12888-017-1253-y

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Lundin A, Hallgren M, Theobald H, Hellgren C, Torgen M. Validity of the 12-item version of the General Health Questionnaire in detecting depression in the general population. Public Health. (2016) 136:66–74. doi: 10.1016/j.puhe.2016.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Liang Y, Wang L, Yin XC. The factor structure of the 12-item general health questionnaire (GHQ-12) in young Chinese civil servants. Health Qual Life Out. (2016) 14:136. doi: 10.1186/s12955-016-0539-y

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Li WHC, Chung JOK, Chui MML, Chan PSL. Factorial structure of the Chinese version of the 12-item General Health Questionnaire in adolescents. J Clin Nurs. (2009) 18:3253–61. doi: 10.1111/j.1365-2702.2009.02905.x

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Ye S. Factor structure of the General Health Questionnaire (GHQ-12): the role of wording effects. Pers Individ Dif. (2009) 46:197–201. doi: 10.1016/j.paid.2008.09.027

CrossRef Full Text | Google Scholar

24. Haoka T, Sasahara SI, Tomotsune Y, Yoshino S, Maeno T, Matsuzaki I. The effect of stress-related factors on mental health status among resident doctors in Japan. Med Educ. (2010) 44:826–34. doi: 10.1111/j.1365-2923.2010.03725.x

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Zhou XY, Pu JC, Zhong XN, Zhu D, Yin DH, Yang LN, et al. Burnout, psychological morbidity, job stress, and job satisfaction in Chinese neurologists. Neurology. (2017) 88:1727–35. doi: 10.1212/WNL.0000000000003883

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Goldberg D, Williams P. A User's Guide to the General Health Questionnaire. Windsor: Nfer-Nelson Publishing(1988).

Google Scholar

27. Valles J, Guilera M, Briones Z, Gomar C, Canet J, Alonso J, et al. Validity of the Spanish 8-item Short-form Generic Health-related Quality-of-Life Questionnaire in Surgical Patients A Population-based Study. Anesthesiology. (2010) 112:1164–74. doi: 10.1097/ALN.0b013e3181d3e017

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Nunnally JC, Bernstein IH. Psychometric Theory. 3rd ed. New York: McGraw-Hill (1994).

Google Scholar

29. Hinkle D, Jurs S, Wiersma W. Applied Statistics for the Behavior-Ral Sciences. Boston: Houghton Mifflin (1988).

Google Scholar

30. Papathanasiou A, Koutsovasilis A, Shea S, Philalithis A, Papavasiliou S, Melidonis A, et al. The Problem Areas in Diabetes (PAID) scale: psychometric evaluation survey in a Greek sample with type 2 diabetes. J Psychiatr Ment Hlt. (2014) 21:345–53. doi: 10.1111/j.1365-2850.2012.01875.x

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Zhao QC, Yang C, Tang SS, Zhao YJ, Dou HZ, Chen YH, et al. Developing and Testing the Reliability and Validity of the Brief Haze Weather Health Protection Behavior Assessment Scale-Adolescent Version (BHWHPBAS-AV). Front Pediatr. (2020) 8:498885. doi: 10.3389/fped.2020.498885

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Tian L, Li HX, Dong B, Xie CY, Wang H, Lin L. The supportive supervisory scale: psychometric properties in Chinese health care aides samples. Health Qual Life Out. (2021) 19:60. doi: 10.1186/s12955-021-01706-y

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Wang YM Li K, Li HQ, Zhao WB, Chen YR, Shang HL, et al. Development, reliability, and validity of the home blood pressure monitoring adherence scale for patients with chronic kidney disease. Patient Prefer Adher. (2020) 14:1863–72. doi: 10.2147/PPA.S264287

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Rodríguez Coll P, Casañas R, Collado Palomares A, Maldonado Aubian G, Salgado Poveda MI, Espada-TresPalacios X, et al. Validation and psychometric properties of the Spanish version of the questionnaire for assessing the childbirth experience (QACE). Sex Reprod Healthc. (2021) 27:100584. doi: 10.1016/j.srhc.2020.100584

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Bentler PM, Bonett D. Significance tests and goodness-of-fit in analysis of covariance structures. Psychol Bull. (1980) 88:588–606. doi: 10.1037/0033-2909.88.3.588

CrossRef Full Text | Google Scholar

36. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. (1990) 107:238–46. doi: 10.1037/0033-2909.107.2.238

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Yuan J, Zhang YH, Xu T, Zhang H, Lu Z, Yang XJ, et al. Development and preliminary evaluation of Chinese Preschoolers' Caregivers' Feeding Behavior Scale. J Acad Nutr Diet. (2019) 119:1890–902. doi: 10.1016/j.jand.2019.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Ishiyaku B, Kasim R, Harir AI. Confirmatory factoral validity of public housing satisfaction constructs. Cogent Bus Manage. (2017) 4:1359458. doi: 10.1080/23311975.2017.1359458

CrossRef Full Text | Google Scholar

39. Sweeting H, Young R, West P, GHQ. increases among Scottish 15 year olds 1987-2006. Soc Psychiatry Psychiatr Epidemiol. (2009) 44:579–86. doi: 10.1007/s00127-008-0462-6

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Schmitz N, Kruse J, Tress W. Psychometric properties of the General Health Questionnaire (GHQ-12) in a German primary care sample. Acta Psychiat Scand. (1999) 100:462–8. doi: 10.1111/j.1600-0447.1999.tb10898.x

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Politi PL, Piccinelli M, Wilkinson G. Reliability, validity and factor structure of the 12-item General Health Questionnaire among young males in Italy. Acta Psychiatr Scand. (1994) 90:432–7. doi: 10.1111/j.1600-0447.1994.tb01620.x

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Graetz B. Multidimensional properties of the General Health Questionnaire. Soc Psychiatry Psychiatr Epidemiol. (1991) 26:132–8. doi: 10.1007/BF00782952

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Farrell GA. The mental health of hospital nurses in Tasmania as measured by the 12-item General Health Questionnaire. J Adv Nurs. (1998) 28:707–12. doi: 10.1046/j.1365-2648.1998.00735.x

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Daradkeh TK, Ghubash R. el-Rufaie OE. Reliability, validity, and factor structure of the Arabic version of the 12-item general health questionnaire. Psychol Rep. (2001) 89:85–94. doi: 10.2466/pr0.2001.89.1.85

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Wang S, Li LZ, van Antwerpen N, Suparman S, Gayatri M, Sari NP, et al. Hand hygiene and mask-wearing practices during COVID-19 among healthcare workers: misinformation as a predictor. Am J Trop Med Hyg. (2021). doi: 10.4269/ajtmh.21-0463

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Ghafouri R, Lotfi-Bajestani S, Nasiri M, Ohnishi K, Atashzadeh-Shoorideh F. Psychometrics of the moral distress scale in Iranian mental health nurses. BMC Nurs. (2021) 20:166. doi: 10.1186/s12912-021-00674-4

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Li LZ, Wang S. Do work-family initiatives improve employee mental health? Longitudinal evidence from a nationally representative cohort. J Affect Disord. (2021) 297:407–14. doi: 10.1016/j.jad.2021.10.112

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Olagunju AT, Bioku AA, Olagunju TO, Sarimiye FO, Onwuameze OE, Halbreich U. Psychological distress and sleep problems in healthcare workers in a developing context during COVID-19 pandemic: Implications for workplace wellbeing. Prog Neuro-Psychoph. (2021) 110:110292. doi: 10.1016/j.pnpbp.2021.110292

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Tanaka K, Tahara M, Mashizume Y, Takahashi K. Effects of Lifestyle Changes on the Mental Health of Healthcare Workers with Different Sense of Coherence Levels in the Era of COVID-19 Pandemic. Int J Env Res Pub He. (2021) 18:2801. doi: 10.3390/ijerph18062801

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Roberts T, Daniels J, Hulme W, Hirst R, Horner D, Lyttle MD, et al. Psychological distress during the acceleration phase of the COVID-19 pandemic: a survey of doctors practising in emergency medicine, anaesthesia and intensive care medicine in the UK and Ireland. Emerg Med J. (2021) 38:450–9. doi: 10.1136/emermed-2020-210438

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Del Piccolo L, Donisi V, Raffaelli R, Garzon S, Perlini C, Rimondini M, et al. The psychological impact of COVID-19 on healthcare providers in obstetrics: a cross-sectional survey study. Front Psychol. (2021) 12:632999. doi: 10.3389/fpsyg.2021.632999

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Hankins M. The reliability of the twelve-item general health questionnaire (GHQ-12) under realistic assumptions. BMC Public Health. (2008) 8:355. doi: 10.1186/1471-2458-8-355

PubMed Abstract | CrossRef Full Text | Google Scholar