The adaption of the Chinese version of the COVID Stress Scales as a screening instrument of stress: Psychometric properties during the COVID-19 pandemic

The COVID Stress Scales (CSS) was used to access related distress concerning Corona Virus Disease 2019 (COVID-19). Based on China's epidemic prevention and control policies during the COVID-19 pandemic, the adaption of the Chinese version of the CSS was developed. Our study evaluated the reliability and validity of the Chinese adapted version of the CSS during the COVID-19 pandemic. An online survey was employed to construct a national sample of 2,116 participants in Chinese mainland. We examined the factor structure, internal consistency, convergent validity, discriminant validity, and concurrent validity. The results demonstrated that the six-factor solution for the Chinese adaptation of the CSS proved a good fit with the data after comparing the factor structure with the five-factor model. The six-factor model had good reliability and supported good convergent, discriminant, and concurrent validity of the CSS Chinese adaption. Overall, our findings supported the Chinese adapted version of the CSS as a psychometrically sound measure of stress during the COVID-19 pandemic in China.

Introduction

The coronavirus disease 2019 (COVID-19) outbreak became a worldwide health emergency. In December 2019, the first cases of pneumonia of unknown origin were identified in Wuhan, China. The WHO confirmed a new coronavirus was the cause of pneumonia in Wuhan (1). The complexity and uncertainty of the pandemic threatened human physical health and mental health (2). Plenty of studies reported negative psychological effects such as posttraumatic stress symptoms, fear, and confusion (3). The stress caused by the virus increased during a lock-down period at home (4, 5). Besides, the resurgence of COVID-19 could exacerbate the psychological impacts of the pandemic (6). Due to the social and economic uncertainty associated with the COVID-19 pandemic, certain groups such as young people and women had a risk of suicide during the height of the pandemic (7). Another study reported that people might experience mild to severe depression and anxiety symptoms while the outbreak continued (8). Their positive emotion progressively decreased over time (9).

In response to the unprecedented psychological impact of the current pandemic, Taylor suggested that developing a measure of COVID-19-related stress and anxiety was an urgent need during the pandemic (10). After examining the relevant literature and consulting experts, they developed the COVID Stress Scales (CSS) to measure COVID-19-related stress (10). The CSS contained six domains: (1) the worries about the dangerousness of COVID-19, (2) fears about COVID socio-economic consequences (e.g., fears of disruption in the supply chain), (3) COVID-19-xenophobia (e.g., fears that foreigners are sources of COVID-19), (4) the worries about being infected by COVID-19-related contamination (i.e., objects, surfaces), (5) traumatic stress symptoms (e.g., nightmares relating to COVID-19), and (6) COVID-19-related checking (e.g., checking news media). The initial psychometric evaluation of the CSS demonstrated good reliability, convergent and discriminant validity (10). The CSS total and subscale scores had shown high internal consistency and test-retest reliability, and the internal consistency of the total score had been reported as 0.95 (10). There were controversial about the factor structure. The initial development was constructed by examining relevant literature and consulting experts, which contained six domains. However, the results from the exploratory factor analysis (EFA) indicated a five-factor solution, COVID-related danger scale and contamination scale loaded on the same factor (10). The five-factor solution was also confirmed in other studies (11, 12). Although Taylor proposed a five-factor model of CSS that was based on a large collection of research evidence, plenty of studies applying confirmatory factor analysis (CFA) had shown a good fit for the six-factor model with the studied population (13–16). Besides, we found that the CSS-Arabic version supported the five-factor solution in the Egyptian and Saudi contexts (11). While it also supported a six-factor solution in a Palestinian context after excluding five items (14). Nevertheless, the CSS Spanish translation proved that the six-factor model was better than the five-factor model and the six-factor model fitted the data well (16). Above all the current studies which were proved to be valid and reliable were conducted in other cultural contexts, which hadn't been adapted and validated in the Chinese population during the COVID-19 pandemic. Standards for Educational and Psychological Testing recommended that evidence of validity should be clarified for each intended use of the test score among the targeted population and for specific procedures (17). It was necessary to confirm the factor structure in the Chinese context.

To reduce the rapid spread of COVID-19 across the world, China decided to suspend the entry into China by foreign nationals holding visas or residence permits temporarily on 26 March 2020 (18). Then a portion of foreign nationals was allowed to enter China after 23 September 2020 according to the announcement, but other measures in the Announcement issued on March 26 continued to be implemented (19). Regarding the epidemic prevention and control policies in China, the foreigners in the CSS xenophobia domain were defined as COVID-19-related personnel, including personnel living in high-risk areas or passing through them, the frontline medical staff, epidemic prevention workers, and other relevant personnel.

Briefly, the main goal of our study was to adapt the CSS to Chinese and to determine the psychometric properties of the Chinese population during the COVID-19 epidemic. Firstly, we explored the factor structure and compared the different factor structures of the adaptation of the Chinese version of the CSS. Additionally, internal consistency values were calculated for the reliability of the Chinese adapted version of the CSS. Psychometric studies of the Persian and Arabic versions of the CSS were based on the Classical Theory of Tests (CTT), which emphasized the evaluation of internal consistency and construct validity of an instrument in a general way (20). In our study, we used the CTT to evaluate the internal consistency and construct validity of the CSS Chinese adaptation (21). Average Variance Extracted (AVE), Composite Reliability (CR), and the square root of the AVE were used to assess convergent validity and discriminant validity. Finally, we calculated the correlations between the CSS and other scales including the Coronavirus Anxiety Scale (CAS), the Fear of COVID-19 Scale (FCV-19S), and depression anxiety stress scales (DASS-21) to examine its concurrent validity.

Methods

Participants and procedure

Over 1 week period in August 2021, a cross-sectional online survey was conducted in Chinese mainland through the WeChat public platform following the electronic research methodology guideline (22) to prevent the spread of COVID-19 through contact. All participants using WeChat might see this survey, and answer the questionnaire by scanning the two-dimensional barcodes of the questionnaire address or clicking the relevant link. Electronic informed consent was obtained from each participant before starting the investigation. All participants were provided with anonymity and confidentiality of their data, and they were informed about the nature, purpose, and procedure of the study. Participants could withdraw from the survey at any moment without providing any justification. This web-based questionnaire was completely voluntary and non-commercial. Participants agreed to the online informed consent statement and completed the questionnaires. After completing the scales, every participant would receive a reward, which contained an individual report and reward (1-3 CNY randomly). The research was approved by the ethics committee of the Second Xiangya Hospital of Central South University.

To ensure the data quality, questionnaires were valid if they met the following criteria. The inclusion criteria included: (1) time for each item completion more than 2 seconds; (2) the participants aged 18 and above, and (3) the item responses were not consecutive identical. Finally, 2,116 questionnaires were included in the final analysis.

Measures

The COVID Stress Scales (CSS)

The CSS was a 36-item self-report instrument designed to assess COVID-19-related stress and anxiety symptoms over the past week. Items were scored on a 5-point Likert scale from 0 (not at all) to 4 (extremely). Higher scores indicated a higher level of stress. The transcultural and lingual adaption from English to Chinese was performed using Beaton and colleagues' methods (23). The translation process was composed of several steps based on the Brislin translation model (24).

We used a forward-backward translation. The translation procedures were as follows. Firstly, the CSS was translated from English into Chinese by two native Chinese-speaking researchers with high English proficiency from the research team independently. Subsequently, two psychology experts reviewed the translated version concerning its content accuracy, semantic equivalence, and sentence structure. Modifications were made after the group reached a consensus. Some minor revisions were made during the translation process. Lastly, the Chinese-adapted version of the CSS was back-translated into the English language again. The back-translated and original versions were compared to ensure accuracy. Our research team reviewed and checked the translation. Finally, the adaption of the Chinese version of the CSS was confirmed.

Considering epidemic prevention and control policies in China, the foreigners in the CSS xenophobia domain were changed to COVID-19-related personnel which included personnel living in or passing through high-risk areas, the frontline medical staff, epidemic prevention workers, and other relevant personnel. Six items of the xenophobia domain were adapted. The amended items were (a) “ I am worried that COVID-19-related personnel is spreading the virus”, (b) “If I met COVID-19-related personnel, I'd be worried that they might have the virus”, (c) “I am worried about contacting COVID-19-related personnel because they might carry the virus”, (d) “If the COVID-19-related personnel doesn't take care of their hygiene, I'm worried they'll spread the virus”, (e) “If I went to a restaurant where COVID-19-related personnel has been, I'd be worried about catching the virus”, and (f) “If I was in an elevator with COVID-19-related personnel, I'd be worried that they're infected with the virus“.

The Coronavirus Anxiety Scale (CAS)

The CAS was a 5-item self-report instrument designed to assess the levels of COVID-19 anxiety (25). The CAS had good reliability and validity (2, 26, 27). It was a brief mental health screener to measure current anxiety over the last 2 weeks. Items were rated on a 5-point Likert scale ranging from 0 (not at all) to 4 (nearly every day over the last 2 weeks). In our study, Cronbach's alpha internal consistency coefficient was 0.95.

The Fear of COVID-19 Scale (FCV-19S)

The FCV-19S was a 7-item instrument to measure an individual's fear of COVID-19 (28). Items were scored on a 5-point Likert scale from 1 (strongly disagree) to 4 (strongly agree). The higher score indicated a higher level of fear. The Chinese version of the FCV-19S had good internal consistency reliability (Cronbach's alpha = 0.92) and validity (29). In our study, Cronbach's alpha internal consistency coefficient was 0.95.

Depression Anxiety Stress Scales (DASS-21)

DASS-21 was a 21-item instrument to measure the experiences of depression, anxiety, and stress over the past week (30). It consisted of three subscales: depression, anxiety, and stress. Each subscale included seven items. Items were rated on a 4-point Likert scale ranging from 0 (not at all) to 3 (strongly agree). The scale had been validated in China (31). In our study, Cronbach's alpha internal consistency coefficient was 0.98.

Statistical analysis

Data were analyzed with the SPSS 26.0 and Amos 26.0. We assessed internal consistency reliability using Cronbach's alpha(α), McDonald's omega(ω), and the Spearman-Brown formula(accepted value ≥0.70) using the CTT (21). Values equal to or greater than α = 0.70 ω = 0.70 were considered satisfactory (32, 33). CFA was to test the hypothesized factor structures obtained from the Canadian and American samples (10) and the Arabic sample (14) separately. Model fit was assessed using comparative fit index (CFI), Tucker-Lewis index (TLI), normed fit index (NFI), incremental fit index (IFI), root-mean-square-error of approximation (RMSEA), and standardized root-mean residual (SRMR). The CFI, NFI, IFI, and TLI values ≥0.90 suggested the good fit model (34). Additionally, the RMSEA value between 0.06 and 0.08 and the SRMR value ≤ 0.08 suggested a better-fitted model (35). The smallest Akaike information criterion (AIC), Bayesian information criterion (BIC), and expected cross-validation Index (ECVI) indicated the model with the best fit (36).

Besides, an instrument's convergent validity could be determined by examining two variables, the AVE of the latent variable and the measure's CR. Convergent validity could be considered adequate when the AVE value was ≥0.50, the CR value was ≥0.70 (37, 38). The discriminant validity was measured by the square root of the AVE. The square root of the AVE in each factor was better than the correlation coefficient value of the factor with other factors indicating it had good discriminant validity (39).

Finally, criterion validity was made up of two subcategories: predictive and concurrent. Concurrent validity was demonstrated when two assessments agreed or a new measure was compared with one already considered valid (40). Previous studies had revealed that pandemic-related anxiety was distinct from anxiety-related traits (41, 42), but few studies measured pandemic-related anxiety as concurrent validity. Therefore, concurrent validity was a necessary content in criterion validity. Thus, concurrent validity was measured by calculating the Pearson product-moment correlation coefficient between the CSS and other correlated scales (CAS, FCV-19S, DASS-21).

Results

Demographic variable

We summarized the participants' characteristics in Table 1. A total of 2,116 participants were included in the study. Respondents aged from 18 to 68 (M = 31.21, SD = 9.51) and the females accounted for 59.0% of the total sample. Most of the participants got vaccinated (67.6%). Among the sample, the monthly income of the majority of subjects was 5,000–9,999 CNY (42.9%).

TABLE 1

Table 1. Sample characteristics (n = 2,116).

Construct validity

We tested the models mentioned in the literature to verify the structural validity of the CSS Chinese adapted version. One was the six-factor model, which included danger (D), socio-economic consequences (SE), xenophobia (X), contamination (C), traumatic stress (T), and compulsive checking (CH). The other was the five-factor model, which combined danger (D) and contamination (C) into one factor. When modeling ordered categorical data, the research seemed to indicate that if there were a large number of ordered categories the data could be treated as continuous. Finney and DiStefano recommended treating the data as continuous and employing maximum likelihood estimation if the variables had five categories or more, the data were approximately normally distributed (43). Thus, we used the maximum likelihood to estimate each model in our study. The CFA was conducted to determine the goodness-of-fit of the six-factor model with 36 items (Figure 1).

FIGURE 1

Figure 1. Confirmatory factor analysis of the COVID Stress Scales.

As was shown in Table 2, all resulting incremental indices in the six-factor model were greater than those in the five-factor model and exceeded the 0.90 level, indicating a better model fit in the six-factor model. The RMSEA value was 0.067 (90 % confidence interval:0.065–0.068) in the six-factor model; in the five-factor model, the RMSEA value was 0.085 (90 % confidence interval:0.083–0.086). The SRMR value in the six-factor model was 0.038, while in the five-factor solution was 0.047. The AIC, BIC and ECVI in the six-factor model [AIC = 6,213.648, BIC = 6,705.832, and ECVI = 2.938 (90 % confidence interval: 2.821–3.057)] smaller than those [AIC = 9,610.543, BIC = 10,074.440, and ECVI = 4.544 (90 % confidence interval:4.397–4.694)] in five-factor model indicated the six-factor model with the best fit.

TABLE 2

Table 2. Fit indices of various structural models for confirmatory factor analysis (n = 2,116).

Internal consistency reliability

Cronbach's alpha (α), McDonald's omega (ω), and Spearman-Brown coefficient were calculated for the internal consistency of the CSS Chinese adaptation. Cronbach's alpha was 0.97, McDonald's omega was 0.97, and split-half reliability through the Spearman-Brown formula was 0.90, indicating the high reliability of the CSS Chinese adapted version. The Cronbach's coefficients of subscale scores and McDonald's Omega subscale value in terms of the six-factor model were summarized in Table 3. The dimensions of Danger (α = 0.94; ω = 0.94), Socioeconomic consequences (α = 0.96; ω = 0.96), Xenophobia (α = 0.95; ω = 0.95), Contamination (α = 0.96; ω = 0.96), Traumatic stress (α = 0.95; ω = 0.96), and Compulsive checking (α = 0 .92; ω = 0.92) had adequate reliability indices.

TABLE 3

Table 3. Normative data and reliability indices for the COVID Stress Scales and its subscales (n = 2,116).

Convergent validity and discriminant validity

As shown in Table 4, the factor loading of each item on all corresponding subscales was better than 0.70 in the six-factor model. AVE was better than 0.50, and CR was better than 0.80. These results supported a good convergent validity of the Chinese adapted version of the CSS.

TABLE 4

Table 4. Convergent validity (n = 2,116).

As shown in Table 5, the square root of the AVE in each factor (D, SE, X, C, T, CH) was better than the correlation coefficient value of the factor with the other factor. According to Zait and Bertea, the results showed that it had a good discriminant validity (39).

TABLE 5

Table 5. Discriminant validity (n = 2116).

Concurrent validity

As shown in Table 6, there were strong positive correlations between the FCV-19S, the CAS, the DASS-21, and the CSS. The CSS total score was positively correlated with the FCV-19S (r = 0.73), the CAS (r = 0.66), the DASS-21 (r = 0.63), the DASS_D (r = 0.58), the DASS_A (r = 0.63) and the DASS_S (r = 0.62). The results provided an evidence of criterion validity.

TABLE 6

Table 6. Pearson's correlations between the FCV-19S, the CAS, the DASS-21, and the CSS.

Discussion

During the COVID-19 pandemic, the scales which assessed general mental health might be underestimated or overestimated because they didn't assess specific symptoms associated with COVID-19 (44). In response to the current pandemic, Taylor suggested that developing a measure of COVID-19-related stress and anxiety was an urgent need during the pandemic (10). The current study aimed to examine the psychometric properties of the adaption of the Chinese version of the CSS. The study showed that the Chinese adapted version of the CSS had good internal consistency, convergent validity, discriminant validity, and concurrent validity. Besides, we tested the factor structure and found the Chinese adapted version fit the six-factor model. Overall, the results indicated that the Chinese adapted version of the CSS showed good psychometric properties.

In our study, the six-factor structure was supported by CFA in the Chinese adapted versions of the CSS. The findings echoed the previous researchers (14, 16) and the original authors' six scales construction (10) though inconsistent with the five-factor model reported by the original research (10) and other researchers (12, 45). The six-factor model presented adequate adjustment indices in our sample of participants (RMSEA = 0.067, SRMR = 0.038; CFI = 0.935; and TLI = 0.929). In contrast, five-factor model had worse adjustment indices (RMSEA = 0.085; SRMR = 0.047; CFI = 0.895; and TLI = 0.886). Besides CFI, TLI, RMSEA, and SRMR, we also compared other resulting incremental indices such as AIC, BIC, and ECVI in the study. The six-factor model (AIC = 6,213.648, BIC = 6,705.832; ECVI = 2.938) were smaller than those in the five-factor model (AIC = 9,610.543; BIC = 10,074.440; ECVI = 4.544), indicating the better model fit in the six-factor model. In our study, factor loadings ranged from 0.7–0.93, which was a higher range than reported. It was worth noting that factor loadings were even high, above what was recommended (46). The difference in the factor structure across different surveys might result from the difference in the population. The original validation study, utilizing parallel analyses, demonstrated that the five-factor solution was sufficiently stable for the Canadian and US community-based samples (10). This solution was replicated in Iran among persons with anxiety and obsessive-compulsive disorders (12). While our study was conducted in the Chinese population, similar to the results in the Peruvian context (16). The most confusing was that the CSS Arabic version supported the five-factor solution in the Egyptian and Saudi contexts (11). Meanwhile, it also supported the six-factor solution in a Palestinian context after excluding five items (14). We deduced that the danger and contamination factors were the separated factors in different populations. Danger and contamination subscales were merged and observed as a single construct in the five-factor model. Differences in the factor structure of the CSS might reflect a specific context or purpose of the CSS. Our findings suggested that in the Chinese population, a distinction between perceptions of the pandemic as dangerous and disrupting everyday functioning on the one hand, and getting exposure to virus in the immediate environment on the other hand, should be made. Besides, it was related to what Taylor suggested that people use various psychological factors when facing the threat of a pandemic, presenting adaptive behaviors, emotions, and defensive reactions linked to their psychological vulnerability (47).

The Chinese adapted version of the CSS and its subscales showed good internal consistency. The internal consistency was also been reported using the McDonald's omega coefficient, which was a more appropriate estimation measure that was based on factor loadings and was not influenced by sample size or the number of items on the scale (48). The internal consistency of the total (α = 0.97, ω = 0.97, Spearman-Brown coefficient = 0.90) and subscale scores (α = 0.92–0.96; ω = 0.92–0.96) were similar to those reported in previous psychometric studies of the CSS (11).

The convergent validity and discriminant validity of the Chinese adaption of the CSS were established in the study. The significant correlations between the subscales in the present study were close to the association reported by the original authors (10). In Taylor and his colleagues' (10) study, the convergent validity was evaluated through the correlations of the scales of the CSS with the pre-COVID trait measures such as anxiety and obsessive-compulsive (OC) symptoms. The discriminant validity was evaluated by the correlations of the general and pre-COVID traits. In our study, the convergent validity was assessed through AVE and CR, the factor loading of each item on all corresponding subscales was better than 0.70. AVE was better than 0.50, and CR was better than 0.80. The square root of the AVE in each factor was to access the discriminant validity. The results demonstrated that the square root of the AVE in each factor was greater than the correlation coefficient value of the factor with other factors. The results supported good convergent validity of the Chinese adapted version of the CSS through diverse statistical methods.

According to previous studies (49, 50), mental health was associated with stress measures. In our study, all correlations were significant between the FCV-19S, the CAS, the DASS-21, and the CSS. The findings supported the concurrent validity of the CSS. The correlation coefficients were significant as expected (r = 0.58–0.73). The correlations between the FCV-19S, the CAS, and the CSS were in line with other studies examining COVID-19-related scales (12, 41). Similar results might indicate that COVID-19-related health anxiety was distinct from anxiety traits (41, 42). The correlation with DASS-21 was in line with the previous study (14). The results suggested that more stress perceived by individuals under the COVID-19 pandemic might be associated with higher depression and anxiety symptoms (51).

Our study also had several limitations. Firstly, we conducted the online survey with self-report measurements due to the COVID-19 epidemic. Secondly, this study did not investigate other aspects of reliability and validity such as test-retest reliability and predictive validity. Evidence of test-retest reliability would be a benefit for longitudinal research in the future. Thirdly, future researchers could develop more objective stress measures, such as physiological stress indicators or known-groups validity (10) to investigate the criterion validity of the CSS.

Conclusion

Regardless of all the above limitations, this study provided data supporting the psychometric properties of the adaption of the Chinese version of the CSS. The results supported the six-factor structure proposed in prior research. In conclusion, the Chinese adapted version of the CSS was a reliable and valid tool for assessing the stress of COVID-19 in China. The spread of coronavirus in China could amplify the risk of maladaptive and stressful symptoms in an already compromised living environment. Understanding stress-related and COVID-19-associated outcomes might be an urgent priority in China.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

All procedures performed in this study involving human participants were following the ethical standards of the Committee of the Second Xiangya Hospital of Central South University, the American Psychological Association (APA, 2010), and the 2013 Helsinki Declaration. The patients/participants provided their written informed consent to participate in this study.

Author contributions

DW conceived and designed the study. LX and QL performed the analysis. LX, QL, HY, and DW prepared and modified the manuscript. All authors were involved in the study conduction and contributed substantially to its revision and approved the submitted version.

Funding

This study was supported by the 225 High-level Health Talents Training Project of Hunan Province of China and the Natural Science Foundation of Hunan Province (Grant No: 2016JJ4101).

Acknowledgments

We are grateful to the people who participated in the study, and the professionals who helped us contact the participants.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2022.962304/full#supplementary-material

References

1. Fitzgerald DA, Wong GWK. COVID-19: a tale of two pandemics across the Asia Pacific region. Paediatr Respir Rev. (2020) 35:75–80. doi: 10.1016/j.prrv.2020.06.018

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Choi E, Lee J, Lee SA. Validation of the Korean version of the obsession with COVID-19 scale and the coronavirus anxiety scale. Death Stud. (2020) 46:608–14. doi: 10.1080/07481187.2020.1833383

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: RAPID review of the evidence. Lancet (London, England). (2020) 395:912–20. doi: 10.1016/S0140-6736(20)30460-8

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Ezpeleta L, Navarro JB, de la Osa N, Trepat E, Penelo E. Life conditions during COVID-19 lockdown and mental health in spanish adolescents. Int J Environ Res Public Health. (2020) 17:E7327. doi: 10.3390/ijerph17197327

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Rolland B, Haesebaert F, Zante E, Benyamina A, Haesebaert J, Franck N. Global changes and factors of increase in caloric/salty food intake, screen use, and substance use during the early COVID-19 containment phase in the general population in france: survey study. JMIR Public Health Surveill. (2020) 6:e19630. doi: 10.2196/19630

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Broche-Pérez Y, Fernández-Fleites Z, Fernández-Castillo E, Jiménez-Puig E, Vizcaíno-Escobar AE, Ferrer-Lozano DM, et al. Anxiety, health self-perception, and worry about the resurgence of COVID-19 predict fear reactions among genders in the cuban population. Front Global Women's Health. (2021) 2:634088. doi: 10.3389/fgwh.2021.634088

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Kaushik M, Guha D. Resurgence of second wave of pandemic Covid-19 and its impact on physical and mental well-being of employees & coping mechanism. Eur J Manag Bus Econ. (2021) 13:108–19. doi: 10.7176/EJBM/13-10-12

CrossRef Full Text | Google Scholar

8. Marzo RR, Vinay V, Bahari R, Chauhan S, Ming DAF, Nelson Fernandez SFA, et al. Depression and anxiety in Malaysian population during third wave of the COVID-19 pandemic. Clin Epidemiology Glob Health. (2021) 12:100868. doi: 10.1016/j.cegh.2021.100868

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Fernández-Abascal EG, Martín-Díaz MD. Longitudinal study on affect, psychological well-being, depression, mental and physical health, prior to and during the COVID-19 pandemic in Spain. Pers Individ Dif. (2021) 172:110591. doi: 10.1016/j.paid.2020.110591

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Taylor S, Landry CA, Paluszek MM, Fergus TA, McKay D, Asmundson GJ. Development and initial validation of the COVID Stress Scales. J Anxiety Disord. (2020) 72:102232. doi: 10.1016/j.janxdis.2020.102232

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Abbady AS, El-Gilany A-H, El-Dabee FA, Elsadek AM, ElWasify M, Elwasify M. Psychometric characteristics of the of COVID Stress Scales-Arabic version (CSS-Arabic) in Egyptian and Saudi university students. Middle East Curr Psychiatry. (2021) 28:14. doi: 10.1186/s43045-021-00095-8

CrossRef Full Text | Google Scholar

12. Khosravani V, Asmundson GJG, Taylor S, Sharifi Bastan F, Samimi Ardestani SM. The Persian COVID stress scales (Persian-CSS) and COVID-19-related stress reactions in patients with obsessive-compulsive and anxiety disorders. J Obsessive Compuls Relat Disord. (2021) 28:100615. doi: 10.1016/j.jocrd.2020.100615

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Demirgöz Bal M, Dişsiz M, Bayri Bingöl F. Validity and reliability of the Turkish version of the COVID stress scale. J Korean Acad Nurs. (2021) 51:525–36. doi: 10.4040/jkan.21106

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Mahamid FA, Veronese G, Bdier D, Pancake R. Psychometric properties of the COVID stress scales (CSS) within Arabic language in a Palestinian context. Curr Psychol (New Brunswick, NJ). (2021) 1–10. doi: 10.1007/s12144-021-01794-5

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Milic M, Dotlic J, Rachor GS, Asmundson GJG, Joksimovic B, Stevanovic J, et al. Validity and reliability of the Serbian COVID Stress Scales. PLoS ONE. (2021) 16:e0259062. doi: 10.1371/journal.pone.0259062

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Noe-Grijalva M, Polo-Ambrocio A, Gómez-Bedia K, Caycho-Rodríguez T. Spanish Translation and Validation of the COVID Stress Scales in Peru. Front Psychol. (2022) 13:840302. doi: 10.3389/fpsyg.2022.840302

PubMed Abstract | CrossRef Full Text | Google Scholar

17. American Educational Research Association American Psychological Association National National Council on Measurement in Education Joint Joint Committee on Standards for Educational and Psychological Testing. Standards For Educational and Psychological Testing. Washington, DC: American Educational Research Association (2014).

Google Scholar

18. Announcement Announcement by the Ministry of Foreign Affairs and the National Immigration Administration on the Temporary Suspension of Entry by Foreign Nationals Holding Valid Chinese Visas or Residence Permits. National Immigration Administration of the People's Republic of China. (2020). Available online at: https://www.nia.gov.cn/n741440/n741542/c1267259/content.html (accessed January 12, 2022).

Google Scholar

19. Ministry of Foreign Affairs of The People's Republic of China National Immigration Administration of The People's Republic of China Announcement on Entry by Foreign Nationals Holding Valid Chinese Residence Permits of Three Categories. National Immigration Administration of the People's Republic of China. (2020). Available online at: https://www.nia.gov.cn/n741440/n741542/c1344669/content.html(accessed January 12, 2022).

Google Scholar

20. Hunsley J, Mash EJ. A Guide to Assessments that Work. New York, NY: Oxford University Press (2008). doi: 10.1093/med:psych/9780195310641.001.0001

CrossRef Full Text | Google Scholar

21. Steyer R. Classical (Psychometric) Test Theory. In: Smelser NJ, Baltes PB, editors. International Encyclopedia of the Social & Behavioral Sciences. Pergamon. (2001) p. 1955–1962. doi: 10.1016/B0-08-043076-7/00721-X

CrossRef Full Text | Google Scholar

22. Andrews D, Nonnecke B, Preece J. Electronic survey methodology: a case study in reaching hard-to-involve internet users. Int J Hum-Comput Int. (2003) 16:185–210. doi: 10.1207/S15327590IJHC1602_04

CrossRef Full Text | Google Scholar

23. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. (2000) 25:3186–91. doi: 10.1097/00007632-200012150-00014

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Brislin RW. Back-translation for cross-cultural research. J Cross Cult Psychol. (1970) 1:185–216. doi: 10.1177/135910457000100301

CrossRef Full Text | Google Scholar

25. Lee SA. Coronavirus anxiety scale: a brief mental health screener for COVID-19 related anxiety. Death Studies. (2020) 393–401. doi: 10.1080/07481187.2020.1748481

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Ahmed O, Faisal RA, Sharker T, Lee SA, Jobe MC. Adaptation of the bangla version of the COVID-19 anxiety scale. Int J Ment Health Addict. (2020) 20:284–95. doi: 10.1007/s11469-020-00357-2

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Magano J, Vidal DG, Sousa HFPE, Dinis MAP, Leite Â. Validation and psychometric properties of the portuguese version of the coronavirus anxiety scale (CAS) and fear of COVID-19 Scale (FCV-19S) and associations with travel, tourism and hospitality. Int J Environ Res Public Health. (2021) 18:E427. doi: 10.3390/ijerph18020427

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Ahorsu DK, Lin C-Y, Imani V, Saffari M, Griffiths MD, Pakpour AH. The fear of COVID-19 scale: development and initial validation. Int J Ment Health Addict. (2020) 20:1537–45. doi: 10.1007/s11469-020-00270-8

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Chi X, Chen S, Chen Y, Chen D, Yu Q, Guo T, et al. Psychometric evaluation of the fear of COVID-19 scale among Chinese population. Int J Ment Health Addict. (2022) 20:1273–88. doi: 10.1007/s11469-020-00441-7

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Henry JD, Crawford JR. The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. Br J Clini Psychol. (2005) 44:227–39. doi: 10.1348/014466505X29657

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Wang K, Shi H-S, Geng F-L, Zou L-Q, Tan S-P, Wang Y, et al. Cross-cultural validation of the Depression Anxiety Stress Scale-21 in China. Psychol Assess. (2016) 28:e88–e100. doi: 10.1037/pas0000207

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Nunnally JD. Psychometric Theory (2nd ed). New York, NY: McGraw-Hill. (1978).

Google Scholar

33. Trizano-Hermosilla I, Alvarado JM. Best alternatives to Cronbach's alpha reliability in realistic conditions: congeneric and asymmetrical measurements. Front Psychol. (2016) 7. doi: 10.3389/fpsyg.2016.00769

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model. (1999) 6:1–55. doi: 10.1080/10705519909540118

CrossRef Full Text | Google Scholar

35. Schreiber JB, Nora A, Stage FK, Barlow EA, King J. Reporting structural equation modeling and confirmatory factor analysis results: a review. J Educ Res. (2006) 99:323–38. doi: 10.3200/JOER.99.6.323-338

CrossRef Full Text | Google Scholar

36. Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods Psychol Res. (2003) 8:23–74.

Google Scholar

37. Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. J Market Res. (1981) 18:39–50. doi: 10.1177/002224378101800313

CrossRef Full Text | Google Scholar

38. Wuang YP, Su CY, Huang MH. Psychometric comparisons of three measures for assessing motor functions in preschoolers with intellectual disabilities. JIDR. (2012) 56:567–78. doi: 10.1111/j.1365-2788.2011.01491.x

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Zait A, Bertea P. Methods for testing discriminant validity. Manage Mark J. (2011) IX. 217–224.

Google Scholar

40. Fink A. Survey research methods. In: Peterson P, Baker E, McGaw B, editors. International Encyclopedia of Education (Third Edition). Elsevier. (2010) p. 152–160. doi: 10.1016/B978-0-08-044894-7.00296-7

CrossRef Full Text | Google Scholar

41. Ardestani Samimi SM, Khosravani V, Sharifi Bastan F, Baloglu M. The Persian version of the COVID-19 Phobia Scale (Persian-C19P-S) and the differences in COVID-19-related phobic reactions in patients with anxiety disorders. Int J Ment Health Addict. (2021) 1–17. doi: 10.1007/s11469-021-00523-0

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Kubb C, Foran HM. Measuring COVID-19 related anxiety in parents: psychometric comparison of four different inventories. JMIR Mental Health. (2020) 7:e24507. doi: 10.2196/24507

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Finney S, DiStefano C. Non-normal and categorical data in structural equation modeling. In: Hancock GR, Mueller RO, editors. Structural Equation Modeling: A Second Course. IAP Information Age Publishing (2013). p. 439–92.

PubMed Abstract | Google Scholar

44. Ransing R, Ramalho R, Orsolini L, Adiukwu F, Gonzalez-Diaz JM, Larnaout A, et al. Can COVID-19 related mental health issues be measured? Brain Behav Immun. (2020) 88:32–4. doi: 10.1016/j.bbi.2020.05.049

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Delgado-Gallegos JL, Montemayor-Garza R, de Padilla-Rivas GR, Franco-Villareal H, Islas JF. Prevalence of stress in healthcare professionals during the COVID-19 Pandemic in Northeast Mexico: a remote, fast survey evaluation, using an adapted COVID-19 stress scales. Int J Environ. Res. Public Health. (2020) 17: E7624. doi: 10.3390/ijerph17207624

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Dominguez Lara S. Propuesta de puntos de corte para cargas factoriales: Una perspectiva de fiabilidad de constructo. Enfermería Clínica. (2018) 28. doi: 10.1016/j.enfcli.2018.06.002

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Taylor S. (2019). Taylor S. (2019). The psychology of pandemics: Preparing for the next global outbreak of infectious disease. Newcastle upon Tyne, UK: Cambridge Scholars Publishing.p. 98–103. doi: 10.20879/acr.2020.17.2.98

CrossRef Full Text | Google Scholar

48. Ventura Leon J, Caycho-Rodríguez T. El coeficiente Omega: un metodo alternativo para la estimacion de la confiabilidad. Revista Latinoamericana de Ciencias Sociales, Niñez y Juventud. (2017) 15:625–7. doi: 10.11600/1692715x.1522525102016

CrossRef Full Text | Google Scholar

49. Amirkhan JH, Urizar GG, Clark S. Criterion validation of a stress measure: the Stress Overload Scale. Psychol Assess. (2015) 27:985–96. doi: 10.1037/pas0000081

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Amirkhan JH. A brief stress diagnostic tool: the short stress overload scale. Assessment. (2018) 25:1001–13. doi: 10.1177/1073191116673173

PubMed Abstract | CrossRef Full Text | Google Scholar

51. She Z, Li D, Zhang W, Zhou N, Xi J, Ju K. Three versions of the perceived stress scale: psychometric evaluation in a nationally representative sample of Chinese adults during the COVID-19 pandemic. Int J Environ Res Public Health. (2021) 18:8312. doi: 10.3390/ijerph18168312

PubMed Abstract | CrossRef Full Text | Google Scholar