Validity and Reliability of the Stress and Anxiety to Viral Epidemics-6 (SAVE-6) Scale to Measure Viral Anxiety of Healthcare Workers in Spain During the COVID-19 Pandemic

This study examined the validity and applicability of the Spanish version of the Stress and Anxiety to Viral Epidemics-6 items (SAVE-6) scale, which can be usually applied to the general population, to healthcare workers to briefly measure their anxiety responses to the viral epidemic. A total of 135 healthcare workers participated in this online survey from January to July 2021. Participants' sociodemographic characteristics were gathered, and their psychiatric symptoms were rated using SAVE-6, Goldberg Anxiety and Depression Scale (GDAS), and the Pittsburgh Sleep Quality Index (PSQI). The confirmatory factor analysis was conducted to examine the validity of the scales. The single-structure model of the SAVE-6 scale was adopted based on the results of the parallel analysis. We decided on the SAVE-6 scale, as it proved to be a good fit to measure healthcare workers' anxiety response to the viral epidemic. SAVE-6 showed good internal consistency (Cronbach's alpha = 0.827 and McDonald's omega = 0.834) and good convergent validity with Goldberg anxiety (r = 0.434, p < 0.001) and depression (r = 0.193, p = 0.043) scores, and PSQI score (r = 0.262, p = 0.002). The Spanish version of SAVE-6 is a reliable and valid rating scale to assess the anxiety response of healthcare workers specifically to the viral epidemic as a brief measure during the COVID-19 pandemic.

Introduction

The COVID-19 pandemic has caused a challenging situation worldwide with a major health impact on vulnerable populations and populations with high risk for COVID-19 infection, such as healthcare workers. In Spain, the first confirmed case was reported in the Canary Islands on 31 January 2020, about two months after the first case was reported in China (1). The incidence and mortality of COVID-19 have been high in Spain despite a recognized, well-established health system (2). Socio-economic factors such as an aging population, reduced healthcare investment, fewer beds in intensive care compared to other countries of Europe, and even cultural factors may have played a role in the development of the COVID-19 pandemic in Spain (3).

Healthcare workers in this pandemic suffer from psychological problems such as severe work-related stress, burnout, depression, anxiety, or post-traumatic stress (4–6). The viral epidemic disaster is different from other disasters in that the victim of a viral disaster directly transfers the negative impact of the disaster to the front-line healthcare workers in the hospital. Healthcare workers are concerned about being infected while taking care of infected or high risk patients and about spreading the viral infection to their friends or family members (7, 8). The high percentage of COVID-19 infections among Spanish health professionals is associated with increased mental health symptoms investigated during the first wave of the COVID-19 pandemic. Healthcare staff has a 12 times higher risk of COVID-19 infection than the general population (9). Spanish healthcare workers reported that the frequency of anxiety, depression, acute stress, posttraumatic stress disorder, and insomnia varies between 30 and almost 60% (9, 10). The rate of acute mental health symptoms is a cause of concern due to the potential short and long-term work, personal, and social impact.

Therefore, specifically assessing healthcare workers' anxiety responses to the viral epidemic to manage their mental health should be considered important. For that, Chung et al. (11) developed the Stress and Anxiety to Viral Epidemics-9 items (SAVE-9) scale to measure healthcare workers' work-related stress and anxiety responses during the pandemic. It is a 9-item self-rating scale and validated in Korean (11), Russian (12), Italian (13), Japanese (14), Turkish (15), and German (16). The SAVE-6 scale, derived from the SAVE-9 scale, used to assess the anxiety response of the general population, was validated in Korean (17), Lebanese (18), and American samples (19). It was also examined for applicability to a particular population such as medical students (20), public workers (21), or cancer patients (22).

Originally, the SAVE-9 scale was reported to be divided into two factors: factor I (viral anxiety, item 1, 2, 3, 4, 5, and 8) and factor II (work-related stress, item 6, 7, and 9). The factor structure can be different cross-culturally (23). There might be differences in the pandemic situation, language, or healthcare system environment among various countries. The factor structure of the Japanese, Italian, and Turkish versions of the SAVE-9 scale was similar to that of the original version; however, the Russian and German versions were different (12, 16). Originally we developed the SAVE-9 scale to measure work-related stress and viral anxiety specifically to the viral epidemic and specifically to healthcare workers. To compare the differences in the level of viral anxiety of healthcare workers with that of the general population across culture, groups, or workplace, we need to validate the SAVE-6 scale among the various population including even healthcare workers. In addition, the SAVE-6 can also be applied to healthcare workers as a brief measure for anxiety response in the current pandemic situation; however, there is no previous study on its applicability to healthcare workers.

In this study, we aimed to validate the Spanish version of the SAVE-6 scale to confirm its construct validity to measure the anxiety response of healthcare workers in Spain during the COVID-19 pandemic. The validity of the SAVE-6 scale was also examined to explore the applicability to healthcare workers as a sole brief measure of their anxiety response.

Materials and Methods

Participants and Procedure

This study was conducted via an online survey from January to July 2021, and it is part of an ongoing study conducted to assess the stress and sleep problems of healthcare workers in Spain. Data were collected by an online questionnaire throughout social networks, healthcare groups (Quirónsalud group, Catalan Institute of Oncology) and scientific societies (Catalan and Spanish Sleep Society). Inclusion criteria were healthcare professionals, aged 25–69 years, of both genders, physicians and nurses in charge of patients with COVID-19 infection, working in hospitals, health care centers for COVID-19 patients, primary health care centers, geriatrics, and convalescent centers. Exclusion criteria was age <25 and >69 years, not in charge of patients with COVID-19, and unable to complete the survey. An e-agreement consent was obtained from participants in all cases. The study protocol was approved by the Institutional Review Board of Grupo Hospitalario QuirónSalud-Cataluña (2020/52-MSU-TEK). In the first part of the study, it was estimated that the sample size necessary to demonstrate a statistically significant difference of 25% on stress and sleep quality in the group of healthcare staff in charge of the COVID-19 patients vs. 10% in the group of healthcare staff in charge of patients with other pathologies with an alpha value of 0.05 and a power of 0.90 resulted in N = 216 divided into two groups (108.21 healthcare staff for COVID-19 patient and 108.21 for non-COVID-19 patients). In this manuscript, only results regarding healthcare staff on charge of COVID-19 patients were included. Sample size estimation was done by the rule of 10:1 (24), the ideal ratio of respondents to items. The SAVE-9 scale has nine items, so we needed at least 90 samples. In this study, we aimed to gather 150 samples, and we finally collected 135 samples.

Survey

Sociodemographic Data and COVID-19 Related Questions

Participants' sociodemographic characteristics such as age, gender, weight, height, living with a partner (married or cohabiting), profession (physician or nurse), and place of work during the pandemic were collected. Concerning COVID-19, we asked questions such as, “Do you consider health protection measures against COVID-19 infection adequate?” “Do you have symptoms of COVID infection?” or “Did any of your family members contract COVID-19 infection?”

Symptom Assessment

Stress and Anxiety to Viral Epidemics-6 Items (SAVE-6)

The SAVE-6 scale was specifically developed to investigate anxiety to the viral pandemic (17). This scale was derived from factor I of the SAVE-9 scale (11) and developed to assess anxiety responses of the general population. The items are rated on a 5-point Likert scale: 0 (never), 1 (rarely), 2 (sometimes), 3 (often), and 4 (always). Among the Korean population (17), the SAVE-6 scale was shown to be reliable (Cronbach's alpha = 0.815) and valid in line with other existing rating scales. The appropriate cut-off point was explored to be ≥ 15 (sensitivity = 0.70, specificity = 0.60) in accordance with a mild degree of generalized anxiety (GAD-7 score of 5). We translated the English version of the SAVE-6 into Spanish and back-translated it into English. In this study, we applied the Spanish version of the SAVE-6 Scale on healthcare workers in Spain.

Goldberg Anxiety and Depression Scale (GDAS)

GADS is a rating scale widely used in primary care to screen depression and anxiety (25). It consists of nine items for depression and 9 items for anxiety. Assessment of anxiety was done as follows: if at least three questions among the first four questions were answered “yes,” responses of the other five questions were continuously measured. Positive screening for anxiety was considered if four or more answers were positive. An assessment of depression was as follows: if at least one question among the first four questions were answered “yes,” responses of the other five questions were continuously measured. Positive screening for depression was considered in those cases with at least two positive answers. It was reported to be a rating scale with a sensitivity of 83.1% and a specificity of 81.8%. For the current study, the Spanish version of GADS was used (26).

The Pittsburgh Sleep Quality Index (PSQI)

The PSQI is a rating scale design to investigate sleep quality (27). It consists of 19 items grouped into seven components, weighted equally from 0 to 3. The total score of PSQI is summed from the scores of seven components, and it ranges from 0 to 21. A higher score reflects poor sleep quality, and a global PSQI score > 5 is the cut-off point to discriminate between good sleepers and poor sleepers with 89.6% sensitivity and 86.5% specificity. In this study, we applied the Spanish version of the PSQI scale (28).

Statistical Analysis

We conducted the principal component analysis (PCA) using principal axis factoring, to explore the factor structure of the Spanish version of the SAVE-6 scale. The skewness and kurtosis were checked to determine whether each item was distributed within normal limits based on the values within the range ± 2 (29). The Kaiser-Meyer-Olkin (KMO) value and Bartlett's sphericity test were assessed to check data suitability and sampling adequacy. A screening test and parallel analysis (30–32), based on the Minimum Rank Factor Analysis (MRFA) (33), with a 95 percentile threshold based on the polychoric correlations matrix, was conducted using the FACTOR 10.10.03. program (33) to determine the number of factors to retain for the Spanish version of the SAVE-6 scale. We compared the explained real-data eigenvalues to the 95th percentile of random eigenvalues, and we decided where the real-data eigenvalues exceeded the 95th percentile of the random eigenvalues. We conducted a bootstrap (2,000 samples) maximum-likelihood confirmatory factor analysis (CFA) for the Spanish version of the SAVE-6 scale to examine the construct validity and applicability. Satisfactory model fit was defined by a standardized root-mean-square residual (SRMR) value ≤ 0.05, root-mean-square-error of approximation (RMSEA) value ≤ 0.10, and comparative fit index (CFI) and Tucker Lewis index (TLI) values ≥ 0.90 (34, 35). Psychometric properties of the scale were also assessed through the Item Response Theory (IRT) approach (graded response model [GRM]). Before running the GRM, IRT assumptions—unidimensionality, local dependence, and monotonicity were assessed through the R package mokken version 3.0.6 and mirt version 1.34. GRM was also run through the R package mirt version 1.34. In GRM, item misfit information was checked through the S-χ² value. GRM provides slope/ discriminating parameters and threshold/ difficulty parameters. The reliability and internal consistency of the scales were assessed using Cronbach's alpha and McDonald's omega. To explore the convergent validity, Pearson correlation analysis between SAVE scales and depression or anxiety subscale of Goldberg scale and PSQI scale. The SPSS version 21.0 (SPSS, Inc., Chicago, Illinois), JASP version 0.14.1.0 software (JASP Team, Amsterdam, Netherlands), and RStudio were also used for statistical analysis.

Results

All 135 healthcare workers participated in this study (Table 1). Among participants, 83.0% were female, 67.4% were physicians, and their mean age was 42.2 ± 9.1 years old. To the questions related to COVID-19, 21.5% answered that they had experienced symptoms of COVID-19 infection, and 34.1% answered that their family members got infected with COVID-19. The proportion of participants who were rated as having anxiety (Goldberg anxiety score ≥ 4) or depression (Goldberg depression score ≥ 2) were 59.3% (N = 80) or 82.2% (N = 111). Participants reported medical conditions such as asthma (nine subjects) or hypothyroidism (nine subjects).

TABLE 1

Table 1. Demographic characteristics of participants (N = 135).

Factor Structure and Psychometric Properties of the SAVE-6 Scale

The distribution of all six items of SAVE-6 was within the normal limit based on the skewness and kurtosis within the range of ± 2 (Table 2). The KMO measure of the SAVE-6 was 0.76, and Bartlett's test of sphericity showed a p-value of < 0.001, which means that this data was suitable for factor analyses. Parallel analysis using MRFA extraction advised the single-structure model of the SAVE-6 (real-data eigenvalue = 64.32, 95 percentiles of random eigenvalue = 34.33, Figure 1). In addition, the CFA of the SAVE-6 showed a good model fit (CFI = 1.00, TLI = 1.00, RMSEA = 0.00, SRMR = 0.057). Therefore, we can adopt the Spanish version of the SAVE-6 scale to assess the anxiety response of healthcare workers to the viral epidemic.

TABLE 2

Table 2. Factor structure of the SAVE-6 scale and factor loadings.

FIGURE 1

Figure 1. Factor structure of the SAVE-6 scale among healthcare workers in Spain.

Supplementary Table 2 shows the psychometric properties of the SAVE-6 through the GRM (an IRT model for polytomous items). Before running the GRM, IRT assumptions (unidimensionality, local dependence, and monotonicity) were assessed (Supplementary Table 1). Loevinger's H coefficient (0.50; Table 3) suggest the unidimensionality of the scale. Non-significant p-values (adjusted for false discovery rate [FDR]) suggested the absence of local dependence. Supplementary Table 1 suggests there was no violation of monotonicity. These statistics about assumptions suggest that IRT models are applicable.

TABLE 3

Table 3. Scale-level psychometric properties of the SAVE-6.

Supplementary Table 2 shows items' fit statistics (S-χ² and p-values [adjusted for false discovery rate]) of items. These non-significant p-values suggested that these items belong to the full scale (SAVE-6). It demonstrates the slope/discrimination parameters (α) and threshold/difficulty parameters (b). Slope parameters ranged between 0.752 (item 5) and 3.254 (item 2) (mean = 2.127). The slope of item 5 is moderate, item 1 is high, and the rest are very high. These values suggested that all items provide reasonable discriminating information about stress and anxiety that the SAVE-6 assess. Results regarding threshold/ difficulty coefficients (b) suggest that a higher latent trait or theta is required to endorse items 4 and 5 compared to other items. On the other hand, a lower latent trait or theta is required to endorse item 6. In items 4 and 5, only b₁ coefficients were negative, and the rest were positive. This suggests that the above- average level of the latent trait or theta is required to endorse Likert-type response options—from “sometimes” to “often.” In item 6, only b₄ is positive. This suggests that lower latent trait or theta should endorse Likert-type response options—from “never” to “sometimes.” Scale information curve (Supplementary Figure 1) provides an understanding of the information provided by the SAVE-6. From this curve, this scale provides more information about people between −1.5 and 0.75 θ level. There are double peaks in the curve, which might be due to the polytomous nature of the data.

Reliability of the SAVE-6 Scale and Evidence Based on Relations to Other Variables

The SAVE-6 scale showed good internal consistency (Cronbach's alpha = 0.827, McDonald's omega = 0.834, split-half reliability [odd-even] = 0.895). A Cronbach's alpha if item dropped were 0.777 ~ 0.848. Mean inter-item correlation of this scale (0.444) was in recommended rage (between 0.15 and 0.50). This scale also had good discrimination power (Ferguson delta = 0.99). This scale also good IRT reliability (0.880) and rho coefficient (0.836). The SAVE-6 score was significantly correlated with Goldberg anxiety score (r = 0.434, p < 0.001), depression score (r = 0.193, p = 0.043), and PSQI score (r = 0.262, p = 0.002). However, the correlation between the SAVE-6 and Goldberg depression scale disappear, if we define the significance level as two-tailed p < 0.0167 (as 0.05/3) due to multiple comparison. The SAVE-6 score was significantly higher among participants who were assessed as having anxiety symptoms [Goldberg anxiety score ≥ 4, t₍₁₃₃₎ = 3.368, p < 0.001] and depression [Goldberg depression score ≥ 2, t₍₁₃₃₎ = 2.647, p < 0.001]. Furthermore, the SAVE-6 scale score was higher among younger individuals (20–39 years old) vs. older individuals (over 40 years old) [t₍₁₃₃₎ = 3.131, p = 0.002], nurses vs. physicians [t₍₁₃₃₎ = 2.296, p = 0.023], and those who had symptoms of COVID-19 infection [t₍₁₃₃₎ = 2.621, p = 0.01]. However, the SAVE-6 score was not significantly higher across gender [t₍₁₃₃₎ = 0.851, p = 0.396], living with partners [t₍₁₃₃₎ = 0.605, p = 0.546], or having family members who were infected with COVID-19 [t₍₁₃₃₎ = 0.569, p = 0.570)].

Discussion

In this study, we found that the SAVE-6 scale, derived from the SAVE-9 scale for the general population, can be a reliable and valid rating scale for measuring the viral anxiety of healthcare workers, specifically in response to the viral epidemic. In this sample of Spanish healthcare workers, we observed that the SAVE-6 scale, a rating scale used to assess the anxiety response of the general population (17–19), can also be applied to measure the anxiety response of healthcare workers, specifically to the viral epidemic.

The original SAVE-9 scale was developed to assess work-related stress and anxiety response to the viral epidemic during this COVID-19 pandemic (11). It was clustered into two factors: factor I (item 1, 2, 3, 4, 5, and 8) and factor II (item 6, 7, and 9), and this clustering was parallel to the other validation studies of Italian, Japanese, and Turkish versions of the SAVE-9 scale. However, that clustering was not observed in other languages such as Russian (12) and German (16); factor I (item 2, 3, 4, and 8) and factor II (item 1, 5, 6, 7, and 9). The SAVE-6 was originally derived from the SAVE-9 scale; all items of SAVE-6 were selected to measure the anxiety response of healthcare workers. Previously it was applied to medical students (20) and public workers (21) whose social role is close to healthcare workers by serving patients or clients at higher risk of viral infection. Applicability of the SAVE-9 Scale failed for public workers (22). Rather, Chung et al. (21) observed that the SAVE-6 could be applied to measure the anxiety response of public workers.

The construct validity of the SAVE-6 scale is better in the Spanish healthcare workers sample compared to those from our previous studies on the general population (17, 18), public workers (21), or medical students (20). The reliability of the SAVE-6 scale, based on Cronbach's alpha and McDonald's omega, was comparable to that of previous studies. Unfortunately, we could not compare the cut-off point of the SAVE-6 scale among healthcare workers in Spain with those of previous studies since we did not apply the Generalized Anxiety Disorder-7 items (GAD-7) scale in this study. Previously, 12 ~ 16 points were defined as a cut-off score of SAVE-6 in accordance with 5 points on the GAD-7 scale (mild degree of general anxiety) from the Receiver-Operating Curve analysis results. It is important to maintain the compatibility of rating scale across cultures, groups, or countries. Since we developed the SAVE-9 or SAVE-6 scale as a measuring tool in accordance with mild degree of generalized anxiety (GAD-7 ≥ 5), we decided to define the cut-off score in accordance with GAD-7 scale in the future study.

During the COVID-19 pandemic, healthcare workers may suffer from related anxiety symptoms. However, such symptoms are non-pathological unlike the pathological symptoms of Generalized Anxiety Disorder (36). Nevertheless, it is important to monitor and manage the psychological stress of healthcare workers not only for their own safety but also for that of their patients (37, 38). Therefore, the SAVE-6 scale will be beneficial in measuring and screening healthcare workers' viral anxiety, which is often neglected. We can (1) measure anxiety responses using a brief and short rating scale, and (2) compare the anxiety response of healthcare workers to the viral epidemic with various population groups such as the general population (17, 18), public workers (21), medical students (20), or cancer patients (22). It will be helpful to develop a psychological support system for specific groups after comparing the level of anxiety during a pandemic. However, applying the SAVE-6, rather than the SAVE-9, scale to healthcare workers, might lead to a decrease in the value of “specificity to the HCW.” The items of factor II of the original SAVE-9 scale (Item 6—Do you feel skeptical about your job after going through this experience? Item 7—After this experience, do you think you will avoid treating patients with viral illness? Item 9—Do you think your colleagues would have more work to do due to your absence from a possible quarantine and might blame you?) were highly specific to work-related stress. We now try to apply factor II as a brief measure for work-related stress. We can guess the value of “specificity to healthcare workers” of the SAVE-9 come from the SAVE-3 rather than SAVE-6. The items of the SAVE-6 seem non-specific to healthcare workers but specific to the viral epidemic. Further study will explore the factorial validity of the SAVE-3 scale as a tool for measuring work-related stress of healthcare workers related to the viral epidemic (39). Therefore, if we aimed to measure the anxiety response of healthcare workers specifically to the viral epidemic, applying the SAVE-6 might be enough.

This study had several limitations. First, one of the limitations could be that it was an online survey. Due to the pandemic, we conducted an online survey rather than face-to-face interviews. It can lead to biased data. Second, a higher proportion of depression (82.2%) may influence the results. We should consider the possibility that the depression subcategory of the Goldberg scale can be highly sensitive and less specific for assessing depression. We can expect 50% of possible depression in the original literature using the cut-off point (25). Third, sampling bias should be considered. As previously addressed, the skewness and kurtosis of item 7 are relatively high. Further studies need to be conducted to explore whether the applicability of the Spanish version of the SAVE-9 scale to healthcare workers can be supported by the CFA in other Spanish healthcare workers' samples. Last, we could not assess past history of psychiatric disorder or current psychiatric problems of participants. It might influence the understanding of the results of this study.

In conclusion, the Spanish version of SAVE-6 is a reliable and valid rating scale to assess the anxiety response of healthcare workers specifically to the viral epidemic as a brief measure during this COVID-19 pandemic. It showed that we can apply the brief version of SAVE-9 to HCW with good validity and reliability. Applying the SAVE-6 to HCW may lose the value of “specificity to the HCW,” but we can keep the value of “specificity to the viral epidemic” of the scale. However, we believe that these results might not decrease the value of the SAVE-9 scale for the HCW, since the score might have been influenced by the prolonged COVID-19 pandemic and the healthcare workers' call of duty for the patients.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The study protocol was approved by the Institutional Review Board of Grupo Hospitalario QuirónSalud-Cataluña (2020/52-MSU-TEK). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

MM-C and SC conceived the study. MM-C and JA-T obtained ethics approval, recruited participants and obtained data. MM-C, SC, and OA organized the database and performed statistical analyses. All authors contributed to writing a draft, revision, and final approval of the submitted version to be published.

Funding

MM-C was supported by a research grant of the Spanish Sleep Society (SES).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank all the Spanish healthcare workers who participated in this study.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2021.796225/full#supplementary-material

References

1. Spiteri G, Fielding J, Diercke M, Campese C, Enouf V, Gaymard A, et al. First cases of coronavirus disease 2019 (COVID-19) in the WHO European Region, 24 January to 21 February 2020. Euro Surveill. (2020) 25:2000178. doi: 10.2807/1560-7917.ES.2020.25.9.2000178

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Nunez-Gil IJ, Fernandez-Perez C, Estrada V, Becerra-Munoz VM, El-Battrawy I, Uribarri A, et al. Mortality risk assessment in Spain and Italy, insights of the HOPE COVID-19 registry. Intern Emerg Med. (2021) 16:957–66. doi: 10.1007/s11739-020-02543-5

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Borrell C, Pons-Vigues M, Morrison J. Diez E. Factors and processes influencing health inequalities in urban areas. J Epidemiol Community Health. (2013) 67:389–91. doi: 10.1136/jech-2012-202014

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Qiu Y, Wu Q, Chen R. Guan C. Research on psychological stress and mental health of medical staff in COVID-19 prevention and control. Int J Disaster Risk Reduct. (2021) 65:102524. doi: 10.1016/j.ijdrr.2021.102524

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Dutta A, Sharma A, Torres-Castro R, Pachori H. Mishra S. Mental health outcomes among health-care workers dealing with COVID-19/severe acute respiratory syndrome coronavirus 2 pandemic: a systematic review and meta-analysis. Indian J Psychiatry. (2021) 63:335–47. doi: 10.4103/psychiatry.IndianJPsychiatry_1029_20

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Bahadirli S. Sagaltici E. Post-traumatic stress disorder in healthcare workers of emergency departments during the pandemic: A cross-sectional study. Am J Emerg Med. (2021) 50:251–5. doi: 10.1016/j.ajem.2021.08.027

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Ahn MH, Shin YW, Suh S, Kim JH, Kim HJ, Lee KU, et al. High Work-related stress and anxiety response to COVID-19 among healthcare workers: a cross-sectional online survey study in South Korea. JMIR Public Health Surveill. (2021) 7:e25489. doi: 10.2196/25489

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Lee J, Lee HJ, Hong Y, Shin YW, Chung S. Park J. Risk perception, unhealthy behavior, and anxiety due to viral epidemic among healthcare workers: the relationships with depressive and insomnia symptoms during COVID-19. Front Psychiatry. (2021) 12:615387. doi: 10.3389/fpsyt.2021.615387

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Dosil Santamaria M, Ozamiz-Etxebarria N, Redondo Rodriguez I, Jaureguizar Albondiga-Mayor J, Picaza Gorrochategi M. Psychological impact of COVID-19 on a sample of Spanish health professionals. Rev Psiquiatr Salud Ment (Engl Ed). (2021) 14:106–12. doi: 10.1016/j.rpsmen.2020.05.002

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Luceno-Moreno L, Talavera-Velasco B, Garcia-Albuerne Y. Martin-Garcia J. Symptoms of posttraumatic stress, anxiety, depression, levels of resilience and burnout in Spanish health personnel during the COVID-19 pandemic. Int J Environ Res Public Health. (2020) 17:5514. doi: 10.3390/ijerph17155514

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Chung S, Kim HJ, Ahn MH, Yeo S, Lee J, Kim K, et al. Development of the stress and anxiety to viral epidemics-9 (SAVE-9) scale for assessing work-related stress and anxiety in healthcare workers in response to viral epidemics. J Korean Med Sci. (2021) 36:e319. doi: 10.3346/jkms.2021.36.e319

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Mosolova E, Chung S, Sosin D. Mosolov S. Stress and anxiety among healthcare workers associated with COVID-19 pandemic in Russia. Psychiatr Danub. (2020) 32:549–56. doi: 10.24869/psyd.2020.549

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Tavormina G, Tavormina MGM, Franza F, Aldi G, Amici P, Amorosi M, et al. A New Rating Scale (SAVE-9) to demonstrate the stress and anxiety in the healthcare workers during the COVID-19 viral epidemic. Psychiatr Danub. (2020) 32:5–9.

PubMed Abstract | Google Scholar

14. Okajima I, Chung S, Suh S. Validation of the Japanese version of Stress and Anxiety to Viral Epidemics-9 (SAVE-9) and relationship among stress, insomnia, anxiety, and depression in healthcare workers exposed to coronavirus disease 2019. Sleep Med. (2021) 84:397–402. doi: 10.1016/j.sleep.2021.06.035

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Uzun N, Akca OF, Bilgic A, Chung S. The validity and reliability of the stress and anxiety to viral epidemics-9 items scale in Turkish health care professionals. J Community Psychol. (2021). doi: 10.1002/jcop.22680. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Konig J, Chung S, Ertl V, Doering BK, Comtesse H, Unterhitzenberger J, et al. The German translation of the Stress and Anxiety to Viral Epidemics-9 (SAVE-9) scale: results from healthcare workers during the second wave of COVID-19. Int J Environ Res Public Health. (2021) 18:9377. doi: 10.3390/ijerph18179377

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Chung S, Ahn MH, Lee S, Kang S, Suh S, Shin WY. The Stress and Anxiety to Viral Epidemics-6 Items (SAVE-6) scale: a new instrument for assessing the anxiety response of general population to the viral epidemic during the COVID-19 pandemic. Front Psychol. (2021) 12:669606. doi: 10.3389/fpsyg.2021.669606

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Hong Y, Yoo S, Mreydem HW, Abou Ali BT, Sale NO, Hammoudi SF, et al. Factorial validity of the Arabic version of the Stress and Anxiety to Viral Epidemics-6 Items (SAVE-6) scale among the general population in Lebanon. J Korean Med Sci. (2021) 36:e168. doi: 10.3346/jkms.2021.36.e168

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Lee S, Lee J, Yoo S, Suh S, Chung S, Lee SA. The psychometric properties of the stress and anxiety to viral epidemics-6 items: a test in the US general population. Front Psychiatry. (2021) 12:746244. doi: 10.3389/fpsyt.2021.746244

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Ahn J, Lee J, Hong Y, Park J. Chung S. Stress and Anxiety to Viral Epidemics-6 for Medical Students: Psychometric Properties of the Anxiety Measure for the COVID-19 Pandemic Front Psychiatry. (2021) 12:705805. doi: 10.3389/fpsyt.2021.705805

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Park CHKJ G., Yi K, Lee S, Suh S, Chung S. Application of Stress and Anxiety to Viral Epidemics-6 Items (SAVE-6) to public workers for measuring their anxiety response during the COVID-19 pandemic. Front Psychiatry. (2021) 12:701543. doi: 10.3389/fpsyt.2021.701543

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Ahn MH, Lee J, Suh S, Lee S, Kim HJ, Shin YW, et al. Application of the Stress and Anxiety to Viral Epidemics-6 (SAVE-6) and Coronavirus Anxiety Scale (CAS) to measure anxiety in cancer patient in response to COVID-19. Front Psychol. (2020) 11:604441. doi: 10.3389/fpsyg.2020.604441

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Gonzalez-Calvo J, Gonzalez VM, Lorig K. Cultural diversity issues in the development of valid and reliable measures of health status. Arthritis Care Res. (1997) 10:448–56. doi: 10.1002/art.1790100613

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Nunnally JC. Psychometric Theory. New York, NY: McGraw-Hill (1978).

Google Scholar

25. Goldberg D, Bridges K, Duncan-Jones P, Grayson D. Detecting anxiety and depression in general medical settings. BMJ. (1988) 297:897–9. doi: 10.1136/bmj.297.6653.897

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Montón C, Pérez-Echevarría MJ Campos RGB. Escalas de ansiedad y depresión de Goldberg: una guía de entrevista eficaz para la detección del malestar psíquico. Aten Primaria. (1993) 12:345–9.

Google Scholar

27. Buysse DJ, Reynolds CF 3rd, Monk TH, Berman SR, Kupfer DJ. The pittsburgh sleep quality index: a new instrument for psychiatric practice and research. Psychiatry Res. (1989) 28:193–213. doi: 10.1016/0165-1781(89)90047-4

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Hita-Contreras F, Martinez-Lopez E, Latorre-Roman PA, Garrido F, Santos MA, Martinez-Amat A. Reliability and validity of the Spanish version of the Pittsburgh Sleep Quality Index (PSQI) in patients with fibromyalgia. Rheumatol Int. (2014) 34:929–36. doi: 10.1007/s00296-014-2960-z

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Gravetter F, Wallnau L. Essentials of Statistics for the Behavioral Sciences, 8th Edn. Belmont, CA: Wadsworth (2014).

Google Scholar

30. Horn J. A rationale and test for the number of factors in factor analysis. Psychometricka. (1965) 30:179–85. doi: 10.1007/BF02289447

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Glorfeld LW. An improvement on Horn's parallel analysis methodology for selecting the correct number of factors to retain. Educ Psychol Meas. (1995) 55:377–93. doi: 10.1177/0013164495055003002

CrossRef Full Text | Google Scholar

32. Timmerman ME, Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychol Methods. (2011) 16:209–20. doi: 10.1037/a0023353

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Lorenzo-Seva U, Ferrando PJ. FACTOR: a computer program to fit the exploratory factor analysis model. Behav Res Methods. (2006) 38:88–91. doi: 10.3758/BF03192753

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Brown TA. Confirmatory Factor Analysis for Applied Research. New York, NY: Guilford Press (2006).

Google Scholar

35. Byrne BM. Structural Equation Modeling With AMOS: Basic Concepts, Applications, and Programming. Mahwah, NJ: Lawrence Erlbaum Associates (2001).

Google Scholar

36. Tyrer P. COVID-19 health anxiety. World Psychiatry. (2020) 19:307–8. doi: 10.1002/wps.20798

PubMed Abstract | CrossRef Full Text | Google Scholar

37. McDaid D. Viewpoint: investing in strategies to support mental health recovery from the COVID-19 pandemic. Eur Psychiatry. (2021) 64:e32. doi: 10.1192/j.eurpsy.2021.28

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Stewart DE, Appelbaum PS. COVID-19 and psychiatrists' responsibilities: a WPA position paper. World Psychiatry. (2020) 19:406–7. doi: 10.1002/wps.20803

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Son HS, Ahn MH, Kim K, Cho IK, Lee J, Suh S, et al. Utility of the Stress and Anxiety to Viral Epidemic-3 Items as a Tool for Assessing Work-Related Stress, and Relationship With Insomnia and Burnout of Healthcare Workers During the COVID-19 Pandemic. Sleep Med Res. (2021) 12(2): 161-8. doi: 10.17241/smr.2021.01025

CrossRef Full Text | Google Scholar