The Stress and Anxiety to Viral Epidemics-6 Items (SAVE-6) Scale: A New Instrument for Assessing the Anxiety Response of General Population to the Viral Epidemic During the COVID-19 Pandemic

The general population has reported experiencing anxiety due to the COVID-19 pandemic. This study explored the validity and utility of the Stress and Anxiety to Viral Epidemics-6 items (SAVE-6) scale for measuring the anxiety response of the general population to the viral epidemic. About 1,009 respondents participated in an online survey. Of these, 501 (49.7%) participants were rated as having at least a mild degree of anxiety response to the viral epidemic (SAVE-6 score ≥ 15), while 90 (8.9%) and 69 (6.8%) participants were rated as having moderate degree of depression and anxiety, respectively. The SAVE-6 scale showed a good internal consistency (Cronbach’s alpha = 0.815). Parallel analysis suggested a one-factor structure for the measure. The SAVE-6 scale was found to be a reliable, valid, and useful brief measure that can be applied to the general population.

Introduction

The COVID-19 pandemic has disrupted the daily lives of many individuals, with them experiencing various psychiatric issues, including depression, anxiety, insomnia, and post-traumatic stress. Particularly, people experience the fear of infection, both for themselves and their loved ones, or of spreading the infection to others. The prevalence of anxiety symptoms in the pandemic era reported ranging from 6.33 to 50.9% (Xiong et al., 2020). Studies have assessed the anxiety levels of individuals using various rating scales, such as the Generalized Anxiety Disorder-seven items (GAD-7; Hou et al., 2021), Zung’s Self-rating Anxiety Scale (SAS; Ran et al., 2020), and the Depression Anxiety and Stress Scale (Wang et al., 2020). However, these scales do not specifically assess anxiety dealing with the COVID-19 pandemic. Therefore, a rating scale specific to the viral epidemic needs to be developed to determine the actual effects of the COVID-19 pandemic on the anxiety of an individual.

Several rating scales were developed and applied in 2020 in response to the pandemic: the five-item Coronavirus Anxiety Scale developed by Lee (2020a), the COVID-19-Anxiety Questionnaire modified by Petzold et al. (2020) based on the DSM-5 Severity Measure for Specific Phobia-Adult Scale, the seven-item Fear of COVID-19 Scale developed by Ahorsu et al. (2020), the four-item Obsession with COVID-19 Scale developed by Lee (2020b), the 11-item Coronavirus Pandemic Anxiety Scale developed by Bernardo et al. (2020), the two-factor nine-item COVID-19 Anxiety Syndrome Scale (six items for perseverance and three items for avoidance) developed by Nikcevic and Spada (2020), and the seven-item COVID-19 Anxiety Scale developed by Silva et al. (2020). These scales inquired about the anxiety of, repetitive thoughts of, or anxiety-related physiological arousal symptoms in an individual. Originally, we developed the Stress and Anxiety to Viral Epidemics-9 (SAVE-9) items scale specifically for healthcare workers. It consisted of items inquiring about apprehension or thoughts of an individual about the risk of infection, about the consequent influence on their physical health, or about avoidance of others (Chung et al., 2020). The SAVE-9 scale was designed to have two factors: first, anxiety about the viral epidemic factor, including six items (SAVE-6) and second, work-related stress associated with the viral epidemic, including three items (SAVE-3). We previously validated the SAVE-9 questionnaire and verified its utility among healthcare workers. In this study, we hypothesized that the SAVE-6 scale can be used for measuring anxiety in response to the viral epidemic among the general population. We also explored the psychometric properties of the SAVE-6 scale and determined the appropriate cut-off point of the scale with respect to the general anxiety symptoms.

Materials and Methods

Participants and Procedure

This study was conducted via an anonymous online survey system through EMBRAIN, a professional research company.¹ The survey collected 1,009 responses from January 14 to 20, 2021. The participants voluntarily responded to the survey. The mean age of the participants was 44.3 (±13.5) years, with 51% (n = 515) male population. The study protocol was approved by the Institutional Review Board of Sungshin Women’s University, Seoul, South Korea (SSWUIRB-2020-040). Written informed consent was waived.

Assessment of Symptoms

SAVE-6

The SAVE-6 scale is a subcategory of the SAVE-9 scale² developed originally for measuring stress and anxiety due to the viral epidemic among healthcare workers (Chung et al., 2020). Each of the six items is rated on a five-point Likert scale ranging from 0 (never) to 4 (always). The cut-off score of the SAVE-6 scale has been reported to be 15, equivalent to at least a mild degree or ≥ 5 on the GAD-7 scale. The total score on the SAVE-6 scale ranges from 0 to 24, with higher scores reflecting higher levels of anxiety response to the viral epidemic.

GAD-7

The GAD-7 scale is a self-report questionnaire for measuring general anxiety (Spitzer et al., 2006). Each item is scored on a four-point Likert scale (0 = not at all to 3 = nearly every day). Scores range from 0 to 21, with higher scores reflecting higher levels of anxiety. The cut-off points for anxiety are 0–4 (minimal), 5–9 (mild anxiety), 10–14 (moderate), and 15–21 (severe).

Patient Health Questionnaire-9

The PHQ-9 scale is a self-report questionnaire for measuring depression (Kroenke et al., 2001). Each item is rated on a four-point Likert scale (0 = not at all to 3 = nearly every day). Scores range from 0 to 27, with higher scores reflecting severe depression. The cut-off points for depression are 0–4 (minimal), 5–9 (mild), 10–14 (moderate), 15–19 (moderately severe), and 20–27 (severe).

Statistical Analyses

We conducted an independent t-test and the chi-square test to examine the gender differences in clinical variables or rating scale scores using the IBM SPSS Statistics for Windows, version 21.0. We also performed Spearman’s correlation to examine the association of scores from the SAVE-6 scale with demographic variables and rating scale scores since the distribution of PHQ-9 and GAD-7 scores were not within the normal limit. We hypothesized a one-factor model for the SAVE-6 scale based on the previous analysis on healthcare workers (Chung et al., 2020). The normality assumption was checked by using skewness and kurtosis for an acceptable limit of range ±2 (Gravetter and Wallnau, 2014). After examining the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity to explore the data suitability, exploratory factor analysis (EFA) was conducted to evaluate the construct validity. In EFA, we used the principal axis factor (PAF) extraction method with a Pearson’s correlation matrix and promax rotation. To determine the number of factors to be retained, the scree test and the parallel analysis test (Horn, 1965; Glorfeld, 1995; Timmerman and Lorenzo-Seva, 2011), based on minimum rank factor analysis (MRFA; Lorenzo-Seva and Ferrando, 2006), with a 95-percentile threshold, based on the polychoric correlation matrix, were conducted using FACTOR, version 10.10.03 (Lorenzo-Seva and Ferrando, 2006) program. The reliability and internal consistency of the factor were examined using Cronbach’s alpha and McDonald’s omega coefficient to verify the dimensionality of the SAVE-6 scale. Finally, the receiver operating characteristic (ROC) analysis was performed to explore the appropriate cut-off score of the SAVE-6 scale in accordance with generalized anxiety symptoms.

Results

Demographic Characteristics

Table 1 presents the demographic characteristics of the patients. There is no significant gender difference in clinical variables and rating scale scores except in the SAVE-9 scale score. Among the sample, 90 (8.9%) and 69 (6.8%) participants scored above the cut-off for clinical depression symptoms (PHQ-9 ≥ 10) and generalized anxiety (GAD-7 ≥ 10), respectively. Among the respondents, 222 (20.0%) reported knowing a person that had been infected, 80 (7.9%) reported having the experience of being quarantined, 6 (0.6%) reported the experience of being infected themselves, and 186 (18.4%) reported having a serious medical illness.

TABLE 1

Table 1. Demographic characteristics of participants (N = 1,009).

The SAVE-6 scores were significantly higher among respondents who were rated as having depression [PHQ-9 ≥ 10, t (1,007) = 9.29, and p < 0.001] and generalized anxiety [GAD-7 ≥ 10, t (1,007) = 8.34, and p < 0.001]. Moreover, the SAVE-6 scale scores were significantly higher among women [t (1,007) = 2.38 and p = 0.018] when compared with men, among people with a serious disease [t (1,007) = 2.11 and p = 0.035], and among people who knew a person infected with COVID-19 (t (1,007) = 2.07 and p = 0.038). However, no significant differences were observed with respect to the area of residence (p = 0.19), to the experience of being infected (p = 0.55), and to the experience of being quarantined (p = 0.09).

Factor Structure of the SAVE-6 Scale

The normality assumption was checked. It revealed that the distribution of each of the six items was within the normal limit (Table 2). The KMO measure (0.82) and the Bartlett’s test of sphericity (p < 0.001) showed adequacy for running EFA. The EFA with PAF extraction, the polychoric correlation, and the promax rotation suggested a one-factor model of the SAVE-6 scale based on the Kaiser Criterion method with an eigenvalue above 1.00 (eigenvalue = 2.635, 42.3% of the variance).

TABLE 2

Table 2. Frequencies of answers of participants to each of the SAVE-6 item.

The scree test and parallel analysis using the MRFA extraction and polychoric correlation were used to identify the adequate number of factors for the scale. We compared the explained real-data eigenvalues with the 95th percentile of random eigenvalues and made a decision where the real-data eigenvalues exceeded the 95th percentile of random eigenvalues. The results suggested that the single-factor structure (real-data eigenvalue = 69.99, 95th percentile of random eigenvalue = 45.40) of the SAVE-6 scale similar to that of the previous study (Chung et al., 2020).

Reliability of the Scores and Evidence Based on Relations to Other Variables

The SAVE-6 scale showed a good internal consistency reliability (McDonald’s ϖ = 0.818 and Cronbach’s α = 0.815). In this sample, the Cronbach’s α of PHQ-9 and GAD-7 were 0.869 and 0.929, respectively. The high scores of SAVE-6 scale scores were significantly correlated with PHQ-9 scores (ρ = 0.37, p < 0.001) and GAD-7 scores (ρ = 0.37, p < 0.001). In this study, the ROC analysis revealed that the 15 point of the SAVE-6 scale is appropriate (area under the curve, AUC = 0.706, sensitivity = 70.7%, and specificity = 60.0%) for at least a mild degree of GAD-7 score (≥5), and almost half of the 1,009 respondents (n = 501, 49.7%) scored ≥ 15 on the SAVE-6 scale. We also observed that the 17 point of the SAVE-6 scale is in accordance with the moderate degree of GAD-7 (≥10, AUC = 0.768, sensitivity = 72.5%, and specificity = 71.3) and 320 (31.7%) respondents were scored ≥ 17 on the SAVE-6 scale.

Discussion

We originally developed the SAVE-9 scale for healthcare workers during the pandemic (Chung et al., 2020). We previously found that the SAVE-9 scale could be clustered into two factors: anxiety about the viral epidemic (six items, SAVE-6) and work-related stress associated with the viral epidemic (three items, SAVE-3). In the present study, we investigated the utility of the six-item factor when applied to the general population, labeled as the SAVE-6 scale. We observed that EFA supported a one-factor model of the SAVE-6 scale, consistent with the result of the parallel analysis. The SAVE-6 scale showed good internal consistency reliability. In addition, the ROC analysis revealed that the 15 point of the SAVE-6 scale is appropriate for at least a mild degree of GAD-7 score (≥5).

The SAVE-6 scale was extracted from the original SAVE-9 scale for measuring the behavior or thoughts of healthcare workers during the COVID-19 pandemic. Previous rating scales were developed to inquire about physiological arousal symptoms of individuals associated with clinically elevated fear and anxiety (the Coronavirus Anxiety Scale, Lee, 2020a); feelings of anxiety, nervousness, muscle tension, and behaviors of avoidance (the COVID-19-Anxiety Questionnaire, Petzold et al., 2020); worry, increased heartbeat, or repetitive thoughts (the Fear of COVID-19 scale, Ahorsu et al., 2020; the Coronavirus Pandemic Anxiety Scale, Bernardo et al., 2020; the COVID-19 Anxiety Scale, Silva et al., 2020); or behaviors of avoidance, checking, and worrying (the COVID-19 Anxiety Syndrome Scale, Nikcevic and Spada, 2020). The SAVE-9 scale consists of items inquiring about the apprehension of an individual during the current pandemic situation, work-related stress of healthcare workers, worry about avoidance behavior of others, and concern about their own health and the health of their family members.

Though the results of this study showed a good single model of the SAVE-6 scale with good reliability, we observed a gender difference in the scores of SAVE-6 scale. In this pandemic era, the level of stress or anxiety due to the viral epidemic was reported to be higher among women compared to men in the general population (Hou et al., 2020; Mohammadpour et al., 2020), and even in the special population, such as healthcare workers (Huang et al., 2021; Lee et al., 2021). Silva et al. (2020) also observed the higher level of anxiety among female participants while developing their new rating scale, the COVID-19 anxiety scale. Female preponderance in the anxiety level needs to be considered while developing an anxiety scale targeting the viral epidemic may be expected. Moreover, female preponderance in the anxiety level needs to be considered while developing an anxiety scale targeting the viral epidemic.

The SAVE-9 scale for healthcare workers was originally developed to be brief and practical and to identify individuals who need psychological support. The appropriate cut-off score of the SAVE-9 scale was defined in accordance with at least a mild degree of GAD-7 score to screen healthcare workers who may be vulnerable to COVID-19 infection and consequent work-related stress (Chung et al., 2020). In the previous study, the appropriate cut-off score of factor I of the SAVE-9 scale was defined as point 15 (AUC = 0.728, sensitivity = 0.72, and specificity = 0.61) among healthcare workers. In parallel with the current study, we also observed point 15 of the SAVE-6 scale as a cut-off among the general population (AUC = 0.706, sensitivity = 70.7%, and specificity = 60.0%). In the current study, 49.7% of the participants were rated as having at least a mild degree of anxiety to the viral epidemic using the SAVE-6 scale, while 31.4% of participants were rated as having a mild degree of anxiety with a GAD-7 score ≥ 5. Although the data were not shown in the results, an additional 27.5% of the participants were screened using the SAVE-6 scale among those who were not rated as having anxiety (GAD-7 < 5).

This study had several limitations. First, we did not measure test-retest reliability. Therefore, it was difficult to state the stability of the measure. Second, we could not gather information concerning the employment of the participants. Given that people from certain professions, such as healthcare workers, government officials, and school teachers, are at a higher risk of infection in this pandemic era, the analysis could have benefited from considering the jobs or workplaces of the participants. Last, the results of this study should be interpreted with caution as it is a cross-sectional study. Further studies are needed to generate more information about the general population.

In conclusion, we observed that the SAVE-6 scale is a reliable, valid, and useful brief measure. Future studies should explore the utility of the SAVE-6 scale among the general population using a more representative sample.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Institutional Review Board of Sungshin Women’s University, No. SSWUIRB-2020-040. Written informed consent was waived.

Author Contributions

SC, MA, Y-WS, and SS conceived the study. SS obtained ethics approval. SK, SS, and SL recruited participants and obtained data. SC organized the database and performed statistical analyses. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the Framework of International Cooperation Program managed by the National Research Foundation of Korea (FY2020K2A9A1A01094956).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to appreciate all the people who voluntarily participated in the survey.

Footnotes

1. ^www.embrain.com

2. ^www.save-viralepidemic.net

References

Ahorsu, D. K., Lin, C. Y., Imani, V., Saffari, M., Griffiths, M. D., and Pakpour, A. H. (2020). The fear of COVID-19 scale: development and initial validation. Int. J. Ment. Heal. Addict. 1–9. doi: 10.1007/s11469-020-00270-8 [Epub ahead of print]

PubMed Abstract | CrossRef Full Text | Google Scholar

Bernardo, A. B. I., Mendoza, N. B., Simon, P. D., Cunanan, A. L. P., Dizon, J., Tarroja, M. C. H., et al. (2020). Coronavirus pandemic anxiety scale (CPAS-11): development and initial validation. Curr. Psychol. 1–9. doi: 10.1007/s12144-020-01193-2 [Epub ahead of print]

PubMed Abstract | CrossRef Full Text | Google Scholar

Chung, S., Kim, H. J., Ahn, M. H., Yeo, S., Lee, J., Kim, K., et al. (2020). Development of the stress and anxiety to viral Epidemics-9 (SAVE-9) scale for assessing work-related stress and anxiety in healthcare workers in response to COVID-19. PsyArXiv [Preprint]. doi: 10.31234/osf.io/a52b4

CrossRef Full Text | Google Scholar

Glorfeld, L. W. (1995). An improvement on horn’s parallel analysis methodology for selecting the correct number of factors to retain. Educ. Psychol. Meas. 1995, 377–393.

Google Scholar

Gravetter, F., and Wallnau, L. (2014). Essentials of Statistics for the Behavioral Sciences. 8th Edn. Belmont, CA: Wadsworth.

Google Scholar

Horn, J. (1965). A rationale and test for the number of factors in factor analysis. Psychom. Theory 30, 179–185.

Google Scholar

Hou, F., Bi, F., Jiao, R., Luo, D., and Song, K. (2020). Gender differences of depression and anxiety among social media users during the COVID-19 outbreak in China: a cross-sectional study. BMC Public Health 20:1648. doi: 10.1186/s12889-020-09738-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Hou, W. K., Tong, H., Liang, L., Li, T. W., Liu, H., Ben-Ezra, M., et al. (2021). Probable anxiety and components of psychological resilience amid COVID-19: a population-based study. J. Affect. Disord. 282, 594–601. doi: 10.1016/j.jad.2020.12.127

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, Q., Luo, L. S., Wang, Y. Y., Jin, Y. H., and Zeng, X. T. (2021). Gender differences in psychological and behavioral responses of infected and uninfected health-care workers during the early COVID-19 outbreak. Front. Public Health 9:638975. doi: 10.3389/fpubh.2021.638975

PubMed Abstract | CrossRef Full Text | Google Scholar

Kroenke, K., Spitzer, R. L., and Williams, J. B. (2001). The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 16, 606–613. doi: 10.1046/j.1525-1497.2001.016009606.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, J., Lee, H. J., Hong, Y., Shin, Y. W., Chung, S., and Park, J. (2021). Risk perception, unhealthy behavior, and anxiety due to viral epidemic among healthcare workers: the relationships with depressive and insomnia symptoms during COVID-19. Front. Psych. 12:615387. doi: 10.3389/fpsyt.2021.615387

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, S. A. (2020a). Coronavirus anxiety scale: a brief mental health screender for COVID-19 related anxiety. Death Stud. 44, 393–401. doi: 10.1080/07481187.2020.1748481

CrossRef Full Text | Google Scholar

Lee, S. A. (2020b). How much “thinking” about COVID-19 is clinically dysfunctional? Brain Behav. Immun. 87, 97–98. doi: 10.1016/j.bbi.2020.04.067

CrossRef Full Text | Google Scholar

Lorenzo-Seva, U., and Ferrando, P. J. (2006). FACTOR: A computer program to fit the exploratory factor analysis model. Behav. Res. Methods 38, 88–91. doi: 10.3758/BF03192753

PubMed Abstract | CrossRef Full Text | Google Scholar

Mohammadpour, M., Ghorbani, V., Khoramnia, S., Ahmadi, S. M., Ghvami, M., and Maleki, M. (2020). Anxiety, self-compassion, gender differences and COVID-19: predicting self-care behaviors and fear of COVID-19 based on anxiety and self-compassion with an emphasis on gender differences. Iran. J. Psychiatry 15, 213–219. doi: 10.18502/ijps.v15i3.3813

PubMed Abstract | CrossRef Full Text | Google Scholar

Nikcevic, A. V., and Spada, M. M. (2020). The COVID-19 anxiety syndrome scale: development and psychometric properties. Psychiatry Res. 292:113322. doi: 10.1016/j.psychres.2020.113322

PubMed Abstract | CrossRef Full Text | Google Scholar

Petzold, M. B., Bendau, A., Plag, J., Pyrkosch, L., Maricic, L. M., Rogoll, J., et al. (2020). Development of the COVID-19-anxiety questionnaire and first psychometric testing. BJPsych Open 6:e91. doi: 10.1192/bjo.2020.82

PubMed Abstract | CrossRef Full Text | Google Scholar

Ran, M. S., Gao, R., Lin, J. X., Zhang, T. M., Chan, S. K. W., Deng, X. P., et al. (2020). The impacts of COVID-19 outbreak on mental health in general population in different areas in China. Psychol. Med. 1–10. doi: 10.1017/S0033291720004717 [Epub ahead of print]

PubMed Abstract | CrossRef Full Text | Google Scholar

Silva, W. A. D., de Sampaio Brito, T. R., and Pereira, C. R. (2020). COVID-19 anxiety scale (CAS): development and psychometric properties. Curr. Psychol. 1–10. doi: 10.1007/s12144-020-01195-0 [Epub ahead of print]

PubMed Abstract | CrossRef Full Text | Google Scholar

Spitzer, R. L., Kroenke, K., Williams, J. B., and Lowe, B. (2006). A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med. 166, 1092–1097. doi: 10.1001/archinte.166.10.1092

PubMed Abstract | CrossRef Full Text | Google Scholar

Timmerman, M. E., and Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychol. Methods 16, 209–220. doi: 10.1037/a0023353

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, C., Pan, R., Wan, X., Tan, Y., Xu, L., Ho, C. S., et al. (2020). Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China. Int. J. Environ. Res. Public Health 17:1729. doi: 10.3390/ijerph17051729

PubMed Abstract | CrossRef Full Text | Google Scholar

Xiong, J., Lipsitz, O., Nasri, F., Lui, L. M. W., Gill, H., Phan, L., et al. (2020). Impact of COVID-19 pandemic on mental health in the general population: a systematic review. J. Affect. Disord. 277, 55–64. doi: 10.1016/j.jad.2020.08.001

PubMed Abstract | CrossRef Full Text | Google Scholar