The Schoolteachers' Version of the Stress and Anxiety to Viral Epidemics-9 (SAVE-9) Scale for Assessing Stress and Anxiety During the COVID-19 Pandemic

This study aimed to validate the schoolteachers' version of the Stress and Anxiety to Viral Epidemics-9 (SAVE-9) scale. This scale assessed the work-related stress and anxiety response of schoolteachers to the COVID-19 pandemic. A total of 400 schoolteachers participated in an online survey between March 4 and 15, 2021. The survey questionnaire included the schoolteachers' version of the SAVE-9, Patient Health Questionnaire-9 (PHQ-9), and Generalized Anxiety Disorders-7 (GAD-7) scales. A scree test and parallel analysis suggested a single-factor structure model for the schoolteachers' version of the SAVE-9 scale (real-data eigenvalue = 68.89, 95th percentile of the random eigenvalues = 27.56). The SAVE-9 scale showed good internal consistency (Cronbach's alpha = 0.853) and good convergent validity with GAD-7 (rho = 0.545, p < 0.001) and PHQ-9 (rho = 0.434, p < 0.001) scale scores. This, schoolteachers' version of the SAVE-9 scale is a reliable and valid rating scale that can be applied to teachers in a pandemic situation.

Introduction

In 2020, people experienced psychological issues such as depression, anxiety, and post-traumatic stress resulting from the coronavirus disease 2019 (COVID-19) pandemic and the subsequent economic crises. COVID-19-related lockdowns, social distancing, and quarantine disrupted their daily lives, business dealings, and interpersonal relationships (1). Although COVID-19 vaccines have been developed and available since December 2020¹, 185,291,530 confirmed cases and 4,010,834 deaths have been recorded globally. These are staggering and tragic numbers².

School environments were also severely disrupted and changed rapidly during the pandemic³. This resulted in schoolteachers experiencing high levels of stress or burnout (2, 3). When schools reopened, teachers who were not trained in infection control had the additional responsibility of preventing viral spread. This occurred by checking students' temperatures or rating their suspected symptoms. Moreover, they were required to direct students to wear masks and practice social distancing while at school. These tasks most likely imposed a severe burden on schoolteachers. Even when schools were closed owing to the lockdown, schoolteachers were required to adapt to a new educational environment and prepare for online education (4). They monitored students' academic abilities and supervised their progress while providing online classes. They also assisted in overcoming children's obstacles to learning (5) such as digital education inequity, absence of parents while students remained at home, and the lack of in-person teacher-student-parent relationships.

Reportedly, teachers had symptoms of stress, anxiety, and depression during the pandemic. This was especially evident among female teachers and those who had children (6). Their age, job stability, level of education, information sources, and level of apprehension also influenced their symptoms (2, 7). They experienced severe burnout (8), and their quality of life was hampered by work overload, feelings of uncertainty, and/or fear (9). Teachers played a key role in the educational environment during this pandemic. However, little research has been conducted focusing on their mental health, though they served as the forgotten frontline heroes of the COVID-19 pandemic (10). Further, to the best of our knowledge, there is no developed rating scale that can measure the anxiety and/or stress of teachers in response to specific viral epidemics. Therefore, we have attempted to develop the schoolteachers' version of the anxiety assessment scale using the SAVE-9 scale that has been developed for healthcare workers.

During the COVID-19 pandemic, healthcare workers, public workers, and schoolteachers cared for others at risk of COVID-19. They played roles in the prevention of viral spread, experienced severe psychological distress, and endured work-related burnout. They had to avoid becoming infected themselves and had to prevent the spread of viral infection to their family and co-workers. Therefore, it is important to develop epidemic-specific psychological assessment tools for the schoolteachers affected by the current pandemic. We recently developed a new rating scale, the Stress and Anxiety to Viral Epidemics-9 (SAVE-9) scale, to assess work-related stress and anxiety among healthcare workers in response to the COVID-19 pandemic (11). The scale was originally developed for frontline healthcare workers caring for patients during the COVID-19 pandemic. However, schoolteachers also were placed in a situation similar to that of healthcare workers. They cared for students and were also responsible for infection control in schools while being at risk of infection from students. Therefore, to address their mental health, it is important to assess the anxiety or work-related stress of schoolteachers during this pandemic situation. This study explored the validity of the schoolteachers' version of the SAVE-9 scale for assessing work-related stress among schoolteachers during the current COVID-19 pandemic.

Materials and Methods

Participants and Procedures

We enrolled schoolteachers working in elementary, middle, and high schools in various regions in Korea. This study was conducted via an online survey administered by a professional online survey company, EMBRAIN⁴, during March 4–15, 2021. The surveys were completed anonymously. No personal information was gathered. The study protocol was approved and the requirement for written informed consent for participation was waived by the Institutional Review Board of Asan Medical Center (2021-0098). The participants voluntarily responded to this questionnaire. We used only the responses of participants who agreed to provide their data for statistical analysis.

Symptom Assessment

SAVE-9 Scale

The main objective of this study was to develop and validate the schoolteachers' version of the SAVE-9 scale. The SAVE-9 was originally developed to assess the mental health of healthcare workers' during the COVID-19 pandemic (11). The scale comprises nine items that considers two factors: (1) anxiety response to a viral epidemic and (2) work-related stress associated with a viral epidemic. To modify this SAVE-9 scale for schoolteachers, item 7 that stated “After this experience, do you think you will avoid treating patients with viral illnesses?” was changed to “Do you think you will avoid teaching children who have had viral illnesses.” The participants were asked to rate their responses on a five-point Likert scale ranging from 0 (never) to 4 (always)⁵.

Patient Health Questionnaire-9

The Patient Health Questionnaire-9 (PHQ-9) scale is a self-administered questionnaire that is used to assess depressive symptoms. Respondents can rate their symptoms for each of the nine items on a three-point Likert scale (0 = not at all to 3 = nearly every day). The total score of the PHQ-9 scale ranges from 0 to 27. The higher scores reflect more severe degrees of depressive symptoms (12). The present study defined the cut-off score for depression as ≥ 10.

Generalized Anxiety Disorder-7

The Generalized Anxiety Disorder-7 (GAD-7) scale is a self-administered questionnaire used to assess general anxiety. Respondents can rate their anxiety symptoms for each of the seven items on a three-point Likert scale (0 = not at all to 3 = nearly every day). The total score of the GAD-7 scale ranges from 0 to 21. The higher scores reflect more severe degrees of anxiety symptoms (13). The present study defined a score of 5 as the cut-off to identify cases with at least mild anxiety.

Statistical Analysis

We conducted a principal component analysis (PCA) with varimax rotation to explore the factor structure of the schoolteachers' version of the SAVE-9 scale. Prior to conducting this analysis, we checked the normality assumption of each item based on its skewness and kurtosis, with an acceptable limit of range of ± 2 (14). We also evaluated the data suitability and sampling adequacy using Kaiser–Meyer–Olkin values and Bartlett's tests of sphericity. A scree test and parallel analysis (15–17), according to a minimum rank factor analysis (MRFA), with a 95th percentile threshold based on a polychoric correlations matrix was conducted to determine the number of factors to retain using the FACTOR 10.10.03. program (18). To define the factor structure, we compared the explained real-data eigenvalues with the 95th percentile of random eigenvalues. We decided where the real-data eigenvalues exceeded the 95th percentile of random eigenvalues. The reliability and internal consistency of the factors were examined using Cronbach's alpha and McDonald's Omega coefficients to verify the dimensionality of the schoolteacher's version of the SAVE-9 scale. To investigate the psychometric properties, independent t-tests were conducted using IBM SPSS Statistics for Windows, version 21.0. We assessed whether the SAVE-9 scale score differed significantly between groups based on sex (male vs. female), age (juniors vs. seniors), marital status (single vs. married), caring for infected students (yes vs. no), and being quarantined (yes vs. no). To determine the convergent validity, we performed a Spearman correlation analysis of the SAVE-9 scale score with age, years of employment, PHQ-9 scores, and GAD-7 scores, since the PHQ-9 and GAD-7 scales' scores did not show normal distributions. Finally, to define the appropriate cut-off score of the SAVE-9 scale consistent with the severity of the generalized anxiety (GAD-7 score), we conducted a receiver operating characteristic (ROC) analysis.

Results

A total of 400 teachers participated in this study. They were from various regions in Korea, including Seoul (N = 78, 19.5%), Pusan (N = 28, 7.0%), Daegu (N = 28, 7.0%), Daejeon (N = 8, 2.0%), Gwangju (N = 10, 2.5%), Chungcheong Province (N = 34, 8.5%), Jeolla Province (N = 31, 7.7%), Gyeongsang Province (n = 49, 12.3%), Gangwon Province (N = 13, 3.3%), and Jeju Province (N = 4, 1.0%). Table 1 shows the demographic characteristics of the participants. Out of all the participants, 270 (67.5%) were female and the others were male; 177 (44.3%) were single; and 57 (14.2%) had experienced or had been treated for depression, anxiety, or insomnia. Their mean duration of employment was 11.8 ± 8.6 years. Among the participants, 71 (17.7%) had cared for students with confirmed COVID-19, 57 (14.2%) had been quarantined due to COVID-19, and 1 (0.3%) had developed COVID-19.

TABLE 1

Table 1. Demographic characteristics of the participants (N = 400).

Factor Structure of the Schoolteacher's Version of the SAVE-9 Scale

Assessment of the normality assumption showed that the distributions of all nine items were within normal limits (Table 2). The data were suitable, and the sampling was adequate for PCA (Bartlett's test of sphericity, p < 0.001; Kaiser–Meyer–Olkin = 0.897). To explore the factor structure of the schoolteacher's version of the SAVE-9 scale, a PCA was conducted, and a two-factor model was explored based on the eigenvalue plot (first factor eigenvalue = 4.337, second factor eigenvalue = 1.054), which considered two factors: (1) Factor I: anxiety regarding the viral epidemic (items 1, 2, 3, 4, and 8) and (2) Factor II: work-related stress associated with the viral epidemic (items 5, 6, 7, and 9) (Table 2). However, the results of the scree test and parallel analysis using MRFA extraction and polychoric correlation (that was conducted to identify the adequate number of factors for the scale) suggested a single-factor structure model (real-data eigenvalue = 68.89, 95th percentile of random eigenvalue = 27.56) for the schoolteacher's version of the SAVE-9 scale. Therefore, we adopted the single-structure model.

TABLE 2

Table 2. Factor structure of the schoolteachers' version of the SAVE-9 scale and factor loadings.

Reliability of the Scores and Evidence Based on Relationships With Other Variables

The schoolteachers' version of the SAVE-9 scale showed good internal consistency (Cronbach's alpha = 0.853 and McDonald's Omega = 0.850). The Spearman correlation coefficient showed significant correlations between the SAVE-9, GAD-7 (rho = 0.545, p < 0.001), and PHQ-9 (rho = 0.434, p < 0.001) scale scores. The SAVE-9 scale score was significantly higher among healthcare workers who were rated as having generalized anxiety [GAD-7 ≥ 10, t₍₃₉₈₎ = 10.155, p < 0.001] and depression [PHQ-9 ≥10, t₍₃₉₈₎ = 7.896, p < 0.001] than among those who did not. The SAVE-9 scale score was significantly higher for women than for men [t₍₃₉₈₎ = 4.063, p < 0.0001]; however, it did not differ significantly according to marital status [single vs. married, t₍₃₉₈₎ = 1.140, p = 0.255], age [junior 20–39 years vs. senior over 40 years, t₍₃₉₈₎ = 0.923, p = 0.356], experience caring for infected students [t₍₃₉₃₎ = 1.660, p = 0.098], or experience being quarantined [t₍₃₉₈₎ = 0.773, p = 0.440].

Cut-Off Score for the SAVE-9 Scale

To explore the appropriate cut-off score of the SAVE-9 scale that corresponded to at least mild generalized anxiety, we conducted an ROC analysis using a GAD-7 scale score of 5. The ROC analysis revealed that the appropriate SAVE-9 cut-off score was ≥ 22 (area under the curve [AUC] = 0.729, sensitivity = 0.79, specificity = 0.54).

Discussion

The results of this study showed that the schoolteacher's version of the SAVE-9 scale was reliable, with good internal consistency. We adopted the single-factor model of the SAVE-9 scale. Moreover, the schoolteachers' version of the SAVE-9 scale showed good convergent validity with a pre-existing anxiety measurement tool (the GAD-7 scale). It exhibited a threshold score of 22 indicating at least mild generalized anxiety (GAD-7 scale score ≥5).

A schoolteacher's role in this pandemic is like that of a healthcare worker. The teacher must care for his or her students just as the healthcare worker must care for his or her patients. For this sample of schoolteachers, we adopted a single-factor model based on the results of the parallel analysis, though PCA suggested a two-factor model. Parallel analysis was used to determine the number of components to retain from PCA (19). Parallel analysis is conducted based on the rationale of multiplying the total number of variable loadings by the significance level to provide an empirical estimate of the 95th percentile. We can also estimate by checking where the real-data eigenvalues exceeded the 95th percentile of the random eigenvalues. The schoolteachers' version of the SAVE-9 scale showed good internal consistency and convergent validity. In our previous sample of healthcare workers (11), items related to work-related stress (items 6, 7, and 9) were clustered separately, though items 5, 6, 7, and 9 were clustered in this sample in PCA. This difference may be due to differences in the working environments between healthcare workers and schoolteachers, the time frame of the survey, or the sampling variability.

The factor loading values of items 1 and 7 in this sample were 0.509 and 0.565, respectively. Items are generally accepted when their factor loading values are >0.6 (20). However, items with values >0.5 may be acceptable if the consistency of the scale is good (21). Thus, we adopted all the items in the final model.

The appropriate cut-off score of the schoolteachers' version of the SAVE-9 scale was ≥22 to indicate at least a mild degree of generalized anxiety (GAD-7 ≥5) (AUC = 0.729, sensitivity = 0.79, specificity = 0.54). These results are consistent with those reported for the original SAVE-9 scale (11). We originally developed the SAVE-9 scale as a viral epidemic-specific anxiety rating scale especially for workers addressing or caring for people at high risk of infection in a pandemic. We intended to identify a mild degree of anxiety response to the viral epidemic to screen for healthcare workers requiring psychological support during the current pandemic. Similarly, we defined a cut-off score for the schoolteachers' version of the SAVE-9 scale. It corresponded to a GAD-7 score of ≥5 (mild degree) to screen for schoolteachers with at least mild anxiety regarding the current viral epidemic. Thus, the schoolteachers' version of the SAVE-9 scale can be applied as it showed good validity in the educational setting.

This study had several limitations. First, the survey was conducted via an anonymous online survey as it was not easy to interview the participants face-to-face during the current pandemic situation. Though we used the services of a professional survey company, the nature of the survey may have introduced errors. Second, the survey was conducted about 1 year after the outbreak. Thus, the teachers might have adapted to their situations, developed online education skills, and cared for students during the pandemic in 2020. This may influence the validity of the schoolteachers' version of the SAVE-9 scale. In this study, we could not collect the data of participants' race, income, education, or socioeconomic status. The status of the pandemic in Korea might be different from that in other countries in which the pandemic is severe. The school or teaching system, the availability of remote learning, or cultural differences related to education should also be considered. Therefore, the results of this study cannot be generalized to samples from other countries. Finally, as the SAVE-9 scale was originally developed for COVID-19 frontline healthcare workers, there are some issues with applying this scale to schoolteachers. However, the SAVE-9 scale focuses on individual anxiety or stress regarding a viral epidemic rather than on clinical practice. Therefore, it can be applied to a schoolteacher that cares for students.

In conclusion, like the SAVE-9 scale for healthcare workers, the teachers' version of the SAVE-9 scale is a reliable and valid rating scale for assessing the anxiety and work-related stress of schoolteachers. During the current pandemic, we hope that the SAVE-9 scale will be used to measure the work-related stress and anxiety response of schoolteachers who teach and care for students. We also hope that this scale will assist in the development of support systems that will decrease their psychological distress.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The study protocol was approved and the requirement for written informed consent for participation was waived by the Institutional Review Board of Asan Medical Center (2021-0098). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

SC and SS developed the study concept. Testing and data collection were performed by JL, SL, and GJ. Data analysis and interpretation were performed by SC, SS, GJ, and SL. SY and SC drafted the paper. All authors contributed to the study design, provided critical revisions, and approved the final version of the paper for submission.

Funding

This work was supported under the Framework of International Cooperation Program managed by the National Research Foundation of Korea (FY2020K2A9A1A01094956).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^Available online at: https://ourworldindata.org/covid-vaccinations accessed on 2021, July, 12th.

2. ^Available online at: https://covid19.who.int/ accessed on 2021, July, 12th.

3. ^https://en.unesco.org/covid19/educationresponse accessed on 2021, July, 12th.

4. ^http://www.embrain.com.

5. ^http://www.save-viralepidemic.net.

References

1. Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet. (2020) 395:912–20. doi: 10.1016/S0140-6736(20)30460-8

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Ozamiz-Etxebarria N, Berasategi Santxo N, Idoiaga Mondragon N, Dosil Santamaría M. The psychological state of teachers during the COVID-19 crisis: the challenge of returning to face-to-face teaching. Front Psychol. (2021) 11:620718. doi: 10.3389/fpsyg.2020.620718

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Stachteas P, Stachteas C. The psychological impact of the COVID-19 pandemic on secondary school teachers. Psychiatriki. (2020) 31:293–301. doi: 10.22365/jpsych.2020.314.293

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Van Lancker W, Parolin Z. COVID-19, school closures, and child poverty: a social crisis in the making. Lancet Public Health. (2020) 5:e243–e4. doi: 10.1016/S2468-2667(20)30084-0

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Gore J, Fray L, Miller A, Harris J, Taggart W. The impact of COVID-19 on student learning in New South Wales primary schools: an empirical study. Aust Educ Res. (2021). doi: 10.1007/s13384-021-00436-w. [Epub ahead of print].

CrossRef Full Text | Google Scholar

6. Santamaria MD, Mondragon NI, Santxo NB, Ozamiz-Etxebarria N. Teacher stress, anxiety and depression at the beginning of the academic year during the COVID-19 pandemic. Glob Ment Health. (2021) 8:e14. doi: 10.1017/gmh.2021.14

CrossRef Full Text | Google Scholar

7. Li Q, Miao Y, Zeng X, Tarimo CS, Wu C, Wu J. Prevalence and factors for anxiety during the coronavirus disease 2019 (COVID-19) epidemic among the teachers in China. J Affect Disord. (2020) 277:153–8. doi: 10.1016/j.jad.2020.08.017

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Amri A, Abidli Z, Elhamzaoui M, Bouzaboul M, Rabea Z, Ahami AOT. Assessment of burnout among primary teachers in confinement during the COVID-19 period in morocco: case of the Kenitra. Pan Afr Med J. (2020) 35:92. doi: 10.11604/pamj.supp.2020.35.2.24345

CrossRef Full Text | Google Scholar

9. Lizana PA, Vega-Fernadez G, Gomez-Bruton A, Leyton B, Lera L. Impact of the COVID-19 pandemic on teacher quality of life: a Longitudinal study from before and during the health crisis. Int J Environ Res Public Health. (2021) 18:3764. doi: 10.3390/ijerph18073764

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Beames JR, Christensen H, Werner-Seidler A. School teachers: the forgotten frontline workers of covid-19. Australas Psychiatry. (2021). doi: 10.1177/10398562211006145. [Epub ahead of print].

CrossRef Full Text | Google Scholar

11. Chung S, Kim HJ, Ahn MH, Yeo S, Lee J, Kim K, et al. Development of the stress and anxiety to viral epidemics-9 (SAVE-9) scale for assessing work-related stress and anxiety in healthcare workers in response to COVID-19. PsyArXiv [Preprint]. (2020). doi: 10.31234/osf.io/a52b4

CrossRef Full Text | Google Scholar

12. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. (2001) 16:606–13. doi: 10.1046/j.1525-1497.2001.016009606.x

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. (2006) 166:1092–7. doi: 10.1001/archinte.166.10.1092

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Gravetter F, Wallnau L. Essentials of Statistics for the Behavioral Sciences, 8th Ed. Wadsworth, OH; Belmont, CA: Wadsworth (2014).

Google Scholar

15. Horn J. A rationale and test for the number of factors in factor analysis. Psychometricka. (1965) 30:179–85. doi: 10.1007/BF02289447

CrossRef Full Text | Google Scholar

16. Glorfeld LW. An improvement on horn's parallel analysis methodology for selecting the correct number of factors to retain. Educ Psychol Meas. (1995) 55:377–93. doi: 10.1177/0013164495055003002

CrossRef Full Text | Google Scholar

17. Timmerman ME, Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychol Methods. (2011) 16:209–20. doi: 10.1037/a0023353

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Lorenzo-Seva U, Ferrando PJ. FACTOR: a computer program to fit the exploratory factor analysis model. Behav Res Methods. (2006) 38:88–91. doi: 10.3758/BF03192753

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Franklin SB, Gibson DJ, Robertson PA, Pohlmann JT, Fralish JS. Parallel analysis: a method for determining significant principal components. J Veg Sci. (1995) 6:99–106. doi: 10.2307/3236261

CrossRef Full Text | Google Scholar

20. Zainudin A. Structural Equation Modeling using aMOS Graphic. Shah Alam: UiTM Press (2012).

21. Hair JF, Babin AH, Money P. Essentials of Business Research Methods. Hoboken, NJ: John Wiley & Sons (2003).

Google Scholar