- 1Department of Psychology, Faculty of Arts, University of Ljubljana, Ljubljana, Slovenia
- 2Institute of Gerontology, Wayne State University, Detroit, MI, United States
- 3International Institute of Applied Gerontology, Ljubljana, Slovenia
During the outbreak of the COVID-19 epidemic, fear of disease and its consequences, recommended lifestyle changes, and severe restrictions set by governments acted as stressors and affected people’s mood, emotions, mental health, and wellbeing. Many studies conducted during this crisis focused on affective and physiological responses to stress, but few studies examined how the crisis affected cognition. The present cross-sectional study examined the relationship between physiological, affective, and cognitive responses to the epidemic. In an online survey conducted at the height of the first wave of the epidemic in Slovenia (April 15–25, 2020), 830 Slovenian residents aged 18–85 years reported the effects of stressors (confinement, problems at home, problems at work, lack of necessities, and increased workload), experienced emotions, generalized anxiety, perceived stress, changes in health, fatigue and sleep quality, and perceived changes in cognition during the epidemic. Risk factors for stress (neuroticism, vulnerability, general health, gender, and age) were also recorded. We hypothesized that stressors and stress risk factors will be related to subjective cognitive decline, with negative emotions, generalized anxiety, perceived stress, and physical symptoms acting as mediator variables. On average, the results showed a mild subjective cognitive decline during the epidemic. In structural equation modeling, 34% of its variance was predicted by the mediator variables, with negative emotions and physical symptoms having the largest contribution. Stress risk factors were predictably related to the four mediator variables. Among the stressors, confinement showed the strongest effect on the four mediator variables, implying the importance of thoughtful communication about necessary restrictive measures during emergency circumstances. The results of this study indicate that the possibility of altered cognitive function should be considered when planning work and study activities during the epidemic.
Introduction
On March 11, 2020, the World Health Organization declared a COVID-19 pandemic (Ghebreyesus, 2020). Slovenia declared the epidemic on March 12, 2020 (Government of the Republic of Slovenia, 2020a). Due to the sudden nature of the COVID-19 outbreak and high infectiousness of the SARS-CoV-2 virus, the Slovenian Government implemented several restrictions. In addition to the introduction of the minimum physical distance of 1.5 m and the mandatory use of face masks, freedom of movement was restricted to small municipalities, people were expected to self-isolate in their homes, most economic activities were stopped, and public life was closed. Public services such as public transport and health services were restricted. Shops were closed, except for grocery stores. Educational institutions were closed and switched to online teaching. According to Eurofound (2020), in April and May 2020, 10% of people reported that they worked from home every day or several times a week before the epidemic, and 23% reported that they started working from home as a result of the situation, with 17% of residents reporting that their working hours increased and 46% reporting that they decreased. The media covered the crisis extensively, and the amount of information about COVID-19 and the ever-changing measures to limit the outbreak was overwhelming. In retrospect, it could be assumed that Slovenians quickly adapted to the emergency measures and followed strict restrictions and recommended lifestyle changes, making Slovenia the first European country to announce the end of the first wave of the epidemic on May 15, 2020 (Government of the Republic of Slovenia, 2020b). However, the implemented restrictions affected people’s behavior and psychological wellbeing (Lep and Hacin Beyazoglu, 2020).
Stress During the COVID-19 Epidemic
The course of the COVID-19 pandemic as a global, prolonged health crisis was unpredictable and beyond control of individuals. It caused stress to many people, and the measures taken to restrict the spread of the virus had many negative effects. A plethora of studies conducted around the world, including in Slovenia (Lep and Hacin Beyazoglu, 2020), showed that the COVID-19 crisis affected people’s mood and emotions (Xiao et al., 2020), leading to decreased subjective wellbeing (Möhring et al., 2020; Ammar et al., 2021b; Paredes et al., 2021), sleep disturbances (Gualano et al., 2020; Pinto et al., 2020; Saraswathi et al., 2020; Ammar et al., 2021b), and increased prevalence of psychiatric conditions, such as generalized anxiety disorder, depression, or post-traumatic stress symptoms (Fu et al., 2020; Liu et al., 2020). The confinement during the epidemic also resulted in an increase of social isolation, physical inactivity (the hours of daily-sitting) and unhealthy diet behaviors (Ammar et al., 2021b). However, only few studies examined how people’s cognition changed during the COVID-19 crisis. Batty et al. (2020), in a prospective cohort study using UK Biobank data, found that a higher risk of hospitalization for COVID-19 was observed for participants with lower performance on two tests of cognitive function – verbal and numerical reasoning and reaction speed – which the authors suggested could be potential markers of health literacy. Commenting on the consequences of the COVID-19 pandemic, Boals and Banks (2020, p. S255) suggested that increases in stress and anxiety are likely to impair cognitive functioning. They wrote that “anecdotally, in the time since the pandemic, students and colleagues have shared that they have had trouble staying focused and productive.” Our own observation was consistent with this comment – many people in our setting who worked from home reported being more tired and having more trouble multitasking. In the present study, therefore, we wanted to investigate how different stressors, i.e., different aspects of the COVID-19 crisis, such as stress related to changes in living and working conditions, affected the adult population of Slovenia. We wanted to examine the relationship between different types of stress responses, including physiological, affective, and cognitive responses. More specifically, we were interested in subjectively reported change in cognitive performance.
Physiological, Affective and Cognitive Responses to Stress
Stress affects mood and emotions, cognition, behavior, wellbeing, and health (Schneiderman et al., 2005). It activates the hypothalamic–pituitary–adrenal (HPA) axis, resulting in physiological responses such as increased cortisol secretion (Kemeny, 2003), heart rate, respiratory rate, blood pressure and muscle tension, making an organism ready for action. Affective responses to stress include negative affect (e.g., feelings of tension, panic, feeling overwhelmed, irritability, restlessness, anger, guilt, sadness, grief, and depression) or positive affect (e.g., feelings of happiness, enthusiasm, contentment, and excitement; Zhaoyang et al., 2020). Cognitive responses to stress include mental slowing, confusion, narrowing of focus, difficulty concentrating, memory impairment (forgetfulness), increased or decreased awareness of one’s surroundings, general negative thinking, intrusive and repetitive thoughts and images, constant worry, difficulty making decisions, poor abstract thinking, disturbed thinking, difficulty identifying familiar objects or people, loss of orientation in time and place, and changes in learning and memory (Becker et al., 1973; Bryce, 2001; Kemeny, 2003).
The direct effect of stress on cognitive functions is not entirely clear, and research examining the effects of stress on executive functions has yielded counterintuitive results (see Shields et al., 2016). Studies found that acute and chronic psychological stress can induce structural and functional changes in the adult brain and impair memory and executive functions (Diamond, 2013; Chattarji et al., 2015; Shields et al., 2016). Executive functions encompass the higher cognitive processes that enable cognitive control, i.e., planning, thinking ahead, and goal-directed action, and include working memory (the ability to retain information in memory and update it regularly), inhibition (the ability to inhibit thoughts or prepotent responses in order to selectively attend to task-relevant information and engage in goal-directed behavior), and cognitive flexibility (the ability to flexibly switch between cognitive rules or ways of thinking; Miyake et al., 2000; Diamond, 2013). In a meta-analysis on the effects of acute stress on executive functions, Shields et al. (2016) showed that stress impairs working memory and cognitive flexibility and that these effects are moderated by sex; stress was also found to impair cognitive inhibition but increase response inhibition, suggesting that stress contributes to a cognitive state of automatic, reactive processing and more alert executive motor control that allows a person to quickly engage with or escape from the current stressor. Qin et al. (2009)found that experimentally induced acute stress leads to deficits in working memory, increased catecholamine and cortisol levels, reduced activation of the dorsolateral prefrontal cortex, and a reallocation of neural resources away from executive function networks. Liston et al. (2009) found in undergraduates that a month-long psychosocial stress related to exams impaired attentional shifting and disrupted functional connectivity within a frontoparietal network mediating attentional shifting, although these impairments were reversible after the stress ended. In general, stress causes a focus on the here and now, resulting in impaired retrospective and prospective memory (Bourne and Yaroush, 2003). The time span from which knowledge can be easily retrieved and used in a given context shrinks as stress levels increase (Bourne and Yaroush, 2003). Functioning during stress may be adaptive in the short term, biasing processing in favor of a single salient stimulus (Liston et al., 2009) or the current stressor to allow an organism to effectively cope with the current unstable circumstances (Shields et al., 2016).
The Interconnectedness of Different Types of Responses to Stress
Emotion and cognition are deeply intertwined (Okon-Singer et al., 2015). Intense negative emotions can interfere with focusing on, encoding, and retrieving important information. Emotions that cause high arousal can divert attention to specific stimuli and affect attentional focus, working memory, learning, reasoning, problem solving, and cognitive control, especially inhibition (Harlé et al., 2013; Okon-Singer et al., 2015). Individuals with emotional disorders, such as anxiety and depression, show impaired cognitive processes (Mathews and MacLeod, 2005). Anxiety can distort attentional processing, because it narrows the range of perceived stimuli and focuses attention on the threat. It biases evaluations of stimulus valence (Yiend, 2010), as well as individual perceptions of reality (Spielberger, 1966). Conversely, cognition can also alter, activate, and inhibit emotions; effortful cognitive strategies, such as reappraising the situation in a more positive light, can be used to cope with and regulate negative emotions (Kryla-Lighthall and Mather, 2009; Cole et al., 2014; Okon-Singer et al., 2015; Tyng et al., 2017).
Complex relationships exist between stress, fatigue, sleep, self-perceived health status, and performance (De Vries et al., 2003; Taylor and Dorn, 2006; Kocalevent et al., 2011; Khanade and Sasangohar, 2017). Sleep and stress interact in a bidirectional manner. Stress causes changes in metabolism through activation of the HPA axis and increased release of glucocorticosteroids, leading to impaired sleep (Van Reeth et al., 2000). This in turn affects the regulation of HPA axis activity, which indirectly modulates arousal (Hirotsu et al., 2015). Sleep disturbances affect how we respond to emotional events during the day, and conversely, responses to past emotional events affect sleep quality (Altena et al., 2016). Stress, anxiety, and depression are associated with fatigue and poorer subjective sleep quality (Van Reeth et al., 2000; Valerio et al., 2016; Thorsteinsson et al., 2019; Cox and Olatunji, 2020; Xiao et al., 2020). Although some studies found no association between subjective sleep quality and cognitive performance (Zavecz et al., 2020), many studies report that sleep quality also affects cognition. It is associated with problems in attention, working memory, and executive functions (Scullin and Bliwise, 2015). Sleep loss and deprivation have been found to impair performance on cognitive tasks involving vigilance and attention, working and long-term memory, learning, logical reasoning, arithmetic calculations, pattern recognition, complex verbal processing, and decision making (Krueger, 1989; Alhola and Polo-Kantola, 2007). Partial sleep restriction deteriorates memory encoding and the ability to learn declarative information (Cousins et al., 2018). Following sleep deprivation, cognitive impairments are thought to be mediated through decreased alertness, attentional lapses, and slowed responses (Alhola and Polo-Kantola, 2007). Thus, cognition can be impaired by stress, fatigue, and decreased sleep quality.
Individual Differences in Stress Reactivity
Responses to the same stressor are not the same for all individuals. Stress occurs when individuals perceive that environmental demands tax or exceed their adaptive capacity (Cohen et al., 1983, 1997), so stressful experiences can be viewed as person-environment transactions, the outcome of which depends on both the stressor and the individual (Kemeny, 2003). In terms of the nature of stressors, circumstances that are perceived as uncontrollable, ambiguous, novel, and durable are more likely to activate a stress response (Kemeny, 2003; Dickerson and Kemeny, 2004). The influence of the external stimulus is mediated by the characteristics of the individual, such as primary appraisal of the stimulus as a threat vs. challenge (Kemeny, 2003), lack of confidence (Farrer et al., 2016), coping mechanisms, self-esteem and social skills (Uchino, 2009), the efficacy of coping efforts (Schneiderman et al., 2005; Pallavicini et al., 2013), social support (Cohen et al., 2000; Cohen, 2004; Qi et al., 2020) and social capital (Xiao et al., 2020), appraisal of psychosocial resources to cope with the stressor, e.g., appraisal of coping skills, personality factors, intellectual resources, financial resources, environmental resources (Kemeny, 2003), and perceived control over potentially negative events (Gallagher et al., 2014). Stress responses, including threat appraisals, negative and positive affect, and task performance, are also related to personality traits, such as neuroticism (Schneider, 2004), extraversion, and openness (Schneider et al., 2012). In addition, larger stress responses are associated with low socioeconomic status, female gender, younger age (Scott et al., 2013; Novais et al., 2017), and poorer physical wellbeing prior to the onset of the stressor (Kocalevent et al., 2011).
Subjective Cognitive Decline
Subjective cognitive complaints are everyday memory and related cognitive concerns expressed by people with or without objective evidence of cognitive impairment and are common across all age groups (Jacob et al., 2019). Subjective cognitive decline is not only predictive of Alzheimer’s disease dementia (Jessen, 2014), but is also associated with numerous other conditions, including normal aging (dos Santos et al., 2012), depression and anxiety (Hill et al., 2016), pregnancy (Crawley et al., 2008), substance use and medication (Jessen et al., 2014), and physical illness (Jacob et al., 2019). In older individuals, memory complaints without actual cognitive decline have been found to be associated with physical health problems, depressive and anxiety symptoms, higher perceived stress and lower mastery (control of potential problems in life), ineffective coping, and high neuroticism (Comijs et al., 2002; Steinberg et al., 2013). Stenfors et al. (2013) found that subjective cognitive complaints in healthy, working non-elderly adults were related to emotional exhaustion, burnout, mental fatigue, disturbed sleep, awakening problems, depressive symptoms, and poorer executive cognitive functioning. Jacob et al. (2019) found in a large nationally representative survey that subjective concentration and memory complaints were predicted by the number of stressful life events, perceived stress, depression, anxiety disorders, sleep problems, and physical health problems (multiple chronic diseases).
The Aim of This Study
Previous studies have rarely examined the effects of chronic stress on human cognitive functions because it would be difficult and unethical to experimentally manipulate such stress conditions (Shields et al., 2016). Thus, most of the evidence on the effects of stress and emotion on cognition has been obtained in short-term experimental studies, in clinical populations, or in selected samples with long-term exposure to stress, such as certain work groups (nurses and shift workers). Less is known about how prolonged collective situational uncertainty, such as that experienced by society at the time of the COVID-19 epidemic, can affect cognitive functioning and its perception. We therefore aimed to investigate whether subjective cognitive complaints during the crisis can be predicted by the physiological and affective responses to stress.
Based on the literature presented, we developed a model of subjective cognitive change during the COVID-19 epidemic, as shown in Figure 1. We expected that the perceived impact of stressors caused by the COVID-19 epidemic would be related to more intense physiological responses leading to physical symptoms (including fatigue, sleep disturbance, physical pain, and worsening of illness) and affective responses (including negative emotions, generalized anxiety, and perceived stress), which in turn would be related to higher levels of subjective cognitive decline associated with impaired attention, memory, and cognitive control. We also expected stress risk factors, such as neuroticism, vulnerability to stress, poorer general health, female gender, and younger age to contribute positively to physiological and affective responses to stress. In addition, we were interested in the association between subjective cognitive change and various demographic characteristics.
Figure 1. A simplified representation of the structural model for predicting subjective cognitive change. Negative emotions, generalized anxiety (GAD-7), perceived stress (loss of perceived control; PSS-4), and physical symptoms mediate the effects of stressors and stress risk factors on subjective cognitive change (CCQ).
Materials and Methods
Participants
To assess perceived stress and responses to stress during the COVID-19 epidemic, we designed an online survey. The survey was open on the Slovenian online survey platform 1KA (2020) for 10 days, from April 15 to 25, 2020, at the peak of the first wave of the epidemic, when the trend of newly detected COVID-19 cases began to level off. It was accessed by 1,290 individuals. Of 1,135 who began filling it out, 1,072 (94%) completed at least part of the survey, 881 (77%) responded to the final section of the survey, and 830 participants had complete data on the variables included in the analyses. Table 1 shows a description of the final sample by gender, age, education, marital status, and employment status. In the general population of Slovenia, the share of the same age categories as in our study was 8, 15, 18, 17, 17, 13, and 11% in the first half of 2020 (Statistical Office of the Republic of Slovenia, 2020). In our sample, the oldest age category was underrepresented, which is most likely related to the online administration of the survey.
Table 1. Description of nominal variables and comparison of subgroups on the Cognitive Change Questionnaire (CCQ) score.
Instruments
This research was planned and conducted in an international group of researchers from China, the United States, and Slovenia, who sought to gain insight into a wide range of experiences with the COVID-19 outbreak among participants from different countries who faced different measures to prevent the spread of the SARS-CoV-2 virus. The common core of the survey was negotiated and took into account the different contexts in the participating countries. It included a combination of self-constructed questions and questions from previously validated questionnaires. The aim was to cover many different aspects of the experiences while being manageable and time efficient for participants. In this paper, we report only the results of the study conducted in Slovenia on selected variables related to our defined research problem.
Perception of Stressors
Participants rated on a 5-point scale how much they experienced various stressors or difficulties that negatively affected their mood or emotions during the COVID-19 pandemic (1 – not at all, 2 – a little, 3 – a moderate amount, 4 – a lot, and 5 – a great deal; ‘Not applicable’ (N/A) was also added but later changed to 1 because if a particular factor was not relevant to the participants, it did not affect them). The first type of stressor related to the specifics of the epidemic crisis and lockdown: time spent indoors, media coverage, and restricted movement. The second type of stressor was related to difficulties at home: family relationships, intimate partner relationship, and reduced privacy. The third type of stressor included work-related problems, academic problems, and economic problems. The fourth type of stressor was lack of necessities: personal protective equipment, food, medicine, and access to a doctor. Participants also reported whether their workload had increased during the epidemic (Yes/No).
Physical Symptoms
To assess participants’ physiological reactions to stress, we asked them to rate on a 5-point scale (1 – not at all, 2 – a little, 3 – a moderate amount, 4 – a lot, and 5 – a great deal; N/A was changed to 1) how much they experienced physical pain, worsening of illness, and fatigue/sleepiness during the epidemic. They also rated how tired they were and how well they slept during the COVID-19 outbreak compared to before the outbreak. These changes in fatigue and sleep quality were rated on 5-point scales (1 – much less/better, 2 – less/better, 3 – same, 4 – more/worse, and 5 – much more/worse).
Affective Responses to Stress
To assess participants’ affective state and stress during the COVID-19 epidemic, we used the GAD-7 scale and self-constructed inventories of emotional states and vulnerability indicators. A Brief Measure for Assessing Generalized Anxiety Disorder – GAD-7 (Spitzer et al., 2006) – is a clinical screening self-report measure consisting of 7 items. Participants rated on a 4-point scale how often they experienced the listed symptoms since the beginning of the epidemic (0 – never, 1 – several days, 2 – over half the days, and 3 – nearly every day). When used for clinical assessment, item responses are summed. In the study by Spitzer et al. (2006), the instrument demonstrated good reliability (Cronbach’s alpha coefficient was 0.92 and test–retest intraclass correlation was 0.83), construct validity (a higher score was strongly associated with multiple domains of functional impairment and disability days), and factorial validity (the scale differentiated symptoms of generalized anxiety from those of depression).
In addition to GAD-7, participants rated how much they experienced the following emotions on a 5-point response scale (1 – not at all, 2 – a little, 3 – a moderate amount, 4 – a lot, and 5 – a great deal): Anger at others, anger at self, sadness, fear, worry, annoyance, depression, distracted thinking, longing for normality, and loneliness.
Participants also completed the Perceived Stress Scale – PSS-4 (Warttig et al., 2013). PSS-4 is a 4-item version of a self-report questionnaire developed by Cohen et al. (1983) to measure how often in the past month the person felt nonspecific appraised stress and was unable to control the important things in their life (1 – never, 2 – almost never, 3 – sometimes, 4 – fairly often, and 5 – very often). The higher the score on the PSS-4, the more the respondents perceive that the demands exceed their ability to cope (Warttig et al., 2013). We therefore considered the responses on this scale to be most indicative of the loss of perceived control during the crisis. Cohen et al. (1983) found high correlations of PSS scores with depressive symptomatology and stress measures in samples of college students. Warttig et al. (2013) found acceptable psychometric properties of the PSS-4 on an English sample (Cronbach’s alpha was 0.77) and low to moderate negative correlations with perceived health status, social support, and age, with women reporting higher stress than men.
Cognitive Responses to Stress
Because we were in a lockdown and faced with emergent and rapidly changing situations, we could not conduct objective cognitive tests. There were also no data available on residents’ cognitive function prior to the epidemic. Therefore, we opted for self-report of the changes people observed in their everyday cognitive function. We looked for questionnaires that were general enough and did not ask about instrumental daily activities, since we targeted healthy adults. We could not find a questionnaire that could be easily applied to the situation, so we decided to develop a new instrument. The survey was long and asked about many different variables, so we wanted to keep the instrument on cognitive changes short. To capture self-perceived changes in cognition, we drew on the Working Memory Questionnaire (Vallat-Azouvi et al., 2012) and the Teenage Executive Functioning Inventory – TEXI (Thorell et al., 2020), and compiled nine questions asking about speed of information processing, short-term storage, prospective memory, attention, and executive control (see Table 2). We had no information about participants’ previous cognitive functioning and were not interested in absolute levels of functioning, so we decided to ask participants directly about the changes they observed in their cognition during the COVID-19 epidemic. They were asked to compare their current state (the state during the epidemic) with the state before the epidemic, using a 7-point scale. For items 1–5, the following scale was used: 1 – much less often, 2 – less often, 3 – a little less often, 4 – same as before, 5 – a little more often, 6 – more often, and 7 – much more often (than before). For items 6–9, the following scale was used: 1 – much easier, 2 – easier, 3 – a little easier, 4 – same as before, 5 – a little harder, 6 – harder, and 7 – much harder (than before). We will refer to these questions as the Cognitive Change Questionnaire (CCQ).
Table 2. Item content, descriptive statistics, and standardized factor loadings from the one-factor measurement model of the CCQ.
Stress Risk Factors
Participants’ vulnerability to stress was assessed by their ratings on a 5-point scale (1 – not at all; 5 – a great deal) of the extent to which their sense of imbalance, self-denial, lack of resilience, vulnerability, tendency to suppress emotions, lack of family support, lack of social support, perfectionistic tendencies, poor adaptability, lack of self-confidence, and lack of coping skills played a role in their negative emotions related to the COVID-19 epidemic. Their neuroticism was rated on the same scale based on their responses to questions how much it describes them as a person to be emotionally stable and worry free. They also rated their general health (not limited to the period of the epidemic) on a 4-point scale (1 – poor, 2 – fair, 3 – good, and 4 – excellent).
The survey also asked participants about their gender, age category, employment status, relationship status, and education level.
Procedure
The Ethics Committee of the Faculty of Arts at University of Ljubljana approved the study (approval No. 184-2020). Snowball sampling was used to recruit participants. The researchers AP and VK sent invitations to their personal email contacts and posted announcements on several Facebook pages and on websites of various organizations. Participants were asked to forward the invitation to their relatives, friends, and acquaintances. Participation in the survey was voluntary. Participants received no benefits for participating in the study. They were introduced to the purpose of the survey and gave their consent to participate in the study by clicking a specific button on a survey webpage. To avoid loss of motivation and dropout from participation in the survey, answering most questions was not mandatory and could be skipped if desired. On average, participation in the survey took 15 min.
Data Analysis
Frequency distributions for each item were examined and descriptive statistics were calculated.
Structural equation modeling was used to evaluate the following theoretical model (Figure 1): (i) Each of the four types of responses to stress (negative emotions, generalized anxiety, perceived stress or loss of perceived control, and physical symptoms) was predicted by the perceived impact of external stressors that occurred during the epidemic (confinement, problems at home, problems at work, lack of necessities, and increased workload); (ii) each of the four types of responses to stress was also predicted by stress risk factors: vulnerability to stress, neuroticism, general health, gender, and age (age category means were analyzed); and (iii) four types of responses to stress predicted subjective cognitive change during the COVID-19 epidemic. Perceptions of stressors and stress risk factors were thus entered as predictors (exogenous variables) in the structural equation model. Physical symptoms, negative emotions, generalized anxiety, and perceived stress were treated as endogenous variables and were also considered mediators between predictors and subjective cognitive change. All observed variables (variables listed in Supplementary Table 1, along with gender, age, general health, and increased workload) were entered into the model simultaneously. Due to the ordinal nature of the observed variables we used the robust weighted least squares estimator (WLSMV), implemented by the cfa function in the R lavaan package (Rosseel, 2012). The following cutoff values were considered indicative of acceptable fit of the model to the data (Marsh et al., 2004): CFI and TLI > 0.90; RMSEA and SRMR < 0.08.
The reliability of the scales was calculated using the omega function in the R psych package (Revelle, 2015). Because the measurement models supported the unidimensional structure of the latent constructs under study, we calculated Cronbach’s alpha coefficient as a measure of internal consistency. We also report the McDonald’s omega total, which is a better choice for reliability estimation in the presence of skewed item distributions and the absence of tau-equivalence, i.e., in the case of different factor loadings (Trizano-Hermosilla and Alvarado, 2016).
Responses to items measuring a specific construct were averaged and descriptive statistics were calculated for such scale scores. Welch’s t test and ANOVAs were used to compare the CCQ score in subsamples based on demographic variables (gender, age, education, relationship status, and employment status) and general health status.
All statistical hypotheses were tested at the significance level of 5%.
Results
The Fit of the Proposed Structural Equation Model
The model tested fit the observed data closely enough, χ2(1948) = 5653.79, p < 0.001, CFI = 0.906, TLI = 0.912, RMSEA = 0.048, 95% confidence interval for RMSEA = 0.046–0.049, p(RMSEA ≤ 0.05) = 0.991, and SRMR = 0.057.
Testing the measurement models of all included constructs confirmed their one-dimensional structure. Table 2 shows the estimated parameters in the measurement model related to the CCQ, and Supplementary Table 1 shows the estimated parameters in the measurement models for other latent constructs. For most items, factor loadings were high and consistent with expectations. In the CCQ, all items except item 1 loaded highly on the general factor. Excluding this item would not increase the reliability of the instrument, so we decided to keep it.
The scales measuring subjective cognitive change, negative emotions, generalized anxiety, and vulnerability to stress showed good reliability (Table 3), with Cronbach’s alpha coefficients of internal consistency exceeding the value of 0.88. The reliability of other scale scores was lower, but considering a small number of items on these scales, we concluded that their reliability was also acceptable (Table 3).
Responses to Stress and the Perceived Impact of Stressors
Regarding the perceived impact of stressors during the COVID-19 epidemic, participants perceived the impact of confinement on their emotions as moderate, while they reported a low impact of problems at home and at work and no to a low impact of lack of necessities (Table 3). A minority of participants (25%) reported experiencing increased workload during the epidemic (Table 1).
On average, participants were characterized by moderate levels of neuroticism and low levels of vulnerability to stress (Table 3). The frequency distribution of ratings of their general health is shown in Table 1. A large majority reported good or excellent health.
Among other constructs, Table 3 also shows descriptive statistics for constructs related to emotional responses to stress. On average, participants reported experiencing low levels of negative emotions and infrequent to occasional perceived stress (loss of perceived control). They rarely felt anxious during the epidemic. Summing responses to GAD-7 items yielded an average total score of 4.28 points (SD = 3.81) on the 0–21 scale. A large percentage of participants scored 0 on GAD-7. For 76 (9%) of participants, the scale sum was above 10, which is considered the cutoff point for identifying moderate generalized anxiety. Thus, we conclude that our sample generally exhibited only mildly negative emotions at the time of our study. In terms of physical responses to stress, participants generally reported experiencing fatigue and sleepiness to a low degree, no (or only mild) physical pain and worsening of illness, and a slight increase in fatigue and decrease in sleep quality compared to pre-epidemic times (see Supplementary Table 1, part Physical Symptoms).
On average, participants reported mildly impaired cognitive function during the COVID-19 lockdown (Table 3). During the epidemic, they were slightly slower in performing their usual activities than before the epidemic, and they noticed slight negative changes in their speed of information processing, attention, memory, and executive control (Table 2). The mean CCQ score (4.18) was statistically significantly larger than 4 (the response indicating no change), t(829) = 8.90, p < 0.001, d = 0.31, 95% confidence interval for d = [0.17, 0.45], 1 − β = 1.00.
No statistically significant differences were found in subjective cognitive change by gender, age, education, relationship status, and employment status (Table 1).
Predictors of Stress Responses and Subjective Cognitive Change
Supplementary Table 2 shows the correlations between different constructs used in the structural model for predicting physical symptoms and emotional responses to stress. Table 4 shows the standardized regression coefficients in the structural model. Among the COVID-19 crisis stressors, confinement showed the largest effect on all four predicted constructs (negative emotions, generalized anxiety, perceived stress, and physical symptoms). Perceived increased workload contributed to more intense physical symptoms. Problems at home and at work and a lack of necessities did not appear to contribute to the emotional and physiological responses to stress. Risk factors for stress showed an expected contribution to stress responses. Stress vulnerability, poor general health, female gender, and younger age contributed to all four types of stress responses, while neuroticism contributed only to emotional responses to stress but not to physical responses.
Table 4. Regression coefficients in the structural model for predicting the four mediator variables.
In the model studied, the four types of stress reactions predicted subjective cognitive change and were able to explain 34% of variance in the CCQ score. The contributions of negative emotions (b = 0.14, SEb = 0.05, z = 2.65, p = 0.004, β = 0.32) and physical symptoms (b = 0.40, SEb = 0.05, z = 8.27, p < 0.001, β = 0.44) were statistically significant, whereas the contributions of generalized anxiety (b = −0.05, SEb = 0.06, z = −0.78, p = 0.434, β = −0.07) and perceived stress in terms of loss of perceived control (b = −0.05, SEb = 0.05, z = −0.98, p = 0.327, β = −0.07) did not reach statistical significance.
Discussion
Our model of subjective cognitive change during the COVID-19 epidemic showed acceptable fit to the data collected. This suggests that the COVID-19 represented a stressful situation that elicited similar responses to those in other types of stressful situations. Confinement (including media coverage and worries about the latest news and other issues) and increased workload during the COVID-19 crisis – potentiated by vulnerability to stress, neuroticism, and poor general health – led to affective, physiological, and cognitive responses that resulted in subjective cognitive decline.
A very small, but statistically significant subjective cognitive decline was reported on average by our participants. Boals and Banks (2012) suggested that perceived stress during the COVID-19 crisis could lead to intrusive thoughts that compete for limited cognitive resources, cause mind wandering, and decrease academic, occupational, and daily life tasks performance. Among the most important factors for mind wandering, they cited worries about the latest news regarding the pandemic, worries about loved ones and others who might be at risk health-wise or financially, and worries about themselves. Overall, our results are consistent with their conjecture, but also reveal some further details.
The Impact of Stressors on COVID-19 Stress Responses
Previous studies have shown that stressors have an important impact on negative emotions, depression, and anxiety (Stein and Lang, 2002; Kemeny, 2003; Scott et al., 2013; Chattarji et al., 2015; Zhaoyang et al., 2020). In our study, increased workload, most likely due to increased teleworking and the need to adapt to the new situation (adjusting daily schedule and work process to work from home, helping children with online learning, increased use of computers and digital communication, etc.) contributed to physical symptoms (i.e., increased fatigue, decreased sleep quality, physical pain, and exacerbation of illness). Among the specific stressors associated with COVID-19 crisis, confinement was found to be the single most important origin of affective responses to stress and physical symptoms. It is possible that this predictor covered other stressful aspects of COVID-19 lockdown, as it was positively associated with changes in work or study conditions, income reductions, and problems with relationships at home (see Supplementary Table 2).
Other studies also found confinement or its variants to be important stressors during the COVID-19 epidemic. For example, Xin et al. (2020) found that mandatory quarantine during the initial COVID-19 outbreak in China was associated with negative thoughts (perceived discrimination) and emotional distress. Tang et al. (2020) found that the likelihood of exhibiting generalized anxiety and depression was higher among respondents who were quarantined than those who were not. Ammar et al. (2021b) reported that COVID-19 home confinement negatively affects mental wellbeing and emotional status and leads to unhealthy diet behaviors. Trabelsi et al. (2021) confirmed the effects of confinement on impaired sleep quality and decreases in physical activity. Bai et al. (2004), who studied the effects of quarantine during the outbreak of various diseases, found an increase in exhaustion, anxiety, irritability, and insomnia. Similarly, our results show a very general effect of this stressor on emotional, cognitive, and physical functioning during the epidemic. The negative effect of confinement could be attributed to quarantine-induced boredom, frustrations, perceived loss of freedom, decreased physical activity, (daily) travel restrictions, altered schedule due to working or studying from home, altered sleep–wake rhythms, and anxiety due to myths, misinformation, erroneous news reports in the media, and misunderstanding of health-related messages (Bao et al., 2020; Saraswathi et al., 2020). The media could also contribute to the stigmatization of those infected and those who leave their homes (Gualano et al., 2020), contributing to distress. In addition, Gualano et al. (2020) found that internet use increased during the COVID-19 lockdown for three-quarters of participants, and using the internet as a source of information led to a higher likelihood of anxiety. In terms of the harmful effects of confinement, it is interesting to note that a higher prevalence of generalized anxiety disorder has also been found in prisoners (Costa et al., 2010; Dadi et al., 2016), where it has been attributed to increased exposure to deprivation of social interaction, deprivation of liberty, rigid rules, constant control of individuals and stressful situations, among other factors (Costa et al., 2010), and similar characteristics could be attributed to the COVID-19 lockdown.
Other Factors of COVID-19 Stress Responses
In addition to the aforementioned effects of stressors, we observed an independent contribution of stress vulnerability, neuroticism, and poor general health to negative emotions, generalized anxiety, and perceived stress. Similar findings have been reported by other studies (Kemeny, 2003; Sexton et al., 2003; Schneider, 2004; Schneider et al., 2012; Scott et al., 2013; Warttig et al., 2013; Gallagher et al., 2014; Klainin-Yobas et al., 2014). Further, women reported more affective responses to stress and more physical symptoms. Gender differences in emotional, physiological, and cognitive responses to stress have also been observed in other studies (Warttig et al., 2013; Novais et al., 2017; Hodes and Epperson, 2019; Gualano et al., 2020; Liu et al., 2020). Several alternative explanations for these differences have been provided, ranging from neurobiological (Novais et al., 2017; Hodes and Epperson, 2019) to psychological in the sense that women are exposed to more stressors or perceive stressors as more stressful than men (Warttig et al., 2013). Finally, age acted as a preventive factor against affective responses to stress, as also found in several other studies (Warttig et al., 2013; Gualano et al., 2020). According to Warttig et al. (2013), older adults report fewer stressors than their younger counterparts because they are less active and redirect their preferences toward satisfying goals and emotion regulation to maximize positive emotional experiences and minimize negative ones. Also, physical symptoms may be perceived as normative in old age, so older people have a higher threshold for reporting them as potential stressors. In addition, the COVID-19 crisis likely brought fewer changes to the lives of older people (especially retirees) than to younger people.
More severe physical symptoms (i.e., greater increase in fatigue and worsening of sleep quality and health status) were reported by younger and more vulnerable individuals, by women, and by participants who reported increased workload, greater impact of confinement and poorer health. This is consistent with Wang et al.’s (2020) finding that anxiety was higher among students who reported poor health, and suggests that restricted movement during the epidemic may have exacerbated pre-existing health problems.
The Association of Physical and Affective Stress Responses With Subjective Cognitive Decline
In our model, physical symptoms and affective responses to stress were considered mediator variables in the relationship between stressors and subjective cognitive change. The results are consistent with other studies that have found subjective cognitive complaints to be associated with physical health problems (Comijs et al., 2002; Jacob et al., 2019) and sleep problems (Stenfors et al., 2013; Miley-Akerstedt et al., 2018; Jacob et al., 2019); the variables included in the construct Physical Symptoms in our study. Comijs et al. (2002) speculated that physical problems may contribute to lower wellbeing and motivation, leading to poor performance on cognitive tasks and memory complaints.
Both subjective and objective cognitive decline have also previously been associated with negative emotions and anxiety (Comijs et al., 2002; Ouimet et al., 2009; Boals and Banks, 2012; Harlé et al., 2013; Okon-Singer et al., 2015; Hill et al., 2016; Jacob et al., 2019). In our study, we found no evidence of the association between subjective cognitive decline and generalized anxiety. One possibility for such a result could be that we found a prevalence of generalized anxiety disorder of 9% in our sample (this was the percentage of participants with GAD-7 sum greater than 10 points), which is much lower than what has been found in some other countries and subject groups, where typically about one-third of the samples had an anxiety disorder and about one-third to one-half had sleep disorders during the COVID-19 epidemic (Fu et al., 2020; Gualano et al., 2020; Huang and Zhao, 2020; Hyland et al., 2020; Twenge and Joiner, 2020; Fiorenzato et al., 2021). Unfortunately, there are no data for the prevalence of generalized anxiety disorder in Slovenia in normal times, but a 12-month prevalence of threshold GAD of about 2% was observed in the European community before the COVID-19 pandemic (Lieb et al., 2005) and a 1-month prevalence of about 8% was observed in primary care patients worldwide (Maier et al., 2000), suggesting that the prevalence of GAD was only slightly increased in our sample. The use of other, more discriminating measures of anxiety could lead to different results. The same could be true for the measure of perceived stress (PSS-4), which had low reliability. This could be one of the reasons why we found no association between the PSS-4 scale score and the CCQ score, which is not consistent with previous studies reporting that loss of perceived control is related to subjective cognitive complaints (Comijs et al., 2002; Boals and Banks, 2012; dos Santos et al., 2012; Steinberg et al., 2013).
Study Limitations and Strengths
Our study has several limitations. We used only self-report scales, which may have led to social desirability and other response biases. Second, the study was cross-sectional. At the time of our study, the epidemic had been declared in Slovenia for just over a month. The differences between our study and others in the expression of affective responses to stress could be explained by the different timing during the lockdown and the different measurement instruments used. Third, participation was voluntary, and stressed individuals may have a greater need to participate in studies, such as ours to express their concerns and problems. Fourth, snowball sampling was used, resulting in an unbalanced gender and age structure of the sample (with males and individuals older than 75 years underrepresented), so our results may not be generalizable to the general population. Fifth, as is common with online surveys, a number of participants (27%) left the survey before completing it or did not answer all questions, so attrition bias may be present (see Supplementary Table 3 and accompanying text for more information on this). Sixth, other relevant stressors (e.g., reduction in physical activity and social interactions) and constructs (e.g., depression) could be included in the model and instruments with better psychometric properties could be used instead of single indicator variables, but this would increase the length of the already long survey and lead to additional dropouts. Seventh, our model is unidirectional and predicts subjective cognitive change based on physical symptoms and affective responses to stress. However, the relationship between the constructs under study may be bidirectional. For example, negative emotions may increase subjective cognitive decline, and subjective cognitive decline may increase negative emotions. The lack of temporal order in the measurement of stress and subjective cognitive decline prohibits causal inferences, and the mediations in our structural model should not be interpreted as causal mediations (Maxwell and Cole, 2007). Longitudinal observation would be desirable to shed more light on psychological responses to the COVID-19 epidemic. Qualitative studies should be conducted as a complement to quantitative studies to investigate in more detail how participants experienced the lockdown and in which situations subjective cognitive decline occurred, how it changed over time and why, how the intensive use of digital technology affected it, etc. Finally, using objective measures of stress (e.g., measuring cortisol levels) to monitor physiological changes during the epidemic would provide a deeper understanding of the impact of the epidemic on physical and mental health. Future studies should also use tests of attention, memory, and executive function and address the potential objective cognitive decline caused by chronic stress due to the COVID-19 epidemic, as subjective and objective cognitive decline do not necessarily overlap in healthy adults (Markova et al., 2017; Barbe et al., 2018).
Nevertheless, we can say that our research is important because we collected the data during the critical period of the first wave of the COVID-19 epidemic, i.e., under lockdown and particular psychological circumstances, and our sample was large. A majority of our results were consistent with findings in the literature, which gives us confidence. Additional support for our findings comes from a very recently published study by Fiorenzato et al. (2021). They investigated the effect of lockdown on the mental health and cognitive functioning of Italian residents. Some findings overlap with ours. For example, their participants reported increased distress and decreased sleep quality. They also complained about their attention, temporal orientation, and executive functions during lockdown. Subjective cognitive complaints were associated with home confinement. In their study, subjective cognitive complaints were also associated with increases in anxiety and depression, female gender, younger age, and underemployment. In our study, gender and age had similar effects on mediator variables (physical symptoms and affective stress responses) but not directly on subjective cognitive decline.
An important contribution of our study is also the developed CCQ, a brief measure of recent subjective cognitive changes. The instrument showed adequate psychometric properties and could be used and further validated in the future studies on the influence of crisis situations on subjective cognitive decline.
Conclusion
Our study showed that prolonged confinement can cause distress and lead to generalized anxiety, negative emotions, loss of perceived control, increased physical symptoms, and subjective cognitive decline. These results have several practical implications. First, governments should ensure that the experience of confinement is as tolerable as possible by setting a reasonable duration of lockdown and providing basic supplies and services (Li et al., 2020). Regulations designed to prevent the further spread of the virus must be well thought out and properly communicated to mitigate stress and prevent stress reactions. Interventions delivered via the Internet, mobile devices, or other types of media should be offered during confinement to monitor physical, mental, and psychosocial health, promote healthy lifestyles, and provide psychosocial support, especially for vulnerable groups, such as the elderly (see Ammar et al., 2021a). Second, cognitive impairment during the pandemic is inevitable for even the most resilient individuals (Boals and Banks, 2020). Therefore, attention should be paid to subjective (and objective) cognitive decline and to adapting work or learning processes during the epidemic. Expectations of what is realistic in times of crisis should be scaled down. In relation to stress, cognitive activities may have a preventive effect, as it has been shown that higher cognitive load focuses attention more on the neutral or positive non-threatening stimuli and reduces mind wandering toward worry (Najmi et al., 2015). However, according to the results of our study, increased workload can increase physical symptoms, such as fatigue and sleep problems. Therefore, a carefully planned, balanced level of work or study activities would be preferable during these difficult times.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
The studies involving human participants were reviewed and approved by The Ethics Committee of the Faculty of Arts at University of Ljubljana. The patients/participants provided their written informed consent to participate in this study.
Author Contributions
AP and VK were involved in developing the theory and survey, planning the study, and distributed the invitations. AP set up the online survey, analyzed the data with the help of LK, and wrote the manuscript with assistance from VK. AP, LK, and VK discussed the results. LK prepared Figure 1. LK and VK commented on the manuscript. All authors contributed to the article and approved the submitted version.
Funding
The authors acknowledge financial support from the Slovenian Research Agency (research core funding No. P5-0110).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
The study was a part of an international research conducted in three countries: China, United States (Kentucky and Michigan), and Slovenia. The authors acknowledge the work of Rui Xu, University of Beijing, and Terry Stratton and Yang Yiang, University of Kentucky, in developing the joint survey. Portions of this data focusing on other research problems have also been included in other papers: Kavčič and Podlesek (2020) and Podlesek and Kavčič (2020, 2021). We would like to thank Bruno Giordani for his comments on the manuscript.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2021.647971/full#supplementary-material
References
1KA (ver. 20.03.02) [online software] (2020). Ljubljana: Fakulteta za družbene vede. Available at: https://www.1ka.si (Accessed December 1, 2020)
Alhola, P., and Polo-Kantola, P. (2007). Sleep deprivation: impact on cognitive performance. Neuropsychiatr. Dis. Treat. 3, 553–567.
Altena, E., Micoulaud-Franchi, J. A., Geoffroy, P. A., Sanz-Arigita, E., Bioulac, S., and Philip, P. (2016). The bidirectional relation between emotional reactivity and sleep: From disruption to recovery. Behav. Neurosci. 130, 336–350. doi: 10.1037/bne0000128
Ammar, A., Bouaziz, B., Trabelsi, K., Glenn, J., Zmijewski, P., Müller, P., et al. (2021a). Applying digital technology to promote active and healthy confinement lifestyle during pandemics in the elderly. Biol. Sport 38, 391–396. doi: 10.5114/biolsport.2021.100149
Ammar, A., Trabelsi, K., Brach, M., Chtourou, H., Boukhris, O., Masmoudi, L., et al. (2021b). Effects of home confinement on mental health and lifestyle behaviours during the COVID-19 outbreak: insights from the ECLB-COVID19 multicentre study. Biol. Sport 38, 9–21. doi: 10.5114/biolsport.2020.96857
Bai, Y. M., Lin, C. C., Lin, C. Y., Chen, J. Y., Chue, C. M., and Chou, P. (2004). Survey of stress reactions among health care workers involved with the SARS outbreak. Psychiatr. Serv. 55, 1055–1057. doi: 10.1176/appi.ps.55.9.1055
Bao, Y., Sun, Y., Meng, S., Shi, J., and Lu, L. (2020). 2019-nCoV epidemic: address mental health care to empower society. Lancet 395, e37–e38. doi: 10.1016/S0140-6736(20)30309-3
Barbe, T., Kimble, L. P., and Rubenstein, C. (2018). Subjective cognitive complaints, psychosocial factors and nursing work function in nurses providing direct patient care. J. Adv. Nurs. 74, 914–925. doi: 10.1111/jan.13505
Batty, G. D., Deary, I. J., Luciano, M., Altschul, D. M., Kivimäki, M., and Gale, C. R. (2020). Psychosocial factors and hospitalisations for COVID-19: Prospective cohort study based on a community sample. Brain Behav. Immun. 89, 569–578. doi: 10.1016/j.bbi.2020.06.021
Becker, S. S., Horowitz, M. J., and Campbell, L. (1973). Cognitive responses to stress: effects of changes in demand and sex. J. Abnorm. Psychol. 82, 519–522. doi: 10.1037/h0035215
Boals, A., and Banks, J. B. (2012). Effects of traumatic stress and perceived stress on everyday cognitive functioning. Cognit. Emot. 26, 1335–1343. doi: 10.1080/02699931.2011.651100
Boals, A., and Banks, J. B. (2020). Stress and cognitive functioning during a pandemic: thoughts from stress researchers. Psychol. Trauma 12, S255–S257. doi: 10.1037/tra0000716
Bourne, L. E., and Yaroush, R. A. (2003). Stress and cognition: a cognitive psychological perspective. NASA Technical Reports Server (NTRS), February 1, 1–123. Available at: http://psych.colorado.edu/~lbourne/StressCognition.pdf (Accessed December 2, 2020)
Bryce, C. P. (2001). Insights into the concept of stress. Available at: http://helid.digicollection.org/en/d/Js2924e/5.html (Accessed December 20, 2020)
Chattarji, S., Tomar, A., Suvrathan, A., Ghosh, S., and Rahman, M. M. (2015). Neighborhood matters: divergent patterns of stress-induced plasticity across the brain. Nat. Neurosci. 18, 1364–1375. doi: 10.1038/nn.4115
Cohen, S. (2004). Social relationships and health. Am. Psychol. 59, 676–684. doi: 10.1037/0003-066X.59.8.676
Cohen, S., Gottlieb, B. H., and Underwood, L. G. (2000). “Social relationships and health,” in Social Support Measurement and Interventions: A Guide for Health and Social Scientists. eds. S. Cohen, L. G. Underwood, and B. H. Gottlieb (Oxford University Press, New York), 3–25. doi: 10.1093/med:psych/9780195126709.003.0001
Cohen, S., Kamarck, T., and Mermelstein, R. (1983). A global measure of perceived stress. J. Health Soc. Behav. 24, 385–396. doi: 10.2307/2136404
Cohen, S., Kessler, R. C., and Underwood Gordon, L. (1997). “Strategies for measuring stress in studies of psychiatric and physical disorders,” in Measuring Stress: A Guide for Health and Social Scientists. eds. S. Cohen, R. C. Kessler, and L. Underwood Gordon (Oxford University Press, New York), 3–26.
Cole, M. W., Repovš, G., and Anticevic, A. (2014). The frontoparietal control system: a central role in mental health. Neuroscientist 20, 652–664. doi: 10.1177/1073858414525995
Comijs, H. C., Deeg, D. J. H., Dik, M. G., Twisk, J. W. R., and Jonker, C. (2002). Memory complaints; the association with psycho-affective and health problems and the role of personality characteristics: a 6-year follow-up study. J. Affect. Disord. 72, 157–165. doi: 10.1016/S0165-0327(01)00453-0
Costa, A., Fuentes, L. J., and Vivas, A. B. (2010). The impact of bilingualism on the executive control and orienting networks of attention. Biling. Lang. Congn. 13, 315–325. doi: 10.1017/S1366728909990010
Cousins, J. N., Sasmita, K., and Chee, M. W. L. (2018). Memory encoding is impaired after multiple nights of partial sleep restriction. J. Sleep Res. 27, 138–145. doi: 10.1111/jsr.12578
Cox, R. C., and Olatunji, B. O. (2020). Sleep in the anxiety-related disorders: a meta-analysis of subjective and objective research. Sleep Med. Rev. 51:101282. doi: 10.1016/j.smrv.2020.101282
Crawley, R., Grant, S., and Hinshaw, K. I. M. (2008). Cognitive changes in pregnancy: Mild decline or societal stereotype?. Appl. Cogn. Psychol. 22, 1142–1162. doi: 10.1002/acp.1427
Dadi, A. F., Dachew, B. A., Kisi, T., Yigzaw, N., and Azale, T. (2016). Anxiety and associated factors among prisoners in north west of Amhara regional state, Ethiopia. BMC Psychiatry 16:83. doi: 10.1186/s12888-016-0792-y
De Vries, J., Michielsen, H. J., and Van Heck, G. L. (2003). Assessment of fatigue among working people: a comparison of six questionnaires. Occup. Environ. Med. 60(Suppl. 1), i10–i15. doi: 10.1136/oem.60.suppl_1.i10
Diamond, A. (2013). Executive functions. Annu. Rev. Psychol. 64, 135–168. doi: 10.1146/annurev-psych-113011-143750
Dickerson, S. S., and Kemeny, M. E. (2004). Acute stressors and cortisol responses: a theoretical integration and synthesis of laboratory research. Psychol. Bull. 130, 355–391. doi: 10.1037/0033-2909.130.3.355
dos Santos, A. T., Leyendecker, D. D., Costa, A. L. S., and de Souza-Talarico, J. N. (2012). Subjective memory complain in healthy elderly: influence of depressive symptoms, perceived stress and self-esteem. Rev. Esc. Enferm. USP 46, 24–29. doi: 10.1590/S0080-62342012000700004
Eurofound (2020). Living, working and COVID-19 data. Eurofound. Available at: https://www.eurofound.europa.eu/data/covid-19
Farrer, L. M., Gulliver, A., Bennett, K., Fassnacht, D. B., and Griffiths, K. M. (2016). Demographic and psychosocial predictors of major depression and generalised anxiety disorder in Australian university students. BMC Psychiatry 16:241. doi: 10.1186/s12888-016-0961-z
Fiorenzato, E., Zabberoni, S., Costa, A., and Cona, G. (2021). Cognitive and mental health changes and their vulnerability factors related to COVID-19 lockdown in Italy. PLoS One 16:e0246204. doi: 10.1371/journal.pone.0246204
Fu, W., Wang, C., Zou, L., Guo, Y., Lu, Z., Yan, S., et al. (2020). Psychological health, sleep quality, and coping styles to stress facing the COVID-19 in Wuhan, China. Transl. Psychiatry 10:225. doi: 10.1038/s41398-020-00913-3
Gallagher, M. W., Bentley, K. H., and Barlow, D. H. (2014). Perceived control and vulnerability to anxiety disorders: a meta-analytic review. Cogn. Ther. Res. 38, 571–584. doi: 10.1007/s10608-014-9624-x
Ghebreyesus, T. A. (2020). WHO Director-General’s opening remarks at the media briefing on COVID-19 - 11 March 2020. WHO Director General’s Speeches. Available at: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020 (Accessed December 5, 2020)
Government of the Republic of Slovenia (2020a). Slovenia declares coronavirus epidemic. GOV.SI Portal. Available at: https://www.gov.si/en/news/2020-03-12-slovenia-declares-coronavirus-epidemic/ (Accessed December 5, 2020)
Government of the Republic of Slovenia (2020b). Slovenia declares the end of COVID-19 epidemic; no quarantine for EU citizens from today. GOV.SI Portal. Available at: https://www.gov.si/en/news/2020-05-15-slovenia-declares-the-end-of-covid-19-epidemic-no-quarantine-for-eu-citizens-from-today/ (Accessed December 5, 2020)
Gualano, M. R., Lo Moro, G., Voglino, G., Bert, F., and Siliquini, R. (2020). Effects of COVID-19 lockdown on mental health and sleep disturbances in Italy. Int. J. Environ. Res. Public Health 17:4779. doi: 10.3390/ijerph17134779
Harlé, K. M., Shenoy, P., and Paulus, M. P. (2013). The influence of emotions on cognitive control: feelings and beliefs-where do they meet? Front. Hum. Neurosci. 7:508. doi: 10.3389/fnhum.2013.00508
Hill, N. L., Mogle, J., Wion, R., Munoz, E., DePasquale, N., Yevchak, A. M., et al. (2016). Subjective cognitive impairment and affective symptoms: a systematic review. Gerontologist 56, e109–e127. doi: 10.1093/geront/gnw091
Hirotsu, C., Tufik, S., and Andersen, M. L. (2015). Interactions between sleep, stress, and metabolism: From physiological to pathological conditions. Sleep Sci. 8, 143–152. doi: 10.1016/j.slsci.2015.09.002
Hodes, G. E., and Epperson, C. N. (2019). Sex differences in vulnerability and resilience to stress across the life span. Biol. Psychiatry 86, 421–432. doi: 10.1016/j.biopsych.2019.04.028
Huang, Y., and Zhao, N. (2020). Generalized anxiety disorder, depressive symptoms and sleep quality during COVID-19 outbreak in China: a web-based cross-sectional survey. Psychiatry Res. 288:112954. doi: 10.1016/j.psychres.2020.112954
Hyland, P., Shevlin, M., McBride, O., Murphy, J., Karatzias, T., Bentall, R. P., et al. (2020). Anxiety and depression in the Republic of Ireland during the COVID-19 pandemic. Acta Psychiatr. Scand. 142, 249–256. doi: 10.1111/acps.13219
Jacob, L., Haro, J. M., and Koyanagi, A. (2019). Physical multimorbidity and subjective cognitive complaints among adults in the United Kingdom: a cross-sectional community-based study. Sci. Rep. 9:12417. doi: 10.1038/s41598-019-48894-8
Jessen, F. (2014). Subjective and objective cognitive decline at the pre-dementia stage of Alzheimer’s disease. Eur. Arch. Psychiatry. Clin. Neurosci 264, 3–7. doi: 10.1007/s00406-014-0539-z
Jessen, F., Amariglio, R. E., Van Boxtel, M., Breteler, M., Ceccaldi, M., Chételat, G., et al. (2014). A conceptual framework for research on subjective cognitive decline in preclinical Alzheimer’s disease. Alzheimers Dement 10, 844–852. doi: 10.1016/j.jalz.2014.01.001
Kavčič, V., and Podlesek, A. (2020). “Samoocena anksioznosti med epidemijo COVID-19 v Sloveniji [Self-assessed anxiety during the COVID-19 epidemic in Slovenia],” in Psihologija pandemije: Posamezniki in družba v času koronske krize [Psychology of a Pandemic: Individuals and the Society in Times of Corona Crisis]. eds. Ž. Lep and K. H. Beyazoglu (Ljubljana, Slovenia: Znanstvena založba Filozofske fakultete Univerze v Ljubljani), 105–116.
Kemeny, M. E. (2003). The psychobiology of stress. Curr. Dir. Psychol. Sci. 12, 124–129. doi: 10.1111/1467-8721.01246
Khanade, K., and Sasangohar, F. (2017). Stress, fatigue, and workload in intensive care nursing: a scoping literature review. Proc. Hum. Factors Ergon. Soc. 61, 686–690. doi: 10.1177/1541931213601658
Klainin-Yobas, P., Keawkerd, O., Pumpuang, W., Thunyadee, C., Thanoi, W., and He, H. G. (2014). The mediating effects of coping on the stress and health relationships among nursing students: a structural equation modelling approach. J. Adv. Nurs. 70, 1287–1298. doi: 10.1111/jan.12283
Kocalevent, R. D., Hinz, A., Brähler, E., and Klapp, B. F. (2011). Determinants of fatigue and stress. BMC Res. Notes 4:238. doi: 10.1186/1756-0500-4-238
Krueger, G. P. (1989). Sustained work, fatigue, sleep loss and performance: a review of the issues. Work Stress. 3, 129–141. doi: 10.1080/02678378908256939
Kryla-Lighthall, N., and Mather, M. (2009). “The role of cognitive control in older adults’ well-being,” in Handbook of Theories of Aging. 2nd Edn. eds. V. Bengston, D. Gans, N. Putney, and M. Silverstein (New York: Springer Publishing Company), 323–344.
Lep, Ž., and Hacin Beyazoglu, K. (2020). Psihologija pandemije: Posamezniki in družba v času koronske krize [Psychology of a Pandemic: Individuals and the Society in Times of Corona Crisis]. Ljubljana, Slovenia: Znanstvena založba Filozofske fakultete Univerze v Ljubljani.
Li, H. Y., Cao, H., Leung, D. Y. P., and Mak, Y. W. (2020). The psychological impacts of a COVID-19 outbreak on college students in China: a longitudinal study. Int. J. Environ. Res. Public Health 17:3933. doi: 10.3390/ijerph17113933
Lieb, R., Becker, E., and Altamura, C. (2005). The epidemiology of generalized anxiety disorder in Europe. Eur. Neuropsychopharmacol. 15, 445–452. doi: 10.1016/j.euroneuro.2005.04.010
Liston, C., McEwen, B. S., and Casey, B. J. (2009). Psychosocial stress reversibly disrupts prefrontal processing and attentional control. Proc. Natl. Acad. Sci. U. S. A. 106, 912–917. doi: 10.1073/pnas.0807041106
Liu, N., Zhang, F., Wei, C., Jia, Y., Shang, Z., Sun, L., et al. (2020). Prevalence and predictors of PTSS during COVID-19 outbreak in China hardest-hit areas: Gender differences matter. Psychiatry Res. 287:112921. doi: 10.1016/j.psychres.2020.112921
Maier, W., Gänsicke, M., Freyberger, H. J., Linz, M., Heun, R., and Lecrubier, Y. (2000). Generalized anxiety disorder (ICD-10) in primary care from a cross-cultural perspective: a valid diagnostic entity? Acta Psychiatr. Scand. 101, 29–36. doi: 10.1034/j.1600-0447.2000.101001029.x
Markova, H., Andel, R., Stepankova, H., Kopecek, M., Nikolai, T., Hort, J., et al. (2017). Subjective cognitive complaints in cognitively healthy older adults and their relationship to cognitive performance and depressive symptoms. J. Alzheimers Dis. 59, 871–881. doi: 10.3233/JAD-160970
Marsh, H. W., Wen, Z., and Hau, K. T. (2004). Structural equation models of latent interactions: evaluation of alternative estimation strategies and indicator construction. Psychol. Methods 9, 275–300. doi: 10.1037/1082-989X.9.3.275
Mathews, A., and MacLeod, C. (2005). Cognitive vulnerability to emotional disorders. Annu. Rev. Clin. Psychol. 1, 167–195. doi: 10.1146/annurev.clinpsy.1.102803.143916
Maxwell, S. E., and Cole, D. A. (2007). Bias in cross-sectional analyses of longitudinal mediation. Psychol. Methods 12, 23–44. doi: 10.1037/1082-989X.12.1.23
Miley-Akerstedt, A., Jelic, V., Marklund, K., Walles, H., Åkerstedt, T., Hagman, G., et al. (2018). Lifestyle factors are important contributors to subjective memory complaints among patients without objective memory impairment or positive neurochemical biomarkers for Alzheimer’s disease. Dement. Geriatr. Cogn. Dis. Extra 8, 439–452. doi: 10.1159/000493749
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., and Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis Cogn. Psychol. 41, 49–100. doi: 10.1006/cogp.1999.0734
Möhring, K., Naumann, E., Reifenscheid, M., Wenz, A., Rettig, T., Krieger, U., et al. (2020). The COVID-19 pandemic and subjective well-being: longitudinal evidence on satisfaction with work and family. Eur. Soc. 23(Suppl. 1), S601–S617. doi: 10.1080/14616696.2020.1833066
Najmi, S., Amir, N., Frosio, K. E., and Ayers, C. (2015). The effects of cognitive load on attention control in subclinical anxiety and generalised anxiety disorder. Cognit. Emot. 29, 1210–1223. doi: 10.1080/02699931.2014.975188
Novais, A., Monteiro, S., Roque, S., Correia-Neves, M., and Sousa, N. (2017). How age, sex and genotype shape the stress response. Neurobiol. Stress 6, 44–56. doi: 10.1016/j.ynstr.2016.11.004
Okon-Singer, H., Hendler, T., Pessoa, L., and Shackman, A. J. (2015). The neurobiology of emotion-cognition interactions: fundamental questions and strategies for future research. Front. Hum. Neurosci. 9:58. doi: 10.3389/fnhum.2015.00058
Ouimet, A. J., Gawronski, B., and Dozois, D. J. A. (2009). Cognitive vulnerability to anxiety: a review and an integrative model. Clin. Psychol. Rev. 29, 459–470. doi: 10.1016/j.cpr.2009.05.004
Pallavicini, F., Gaggioli, A., Raspelli, S., Cipresso, P., Serino, S., Vigna, C., et al. (2013). Interreality for the management and training of psychological stress: study protocol for a randomized controlled trial. Trials 14:191. doi: 10.1186/1745-6215-14-191
Paredes, M. R., Apaolaza, V., Fernandez-Robin, C., Hartmann, P., and Yañez-Martinez, D. (2021). The impact of the COVID-19 pandemic on subjective mental well-being: the interplay of perceived threat, future anxiety and resilience. Personal. Individ. Differ. 170:110455. doi: 10.1016/j.paid.2020.110455
Pinto, J., van Zeller, M., Amorim, P., Pimentel, A., Dantas, P., Eusébio, E., et al. (2020). Sleep quality in times of Covid-19 pandemic. Sleep Med. 74, 81–85. doi: 10.1016/j.sleep.2020.07.012
Podlesek, A., and Kavčič, V. (2020). “Stres in samozaznane kognitivne spremembe med epidemijo COVID-19 [Stress and self-perceived cognitive changes during the COVID-19 epidemic],” in Psihologija pandemije: Posamezniki in družba v času koronske krize [Psychology of a Pandemic: Individuals and the Society in Times of Corona Crisis]. eds. Ž. Lep and K. H. Beyazoglu (Ljubljana, Slovenia: Znanstvena založba Filozofske fakultete Univerze v Ljubljani), 143–154.
Podlesek, A., and Kavčič, V. (2021). Generalized anxiety in Slovenian university students during the COVID-19 epidemic. Sodobna Pedagogika 72, 322–341.
Qi, M., Zhou, S.-J., Guo, Z.-C., Zhang, L.-G., Min, H.-J., Li, X.-M., et al. (2020). The effect of social support on mental health in Chinese adolescents during the outbreak of COVID-19. J. Adolesc. Health 67, 514–518. doi: 10.1016/j.jadohealth.2020.07.001
Qin, S., Hermans, E. J., Van Marle, H. J., Luo, J., and Fernández, G. (2009). Acute psychological stress reduces working memory-related activity in the dorsolateral prefrontal cortex Biol. Psychiatry. 66, 25–32. doi: 10.1016/j.biopsych.2009.03.006
Revelle, W. (2015). Package “psych” - Procedures for psychological, psychometric and personality research. R Package, 1–358. Available at: https://personality-project.org/r/psych/ (Accessed December 29, 2020)
Rosseel, Y. (2012). Lavaan: an R package for structural equation modeling. J. Stat. Softw. 48, 1–36. doi: 10.18637/jss.v048.i02
Saraswathi, I., Saikarthik, J., Kumar, K. S., Srinivasan, K. M., Ardhanaari, M., and Gunapriya, R. (2020). Impact of COVID-19 outbreak on the mental health status of undergraduate medical students in a COVID-19 treating medical college: a prospective longitudinal study. PeerJ 8:e10164. doi: 10.7717/peerj.10164
Schneider, T. R. (2004). The role of neuroticism on psychological and physiological stress responses. J. Exp. Soc. Psychol. 40, 795–804. doi: 10.1016/j.jesp.2004.04.005
Schneider, T. R., Rench, T. A., Lyons, J. B., and Riffle, R. R. (2012). The influence of neuroticism, extraversion and openness on stress responses. Stress. Health 28, 102–110. doi: 10.1002/smi.1409
Schneiderman, N., Ironson, G., and Siegel, S. D. (2005). Stress and health: psychological, behavioral, and biological determinants. Annu. Rev. Clin. Psychol. 1, 607–628. doi: 10.1146/annurev.clinpsy.1.102803.144141
Scott, S. B., Sliwinski, M. J., and Blanchard-Fields, F. (2013). Age differences in emotional responses to daily stress: the role of timing, severity, and global perceived stress. Psychol. Aging 28, 1076–1087. doi: 10.1037/a0034000
Scullin, M. K., and Bliwise, D. L. (2015). Sleep, cognition, and normal aging: integrating a half century of multidisciplinary research. Perspect. Psychol. Sci. 10, 97–137. doi: 10.1177/1745691614556680
Sexton, K. A., Norton, P. J., Walker, J. R., and Norton, G. R. (2003). Hierarchical model of generalized and specific vulnerabilities in anxiety. Cogn. Behav. Ther. 32, 82–94. doi: 10.1080/16506070302321
Shields, G. S., Sazma, M. A., and Yonelinas, A. P. (2016). The effects of acute stress on core executive functions: A meta-analysis and comparison with cortisol. Neurosci. Biobehav. Rev. 68, 651–668. doi: 10.1016/j.neubiorev.2016.06.038
Spielberger, C. D. (1966). “Theory and research on anxiety,” in Anxiety and Behavior. ed. C. D. Spielberger (New York: Academic Press), 3–20.
Spitzer, R. L., Kroenke, K., Williams, J. B., and Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med 166, 1092–1097. doi: 10.1001/archinte.166.10.1092
Statistical Office of the Republic of Slovenia (2020). SiStat database. Available at: https://pxweb.stat.si/SiStatData/pxweb/sl/Data/Data/05C4004S.px/ (Accessed December 10, 2020)
Stein, M. B., and Lang, A. J. (2002). “Anxiety and stress disorders: course over the lifetime,” in Neuropsychopharmacology. The Fifth Generation of Progress. eds. K. L. Davis, D. Charney, J. T. Coyle, and C. Nemeroff (Philadelphia, PA: Lippincott Williams & Wilkins) 859–866.
Steinberg, S. I., Negash, S., Sammel, M. D., Bogner, H., Harel, B. T., Livney, M. G., et al. (2013). Subjective memory complaints, cognitive performance, and psychological factors in healthy older adults. Am. J. Alzheimers Dis. Other Dement. 28, 776–783. doi: 10.1177/1533317513504817
Stenfors, C. U. D., Marklund, P., Magnusson Hanson, L. L., Theorell, T., and Nilsson, L.-G. (2013). Subjective cognitive complaints and the role of executive cognitive functioning in the working population: a case-control study. PLoS One 8:e83351. doi: 10.1371/journal.pone.0083351
Tang, F., Liang, J., Zhang, H., Kelifa, M. M., He, Q., and Wang, P. (2020). COVID-19 related depression and anxiety among quarantined respondents. Psychol. Health 36, 164–178. doi: 10.1080/08870446.2020.1782410
Taylor, A. H., and Dorn, L. (2006). Stress, fatigue, health, and risk of road traffic accidents among professional drivers: the contribution of physical inactivity. Annu. Rev. Public Health 27, 371–391. doi: 10.1146/annurev.publhealth.27.021405.102117
Thorsteinsson, E. B., Brown, R. F., and Owens, M. T. (2019). Modeling the effects of stress, anxiety, and depression on rumination, sleep, and fatigue in a nonclinical sample. J. Nerv. Ment. Dis. 207, 355–359. doi: 10.1097/NMD.0000000000000973
Thorell, L. B., Lazarević, N. F., Milovanović, I., and Bugarski Ignjatović, V. (2020). Psychometric properties of the Teenage Executive Functioning Inventory (TEXI): A freely available questionnaire for assessing deficits in working memory and inhibition among adolescents. Child Neuropsychol. 26, 857–864. doi: 10.1080/09297049.2020.1726885
Trabelsi, K., Ammar, A., Masmoudi, L., Boukhris, O., Chtourou, H., Bouaziz, B., et al. (2021). Globally altered sleep patterns and physical activity levels by confinement in 5056 individuals: ECLB COVID-19 international online survey. Biol. Sport 38, 495–506. doi: 10.5114/biolsport.2021.101605
Trizano-Hermosilla, I., and Alvarado, J. M. (2016). Best alternatives to Cronbach’s alpha reliability in realistic conditions: congeneric and asymmetrical measurements. Front. Psychol. 7:769. doi: 10.3389/fpsyg.2016.00769
Twenge, J. M., and Joiner, T. E. (2020). U.S. Census Bureau-assessed prevalence of anxiety and depressive symptoms in 2019 and during the 2020 COVID-19 pandemic. Depress. Anxiety 37, 954–956. doi: 10.1002/da.23077
Tyng, C. M., Amin, H. U., Saad, M. N. M., and Malik, A. S. (2017). The influences of emotion on learning and memory. Front. Psychol. 8:1454. doi: 10.3389/fpsyg.2017.01454
Uchino, B. N. (2009). Understanding the links between social support and physical health: a life-span perspective with emphasis on the separability of perceived and received support. Perspect. Psychol. Sci. 4, 236–255. doi: 10.1111/j.1745-6924.2009.01122.x
Valerio, T. D., Kim, J., Sexton-Radek, K., and Kim, M. J. (2016). Association of stress, general health, and alcohol use with poor sleep quality among U.S. college students. Am. J. Health Educ. 47, 17–23. doi: 10.1080/19325037.2015.1111173
Vallat-Azouvi, C., Pradat-Diehl, P., and Azouvi, P. (2012). The Working Memory Questionnaire: A scale to assess everyday life problems related to deficits of working memory in brain injured patients. Neuropsychol. Rehabil. 22, 634–649. doi: 10.1080/09602011.2012.681110
Van Reeth, O., Weibel, L., Spiegel, K., Leproult, R., Dugovic, C., and Maccari, S. (2000). Interactions between stress and sleep: from basic research to clinical situations. Sleep Med. Rev. 4, 201–219. doi: 10.1053/smrv.1999.0097
Wang, Z. H., Yang, H. L., Yang, Y. Q., Liu, D., Li, Z. H., Zhang, X. R., et al. (2020). Prevalence of anxiety and depression symptom, and the demands for psychological knowledge and interventions in college students during COVID-19 epidemic: a large cross-sectional study. J. Affect. Disord. 275, 188–193. doi: 10.1016/j.jad.2020.06.034
Warttig, S. L., Forshaw, M. J., South, J., and White, A. K. (2013). New, normative, English-sample data for the short form perceived stress scale (PSS-4). J. Health Psychol. 18, 1617–1628. doi: 10.1177/1359105313508346
Xiao, H., Zhang, Y., Kong, D., Li, S., and Yang, N. (2020). Social capital and sleep quality in individuals who self-isolated for 14 days during the coronavirus disease 2019 (COVID-19) outbreak in January 2020 in China. Med. Sci. Monit. 26:e923921. doi: 10.12659/MSM.923921
Xin, M., Luo, S., She, R., Yu, Y., Li, L., Wang, S., et al. (2020). Negative cognitive and psychological correlates of mandatory quarantine during the initial COVID-19 outbreak in China. Am. Psychol. 75, 607–617. doi: 10.1037/amp0000692
Yiend, J. (2010). The effects of emotion on attention: a review of attentional processing of emotional information. Cognit. Emot. 24, 3–47. doi: 10.1080/02699930903205698
Zavecz, Z., Nagy, T., Galkó, A., Nemeth, D., and Janacsek, K. (2020). The relationship between subjective sleep quality and cognitive performance in healthy young adults: Evidence from three empirical studies. Sci. Rep. 10:4855. doi: 10.1038/s41598-020-61627-6
Keywords: COVID-19, SARS-CoV-2, stress, emotions, cognitive complaints, physical health, cognitively normal adults
Citation: Podlesek A, Komidar L and Kavcic V (2021) The Relationship Between Perceived Stress and Subjective Cognitive Decline During the COVID-19 Epidemic. Front. Psychol. 12:647971. doi: 10.3389/fpsyg.2021.647971
Edited by:
Vasileios Maroulas, The University of Tennessee, Knoxville, United StatesReviewed by:
Antonia Meyer, University Hospital of Basel, SwitzerlandHamdi Chtourou, University of Sfax, Tunisia
Khaled Trabelsi, University of Sfax, Tunisia
Copyright © 2021 Podlesek, Komidar and Kavcic. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Anja Podlesek, anja.podlesek@ff.uni-lj.si