Alexandra Laurent1,2*
Florent Lheureux3
Magali Genet3
Maria Cruz Martin Delgado4Maria G. Bocci5
Alessia Prestifilippo6Guillaume Besch7Gilles Capellier8- 1Le Laboratoire de Psychologie: Dynamiques Relationnelles Et Processus Identitaires (Psy DREPI), University of Bourgogne Franche-Comté, Dijon, France
- 2La Maison des Sciences de l'Homme et de l'Environnement (MSHE) C. N. Ledoux, University of Bourgogne Franche-Comté, Besançon, France
- 3Laboratory of Psychology, University of Bourgogne Franche-Comté, Besançon, France
- 4Intensive Care Unit, Hospital Universitario de Torrejón, Madrid, Spain
- 5Department of Anesthesiology and Intensive Care, Fondazione Policlinico Universitario Agostino Gemelli IRCCS, Rome, Italy
- 6Fondazione Policlinico Universitario Agostino Gemelli IRCCS, Rome, Italy
- 7Department of Anesthesiology and Intensive Care Medicine, University Hospital of Besançon, University of Bourgogne Franche-Comté, Besançon, France
- 8Medical Intensive Care Unit, University Hospital of Besançon, Besançon, France
Background: Many studies have been conducted in intensive care units (ICUs) to identify the stress factors involved in the health of professionals and the quality and safety of care. The objectives are to identify the psychometric scales used in these studies to measure stressors and to assess their relevance and validity/reliability.
Methods: All peer-reviewed full-text articles published in English between 1997 and 2016 and focusing on an empirical quantitative study of job stressors were identified through searches on seven databases and editorial portals.
Results: From the 102 studies analyzed, we identified 59 different scales: 17 “all settings scales” (16 validated scales), 20 “healthcare settings scales” (13 validated scales), and 22 “ICU settings scales” (two validated scales). All these scales used measured stressors from at least one of the following eight broad categories: High job demands, Problematic relationships with other professionals, Lack of control over work situations and career, Lack of organizational resources, Problematic situations with users and relatives, Dealing with ethical- and moral-related situations, Risk management issues, and Disadvantages in comparison to other occupational situations. The “all settings scales” and “healthcare settings scales,” the most often validated, did not measure, or only slightly measured, the stressors most specific to ICUs. Where these were taken into account, the authors were forced to develop their own tools or modify existing scales without testing the validity of the tool used.
Conclusions: This review highlights the lack of a tool that meets both the criteria of validity and relevance with regard to the specificity of work in ICUs. Future research must focus on developing reliable/valid tools covering all types of relevant stressors to ensure the quality of the studies carried out in this field.
Introduction
Intensive care unit (ICU) professionals must deal with patients with serious medical conditions that require complex diagnostic and therapeutic procedures. This often requires a considerable level of coordination of human resources. Furthermore, end-of-life decisions are frequent and contribute to an intense emotional charge (Teixeira et al., 2014; Flannery et al., 2016). The ICU is thus fertile ground for the emergence of professional stressors (Donchin and Seagull, 2002; Embriaco et al., 2007a). Assessed by individuals as situations that weaken or are beyond their resources (Lazarus and Folkman, 1984), work-related stressors impact the mental and physical health of workers and the quality and safety of care (Sochalski, 2004; de Cássia Pereira Fernandes et al., 2016; Krämer et al., 2016; Dragano et al., 2017; Vandevala et al., 2017).
To assess these professional stressors, various surveys have been developed that cover either generic scales addressing all professional activities or more specific scales focusing primarily on the healthcare field or on a specific healthcare sector. These healthcare scales are used to measure specific stressors such as “end-of-life decisions” (Ozden et al., 2013; Teixeira et al., 2014), “conflicts,” or the “health culture” (Profit et al., 2014; Garrouste-Orgeas et al., 2015) particularly studied for their involvement in burnout or anxiety–depression.
While the identification of professional stressors is in line with the promotion of well-being at work and the quality of healthcare, the reliability of the psychometric tools is also of great importance. However, it appears that the multiplication of stress assessment tools makes it difficult to choose the most appropriate scale. In this sense, Bonneterre et al. (2008) denounce the use of unsuitable tools or the use of tools with insufficient psychometric qualities, which make the predictive validities between stressors and epidemiological indicators unreliable. Therefore, when researchers or clinicians wish to evaluate the stressors present in the ICU, what types of tools can they find in the literature? First, are these tools psychometrically valid? In addition, are they able to measure all relevant stressors, including those most specific to ICUs? This latter question deals with the issue of ecological validity. The ecological validity concept examined whether a study and its findings are representative of real-life situations (Brewer and Crano, 2014). In particular, ecologically valid studies used material and procedure that satisfy three parameters (Schmuckler, 2001): they reflect situations or events that can actually occur in participants' everyday lives, reproduce or they refer to features of participants' current living environments, and stimulate from participants reactions that are already available in their response repertoire. Applied to the measurement of stressors in ICUs, an ecologically valid tool should cover all types of situations actually occurring in ICUs that are likely to cause stress to professionals, with item content that refers to their current working environment (e.g., service functioning, patients and families, tasks to perform). Thus, the ideal tool would respect these principles in addition to present good psychometric properties (e.g., reliability, construct validity). But is it possible to find such an ideal tool in peer-reviewed journals?
To answer these questions, our literature review was guided by the following research objectives:
1) To comprehensively identify the scales and questionnaires that have been used to date to measure perceived job stressors in the ICU (Are there few or numerous measurement tools in the peer-reviewed literature?).
2) To determine the most frequently and least frequently used scales/questionnaires (Is there a consensual use of one or several tools? If so, what may explain this?).
3) To critically examine the ecological validity of these studies, i.e., their ability to take into account all stressors relevant to ICU settings; this means specific stressors as well as more general stressors such as job demands, lack of social support, etc.
4) To critically examine their basic psychometric reliability/validity as evidenced by the use of suitable methodological and statistical procedures such as factor analysis, Cronbach's alpha, etc.
Methods
Study Design
We considered all peer-reviewed full-text articles reporting an empirical study, a literature review, or a meta-analysis published in English between 1997 and 2016. The choice of article in English corresponds to the objective of reporting on the most commonly used stress scales in intensive care at an international level. In this sense, English publications, the common language of international researchers and practitioners, include the largest number of journals with high international visibility. Only the empirical studies that used a quantitative methodology to explicitly measure job/occupational stress factors for healthcare professionals in ICUs were selected.
Noncompliant articles (commentaries, case reports, posters, and editorials) simply reporting perceived stress or focusing on patients instead of healthcare providers were excluded.
Search Strategy and Study Eligibility
The search for and selection of articles were conducted between March and May 2017 (according to PRISMA guidelines). Seven databases and editorial portals (Medline via Pubmed; PsycInfo and Psychology, and behavioral science collection via EBSCOhost; Elsevier's ScienceDirect, SpringerLink, Sage Journals, and Wiley Online Library) were screened using the following terms in the title, the abstract, or the keywords: (intensive OR critical) AND care AND (professional OR job OR work OR occupational) AND (stress OR stressors OR burnout). The references of 10 literature review articles (Dunser et al., 2006; Embriaco et al., 2007b; Fassier and Azoulay, 2010; Adriaenssens et al., 2015; Karanikola et al., 2015; Van Mol et al., 2015; Chuang et al., 2016; Flannery et al., 2016) were manually screened to identify other relevant studies that had not been initially retrieved.
The initial search, conducted by MG, identified 1,330 records (Figure 1), and 34 were added following a manual search in the references of the articles retrieved. Duplicates (n = 230), articles in a language other than English (n = 328), or those not conforming to the search criteria (n = 110) were removed. The abstracts were then screened for eligibility separately by two authors (MG and FL). When disagreement or the need for further analysis arose, the entire article was (n = 96) examined and discussions were held between AL, FL, MG, MM, MB, and AP to reach an agreement. From the 696 articles thus selected, 594 were excluded as they did not focus on stress in the ICU (n = 207), did not measure stressors (n = 276), did not include healthcare providers (n = 33), did not use a quantitative methodology (n = 77), or could not be retrieved (n = 1). In total, our review focused on 102 empirical studies (references in Supplementary Material).
Figure 1. Flow diagram of selected articles based on relevance search criteria.
Data Extraction and Criteria Used to Assess the Quality and the Relevance of Existing Scales in Intensive Care Unit Contexts
Following the selection of the 102 relevant articles, data were extracted and coded by AL and FL for each scale according to five criteria:
(a) Identification of the scale used and the number of articles that used it.
(b) The origin and metrological reliability/validity. At that level, we made a distinction between ante hoc and ad hoc scales, with subdivisions within these two broad categories of scales. This is because “previously-validated scales are generally preferable to ad hoc scales” (Furr, 2011, p. 8) given that the quality assessment of ad hoc scales rarely goes beyond face validity as perceived by the researchers (i.e., lack of independent evidence of validity from other sources such as experts or participants). Furthermore, the use of ad hoc measures reduces the comparability of studies as it introduces a possible confounding factor. Moreover, they generally lack a complete inspection of other forms of validity and reliability, as estimated by internal consistency coefficients (e.g., Cronbach's alpha), high test–retest correlation coefficients, theory-consistent and interpretable factor structure without problematic loadings, or correlations with similar measures (convergent validity) or logically related phenomena (nomological validity). In this respect, “rigorously developed measures have a lower probability of being based on chance or method variance than ad hoc measure” Peter and Churchill (1986, p. 3). Finally, ante hoc scales offer more the guarantee that they were elaborated independently from any hypothesis-testing purpose (i.e., absence of hypothesis confirmation bias in the generation of items). In addition, we also consider whether ante hoc validated scales were modified—or not—by ICU researchers. In fact, as pointed out by Furr (2011, p. 9) “well-validated original scales are preferable to modified scales. Because a modified scale's psychometric properties and quality might differ from those of the original scale, the modified scale is—to some degree—an ad hoc scale. As such, its psychometric properties and quality are unclear and suspect.” Nevertheless, because some authors using an ad hoc scale have provided validity-related information regarding the origin of items and/or basic statistics (generally Cronbach's alpha and sometimes factor analysis results), we differentiated ad hoc scales according to the presence or absence of such information. As a result of the combination of these criteria, we differentiated five types of scales ranging from the probably more reliable/valid to the less reliable/valid: ante hoc validated scales (Type 1); ante hoc validated scales with ad hoc modifications (items removed or rewritten) (Type 2); ad hoc scales with validity-related information regarding the origin of items and basic statistics (Type 3); ad hoc scales with validity-related information regarding the origin of items only or report of basic statistics only (Type 4); ad hoc scales without report of any validity-related information (Type 5).
(c) The target population and the level of generality or specificity according to three levels: (1) scales designed to measure stressors in all work settings (i.e., all settings scales); (2) scales specific to healthcare providers from several types of clinical units/settings (i.e., healthcare settings scales); (3) scales tailored to measure stressors specific to ICU settings.
(d) The number of items and, when relevant and available, their subscales.
(e) The types of stressors measured. An in-depth assessment of the subscales and items enabled us to develop a typology of all stressors that had been measured by the different scales. FL, AL, and MG examined in detail each scale with regard to its structure (presence and labels of subscales) and item content and subsequently made iterative comparisons between them to identify the broad types and subtypes of stressors measured when considering all scales together. The choice of an inductive methodology to define the typology of stressors made it possible to allow oneself not to be limited to a typology preconceived by a theoretical model, which would risk not covering all the stressors explored in the literature. This typology was then critically examined by AL, GC, and GB and revised subsequently to reach a consensus. Then, each scale was assessed using a binary format “yes/no presence” for each type and subtype of stressors. This criterion was especially considered to assess the ecological validity of scales. Because ICU professionals can be jointly exposed to widely common stressors (e.g., workload, lack of support from colleagues), stressors shared with many other health professions (e.g., lack of recognition from patients and families, administrative hassles), and stressors more typical of ICUs (e.g., end-of-life decisions, constant monitoring of critically ill patients), we consider that the more a scale covers exhaustively the different types and subtypes of stressors, the more it was considered as ecologically valid.
Results
What Types of Scales Are Used in the Intensive Care Unit?
From the 102 studies analyzed in our literature review, we identified 59 different scales according to three main categories (Table 1): 17 “generic scales” used in all sectors of professional activities, 20 “healthcare scales” used in the field of human health, and 22 scales specific to ICU settings. Only 28 out of the 59 scales were Type 1 scales (ante hoc validated). The majority of them were generic scales (13/17) or healthcare settings scales (13/20); there were only two ICU-specific scales (2/22).
Table 1. Scale description according to assessment criteria.
Overall, among the 102 studies seeking to identify stressors in the ICU, 36 used a more or less problematic scale in terms of validity (Types 2–5).
Types of Stressors Assessed in Intensive Care Unit Studies
Eight major types of stressors, grouping 58 subtypes, have been identified (see Table 2; for the coding of each scale, see Table 1): (1) High job demands (40 scales); (2) Problematic relationships with other professionals (39 scales); (3) Lack of control over work situations and career (31 scales); (4) Lack of organizational resources (29 scales); (5) Problematic situations with users and relatives (20 scales); (6) Dealing with ethical and moral-related situations (19 scales); (7) Risk management issues (14 scales); (8) Disadvantages in comparison to other occupational situations (11 scales). Two types of stressors do not fall within these eight main categories and have been classified as “other” (measured by three different scales).
Table 2. Broad types and subtypes of stressors measured by each type of scales used in ICU studies.
The High job demands category refers to taxing task-, workflow-, and role-related situations and distinguishes between eight subtypes. The problematic relationships with other professionals category includes 11 subtypes of stressors related to the lack of support, communication/collaboration problems, or conflicts with colleagues, superiors, or other departments. The lack of control over work situations and career category includes the different control and skill-related factors which may prevent professionals from being responsive, efficient, and committed. This category of stressors includes eight subtypes. The lack of organizational resources category includes nine subtypes of resources the healthcare organization might fail to sufficiently provide to promote the quality of life and the quality of care of ICU professionals. The problematic situations with service users and relatives category includes six subtypes of stressors which all involve one or more people whose behavior or health condition is likely to exceed the resources of healthcare professionals. The dealing with ethical- and moral-related situations category refers to eight subtypes of situations where patients' well-being and safety are likely to be harmed by unsuitable decisions/actions of the ICU team. The risk management issues category refers to risky situations for both patients and professionals with a substantial probability of making critical errors (three subtypes included). Lastly, the disadvantages in comparison to other occupational situations category of stressors concerns the extent to which the professional considers his/her current occupational situation is disadvantaged in comparison to other career choices (e.g., other jobs, other employers, other professions) (three measured subtypes).
Ability of Scales to Take Into Account the Specificity of Intensive Care Unit Stressors
There are important differences concerning the extent to which the three types of scales (i.e., all settings, healthcare settings, and ICU settings scales) address the broad range of the types and subtypes of stressors mentioned above. To highlight these differences, we compared the proportion of each type of scale that measured a given stressor relative to the overall number of scales that measured it. However, given that there were more ICU settings scales (N = 22) than healthcare settings scales (N = 20) and all settings scales (N = 17), we applied a correction to make the comparisons valid1.
As illustrated by Figure 2, high job demands (52%), disadvantages in comparison to other occupational situations (55%), and lack of control over work situations and career (53%) were more frequently measured by the all settings scales than by the other two scales (ranging from 15 to 32%). However, all settings scales were rarely used to measure risk management issues (24%) and problematic situations with service users and relatives (18%) and were hardly used to measure dealing with ethical- and moral-related situations (3%); conversely, ICU settings scales were used more frequently to measure these broad types of stressors (51, 45, and 72%, respectively). Healthcare settings scales were used to measure problematic situations with service users and relatives (37%) almost as often as ICU settings scales, but less frequently to measure risk management issues (25%) and dealing with ethical- and moral-related situations (26%). The three types of scales measured lack of organizational resources (30–38%) and problematic relationships with other professionals (31–37%) to the same extent.
Figure 2. Comparison of the three types of scales regarding their propensity to measure the eight broad types of stressors (% is the corrected proportion of scales of a type covering at least one stressor in the target category).
Beyond these eight broad types of stressors, there were a number of important differences concerning their subtypes. For instance, some stressors were more frequently measured by ICU scales than by the other two scales: decisional dilemmas/uncertainty regarding patients' survival and end-of-life care (100%), unsuitability of care (futility or over/under aggressiveness of therapeutics) (86%), conflict and inappropriate expectations or behavior from relatives (82%), or constant alert and sudden emergencies due to the patient's condition (65%).
In addition, in terms of item content, ICU settings scales and, to a lesser extent, healthcare settings scales generally refer more to precise and tailored features of the working environment of ICU professionals than all settings scales.
Scales Most Frequently Used in Intensive Care Unit Studies and Comparison of Their Metrological Properties and Specificities
Approximately two thirds of the retrieved studies used at least one of the eight following scales (out of the 59 scales identified) (Table 1): 17 studies used the Job Content Questionnaire (JCQ) or one of its variants (Karasek et al., 1998; Piers et al., 2011); 15 studies used the Moral Distress Scale (MDS) or one of its variants (Corley et al., 2001; Hamric et al., 2012); eight studies used the Nursing Stress Scale (NSS) (Gray-Toft and Anderson, 1981); six studies used the Safety Attitudes Questionnaire–ICU version (SAQ-ICU) (Sexton et al., 2006); six studies used the Nursing Work Index (NWI) or one of its variants (Kramer and Hafner, 1989; Aiken and Patrician, 2000; Lake, 2002; Bonneterre et al., 2011); four studies used the Nurses Stress Index (NSI) (Harris, 1989); four studies used the scale of Embriaco et al. (2007a); and three studies used the Effort–Reward Imbalance (ERI) questionnaire (Siegrist et al., 2004). Other scales such as the Daily Hassles Questionnaire (DHQ) (Kanner et al., 1981), Workplace Stress Scale (WSS) (The Marlin Company and the American Institute of Stress, 2011), Hospital Survey Of Patient Safety Culture (HSOPSC) (Sorra and Dyer, 2010), Copenhagen Psychosocial Questionnaire V2 (COPSOQ2) (Pejtersen et al., 2010), ICU-related stressors questionnaire (Coomber et al., 2002), and the scales of Tummers et al. (2002) or Teixeira et al. (2013) were used in two studies each. Finally, the 44 other scales were used in one article each, including 10 all settings scales (four Type 1 and four Type 2 scales, Rizzo et al., 1970; Kanner et al., 1978; Cooper and Williams, 1991; Dhondt and Houtman, 1992; Jackson et al., 1993; Veldhoven and Meijman, 1994; Semmer et al., 1995; Shimomitsu et al., 2000; Stinglhamber and Vandenberghe, 2003; Hart, 2006), 15 healthcare settings scales (10 Type 1 scales, Hinshaw and Atwood, 1985; Revicki et al., 1991; Miranda et al., 1997; O'Brien-Pallas et al., 1997; Tyssen et al., 2001; Kitaoka-Higashiguchi and Nakagawa, 2003; Varjus et al., 2003; Olson, 2007; Einarsen et al., 2009; one Type 3, three Type 4, and two Type 5 scales), and 19 ICU settings scales (one Type 1, Bailey et al., 1980; five Type 3, six Type 4, and six Type 5 scales).
An analysis of the 59 scales reveals that no scale covered all the main stressors in ICUs (Table 2), not even those with satisfactory metrological properties. For instance, the JCQ, the most used across the world, is a Type 1 scale (ante hoc with satisfying validity-related statistics) which has primarily been used to measure the following types of stressors: lack of control over work situations and career, high job demands, and problematic relationships with other professionals. However, this scale does not cover stressors such as dealing with ethical- and moral-related situations, problematic situations with service users and relatives, and risk management issues and measures only one stressor in the broad category of the lack of organizational resources (note: the full 49-item JCQ also measures Lack of job security, a stressor associated with the disadvantages in comparison with other occupational situations category, but this version was not used in the retrieved ICU studies). Furthermore, four of the 17 studies retrieved combined it with a healthcare settings scale [NSS, NWI, Nine Equivalent of nursing Manpower use (NEMS)] or an ICU settings scale such as the “Perceived ethical environment questionnaire” and the “Perceived inappropriateness of care questionnaire” developed by Piers et al. (2011).
The second most frequently used scale in ICU studies—the MDS—also presents certain limitations regarding its coverage of the different types of stressors. It is a Type 1 scale, but it is tailored to measure stressors relevant in many healthcare settings which focus primary on stressors related to dealing with ethical- and moral-related situations. It also measures stressors relating to the lack of control over work situations and career, as well as one stressor relating to high job demands. However, this scale does not measure stressors relating to the following categories: problematic relationships with other professionals, problematic situations with service users and relatives, lack of organizational resources, risk management issues, and disadvantages in comparison to other occupational situations. This led authors of two out of the 16 studies which used the MDS to combine it with either a healthcare settings scale (Varjus et al., 2003) or an ad hoc ICU-specific questionnaire (Hamric and Blackhall, 2007) to measure additional stressors.
Similar observations can be made with regard to the third most frequently used scale, i.e., the NSS. This Type 1 scale specifically targets nurses (i.e., it is unsuitable for other healthcare professions). It primarily covers stressors from four broad types: high job demands, lack of organizational resources, problematic relationships with other professionals, and problematic situations with service users and relatives. However, it measures one stressor from the lack of control over work situations and career category to a limited extent and none from the categories dealing with ethical- and moral-related situations, risk management issues, and disadvantages in comparison with other occupational situations.
The same comments were made for all other scales. No scale covered the eight broad types of stressors in a comprehensive manner, and many of them were problematic in terms of validity/reliability. It is worth mentioning that 14 articles analyzed combined two or three scales to increase stressor coverage.
Discussion
From the 102 studies analyzed, we identified 59 different scales. Only 28 out of the 59 scales were validated (Type 1), and two ICU settings scales out of 22 were validated. Our review of the literature highlights the wide variability across scales used to identify stressors in the ICU with regard to their level of generality/specificity (scales for all types of professional contexts or scales targeting healthcare contexts or more specifically ICU professionals), their psychometric qualities (five levels of validity/reliability), and the type of stressors covered by scales.
This variability sheds light on the constraints that appear to guide investigators' methodological choices. The advantage of using a generic scale validated internationally is that authors are able to carry out epidemiological studies allowing interprofessional and international comparisons (El Khamali et al., 2018). However, generic scales, primarily used in studies, appear ill adapted to measure the stressors more specific to the professional activity in ICUs. Indeed, the stressors in the categories Dealing with ethical- and moral-related situations and Risk management issues, represented primarily in specific scales, did not appear or were relatively few within the generic and healthcare scales.
For instance, the scale most commonly used by the authors was the JCQ scale, despite the fact that no factor related to dealing with ethical- and moral-related situations or risk management issues was covered. Many authors, however, raised the question of the importance of the issues related to risk management and patient safety in relation to high-tech care and the severity of the pathology (Aiken and Patrician, 2000; Adriaenssens et al., 2015). The difficulties associated with ethical dimensions have also been extensively studied in the ICU context in relation to end-of-life situations (Laurent et al., 2017). The SAQ-ICU scale combines interesting criteria to identify stress factors in ICU settings. The scale has been validated and covers factors relative to the problematic relationships with other professionals and lack of organizational resources categories. However, factors relating to stressors such as job disadvantages, problematic situations with users and relatives, high job demands, and risk management issues are absent, despite the fact that the latter two factors of stress have been widely reported by ICU professionals (Pastores et al., 2019).
Thus, investigators who chose to use a scale reflecting these more specific stressors of ICUs were forced to develop their own tools whose psychometric qualities were yet to be tried and tested (tools based on literature reviews, interviews with professionals, or items from different scales). Beyond the measurement level, these observations highlight the crucial limitation of the uncritical use of general theories of occupational stress (e.g., the job strain model of Karasek, the effort–reward imbalance model of Siegrist) to analyze job stressors in ICUs. Indeed, as advocated by a number of other scholars (Borteyrou et al., 2014), these theories/models of stress must be contextualized to enhance their ecological validity as they only account for generic stressors in professional settings and not the specific ones.
Our systematic review shows that a number of factors are absent from all the scales, for instance, diagnostic/admission decisions, the training and supervision of students, the lack of space related to rooms for break or family discussions (Blanch et al., 2016; Trevick et al., 2016; Pastores et al., 2019). We noted that the healthcare and ICU scales targeted either all healthcare professionals or nurses and that there was no scale developed specifically for physicians.
It is worth nothing that some scales allow the assessment of the effects on outcomes (e.g., job burnout) of the interaction between some types of stressors. This is the case of the JCQ and its variants which can estimate whether resources like job control (decision latitude) and social support (from colleagues and supervisors) moderate the effect of job demands. This is also the case of the ERI scale, which is based on the postulate that the imbalance (interaction) between the rewards obtained in exchange of job efforts is critical to explain outcomes, more than their isolated main effects. However, accounting for these interaction effects generally did not increase the explained variance of outcomes in empirical studies (e.g., Brough and Biggs, 2015; Gorgievski et al., 2019).
This study has a number of limitations. Several studies, and therefore stress scales, may not have been considered in our review if the objective of identifying ICU stressors was not clearly reflected in the article title, keywords, or abstract. In addition, we have focused mainly on studies published in English. Thus, scales developed at a more local level may have been excluded from our review, and a more extensive search seems warranted to cover more the important issue of international, intercultural, and health-system comparisons in terms of stressor identification and measurement.
Conclusion
Our review of the literature clearly raises the question of the relevance of the scales used in studies measuring stressors in intensive care settings. Indeed, no available tool meets both the criteria of metrological validity and of ecological validity (i.e., covering all relevant stressors in ICUs, particularly those that are the most specific to them). Thus, researchers and practicians currently face a methodological dilemma, as they are forced to make a choice between the two or to make some unfortunate “bricolage,” such as ad hoc elaboration, removal, or modification of items or the combination of different scales.
There is an urgent need to propose a validated tool capable of taking into account the whole professional reality of ICU settings. This tool would make it possible to compare the respective impacts of generic vs. specific stressors in the etiology of outcomes (e.g., burnout, job satisfaction, turnover intentions). A better identification of stress factors should make it possible to define a more appropriate care policy in particular to prevent burnout and its associated effects such as depression, suicidal tendencies, addictive behaviors, and physical impairment (e.g., Lheureux et al., 2016; Vandevala et al., 2017). Finally, the more comprehensive and thus more able to approximate the reality of the activities undertaken by professionals the tool is, the easier it will be to use the factors identified to implement effective training needs and target the necessary prevention and support measures.
Data Availability Statement
All datasets generated for this study are included in the article/Supplementary Material.
Author Contributions
AL: abstracts screened for eligibility, screened full text sectioned, examination in detail of each scale, examination of typology, and writing of the article. FL: abstracts screened for eligibility, examination in detail of each scale, and writing of the article. MG: initial search and examination in detail of each scale. MM: screened full text sectioned and proofreading. MB: screened full text sectioned and proofreading. AP: screened full text sectioned and proofreading. GB: examination of typology and proofreading. GC: examination of typology and proofreading.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2020.00245/full#supplementary-material
Footnote
1. ^The formula was: (N scales of a given type that measure the stressor/Overall N scales that measure the stressor)/(N all settings scales that measure the stressor/17) + (N healthcare settings scales that measure the stressor/20) + (N ICU settings scales that measure the stressor/22). For instance, there were nine scales that measured the “taxing work environment (noisy, hectic, crowded, heated, etc.)” stressor type, including three all settings scales, three healthcare settings scales, and three ICU settings scales. Without the correction, this stressor appeared to be measured to the same extent by the three types of scales (3/9). However, this conclusion is incorrect because the ratio 3/17 (17.65%) for all settings scales is higher than the ratio of 3/20 (15%) for healthcare settings scales and higher than the ratio of 3/22 (13.63%) for ICU settings scales. After the correction which took into account the unequal number of scale types (17 vs. 20 vs. 22), it emerged that this stressor was measured slightly more frequently by all settings scales [(3/17)/((3/17) + (3/20) + (3/22)) = 38%] than by healthcare settings scales [(3/20)/((3/17) + (3/20) + (3/22)) = 35%] and ICU settings scales [(3/22)/((3/17) + (3/20) + (3/22)) = 29%].
References
Adriaenssens, J., De Gucht, V., and Maes, S. (2015). Determinants and prevalence of burnout in emergency nurses: a systematic review of 25 years of research. Int. J. Nurs. Stud. 52, 649–661. doi: 10.1016/j.ijnurstu.2014.11.004
Aiken, L. H., and Patrician, P. A. (2000). Measuring organizational traits of hospitals: the revised nursing work index. Nurs. Res. 49, 146–153. doi: 10.1097/00006199-200005000-00006
Bailey, J. T., Steffen, S. M., and Grout, J. W. (1980). The stress audit: identifying the stressors of ICU nursing. J. Nurs. Educ. 19, 15–25.
Blanch, L., Abillama, F. F., Amin, P., Christian, M., Joynt, G. M., Myburgh, J., et al. (2016). Triage decisions for ICU admission: Report from the Task Force of the World Federation of Societies of Intensive and Critical Care Medicine. J. Crit. Care 36, 301–305. doi: 10.1016/j.jcrc.2016.06.014
Bonneterre, V., Ehlinger, V., Balducci, F., Caroly, S., Jolivet, A., Sobaszek, A., et al. (2011). Validation of an instrument for measuring psychosocial and organisational work constraints detrimental to health among hospital workers: The NWI-EO questionnaire. Int. J. Nurs. Stud. 48, 557–567. doi: 10.1016/j.ijnurstu.2010.09.006
Bonneterre, V., Liaudy, S., Chatellier, G., Lang, T., and de Gaudemaris, R. (2008). Reliability, validity, and health issues arising from questionnaires used to measure psychosocial and organizational work factors (POWFs) among hospital nurses: a critical review. J. Nurs. Meas. 16, 207–230. doi: 10.1891/1061-3749.16.3.207
Borteyrou, X., Truchot, D., and Rascle, N. (2014). Development and validation of the work stressor inventory for nurses in oncology: preliminary findings. J. Adv. Nurs. 70, 443–453. doi: 10.1111/jan.12231
Brewer, M. B., and Crano, W. D. (2014). “Research design and issues of validity,” in Handbook of Research Methods in Social and Personality Psychology, 2nd edn., eds H. T. Reis and C. M. Judd (New York, NY: Cambridge University Press), 11–26. doi: 10.1017/CBO9780511996481.005
Brough, P., and Biggs, A. (2015). Job demands × job control interaction effects: do occupation-specific job demands increase their occurrence? Stress Health 31, 138–149. doi: 10.1002/smi.2537
Chuang, C. H., Tseng, P. C., Lin, C. Y., Lin, K. H., and Chen, Y. Y. (2016). Burnout in the intensive care unit professionals. Medicine 95:e5629. doi: 10.1097/MD.0000000000005629
Coomber, S., Todd, C., Park, G., Baxter, P., Firth-Cozens, J., and Shore, S. (2002). Stress in UK intensive care unit doctors. Br. J. Anaesth. 89, 873–881. doi: 10.1093/bja/aef273
Cooper, C. L., and Williams, J. (1991). A validation study of the OSI on a blue-collar sample. Stress Med. 7, 109–112. doi: 10.1002/smi.2460070210
Corley, M. C., Elswick, R. K., Gorman, M., and Clor, T. (2001). Development and evaluation of a moral distress scale. J. Adv. Nurs. doi: 10.1037/t37284-000
de Cássia Pereira Fernandes, R., da Silva Pataro, S. M., de Carvalho, R. B., and Burdorf, A. (2016). The concurrence of musculoskeletal pain and associated work-related factors: a cross sectional study. BMC Pub. Health 16:628. doi: 10.1186/s12889-016-3306-4
Dhondt, S., and Houtman, I. (1992). NIPG Onderzoeksvragenlijst Arbeidsinhoud: Constructie en Eerste Toets op Betrouwbaarheid en Validiteit. Leiden: TNO, rapportnummer 92.088.
Donchin, Y., and Seagull, F. J. (2002). The hostile environment of the intensive care unit. Curr. Opin. Crit. Care 8, 316–320. doi: 10.1097/00075198-200208000-00008
Dragano, N., Siegrist, J., Nyberg, S. T., Lunau, T., Fransson, E. I., Alfredsson, L., et al. (2017). Effort-reward imbalance at work and incident coronary heart disease: a multicohort study of 90,164 individuals. Epidemiol. Camb. Mass 28, 619–626. doi: 10.1097/EDE.0000000000000666
Dunser, M. W., Baelani, I., and Ganbold, L. (2006). A review and analysis of intensive care medicine in the least developed countries. Crit. Care Med. 34, 1234–1242. doi: 10.1097/01.CCM.0000208360.70835.87
Einarsen, S., Hoel, H., and Notelaers, G. (2009). Measuring exposure to bullying and harassment at work: Validity, factor structure and psychometric properties of the Negative Acts Questionnaire-Revised. Work Stress 23, 24–44. doi: 10.1080/02678370902815673
El Khamali, R., Mouaci, A., Valera, S., Cano-Chervel, M., Pinglis, C., Sanz, C., et al. (2018). Effects of a multimodal program including simulation on job strain among nurses working in intensive care units: a randomized clinical trial. JAMA 320, 1988–1997. doi: 10.1001/jama.2018.14284
Embriaco, N., Azoulay, E., Barrau, K., Kentish, N., Pochard, F., Loundou, A., et al. (2007a). High level of burnout in intensivists: prevalence and associated factors. Am. J. Respir. Crit. Care Med. 175, 686–692. doi: 10.1164/rccm.200608-1184OC
Embriaco, N., Papazian, L., Kentish-Barnes, N., Pochard, F., and Azoulay, E. (2007b). Burnout syndrome among critical care healthcare workers. Curr. Opin. Crit. Care 13, 482–488. doi: 10.1097/MCC.0b013e3282efd28a
Fassier, T., and Azoulay, E. (2010). Conflicts and communication gaps in the intensive care unit. Curr. Opin. Crit. Care 16, 654–665. doi: 10.1097/MCC.0b013e32834044f0
Flannery, L., Ramjan, L. M., and Peters, K. (2016). End-of-life decisions in the Intensive Care Unit (ICU) – Exploring the experiences of ICU nurses and doctors – A critical literature review. Aust. Crit. Care 29, 97–103. doi: 10.1016/j.aucc.2015.07.004
Furr, R. M. (2011). Scale Construction and Psychometrics for Social and Personality Psychology. London: SAGE Publications Ltd. doi: 10.4135/9781446287866
Garrouste-Orgeas, M., Perrin, M., Soufir, L., Vesin, A., Blot, F., Maxime, V., et al. (2015). The Iatroref study: medical errors are associated with symptoms of depression in ICU staff but not burnout or safety culture. Intens. Care Med. 41, 273–284. doi: 10.1007/s00134-014-3601-4
Gorgievski, M. J., Van der Heijden, B. I. J. M., and Bakker, A. B. (2019). Effort-reward imbalance and work-home interference: a two-wave study among european male nurses. Work Stress. 33, 315–333. doi: 10.1080/02678373.2018.1503358
Gray-Toft, P., and Anderson, J. G. (1981). The nursing stress scale: development of an instrument. J. Behav. Assess. 3, 11–23. doi: 10.1007/BF01321348
Hamric, A. B., and Blackhall, L. J. (2007). Nurse-physician perspectives on the care of dying patients in intensive care units: collaboration, moral distress, and ethical climate. Crit. Care Med. 35, 422–429. doi: 10.1097/01.CCM.0000254722.50608.2D
Hamric, A. B., Borchers, C. T., and Epstein, E. G. (2012). Development and testing of an instrument to measure moral distress in healthcare professionals. AJOB Prim. Res. 3, 1–9. doi: 10.1080/21507716.2011.652337
Harris, P. E. (1989). The nurse stress index. Work Stress 3, 335–346. doi: 10.1080/02678378908256952
Hart, S. G. (2006). NASA-task load index (NASA-TLX); 20 years later. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 50, 904–908. doi: 10.1177/154193120605000909
Hinshaw, A. S., and Atwood, J. R. (1985). Anticipated Turnover Among Nursing Staff Study: Final Report (R01NU00908). Bethesda, MD: National Institutes of Health, National Center for Nursing Research.
Jackson, P. R., Wall, T. D., Martin, R., and Davids, K. (1993). New measures of job control, cognitive demand, and production responsibility. J. Appl. Psychol. 78, 753–762. doi: 10.1037/0021-9010.78.5.753
Kanner, A. D., Coyne, J. C., Schaefer, C., and Lazarus, R. S. (1981). Comparison of two modes of stress measurement: daily hassles and uplifts versus major life events. J. Behav. Med. 4, 1–39. doi: 10.1007/BF00844845
Kanner, A. D., Kafry, D., and Pines, A. (1978). Conspicuous in its absence: the lack of positive conditions as a source of stress. J. Human Stress 4, 33–39. doi: 10.1080/0097840X.1978.10545984
Karanikola, M., Giannakopoulou, M., Mpouzika, M., Kaite, C. P., Tsiaousis, G. Z., and Papathanassoglou, E. D. E. (2015). Dysfunctional psychological responses among Intensive Care Unit nurses: a systematic review of the literature. Rev. Esc. Enferm. USP 49, 847–857. doi: 10.1590/S0080-623420150000500020
Karasek, R., Brisson, C., Houtman, I., Bongers, P., and Amick, B. (1998). The Job Content Questionnaire (JCQ): an instrument for internationally comparative assessments of psychosocial job characteristics. J. Occup. Health Psychol. 3, 322–355. doi: 10.1037/1076-8998.3.4.322
Kitaoka-Higashiguchi, K., and Nakagawa, H. (2003). Job strain, coping, and burnout among Japanese nurses. Jpn. J. Health Hum. Ecol. 69, 66–79. doi: 10.3861/jshhe.69.66
Kramer, M., and Hafner, L. P. (1989). Shared values: impact on staff nurse job satisfaction and perceived productivity. Nurs. Res. 38, 172–177. doi: 10.1097/00006199-198905000-00019
Krämer, T., Schneider, A., Spieß, E., Angerer, P., and Weigl, M. (2016). Associations between job demands, work-related strain and perceived quality of care: a longitudinal study among hospital physicians. Int. J. Qual. Health Care J. Int. Soc. Qual. Health Care 28, 824–829. doi: 10.1093/intqhc/mzw119
Lake, E. T. (2002). Development of the practice environment scale of the nursing work index. Res. Nurs. Health 25, 176–188. doi: 10.1002/nur.10032
Laurent, A., Bonnet, M., Capellier, G., Aslanian, P., and Hebert, P. (2017). Emotional impact of end-of-life decisions on professional relationships in the ICU: an obstacle to collegiality? Crit. Care Med. 45, 2023–2030. doi: 10.1097/CCM.0000000000002710
Lazarus, R. S., and Folkman, S. (1984). Stress, Appraisal, and Coping. New-York, NY: Springer Publishing Company.
Lheureux, F., Truchot, D., and Borteyrou, X. (2016). Suicidal tendency, physical health problems and addictive behaviors among general practitioners: the effect of burnout. Work Stress. 30, 173–192. doi: 10.1080/02678373.2016.1171806
Miranda, D. R., Moreno, R., and Iapichino, G. (1997). Nine equivalents of nursing manpower use score (NEMS). Intensive Care Med. 23, 760–765. doi: 10.1007/s001340050406
O'Brien-Pallas, L., Irvine, D., Peereboom, E., and Murray, M. (1997). Measuring nursing workload: understanding the variability. Nurs. Econ. 15, 171–182.
Olson, L. L. (2007). Hospital nurses' perceptions of the ethical climate of their work setting. Image J. Nurs. Sch. 30, 345–349. doi: 10.1111/j.1547-5069.1998.tb01331.x
Ozden, D., Karagozoglu, S., and Yildirim, G. (2013). Intensive care nurses' perception of futility: Job satisfaction and burnout dimensions. Nurs. Ethics 20, 436–447. doi: 10.1177/0969733012466002
Pastores, S. M., Kvetan, V., Coopersmith, C. M., Farmer, J. C., Sessler, C., Christman, J. W., et al. (2019). Workforce, workload, and burnout among intensivists and advanced practice providers: a narrative review. Crit. Care Med. 1:3637. doi: 10.1097/CCM.0000000000003637
Pejtersen, J. H., Kristensen, T. S., Borg, V., and Bjorner, J. B. (2010). The second version of the Copenhagen Psychosocial Questionnaire. Scand. J. Public Health. 38(3 Suppl), 8–24. doi: 10.1177/1403494809349858
Peter, J. P., and Churchill, G. A. (1986). Relationships among research design choices and psychometric properties of rating scales: a meta-analysis. J. Mark. Res. 23, 1–10. doi: 10.1177/002224378602300101
Piers, R. D., Azoulay, E., Ricou, B., Dekeyser Ganz, F., Decruyenaere, J., Max, A., et al. (2011). Perceptions of appropriateness of care among European and Israeli intensive care unit nurses and physicians. JAMA 306, 2694–2703. doi: 10.1001/jama.2011.1888
Profit, J., Sharek, P. J., Amspoker, A. B., Kowalkowski, M. A., Nisbet, C. C., Thomas, E. J., et al. (2014). Burnout in the NICU setting and its relation to safety culture. BMJ Qual. Saf. 23, 806–813. doi: 10.1136/bmjqs-2014-002831
Revicki, D. A., May, H. J., and Whitley, T. W. (1991). Reliability and validity of the work-related strain inventory among health professionals. Behav. Med. 17, 111–120. doi: 10.1080/08964289.1991.9937554
Rizzo, J. R., House, R. J., and Lirtzman, S. I. (1970). Role conflict and ambiguity in complex organizations. Adm. Sci. Q. 15:150. doi: 10.2307/2391486
Schmuckler, M. A. (2001). What is ecological validity? A dimensional analysis. Infancy 2, 419–436. doi: 10.1207/S15327078IN0204_02
Semmer, N., Zapf, D., and Dunckel, H. (1995). “Assessing stress at work: a framework and an instrument,” in Work and Health: Scientific Basis of Progress in the Working Environment, eds. O. Svane and C. Johansen [Bruxelles, BE: Directorate-General for Employment, Social Affairs and Inclusion (European Commission), Directorate-General for Research and Innovation (European Commission)], 105–113.
Sexton, J. B., Helmreich, R. L., Neilands, T. B., Rowan, K., Vella, K., Boyden, J., et al. (2006). The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv. Res. 6:44. doi: 10.1186/1472-6963-6-44
Shimomitsu, T., Yokoyama, K., Ohno, H., Maruta, T., and Tanigawa, T. (2000). [Manual of the Brief Job Stress Questionnaire. Reports on the Study of Job Stress and its Effects on Health in the Workplace: The Research Grant for the Prevention of Work-Related Diseases from the Japan Ministry of Labor]. Tokyo: The Ministry of Labor (2000), 17–27 (in Japanese).
Siegrist, J., Starke, D., Chandola, T., Godin, I., Marmot, M., Niedhammer, I., et al. (2004). The measurement of effort–reward imbalance at work: European comparisons. Soc. Sci. Med. 58, 1483–1499. doi: 10.1016/S0277-9536(03)00351-4
Sochalski, J. (2004). Is more better?: the relationship between nurse staffing and the quality of nursing care in hospitals. Med. Care 42, II67–II73. doi: 10.1097/01.mlr.0000109127.76128.aa
Sorra, J. S., and Dyer, N. (2010). Multilevel psychometric properties of the AHRQ hospital survey on patient safety culture. BMC Health Serv. Res. 10:199. doi: 10.1186/1472-6963-10-199
Stinglhamber, F., and Vandenberghe, C. (2003). Organizations and supervisors as sources of support and targets of commitment: a longitudinal study. J. Organ. Behav. 24, 251–270. doi: 10.1002/job.192
Teixeira, C., Ribeiro, O., Fonseca, A. M., and Carvalho, A. S. (2013). Burnout in intensive care units - a consideration of the possible prevalence and frequency of new risk factors: a descriptive correlational multicentre study. BMC Anesthesiol. 13:38. doi: 10.1186/1471-2253-13-38
Teixeira, C., Ribeiro, O., Fonseca, A. M., and Carvalho, A. S. (2014). Ethical decision making in intensive care units: a burnout risk factor? Results from a multicentre study conducted with physicians and nurses. J. Med. Ethics 40, 97–103. doi: 10.1136/medethics-2012-100619
The Marlin Company the American Institute of Stress (2011). Attitudes in the American Workplace VII. The Seventh Annual Labor Day Survey. Available online at: https://www.stress.org/wp-content/uploads/2011/08/2001Attitude-in-the-Workplace-Harris.pdf
Trevick, S., Kim, M., and Naidech, A. (2016). Communication, leadership, and decision-making in the neuro-ICU. Curr. Neurol. Neurosci. Rep. 16:99. doi: 10.1007/s11910-016-0699-5
Tummers, G. E. R., van Merode, G. G., and Landeweerd, J. A. (2002). The diversity of work: differences, similarities and relationships concerning characteristics of the organisation, the work and psychological work reactions in intensive care and non-intensive care nursing. Int. J. Nurs. Stud. 39, 841–855. doi: 10.1016/S0020-7489(02)00020-2
Tyssen, R., Vaglum, P., Grønvold, N. T., and Ekeberg, Ø. (2001). The impact of job stress and working conditions on mental health problems among junior house officers. A nationwide Norwegian prospective cohort study. Med. Educ. 34, 374–384. doi: 10.1046/j.1365-2923.2000.00540.x
Van Mol, M. M. C., Kompanje, E. J. O., Benoit, D. D., Bakker, J., Nijkamp, M. D., and Seedat, S. (2015). The prevalence of compassion fatigue and burnout among healthcare professionals in intensive care units: a systematic review. PLoS ONE 10:e0136955. doi: 10.1371/journal.pone.0136955
Vandevala, T., Pavey, L., Chelidoni, O., Chang, N. F., Creagh-Brown, B., and Cox, A. (2017). Psychological rumination and recovery from work in intensive care professionals: associations with stress, burnout, depression and health. J. Intensive Care 5:16. doi: 10.1186/s40560-017-0209-0
Keywords: systematic review, intensive care unit (ICU), job stressors, occupational stressors, psychometrics, job stress scales
Citation: Laurent A, Lheureux F, Genet M, Martin Delgado MC, Bocci MG, Prestifilippo A, Besch G and Capellier G (2020) Scales Used to Measure Job Stressors in Intensive Care Units: Are They Relevant and Reliable? A Systematic Review. Front. Psychol. 11:245. doi: 10.3389/fpsyg.2020.00245
Received: 27 June 2019; Accepted: 03 February 2020;
Published: 12 March 2020.
Edited by:
Cesar Merino-Soto, University of San Martín de Porres, PeruReviewed by:
Caterina Ledda, University of Catania, ItalyGiridhara R. Babu, Public Health Foundation of India, India
Copyright © 2020 Laurent, Lheureux, Genet, Martin Delgado, Bocci, Prestifilippo, Besch and Capellier. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Alexandra Laurent, alexandra.laurent@u-bourgogne.fr