The Influence of Depression on the Psychometric Properties of the Maslach Burnout Inventory–Human Services Survey: A Cross-Sectional Study With Nursing Assistants

Background: The Maslach Burnout Inventory-Human Services Survey (MBI-HSS) is the most commonly used instrument to assess burnout. Although various factors have been reported to influence its validity, the influence of major depressive disorder (MDD) has not been previously considered. We developed this study to investigate the influence of MDD on the psychometric properties of the MBI-HSS in nursing assistants.

Results: From a sample of 521 nursing assistants, we found in those with MDD (n = 138, 24.56%) a degree of data misfit into the model, revealed by non-acceptable values for the root mean square error of approximation (RMSEA; 0.073; p = 0.004) and for the comparative fit index (CFI; 0.912), while in the non-MDD group these indices were acceptable and good, respectively, for RMSEA (0.048; p = 0.639) and for CFI (0.951). Also, we found higher coefficients of correlation among MBI-HSS factors and less items loading properly in their respective factors in the MDD subset, when compared to the non-MDD subset. For the total sample, while original 3-factor solution was an acceptable model, the bifactor model fitted data better.

Conclusions: MDD may impair the construct validity of MBI-HSS subscales, by increasing measurement error and decreasing model fitness. Therefore, researchers and health professionals should be aware of potential changes in the psychometric properties of the MBI-HSS when applied in subjects with depression.

Introduction

Burnout is a syndrome including emotional exhaustion (EE), depersonalization (DE) and low personal accomplishment (PA), resulting from prolonged stress at work (1–3). The Maslach Burnout Inventory (MBI) is the most widely used instrument to assess burnout; it has been translated into various languages and has been used in innumerous countries (4). Some inconsistencies in its validation and its possible determinants have been described (5–8). Depression has a significant relationship with burnout; however, the potential influence of major depressive disorder (MDD) on MBI validation has not been specifically investigated.

Background

According to Maslach's description, EE is the syndrome‘s core dimension and refers to feelings of being emotionally overextended and depleted of one's emotional resources (1). DE comprises negative, detached and impersonal attitudes toward other people (i.e., clients and patients). Reduced PA reflects feelings of incompetence at work and negative self-evaluation (1, 9).

Burnout leads to personal suffering including sleep complaints (10), increased use of alcohol/drugs (11), family conflict (12), higher work absenteeism (13), higher staff turnover, early retirement and significant financial impact (14), self-reported unprofessional conduct (15), and suicidal ideation (16, 17).

Nurses play a distinct and critical role in the health system; however they are frequently under increased stress and work overload (18). Burnout is high among nurses and nursing assistants (19–21) in diverse healthcare settings (22–24). Besides the negative impact in the nurse, the overall level of nurse burnout in hospital units has been reported to affect patient satisfaction and other measures of deficient care quality (25). Assessing and minimizing burnout is fundamental to maintain high levels of work performance, decrease nurse shortages (19, 26) and meet recruitment targets for nurses (27).

Several studies using the exploratory and confirmatory factor analysis have validated the MBI in various countries (4–8), including for Brazilian nurses (28, 29). The MBI has a satisfactory reliability, its internal consistency, assessed by the Cronbach's alpha, has been reported to range from 0.70 to 0.90 for its three subscales (30). However, inconsistencies in MBI psychometric properties have been reported (31). For example, relatively low reliability for the DE subscale (32), and a best fit for a four-factor model (6) have been reported. Also, the use of 20 instead of 22 MBI items have been proposed (7, 8). Some factors have been reported to influence the MBI psychometric proprieties including sample characteristics (32) and the presence or absence of burnout (33). Recently, two studies reported the best fit for a bifactor model, composed by a general factor and the three traditional factors (5, 34). In the bifactor model, the items may load simultaneously on a global burnout factor and on the three specific ones (i.e., depersonalization, emotional exhaustion and personal accomplishment). In such approach it is possible to estimate the relevance of the global and the specific dimensions of burnout altogether.

Burnout is highly correlated with depression and an overlap, particularly between EE and depressive symptoms have been highlighted (35). Literature data indicate that burnout partially mediate the relationship between work stress and depression (36), and depression generally follows burnout (37). Emotional exhaustion and depersonalization (38), high levels of psychological demand, low social support at work, and stress due to inadequate work (14) have been shown to be significant predictors of subsequent depression. Importantly, among new graduate nurses, changes in burnout levels have been reported to be accompanied by changes in depressive symptoms and also intention to leave the profession (39). Although they are considered to be distinct entities that complement each other (40), the complexity of their relationship still deserves elucidation.

Depression may potentially influence the validity of a scale, as it has been shown for the Modified Fatigue Impact Scale (41). As above mentioned, some factors have been reported to influence the MBI psychometric properties. Although being highly related with burnout the potential influence of depression on MBI psychometric properties has not been investigated. Thus, we conducted this study with nursing assistants of a university hospital to investigate the influence of depression on the psychometric validity of the MBI-HSS.

Methods

Sample

The study was conducted at the Central Institute of the Clinics Hospital, Faculty of Medicine, University of São Paulo (ICHC-FMUSP). According to Burns and Grove, to perform factor analysis for each item of a scale, at least 10 subjects are needed (42). Therefore, to perform the factor analysis of the MBI-HSS, which has 22 items, it would be necessary to study at least 220 individuals. To evaluate the influence of depression on the factor structure of the MBI-HSS, we estimated that depression affects ~20% of subjects with moderate or severe burnout and 7% of subjects with mild burnout (40, 43). Thus, to detect the factorial validity we would need 430 subjects (323 subjects with mild and 97 subjects with moderate or severe burnout) to achieve a test power of 90%, with a two-tailed alpha of 5%. Estimating a rate of missing data of 15%, we estimated that 520 subjects would be necessary. We enrolled 521 nursing assistants belonging to various medical units: 320 individuals working in the general medical unit; 60 in the surgical unit; 66 in the emergency room; and 75 in the intensive care unit. We included 40% of nursing assistants from the day-shift (from 7 a.m. to 7 p.m., n = 345) and the night-shift (from 7 p.m. to 7 a.m., n = 176) of each unit.

Data Collection

We performed the interviews during participants' work hours, in offices on their respective units. Two researchers, a psychiatrist (T.R.T.) and a psychologist (C.C.S.) went to each medical unit and invited the nursing assistants consecutively as they met them in the nursing workplace.

Ethics Statement

The study was approved by the Hospital Committee of Ethics in Research of the Clinics Hospital, Faculty of Medicine, University of São Paulo (CAPPesq–HC-FMUSP); number 1202/07. All participants started the research procedures only after providing signed informed consent.

Instruments

Maslach Burnout Inventory-Human Services Survey (MBI-HSS)

The MBI-HSS is a self-report instrument comprised of 22 items, grouped in 3 subscales. The scores on each subscale are assessed separately: a 9-item subscale for EE, a 5-item subscale for DE and an 8-item subscale for PA. The severity of each item is assessed on a 7-point ordinal scale, based on its frequency from zero (never) to six (always). Cut off points have been established based on the American normative data or dividing the subscales into tertiles or quartiles (44). The Portuguese translation was performed by two psychiatrists; a consensual version was submitted to 5 healthy volunteers from the hospital staff to evaluate its use, feasibility and clarity. The resulting version was back translated by a native English speaker fluent in Portuguese and approved by the author of the original American MBI-HSS version.

Primary Care Evaluation of Mental Disorders (PRIME-MD)

A psychiatrist (T.R.T.) used the Portuguese version (45) of the Primary Care Evaluation of Mental Disorders (PRIME-MD) to diagnose MDD. The PRIME-MD is a structured interview divided into five modules comprising the most frequent mental illnesses in primary care (mood disorders, anxiety, somatoform disorders, alcohol problems and eating disorders) (46). The mood module presents 9 yes/no questions investigating the depressive symptoms included in the DSM-III-R criteria to diagnose MDD; these criteria have been preserved unchanged by the DSM-IV and DSM-5 (47).

Questions to Obtain Demographic and Occupational Data

A questionnaire was used to collect occupational and demographic data including gender, age, marital status, number of children and professional characteristics (professional experience, weekly working hours and absenteeism over the month prior to the interview–missing days from work). Race was self-reported based on the Brazilian census, including white, black, mixed, Asian, and Indigenous people.

Statistical Analysis

In the descriptive analysis, we show the absolute frequency and percentages for qualitative variables (gender, race, marital status). Absenteeism was analyzed as a dichotomous variable (by comparing subjects without absenteeism with those absent ≥1 day from work in the month prior to the interview). The number of children were categorized in those with “0,” “1 or 2” and “≥3” children. We report the means and standard deviations for quantitative variables (age, time of working experience in healthcare and weekly working hours). We used the non-parametric Mann-Whitney test to compare numerical variables between MDD and non-MDD subsets.

The severity of MBI-HSS items vary from 0 to 6 in an ordinal range. Therefore, we performed the confirmatory factorial analysis using the weighted least squares means and variance adjusted estimation method (WLSMV) (48), and the software MPLUS 7.4 (49, 50). The CFA was performed for the original 3-factor model and for a bifactor model (51). We tested the bifactor model because of the significant correlation between its subscales and the recent reported good performance of this model for the MBI-HSS (5, 34). To test the validity of construct, we evaluated the fit of the model to the data by the exact fit p-value and chi-square/degree of freedom –X²/df and the approximate fit with the root mean square error of approximation (RMSEA) and its p-value. Considering the X²/df, its values indicate good fit if 0 ≤ X²/df ≤ 2, acceptable fit if 2 < X²/df ≤ 3, and poor if X²/df > 3; p-values of the X²/df indicate good fit if 0.05 < p < 1.00, acceptable fit if 0.01 ≤ p ≤ 0.05, and values of p < 0.01 indicate poor fit. Regarding RMSEA, values 0 ≤ RMSEA ≤ 0.05 indicate good fit, values 0.05 < RMSEA ≤ 0.08 indicate acceptable fit and values of RMSEA > 0.08 indicate poor fit (52); p-values of RMSEA indicate good fit if 0.10 < p ≤ 1.00, and acceptable fit if 0.05 ≤ p ≤ 0.10.

We performed the analysis of practical significance by means of the Tucker-Lewis index (TLI or non-normed fit index–NNFI) and the comparative fit index (CFI). Values of the TLI are considered good if ≥0.97 and acceptable if 0.95 ≤ NNFI < 0.97; values of the CFI are considered good if ≥0.97 and acceptable if 0.97 > NNFI ≥ 0.95 (52).

To test the hypothesis that MDD impairs the performance of MBI-HSS scale, we conducted the analysis for the subset of nursing assistants without MDD (n-MDD) and for the subset with MDD (MDD) separately. We performed the configural invariance analysis for the non-MDD and MDD subsets for the 3-factor model. For the bifactor model, we performed the configural invariance analysis, and also the scalar invariance analysis and the analysis of scalar vs. configural invariances.

Factor loadings were initially analyzed by the significance of their standard factor loadings estimates and the correlations between the 3 sub-dimensions of the 3-factor model and of the bifactor model.

To evaluate the internal consistency of each factor in the 3-factor model, we calculated the Cronbach's Alpha using the SPSS Statistics 24 package. We considered values >0.7 as indicative of acceptable indices. For the bifactor model, we included the Explained Common Variance (ECV), the non-hierarchical omega (total omega) and the hierarchical omega indices. These indices evaluate how well the subscale items measure the latent construct, reflecting the reliability of the instrument. We used a significance level of < 0.05.

Results

Sample Characteristics

The sample consisted of 521 nursing assistants, the majority were women (91%), white, married and with children (Table 1). The MDD subset (n = 138, 24, 76%) was more likely to include women, with children, with longer working experience in healthcare and with increased absenteeism in the month preceding the interview (Table 1).

TABLE 1

Table 1. Demographic and occupational characteristics of nursing assistants: total sample, MDD and non-MDD subsets.

Regarding burnout, the MDD subset showed significantly increased scores on EE and DE and decreased scores on PA compared to the non-MDD subset (Table 2).

TABLE 2

Table 2. Severity of burnout according to the MBI-HSS, total sample, MDD and non-MDD subsets.

Construct Validity

For the total sample, almost all indices were good or acceptable in the bifactor model; in the 3-factor model, the X²/df and the RMSEA indices were acceptable, and the CFI and the TLI were below the acceptable cutoff. In the simultaneous analysis (configural), the indices indicated acceptable fit in the bifactor model for MDD and n-MDD subsets, except for CFI and TLI, while in the 3-factor model, acceptable indices were found for the RMSEA only. The scalar analysis for the bifactor model revealed good or acceptable values for all indices, except for the CFI.

In the n-MDD subset, most of the MBI-HSS indices were good or acceptable in the bifactor model (i.e., X²/df, RMSEA and its p-value, and CFI); in the 3-factor model, acceptable and good indices were found for the RMSEA and its p-value, respectively (Table 3). However, in the MDD subset, no index indicated good fit, and acceptable indices were found for X²/df and RMSEA in the 3-factor and in the bifactor models (Table 3).

TABLE 3

Table 3. Factor Analysis of the MBI-HSS.

Factor Loadings

In the 3 factors model, all estimates were significant (Figure 1). In the factorial bifactor model, the PA4 (i.e., I can easily understand how my patients feel about things), PA9 (fell positively influencing other people's lives), and the DE15 (don't really care what happens to patients) did not load significantly in the general factor, and the EE 20 (i.e., I feel like I'm at the end of my rope) did not load significantly in the EE factor; both in MDD and in n-MDD subsets (Figure 2).

FIGURE 1

Figure 1. 3-factor model of MBI-HSS in MDD and non-MDD subjects including the significant estimate values. MDD, major depressive disorder; EE, emotional exhaustion; PA, personal accomplishment; DE, depersonalization; Non-MDD subset, N = 377; MDD subset, N = 136.

FIGURE 2

Figure 2. Bifactor model of MBI-HSS in MDD and non-MDD subjects including the significant estimate values. MDD, major depressive disorder; EE, emotional exhaustion; PA, personal accomplishment; DE, depersonalization; Non-MDD subset, N = 377; MDD subset, N = 136.

Considering the cutoff >0.6 as indicative of adequate loading, the n-MDD subset presented more items loading adequate in the bifactor model (i.e., factors EE, PA and in the general factor), and in the 3-factor model (i.e. factor EE) compared to the MDD subset. A similar picture was found for the cutoff >0.5 as indicative of adequate loading; the n-MDD subset showed more items significantly loading in the bifactor model (i.e., EE and PA factors), and in the 3-factor model (i.e., PA factor). No factor showed more items loading >0.6 or >0.5 in the MDD subset compared to the n-MDD subset, both in the bifactor and 3-factor models.

Correlations

The 3-factor model revealed significant correlations between EE and PA (negative); between EE and DE (positive); and between PA and DE (negative). Significant correlations between EE and DE (positive); and PA and DE (negative) were also found in the bifactor model; however, the bifactor model revealed an absence of correlation between EE and PA both in the MDD and n-MDD subsets. The coefficients of correlation between the subscales were higher in the subset of MDD both for the 3-factor and the bifactor models (Figures 1, 2).

Internal Consistency

The reliability analysis for the 3-factor model indicated acceptable Cronbach's Alpha index for the EE, both for the MDD and for the non-MDD subsets (Table 4).

TABLE 4

Table 4. Reliability of MBI-HSS (Cronbach's Alpha): total sample, MDD and non-MDD subsets.

Considering the bifactor model, acceptable reliability was found for the total sample and n-MDD subset, but not for the MDD; estimated indices for the non-hierarchical were 0.811; 0.810; and 0.774, respectively, for total sample, non-MDD and MDD subsets. The estimated indices for the explained variance were 0.621; 0.641; and 0.672, respectively, for total sample, non-MDD and MDD subsets. The hierarchical omega revealed that most of the variance was explained by the general factor and only a small proportion of variance was explained exclusively by the EE, PA, and DE individually for the total sample and for both subsets non-MDD and MDD individually (Table 5).

TABLE 5

Table 5. Reliability of MBI-HSS for the bifactor model (hierarchical omega coefficient): total sample, MDD and non-MDD subsets.

Discussion

For a sample of 521 nursing assistants at a teaching general hospital, we found that MDD influenced the psychometric properties of the MBI-HSS. Additionally, when considering the total sample, although the original 3-factor model showed an acceptable fit, the bifactor solution provided incremental fitness to the model.

The Influence of Depression on MBI-HSS Properties

The influence of MDD on the performance of MBI-HSS was revealed by results showing non-acceptable fit indices for the MDD subset and acceptable or good indices for the non-MDD subset (i.e., indices of RMSEA and CFI), and higher correlations among MBI-HSS factors in the MDD subset compared to the non-MDD subset.

Our findings corroborate previous data showing differences on the performance of the MBI, as a consequence of sample psychopathology/characteristics. Schaufeli et al. found good fit indices for the 3-factor model in burned-out employees, but six items of PA did not load accordingly among those non-burned-out (33); indicating a weak support for validity of the PA subscale in those without burnout. Of note, by definition, symptoms in burned-out employees had to be restricted to workplace, while symptoms in those non-burned-out did not have that restriction. Considering that their sample was comprised of employees who were seeking psychological treatment, the non-burned-out group surprisingly had higher psychopathology, including higher levels of depression and anxiety symptoms (33). Consequently, in their sample, the performance of the MBI was worse in the group with increased depression symptoms (i.e., the non-burned-out group), as we found here. Another example for the influence of sample characteristics on the MBI performance comes from workers taking care of individuals with intellectual disabilities. In that sample, Chao et al. found a better fit for a 4-factor solution instead of the traditional 3-factor model (32). In particular, items assessing DE loaded in two DE sub-factors (32). According to the authors, such particularity could be explained by taking into account that services for people with intellectual disabilities are person-centered and focused on building family-like relationships (32). In that situation, the wording in the DE subscale would be unacceptable to those workers, leading to inconsistent responses and impeding items to combine in a single DE factor.

The stronger correlations found in the MDD subset suggest that MDD may influence the MBI structure by decreasing the original conceptually proposed independence of its dimensions. Interestingly, among elementary and secondary teachers, also using the bifactor model for the MBI, depressive symptoms were reported to be related to the general factor of burnout, instead of any specific dimension (34). Our MDD subset also showed less items with factor loadings >0.50 or >0.60 in the PA and DE factors compared to the non-MDD subset both in the bifactor and in the 3-factor models. Thus, our results suggest that the presence of MDD could decrease the strength and the independence of the PA and DE dimensions.

It is possible that depressed mood could affect the subjectivity of the symptom perception and consequently limit the validity of an instrument. Such possibility, was proposed by Larson to explain the influence of depressive mood on the validation of an instrument to assess fatigue (41). According to this thinking, MDD could lead to a distinct interpretation of some MBI items, or possibly a qualitative/quantitative change in experiencing some of them, leading to a diverse scoring and consequently influencing MBI-HSS psychometric properties, as we found in our MDD subset of nursing assistants.

MBI-HSS Properties for the Total Sample

For the totality of our sample, our results supported the 3-factor model, but indicated a better fit for the bifactor model, confirming the findings reported by Meszaros et al. (5) and corroborating their proposal of including a general burnout dimension in the conceptualization of burnout (5).

We found no correlation between EE and PA in the bifactor model, both in the MDD and n-MDD subsets. This independence of PA has previously been reported (5, 6), reinforcing that PA cannot be interpreted as an opposite of EE and DE (6).

The presence of a general factor in the bifactor model allowed the delineation of the specificity of an item. For example, in the PA factor the items “I can easily understand how my patients feel about things” (i.e., item PA4) and “feel positively influencing other people's lives” (i.e., item PA9) loaded in the PA but not in the general factor both in MDD and in n-MDD subsets (Figure 2). Such findings support the specificity of these items as markers of PA. Of note, in the study of Meszaros et al. the PA4 item also did not load in the general factor (5). In contrast, the item EE20 (i.e., I feel like I'm at the end of my rope) loaded in the general factor but not in the EE factor; both in MDD and in n-MDD subsets (Figure 2), suggesting that it actually assess a general aspect of burnout and not specifically EE.

The reliability of the 3-factor model assessed by the Cronbach‘s Alpha index was higher for the EE than for the DE and PA factors. The higher reliability of EE compared to DE and PA has been reported by previous studies (30, 53) and for samples from various countries (4). Actually, although instruments capturing burnout have included distinct dimensions [e.g., professional repression, dehumanization, emotional distancing (54, 55); enthusiasm toward the job, indolence, guilt (56)], they tend to maintain the exhaustion dimension (54, 55). Those findings support the original view that EE is the burnout syndrome‘s core dimension (1).

In the bifactor model, the non-hierarchical omega index indicated a good reliability, and the hierarchical omega revealed that most of the variance was explained by the general factor. These results converge with the above-mentioned proposal that a general burnout factor should be considered.

Some limitations of our study should be addressed. Our nursing assistants are from a teaching hospital and it is not possible to generalize our findings to settings outside this context. Additionally, our study recruited only in a healthcare setting in Brazil and cannot address whether cultural issues specific to Brazil moderate the impact of MDD on MBI psychometric properties.

Conclusions

Our data support that the presence of MDD may decrease the construct validity of MBI-HSS, as shown by non-satisfactory values of RMSEA and CFI, and also decrease the independence of its dimensions, as shown by an increase in their correlation, particularly between EE and DE. Such influence may result from distinct interpretation of some MBI-HSS items or from distinct experiences of some burnout symptoms by MDD subjects. Based on our findings, we suggest that researchers consider the influence of MDD when using the MBI-HSS to assess burnout in depressed individuals. We also found a best fit for a bifactor model, including a general factor. To avoid spurious conclusions about MBI-HSS subscales, it is advisable to perform factorial analysis to identify potential influence of the sample, including the influence of depression. Additional studies are warranted to confirm the MDD influence on MBI-HSS validity in different cultures and healthcare settings.

Relevance to Clinical Practice

In this study, assessing burnout in 521 nursing assistants, we found that a current MDD episode results in a negative impact on the MBI-HSS psychometric properties. Consequently, studies and programs intending to assess and reduce burnout should consider checking MBI-HSS psychometric properties, particularly in those with MDD.

Author Contributions

TT worked in the idealization of the study, literature search, wrote the protocol, collected the data, worked in the data analysis and wrote the first draft of the manuscript. CF worked in the literature search, collected the data and contributed to the writing of the manuscript. Y-PW worked in the analysis of the data and contributed to the writing of the manuscript. FR worked in the design of the study and worked in the writing of the manuscript. MdL worked in the design of the study and worked in the writing of the manuscript. JS worked in the analysis of the data and performed the statistical procedures. DI worked in the idealization and design of the study and in the writing of the manuscript. JH worked in the rational of the validation process and writing of the manuscript. RF worked in the idealization and design of the study, supervised the data collecting, worked in the data analysis and writing of the manuscript. All the authors contributed and approved the final version of the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank Dr. Christina Maslach for reviewing the back translation of the MBI-HSS.

References

1. Maslach C, Jackson S. The measurement of experienced burnout. J Occup Behav. (1981) 2:99–113. doi: 10.1002/job.4030020205

CrossRef Full Text | Google Scholar

2. Adriaenssens J, De Gucht V, Maes S. Causes and consequences of occupational stress in emergency nurses, a longitudinal study. J Nurs Manag. (2013) 23:346–58. doi: 10.1111/jonm.12138.

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Zhang LF, You LM, Liu K, Zheng J, Fang JB, Lu MM, et al. The association of Chinese hospital work environment with nurse burnout, job satisfaction, and intention to leave. Nurs Outlook (2013) 62:128–37. doi: 10.1016/j.outlook.2013.10.010

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Poghosyan L, Aiken LH, Sloane DM. Factor structure of the Maslach burnout inventory: an analysis of data from large scale cross-sectional surveys of nurses from eight countries. Int J Nurs Stud. (2009) 46:894–902. doi: 10.1016/j.ijnurstu.2009.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Meszaros V, Adam S, Szabo M, Szigeti R, Urban R. The bifactor model of the Maslach Burnout Inventory-Human Services Survey (MBI-HSS)–an alternative measurement model of burnout. Stress Health (2014) 30:82–8. doi: 10.1002/smi.2481

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Kanste O, Miettunen J, Kyngas H. Factor structure of the maslach burnout inventory among finnish nursing staff. Nurs Health Sci. (2006) 8:201–7. doi: 10.1111/j.1442-2018.2006.00283.x

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Pisanti R, Lombardo C, Lucidi F, Violani C, Lazzari D. Psychometric properties of the maslach burnout inventory for human services among italian nurses: a test of alternative models. J Adv Nurs. (2013) 69:697–707. doi: 10.1111/j.1365-2648.2012.06114.x

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Loera B, Converso D, Viotti S. Evaluating the psychometric properties of the Maslach Burnout Inventory-Human Services Survey (MBI-HSS) among Italian nurses: how many factors must a researcher consider? PLoS ONE (2014) 9:e114987. doi: 10.1371/journal.pone.0114987

CrossRef Full Text

9. World Health Organization. Guidelines for the Primary Prevention of Mental, Neurological and Psychosocial disorders: Staff Burnout, Division of Mental Health World Health Organization (WHO). Germany (1998). pp. 91–110.

10. Brand S, Beck J, Hatzinger M, Harbaugh A, Ruch W, Holsboer-Trachsler E. Associations between satisfaction with life, burnout-related emotional and physical exhaustion, and sleep complaints. World J Biol Psychiatry (2010) 11:744–54. doi: 10.3109/15622971003624205

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Peterson U, Bergstrom G, Demerouti E, Gustavsson P, Asberg M, Nygren A. Burnout levels and self-rated health prospectively predict future long-term sickness absence: a study among female health professionals. J Occup Environ Med. (2011) 53:788–93. doi: 10.1097/JOM.0b013e318222b1dc

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Tayfur O, Arslan M. The role of lack of reciprocity, supervisory support, workload and work-family conflict on exhaustion: evidence from physicians. Psychol Health Med. (2013) 18:564–75. doi: 10.1080/13548506.2012.756535

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Mion G, Libert N, Journois D. Burnout-associated factors in anesthesia and intensive care medicine. 2009 survey of the French Society of anesthesiology and intensive care. Ann Fr Anesth Reanim. (2013) 32:175–88. doi: 10.1016/j.annfar.2012.12.004

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Iacovides, Fountoulakis KN, Kaprinis S, Kaprinis G. The relationship between job stress, burnout and clinical depression. J Affect Disord. (2003) 75:209–21. doi: 10.1016/S0165-0327(02)00101-5

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Dyrbye LN, Massie FS Jr, Eacker A, Harper W, Power D, Durning SJ, et al. Relationship between burnout and professional conduct and attitudes among US medical students. JAMA (2010) 304:1173–80. doi: 10.1001/jama.2010.1318

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Dyrbye LN, Thomas MR, Massie FS, Power DV, Eacker A, Harper W, et al. Burnout and suicidal ideation among U.S. medical students. Ann Int Med. (2008) 149:334–41. doi: 10.7326/0003-4819-149-5-200809020-00008

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Tomas-Sabado J, Maynegre-Santaularia M, Perez-Bartolome M, Alsina-Rodriguez M, Quinta-Barbero R. Granell-Navas S. Burnout syndrome and suicide risk among primary care nurses. Enferm. Clín. (2010) 20:173–8. doi: 10.1016/j.enfcli.2010.03.004

CrossRef Full Text

18. Myhren H, Ekeberg O, Stokland O. Job satisfaction and burnout among intensive care unit nurses and physicians. Crit Care Res Pract. (2013) 2013:786176. doi: 10.1155/2013/786176

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Aiken LH, Sermeus W, Van den Heede K, Sloane DM, Busse R, McKee M, et al. Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and the United States. BMJ (2012) 344:e1717. doi: 10.1136/bmj.e1717

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Moreira DS, Magnago RF, Sakae TM, Magajewski FR. Prevalence of burnout syndrome in nursing staff in a large hospital in south of Brazil. Cad. Saude Publica (2009) 25:1559–68. doi: 10.1590/S0102-311X2009000700014

CrossRef Full Text | Google Scholar

21. Leineweber C, Westerlund H, Chungkham HS, Lindqvist R, Runesdotter S, Tishelman C. Nurses' practice environment and work-family conflict in relation to burn out: a multilevel modelling approach. PLoS ONE (2014) 9:e96991. doi: 10.1371/journal.pone.0096991

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Ksiazek, Stefaniak TJ, Stadnyk M, Ksiazek J. Burnout syndrome in surgical oncology and general surgery nurses: a cross-sectional study. Eur J Oncol Nurs. (2011) 15:347–50. doi: 10.1016/j.ejon.2010.09.002

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Benevides-Pereira AM, Das Neves Alves R. A study on burnout syndrome in healthcare providers to people living with HIV. AIDS Care (2007) 19:565–71. doi: 10.1080/09540120600722775

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Silva AT, Menezes PR. Burnout syndrome and common mental disorders among communit-based health agents. Rev Saúde Pública (2008) 42:921–9. doi: 10.1590/S0034-89102008000500019

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Vahey DC, Aiken LH, Sloane DM, Clarke SP, Vargas D. Nurse burnout and patient satisfaction. Med Care (2004) 42:II57–66. doi: 10.1097/01.mlr.0000109126.50398.5a

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Simon M. Nine per cent of nurses across Europe report intent to leave their profession, with burnout among the associated personal and professional factors. Evid Based Nurs. (2014) 17:54–5. doi: 10.1136/eb-2013-101321

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Aiken LH, Clark SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. J Am Med Assoc. (2002) 288:1987–93. doi: 10.1001/jama.288.16.1987

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Carvalho MB. Professor - Um Profissional, Sua Saúde e a Educação Em Saúde na Escola (Doctoral). Universidade de São Paulo, São Paulo (1995).

29. Tamayo MR. Relação Entre a Síndrome de Burnout e os Valores Organizacionais no Pessoal de Enfermagem de dois Hospitais Públicos (masters). Universidade de Brasília, Brasília (1997).

30. van der Colff JJ, Rothmann S. Burnout of registered nurses in South Africa. J Nurs Manag. (2014) 22:630–42. doi: 10.1111/j.1365-2834.2012.01467.x

CrossRef Full Text | Google Scholar

31. Beckstead JW. Confirmatory factor analysis of the maslach burnout inventory among Florida nurses. Int J Nurs Stud. (2002) 39:785–92. doi: 10.1016/S0020-7489(02)00012-3

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Chao SF, McCallion P, Nickle T. Factorial validity and consistency of the maslach burnout inventory among staff working with persons with intellectual disability and dementia. J Intellect Disabil Res. (2011) 55:529–36. doi: 10.1111/j.1365-2788.2011.01413.x

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Schaufeli WB, Bakker AB, Hoogduin K, Schaap C, Kladler A. On the clinical validity of the maslach burnout inventory and the burnout measure. Psychol Health (2001) 16:565–82. doi: 10.1080/08870440108405527

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Szigeti R, Balazs N, Bikfalvi R, Urban R. Burnout and depressive symptoms in teachers: factor structure and construct validity of the maslach burnout inventory-educators survey among elementary and secondary school teachers in Hungary. Stress Health (2017) 33:530–9. doi: 10.1002/smi.2737

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Valente M, Wang YP, Menezes PR. Structural validity of the maslach burnout inventory and influence of depressive symptoms in banking workplace: unfastening the occupational conundrum. Psychiatry Res (2018) 267:168–74. doi: 10.1016/j.psychres.2018.05.069

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Lin TC, Lin HS, Cheng SF, Wu LM, Ou-Yang MC. Work stress, occupational burnout and depression levels:a clinical study of paediatric intensive care unit nurses in Taiwan. J Clin Nurs. (2016) 25:1120–30. doi: 10.1111/jocn.13119

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Glass DC, McKnight JD, Valdimarsdottir H. Depression, burnout, and perceptions of control in hospital nurses. J Consult Clin Psychol. (1993) 61:147–55. doi: 10.1037/0022-006X.61.1.147

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Hakanen JJ, Schaufeli WB. Do burnout and work engagement predict depressive symptoms and life satisfaction? A three-wave seven-year prospective study. J Affect Disord. (2012) 141:415–24. doi: 10.1016/j.jad.2012.02.043

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Rudman, Gustavsson JP. Early-career burnout among new graduate nurses: a prospective observational study of intra-individual change trajectories. Int J Nurs Stud. (2011) 48:292–306. doi: 10.1016/j.ijnurstu.2010.07.012

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Ahola K, Honkonen T, Isometsa E, Kalimo R, Nykyri E, Aromaa A, et al. The relationship between job-related burnout and depressive disorders–results from the Finnish Health 2000 Study. J Affect Disord. (2005) 88:55–62. doi: 10.1016/j.jad.2005.06.004

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Larson RD. Psychometric properties of the modified fatigue impact scale. Int J MS Care (2013) 15:15–20. doi: 10.7224/1537-2073.2012-019

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Burns N, Grove S. The Practice of Nursing Research: Conduct, Critique and Utilization. Philadelphia, PA: WB Saunders (2001).

43. Ahola K, Honkonen T, Isometsa E, Kalimo R, Nykyri E, Koskinen S, et al. Burnout in the general population. Results from the finnish health 2000 study. Soc Psychiatry Psychiatr Epidemiol. (2006) 41:11–7. doi: 10.1007/s00127-005-0011-5

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Schaufeli W, Enzmann D. The Burnout Companion to Study & Practice. London: Taylor & Francis (1998).

Google Scholar

45. Fraguas R Jr, Henriques SG Jr, De Lucia MS, Iosifescu DV, Schwartz FH, Menezes PR, et al. The detection of depression in medical setting: a study with PRIME-MD. J Affect Disord. (2006) 91:11–7. doi: 10.1016/j.jad.2005.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. primary care evaluation of mental disorders. patient health questionnaire. JAMA (1999) 282:1737–44. doi: 10.1001/jama.282.18.1737

PubMed Abstract | CrossRef Full Text | Google Scholar

47. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th Edn. Text Revision Washington DC (2000).

48. Li CH. Confirmatory factor analysis with ordinal data: comparing robust maximum likelihood and diagonally weighted least squares. Behav Res Methods (2016) 48:936–49. doi: 10.3758/s13428-015-0619-7

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Byrne BM. Structural Equation Modeling With Mplus: Basic Concepts, Applications, and Programming. New York, NY: Routledge Taylor & Francis Group (2012).

Google Scholar

50. Brown TA. Confirmatory Factor Analysis for Applied Research. New York, NY: The Guilford Press (2015).

51. Rodriguez, Reise SP, Haviland MG. Evaluating bifactor models: calculating and interpreting statistical indices. Psychol Methods (2016) 21:137–50. doi: 10.1037/met0000045

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. MPR Online (2003) 8:23–74.

Google Scholar

53. Densten IL. Re-thinking burnout. Organ Behav J. (2001) 22:833–47. doi: 10.1002/job.115

CrossRef Full Text

54. Bortoletti FF, Benevides-Pereira MT, Vasconcellos EG, Siqueira JO, Araujo Junior E, Nardozza LM, et al. Triggering risk factors of the burnout syndrome in OB/GYN physicians from a reference public university of Brazil. ISRN Obstet Gynecol. (2012) 2012:593876. doi: 10.5402/2012/593876

CrossRef Full Text | Google Scholar

55. Benevides-Pereira TM, Pedro Guilherme Basso, Paulo Cesar Porto-Martins, José Antonio Carrobles, José de Oliveira Siqueira. Confirmatory factor analysis of the ISB - burnout syndrome inventory. Psychol Commun Health (2017) 6:5–30. doi: 10.5964/pch.v6i1.165

CrossRef Full Text | Google Scholar

56. Gil-Monte PR, Manzano-Garcia G. Psychometric properties of the Spanish burnout inventory among staff nurses. J Psychiatr Ment Health Nurs. (2015) 22:756–63. doi: 10.1111/jpm.12255

PubMed Abstract | CrossRef Full Text | Google Scholar