Transcultural Adaptation of the Oldenburg Burnout Inventory (OLBI) for Brazil and Portugal

During the last few years, burnout has gained more and more attention for its strong connection with job performance, absenteeism, and presenteeism. It is a psychological phenomenon that depends on occupation, also presenting differences between sexes. However, to properly compare the burnout levels of different groups, a psychometric instrument with adequate validity evidence should be selected (i.e., with measurement invariance). This paper aims to describe the psychometric properties of the Oldenburg Burnout Inventory (OLBI) version adapted for workers from Brazil and Portugal, and to compare burnout across countries and sexes. OLBI's validity evidence based on the internal structure (dimensionality, reliability, and measurement invariance), and validity evidence based on relationships with other variables (work engagement) are described. Additionally, it aims presents a revision of different OLBI's versions—since this is the first version of the instrument developed simultaneously for both countries—it is an important instrument for understanding burnout between sexes in organizations. Data were used from 1,172 employees across two independent samples, one from Portugal and the other from Brazil, 65 percent being female. Regarding the OLBI internal structure, a reduced version (15 items) was obtained. The high correlation between disengagement and exhaustion, suggested the existence of a second-order latent factor, burnout, which presented measurement invariance for country and sex. Confirmatory factor analysis of the Portuguese OLBI version presented good goodness-of-fit indices and good internal consistency values. No statistically significant differences were found in burnout between sexes or countries. OLBI also showed psychometric properties that make it a promising and freely available instrument to measure and compare burnout levels of Portuguese and Brazilian employees.

Introduction

Work organizations and labor relations all over the world are undergoing significant changes, with an impact on workers' lives and health, since the demands of modern working life are increasing pressure to levels never seen before (International Labour Office, 2016). Thus, the workforce must deal with a new landscape where psychosocial risks at work must be addressed (European Agency for Safety and Health at Work, 2018). Stress is a risk which, at extreme levels, can lead to burnout. Burnout has become a global concern, and work-related stress is a big challenge to organizations' performance and to their workers' health. Burnout levels vary depending on country, occupation, and individual characteristics, among which sex, is one of the most important factors (Purvanova and Muros, 2010). Burnout can affect any worker, with consequences not only in terms of health, safety, and well-being, but also for productivity, quality of service, and cost-effectiveness to the organization (Poghosyan et al., 2010; Carod-Artal and Vázquez-Cabrera, 2013). It is a severe reaction to occupational stress, having in its symptomatology changes to physical and psychological health and behavioral-motivational aspects, expressed through a reduction in job satisfaction or even a change of profession (Marques-Pinto et al., 2003). It is a syndrome (psychological in nature) that may occur when workers chronically face a stressful working environment and feel low resources to face high job demands (Maslach et al., 2001; Bakker and Demerouti, 2007; Maslach, 2015). The definition of Burnout has been expanded from a concept associated with human services professions to a concept related to all kinds of professions that can be affected (Lindblom et al., 2006).

From a historical perspective, burnout was initially considered as a psychological phenomenon in the USA, beginning with studies by the psychologist Freudenberger (1974) and the psychologist Maslach (1976). Despite some criticism (Bianchi et al., 2017) and the existence of several other related constructs, such as karōshi (), meaning “death by overwork” (International Labour Organization, 2013) and karōjisatsu (), meaning “suicide from overwork” (Amagasa et al., 2005), burnout became a popular topic in occupational health (Marques-Pinto et al., 2008; Schaufeli, 2017). There is some discussion about burnout (Bianchi, 2015; Bianchi et al., 2017; Epstein and Privitera, 2017; Mion et al., 2018) in terms of its dimensionality. The names attributed to the constructs can vary (Simbula and Guglielmi, 2010; Larsen et al., 2017). The most commonly suggested structure is a tri-factor one (Maslach et al., 2001, 2016), comprising emotional exhaustion (or simply exhaustion), depersonalization (also known as cynicism or disengagement), and reduced sense of personal accomplishment (or professional efficacy) (Halbesleben et al., 2004). It is expected that if a worker has high levels of the first two dimensions, there should be low levels of the third dimension since it is measured in the opposite direction to the other two. Carod-Artal and Vázquez-Cabrera (2013) state that emotional exhaustion is the most important dimension of burnout syndrome—being referred to as a state of having feelings of being emotionally overextended and depleted of one's emotional resources—representing the individual stress component (Bresó et al., 2007). Depersonalization refers to cynical or excessively detached responses to others in the work context; this is the interpersonal component of burnout (Maslach, 1998). Finally, diminished personal accomplishment refers to the decreased sense of competence and of productivity, representing the component of self-evaluation (Maslach, 1998).

There is also a two-dimensional approach to burnout (Demerouti et al., 2000). Based on empirical evidence, some authors consider that disengagement and exhaustion are the core dimensions of burnout, while reduced personal accomplishment plays a less important role (Maslach et al., 2001; Shirom, 2002). In fact, it has been shown that the relation of reduced personal accomplishment to burnout outcomes and antecedents is weaker than the other two dimensions (Lee and Ashforth, 1996). Moreover, while emotional exhaustion leads to disengagement, reduced personal accomplishment develops individually in relation to the other two dimensions (Leiter, 1993). Cordes and Dougherty (1993) suggest that it is an individual difference similar to self-efficacy.

Currently, burnout is becoming increasingly prominent in the literature (Leiter and Maslach, 2017); it has been associated with multiple occupational groups, beyond human services (Maslach and Leiter, 2017). In fact, burnout's prevalence has increased in some occupations, such as physicians in the USA (Shanafelt et al., 2015); it shows a high prevalence in various occupations, including radiology residents (Guenette and Smith, 2017), midwives in Australia (Creedy et al., 2017), nurses in various countries (Gómez-Urquiza et al., 2017). One reason why burnout is so common is due to the high levels of stress and emotional demands present in the job (Demerouti et al., 2001). Job stress can increase absenteeism, affect family roles, productivity, and mental and physical health, and decrease job satisfaction, which, in turn, can lead to reduced personal accomplishment, depersonalization, and emotional exhaustion (Carod-Artal and Vázquez-Cabrera, 2013).

Burnout Measurement With the Oldenburg Burnout Inventory

Despite the existence of several instruments to measure burnout, the Maslach Burnout Inventory (MBI; Maslach et al., 2016) is the most used (Ahola et al., 2017) and is commercially available. However, there are other options, some of them free, such as the Copenhagen Burnout Inventory (Kristensen et al., 2005); the Burnout Measure (Pines and Aronson, 1988; Malach-Pines, 2005); the Educator Burnout Inventory (Wang et al., 2003), the Oldenburg Burnout Inventory (OLBI; Bakker et al., 2004); the Shirom-Melamed Burnout Measure (Shirom and Melamed, 2006); the Bergen Burnout Indicator (Salmela-Aro et al., 2011); the Karolinska Exhaustion Scale (Saboonchi et al., 2013); and the Spanish Burnout Inventory (Gil-Monte et al., 2017).

OLBI seems to be the most prominent alternative to MBI (Demerouti et al., 2000). It was originally developed by Demerouti and Nachreiner (1998), who suggested two burnout dimensions, disengagement and exhaustion, applicable to professionals outside human services occupations. OLBI's versions vary across occupational groups and countries (see Table 1). In some countries—Brazil and Portugal (Campos et al., 2012), Sweden (Dahlin et al., 2007; Rudman et al., 2014), Slovenia (Kogoj et al., 2014), South Africa (Mokgele and Rothmann, 2014), Germany and Greece (Reis et al., 2015), and Malaysia (Mahadi et al., 2018)—OLBI has a version for students. OLBI does not contain any factor correspondent to what the MBI calls “professional efficacy”; this dimension received criticism in some studies (Bresó et al., 2007; Marôco et al., 2014), and, in the opinion of various authors, it is not a core burnout dimension (Bakker et al., 2004; Demerouti and Bakker, 2008) but can be interpreted as a possible burnout consequence (Koeske and Koeske, 1989) related to personality characteristics (Cordes and Dougherty, 1993).

TABLE 1

Table 1. OLBI's versions: validity evidence based on the internal structure.

The exhaustion subscale of OLBI has eight items which relate to feelings of emptiness, work overload, the need to rest, and physical, cognitive, and emotional exhaustion (Demerouti et al., 2003). Differently from the exhaustion concept presented in the MBI, the OLBI approach to exhaustion covers cognitive, physical, and affective aspects of exhaustion, which may facilitate the use of the instrument with workers of different kinds of activity (Demerouti et al., 2003; Bakker et al., 2004). The disengagement subscale has also eight items which refer to distancing oneself from the work, together with negative and cynical behaviors and attitudes in relation to one's job (Demerouti and Bakker, 2008). The OLBI's concept of disengagement differs from MBI's depersonalization in terms of the amplitude of the distancing, since OLBI's concept is broader: it may refer to distancing oneself from work in general or, more specifically, to distancing oneself from the content and object, along with experiencing negative attitudes (Demerouti et al., 2003). Thus, disengagement offers a less restricted view of the lack of interest in work. It is important to note that González-Romá et al. (2006) suggested that two of the three work engagement dimensions measured by the Utrecht Work Engagement Scale (UWES), vigor and dedication (the third dimension is absorption), can be paired with emotional exhaustion and cynicism (burnout dimensions). One dimension, named “identification,” involved dedication and cynicism; the other, named “energy,” comprised vigor and exhaustion factors, this indicates that OLBI's negatively- and positively-worded items can be markers for work engagement and burnout (Halbesleben and Demerouti, 2005).

Table 1 summarizes the different OLBI versions used with different samples found through a search of Embase, Scopus, PubMed, Web of Science, and Google Scholar using the terms: “OLBI,” “Oldenburg Burnout Inventory,” “adaptation,” “version,” “validity,” and “psychometric properties.” The OLBI's total number of items changed since its original structure of 25 items (Demerouti and Nachreiner, 1998) to 15 (Demerouti et al., 2001); today's English language version has 16 items (Bakker et al., 2004; Halbesleben and Demerouti, 2005). It has positively and negatively worded items—an equal number of each kind in the two dimensions—something that is considered an advantage (Price, 1997) since it can diminish acquiescence bias despite diminishing the internal consistency of the instruments (Salazar, 2015). OLBI has been translated into many languages, although not always evaluated in its psychometric properties (Table 1). Some studies use OLBI without taking into consideration the recommended steps to adapt an instrument for a country or culture different from the one for which it was originally developed (van de Vijver, 2016). There is a certain lack of use of adequate guidelines when translating and adapting the instrument for a new sample (International Test Commission, 2018). As can be observed in Table 1, the majority of the new OLBI versions have not evaluated their psychometric properties with the appropriate technique (confirmatory factor analysis [CFA]; Brown, 2015). In fact, some of them have avoided both exploratory and confirmatory factor analysis. Usually, the original two-factor structure is the one with better goodness-of-fit indices—even when compared with the other two-factor structures (e.g., with positive items in one group and negatively-worded items in the other), and with one- or four-factor structures (Demerouti et al., 2001). The measurement invariance/equivalence of the instrument across different groups is essential to properly establish comparisons (Davidov et al., 2014). OLBI invariance has been addressed by Demerouti et al. (2001) in its 15-items version for three different professions, they observed metric invariance (same factor's loadings). Demerouti and Nachreiner (1998) stated that three different groups of professionals obtained a similar OLBI (25-items) structure after a principal component analysis for each of the groups. Others researchers have obtained measurement invariance between countries (Demerouti et al., 2003) and between workers and students (Reis et al., 2015). These findings suggest that burnout is not exclusive to human services professions (Demerouti and Nachreiner, 1998; Demerouti et al., 2001) since various studies have tested burnout levels using OLBI in other occupations (e.g., executive directors, white-collar employees, construction workers). Altogether, few studies tested the measurement invariance of the groups with which they established comparisons. Finally, regarding the reliability of the scores, the internal consistency estimates were acceptable to good in most of the studies, while almost all studies reported only the Cronbach's α (see Table 1).

This study aims to describe the psychometric properties of an OLBI version developed simultaneously for workers from Brazil and Portugal, its validity evidence based on the internal structure (dimensionality, measurement invariance, reliability), and the validity evidence based on the relationship with other variables (work engagement); and to compare burnout across sexes and countries. Additionally, the study seeks to present a revision of OLBI's different versions since this is the first version of the instrument developed simultaneously for Portugal and Brazil, adapting an important instrument for understanding burnout in relation to sexes in the organizations. It will be structured by presenting some considerations about burnout among sexes, followed by burnout measured by OLBI.

Research Hypotheses

Following the recommendations of The Standards for Educational and Psychological Testing (American Educational Research Association, 2014), this paper aims to assess two types of validity evidence for the Portuguese version (PT-BR and PT-PT) of the OLBI (Bakker et al., 2004)—one related to the internal structure, the other based on the relations to other variables (work engagement). Since various studies have successfully confirmed the original two-factor structure of OLBI (Halbesleben and Demerouti, 2005; Peterson et al., 2011; Subburaj and Vijayadurai, 2016), it was hypothesized that the tested OLBI version would present a good fit confirming its original dimensionality of two factors (H1). Burnout has been hypothesized by some authors as a higher-order dimension (Taris et al., 1999; Shirom and Melamed, 2006; Marôco et al., 2008). Thus, a possible second-order latent factor, burnout, was tested for OLBI (H2). Through the review of the different versions (Table 1), the majority of the studies showed acceptable to very good reliability of the scores' evidence in terms of internal consistency (e.g., Demerouti and Bakker, 2008; Innstrand, 2016; Subburaj and Vijayadurai, 2016). Consequently, it was assumed that OLBI would present acceptable internal consistency reliability estimates (H3). Some studies found evidence of measurement invariance for OLBI between occupations (Demerouti et al., 2001; Demerouti and Bakker, 2008; Innstrand, 2016) and sex (Foster, 2015), but none investigated measurement invariance among workers of different countries. H4 hypothesized that OLBI will present evidence of measurement invariance between sexes and countries.

Research has found that burnout levels can vary among sexes, with females usually presenting slightly more exhaustion than males (Purvanova and Muros, 2010; Innstrand et al., 2011), females being more likely to experience burnout (Dimou et al., 2016). However, others suggest that research does not allow one to conclude any sex-specific risks (Seidler et al., 2014; Adriaenssens et al., 2015) considering that the burnout differences can be related with the levels of workload as well as care-load (Bekker et al., 2005; Langballe et al., 2011). Burnout can also vary among countries also (Poghosyan et al., 2010; Alexandrova-Karamanova et al., 2016; Jovanović et al., 2016). North American countries have a tendency to present higher exhaustion and disengagement levels than European countries—differences that can be related to cultural aspects (Maslach et al., 2001). However, regarding the Portugal-Brazil comparison, no differences were reported in a previous study (Dias et al., 2010). Occupations can play a substantial role in burnout levels (e.g., emotional challenges of working in the teaching or caregiving role) (Maslach et al., 2001). Altogether, it was hypothesized that burnout's latent means differ between sexes and countries (H5).

Work engagement is known to be a construct with strong correlations with burnout (Demerouti et al., 2010; Petrović et al., 2017), since both can be considered indicators of well-being (Bakker et al., 2014). Thus, the divergent validity evidence based on the relation to other variables, work engagement, was assessed (H6).

Methods

Sample

A total sample composed of 1,172 participants was collected by combining two independent samples: one sample of Brazilian workers in various occupations (n = 604), and one of Portuguese workers in various occupations (n = 568). Both samples completed the OLBI and the Utrecht Work Engagement Scale (Schaufeli and Bakker, 2003). Participation was anonymous and voluntary. The average age of the total sample was 35.47 years (SD = 9.95), with 65% being female. Workers' occupations were according to the International Standard Classification of Occupations ISCO-08 (International Labour Office, 2012)—mainly professionals or administrative support—and 73% of the sample were, at least, college graduates (Table 2). Regarding children, 59% had none; 45% reported being married or cohabiting.

TABLE 2

Table 2. Sociodemographics, occupational group, and academic level for each country, and total.

A non-probabilistic convenience sampling was used. The inclusion criteria were: (1) all participants were workers with a contract or formal ties with their employers, (2) had easy access to a PC, smartphone, or tablet to access the online platform where the instruments were deployed, and (3) were literate.

Measures

The OLBI was used to assess burnout, through the development of a version transculturally adapted both for Brazil and Portugal (Table 3). The OLBI is a self-report five-point rating scale (1 = “Strongly disagree”; to 5 = “Strongly agree”) with eight questions within each of the two dimensions, disengagement and exhaustion (Demerouti et al., 2001). The disengagement factor refers to distancing from work in terms of both object and content, and to the development of cynical and negative attitudes and behaviors in relation to one's job (Bakker et al., 2004). Exhaustion refers to feelings of physical fatigue, the need to rest, and feelings of overtaxing and emptiness in relation to work (Demerouti and Bakker, 2008). To develop the Portuguese version (Table 3) the English version of the OLBI was used (Bakker et al., 2004) following The ITC Guidelines for Translating and Adapting Tests (International Test Commission, 2018), adapting the items to the Portuguese language according to the Orthographic Agreement signed by both Portugal and Brazil in 2009. The items were discussed with Portuguese and Brazilian psychologists and methodologists to create a version of the items that gathered the consensus of specialists regarding cultural, semantic, and idiomatic equivalence in the two countries. Finally, a small pilot test was done with 15 workers from each country; this did not suggest any modifications and the Portuguese adapted OLBI's 16 items were understood. The final single version (for both countries) had no other changes.

TABLE 3

Table 3. OLBI original and Portuguese versions.

Work engagement refers to a positive motivational state and is composed of vigor, dedication, and absorption. This construct was measured with UWES-9 in its transculturally adapted version to both Brazil and Portugal (Sinval et al., 2018). It is a self-report instrument scored on a seven-point rating scale (0 = “Never”; 6 = “Always”), with three questions in each of its three dimensions. The UWES has shown good divergent validity evidence with the OLBI, since work engagement and burnout are moderately and negatively related (Goering et al., 2017). It was chosen not only for its good psychometric qualities for both countries, but also because it showed measurement invariance between both countries and it is a short instrument that allows for a robust work engagement measure with only a few items (Schaufeli and Bakker, 2003). It is a well-spread measure across many countries (Sinval et al., 2018) and is actually the most used instrument to measure work engagement. However, studies that investigated the relations between burnout and work engagement have mainly used the MBI for burnout and the UWES for work engagement (Schaufeli and de Witte, 2017). This study used OLBI together with UWES, trying to enrich the discussion about the two concepts, rather than just discussing instruments. The UWES dimensions are vigor, referring to the energy and resilience that one has in work; dedication, referring to being enthusiastic, inspired, and proud of one's work; and absorption, referring to being immersed in one's work without the perception of time passing (Schaufeli et al., 2002). It is expected that high levels of work engagement correspond to highly energized workers (Schaufeli and Bakker, 2010).

Procedures

Data were gathered from 2015 to 2017, in both countries, in an effort to have a larger sample, since web surveys present low response rates (Massey and Tourangeau, 2013). Both samples completed the OLBI, a brief sociodemographic questionnaire, and the UWES-9. All the collected data were obtained online using LimeSurvey software (LimeSurvey GmbH, 2017) running on the website of two major universities in each country. Nearly 35 percent of the disseminated questionnaires were completed in both countries. Participants were both contacted individually and through companies which answered positively to the invitation to participate in the study. Before filling out the survey, participants were informed about the study, assuring them that the study was a research study and that the company would not access individual data and that companies simply helped the researchers disseminate the study. Informed consent was obtained online from all participants.

To allow comparative studies, the same procedures were used in both countries. The study was approved by the Ethics Committee of the University of Porto (on 03-18-2015), Portugal, and the University of São Paulo (on 01-09-2014; CAAE no. 33301214.2.0000.5407), Brazil, and followed the usual rules for online surveys, namely, no access of participants' companies to individual results and no direct contact between participants and researchers [A few used the email to clarify some details about access to individual data, but it is not possible to identify whether they participated in the study].

Data Analysis

A confirmatory factor analysis (CFA) was conducted to verify if the original two-factor structure proposed by Bakker et al. (2004), presented an adequate fit to the study sample. Only complete data cases were considered. As goodness-of-fit indices, SRMR (Standardized Root Mean Square Residual), RMSEA (root mean square error of approximation), NFI (Normed Fit Index), CFI (Comparative Fit Index), and the TLI (Tucker Lewis Index) were used. The fit of the model was considered good for TLI, CFI and TLI values above 0.95; SRMR below 0.08; and RMSEA values below 0.08 (Hoyle, 1995; Boomsma, 2000; McDonald and Ho, 2002; Byrne, 2010). All statistical analyses were performed with R (R Core Team, 2018) and RStudio (RStudio Team, 2018). The descriptive statistics were obtained with the skimr package (McNamara et al., 2018), the standard error of the mean (SEM) was calculated with the plotrix (Lemon, 2006) package and the coefficient of variation (CV) was estimated with the package sjstats (Lüdecke, 2019). To assess multivariate normality, Mardia's multivariate kurtosis (Mardia, 1970) was used; it was calculated using the psych package (Revelle, 2018). The lavaan package (Rosseel, 2012) was selected to conduct the CFA analyses using the Weighted Least Squares Means and Variances (WLSMV) estimation method (Muthén, 1983).

To test the proposed structure for OLBI, the cross-validity evidence was assessed to give information about how well the new structure will fit an independent sample of the same population (Cudeck and Browne, 1983). To do so, the sample was randomly split into two sub-samples through the package minDiff (Papenberg, 2018). The workers' age was used as criteria variable for which it was desired to minimize differences between subsamples (Papenberg, 2018). The subsamples were generated using 1,000 repetitions in order to minimize the differences, since the most equal group assignment was selected. Having two independent subsamples with similar properties, one subsample can be used as calibration subsample, and another as validation subsample (Chin and Todd, 1995).

The convergent validity evidence was analyzed using the average variance extracted (AVE) which was estimated as described in Marôco (2014) and Fornell and Larcker (1981). The constructs' convergent validity evidence was assumed for values of AVE ≥ 0.5 (Hair et al., 2009).

The discriminant validity evidence was checked (Fornell and Larcker, 1981; Marôco, 2014) to verify whether the items that represent a dimension were strongly correlated with other dimensions (Marôco, 2014): for two factors, x and y, if AVEx and AVEy ≥ ρ²_xy (squared correlation between the factors x and y), there is discriminant validity evidence. The Heterotrait-monotrait (HTMT) criterion (Henseler et al., 2015) was also used. Values above 0.85 were considered indicative of satisfactory discriminant validity evidence (Kline, 2016). The HTMT ratios of correlations were calculated using the semTools package (Jorgensen et al., 2018).

The reliability of the scores was assessed with various estimates of internal consistency as recommended (Irwing and Hughes, 2018): α_ordinal (Zumbo et al., 2007), and ω_ordinal (Bollen, 1980; Raykov, 2001) using the semTools package (Jorgensen et al., 2018), higher values were indicative of better internal consistency results. Also, the McDonald's hierarchical omega (ω_H; Zinbarg et al., 2005) was estimated; a higher value of ω_H indicates a stronger influence of the latent variable common to all factors, and that the observed scale scores generalize to scores for the common latent variable (Zinbarg et al., 2007). The omega hierarchical subscale (ω_HS) was calculated for each specific factor, it reflects the reliability of each subscale after controlling for the variance due to the general factor (Reise et al., 2013). Both the ω_H and the ω_HS were used for calculating the internal consistency of the bi-factor model. There is some discussion about the use of α_ordinal (Revelle and Condon, 2018) as so we reported other estimates. The α_ordinal was calculated based on the polychoric correlations. However, the ω_ordinal and ω_H accounts for both item covariances and item thresholds (Green and Yang, 2009). The ω_ordinal and the ω_H are different in the denominator, the first assumes a congeneric factor model where measurement errors aren't correlated (Bollen, 1980), the second uses the observed covariance matrix instead of the model-implied covariance matrix (McDonald, 1999; Jorgensen et al., 2018). The CR was calculated by summing the z scores of the item scores. The second-order factor reliability was also calculated using the omega coefficient (Jorgensen et al., 2018). The proportion of observed variance explained by the second-order factor after controlling for the uniqueness of the first-order factor (ω_{partial L1}); the proportion of the second-order factor explaining the variance of the first-order factor level (ω_L2); and the proportion of the second-order factor explaining the total score (ω_L1) were also calculated. The reliability estimates were calculated with the semTools package (Jorgensen et al., 2018).

The measurement invariance of the higher-order model was assessed using the lavaan package (Rosseel, 2012), the categorical items were considered into account through theta-parameterization (Millsap and Yun-Tein, 2004) to compare a group of seven different models based on the recommendations of Millsap and Yun-Tein (2004) and on the second-order models' invariance specificities (Chen et al., 2005): (a) configural invariance; (b) first-order factor loadings; (c) second-order structural loadings; (d) thresholds of measured variables; (e) intercepts of first-order factors; (f) disturbances of first-order factors; and (g) residual variances of observed variables. Mean scores for burnout latent variable were compared within the structural equation modeling framework; effect sizes (Cohen's d) were determined (Cohen, 1988). The raw means, SDs and score percentiles were calculated using the doBy package (Højsgaard and Halekoh, 2018).

Results

The results related to psychometric properties of the OLBI in terms of internal structure are presented first, followed by the latent means comparisons, and finally by the validity evidence based on the relations to other variables.

Validity Evidence Based on Internal Structure

Dimensionality

Items' distributional properties

To judge distributional properties and psychometric sensitivity on the Portuguese and Brazilian samples, summary measures, skewness (Sk), kurtosis (Ku), and a histogram for each of the 16 items were used (Table 4). No strong deviations from the normal distribution (Finney and DiStefano, 2013) were considered for absolute values of Ku smaller than seven (7) and Sk smaller than three (3), assuring that they wouldn't compromise CFA results (Marôco, 2014). Mardia's multivariate kurtosis for the 16 items of OLBI was 48.88; p < 0.001. All possible Likert-scale answer values were observed on all items; no outliers were deleted. These items follow an approximately normal distribution in the normative population under study, since their distributional properties are indicative of appropriate psychometric sensitivity.

TABLE 4

Table 4. OLBI's items: descriptive statistics.

Factor-related validity evidence

To proceed with OLBI's transcultural adaptation to Brazil and Portugal, a cross-validity evidence approach was adopted. The sample was divided in two subsamples: calibration (n = 586) and validation subsamples (n = 586). The first was used to test which is OLBI's best solution in terms of fit to the data, and theoretical sense. The second subsample was used to assess cross-validity evidence of the proposed model. The two-factor OLBI fit to the data was mediocre (${\chi }_{(103)}^2$ = 720.764; p < 0.001; n = 586; CFI = 0.980; CFI_scaled = 0.918; NFI = 0.977; TLI = 0.977; SRMR = 0.072; RMSEA = 0.101; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.094; 0.108[), since CFI, NFI, and TLI values were above 0.95 (good fit), SRMR values were bellow 0.08 (good fit), but RMSEA values were above 0.10 being indicative of poor fit (MacCallum et al., 1996). One item presented a very low loading (λ_{item 13} = 0.220): and thus, this item was deleted. Also, based on the analysis of the modification indices, four correlations between items' residuals of the same factor were added, since it seems reasonable that indicators from the same factor explain shared error variance (Kline, 2016). The reduced model of 15 items showed better goodness-of-fit indices (Figure 1; ${\chi }_{(85)}^2$ = 514.098; p < 0.001; n = 586; CFI = 0.986; CFI_scaled = 0.937; NFI = 0.984; TLI = 0.983; SRMR = 0.064; RMSEA = 0.093; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.085; 0.101[), which indicated an acceptable fit, all items presented loadings above or equal to 0.47 (p < 0.001). The Cheung and Rensvold (2002) criterion (ΔCFI ≤ 0.01) supported the preference for the reduced model (ΔCFI_scaled = −0.019). Thus, H1 was accepted.

FIGURE 1

Figure 1. OLBI's two-factor reduced version (15-item) structure fit. A combined sample of Portuguese (n = 268) and Brazilian (n = 318) workers. Correlations between latent variables, residuals' correlations and factor loadings for each item are shown. ${\chi }_{(85)}^2$ = 514.098; p < 0.001; n = 586; CFI = 0.986; CFI_scaled = 0.937; NFI = 0.984; TLI = 0.983; SRMR = 0.064; RMSEA = 0.093; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.085; 0.101[.

Convergent validity evidence

To check if items contained within each factor are related to each other, the AVE was calculated for disengagement (AVE = 0.57), and for exhaustion (AVE = 0.50). These results suggest acceptable convergent validity evidence for the OLBI-15.

Discriminant validity evidence

The discriminant validity evidence between the two OLBI factors was unsatisfactory. These findings showed that the two factors are strongly related to each other, since AVE_{disengagement} = 0.57 and AVE_exhaustion = 0.50 were smaller than r²_DE = 0.69. Regarding the HTMT ratio of correlations (Henseler et al., 2015) the obtained value (0.80) is below the satisfactory threshold. These findings point to the fact that the two factors' correlation might be explained by a second-order latent factor, by a bi-factor model or by a unidimensional model.

Unidimensional model

A unidimensional model where the factor burnout loads on all 15 items was tested. The four residuals' correlations were maintained. This model assumes that the only latent factor that explains the manifest variables is burnout. As so, it assumes that the other two latent variables (i.e., disengagement and exhaustion) aren't meaningful by themselves since the discriminant validity evidence wasn't satisfactory. The content explained by them is similar, the unidimensional model tests if it is plausible to specify a single latent variable.

The OLBI's unidimensional model presented an mediocre fit (Figure 2; ${\chi }_{(86)}^2$ = 737.139; p < 0.001; n = 586; CFI = 0.979; CFI_scaled = 0.913; NFI = 0.977; TLI = 0.975; SRMR = 0.077; RMSEA = 0.114; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.106; 0.121[). Based on the Cheung and Rensvold (2002) criteria (ΔCFI ≤ 0.01) the two-factor reduced model was found to have a statistically better fit to these data than the unidimensional model (ΔCFI_scaled = −0.024). All the factor loadings and residuals' correlations were statistically significant (p < 0.001). Item 14 had the lowest factor loading (λ_{item 14} = 0.450).

FIGURE 2

Figure 2. OLBI's unidimensional reduced version (15 items) structure fit. A combined sample of Portuguese (n = 268) and Brazilian (n = 318) workers. Residuals' correlations and factor loadings for each item are shown. ${\chi }_{(86)}^2$ = 737.139; p < 0.001; n = 586; CFI = 0.979; CFI_scaled = 0.913; NFI = 0.977; TLI = 0.975; SRMR = 0.077; RMSEA = 0.114; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.106; 0.121[.

Bi-factor¹ model

A bi-factor model (Holzinger and Swineford, 1937; Holzinger and Harman, 1938) is a nested factor model (Gustafsson and Balke, 1993) or direct hierarchical model (Gignac, 2008) that specifies a single general factor among each measured variable that accounts for commonality shared by the related domains; and multiple specific orthogonal factors each of which account for unique variance above and over the general factor (Rios and Wells, 2014; Mansolf and Reise, 2017; Chen and Zhang, 2018). The bi-factor model has advantages (Canivez, 2016; Chen and Zhang, 2018), but also some limitations (Mulaik and Quartetti, 1997; Reise et al., 2010; Murray and Johnson, 2013) in comparison with higher-order models (e.g., second-order models), as so, the choice between them should be carefully weighted.

The OLBI's bi-factor model presented an acceptable fit (Figure 3; ${\chi }_{(75)}^2$ = 392.202; p < 0.001; n = 586; CFI = 0.990; CFI_scaled = 0.937; NFI = 0.987; TLI = 0.986; SRMR = 0.056; RMSEA = 0.085; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.077; 0.093[). The ΔCFI ≤ 0.010 criterion (Cheung and Rensvold, 2002) didn't find meaningful differences between the two-factor reduced model and the bi-factor model (ΔCFI_scaled = 0.000). All factor loading of the general factor (i.e., burnout) were statistically significant (p < 0.001), although the specific factors presented two non-significant loadings (α = 0.05), one on the disengagement subscale (λ_{item 11} = −0.001), and one on the exhaustion subscale (λ_{item 16} = 0.037).

FIGURE 3

Figure 3. OLBI's bi-factor reduced version (15 items) structure fit. A combined sample of Portuguese (n = 268) and Brazilian (n = 318) workers. Latent loadings for each factor; and factor loadings for each item are shown. ${\chi }_{(75)}^2$ = 392.202; p < 0.001; n = 586; CFI = 0.990; CFI_scaled = 0.937; NFI = 0.987; TLI = 0.986; SRMR = 0.056; RMSEA = 0.085; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.077; 0.093[.

Second-order model

A second-order latent factor may be admissible when two factors have high correlations between them, or/and when exists a higher order construct which might explain the lower order factors (Chen et al., 2005; Marôco, 2014). Since the two OLBI factors did not present satisfactory discriminant validity evidence between them, a second-order model was tested. The higher-order construct was named as burnout. Having as a start point the reduced model, and since there were not enough degrees of freedom to test the second-order latent model, the two structural weights between the second-order factor and the first-order factors were constrained to be equal.

The OLBI's second-order latent factor model presented an acceptable fit (Figure 4; ${\chi }_{(85)}^2$ = 514.098; p < 0.001; n = 586; CFI = 0.986; CFI_scaled = 0.937; NFI = 0.984; TLI = 0.983; SRMR = 0.064; RMSEA = 0.093; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.085; 0.101[). The RMSEA value was mediocre, however its confidence interval was precise and point estimates for RMSEA have been shown to depend on sample size and model misspecification and model degrees of freedom (MacCallum et al., 1996; Chen et al., 2008). Nevertheless, other goodness-of-fit indices were used in conjunction to assess models' adequacy. SRMR values were acceptable, which seem to be generally accurate across all conditions (Maydeu-Olivares et al., 2018). The constrained structural weights from burnout to disengagement and exhaustion were high (γ = 0.91; p < 0.001). These results suggest that burnout is a higher order construct reflected on disengagement and exhaustion. The findings show that hypothesis 2 can be confirmed, since the paths from the second-order latent to the first-order ones were statistically significant (p < 0.001) and had high values.

FIGURE 4

Figure 4. OLBI's second-order factor reduced version (15 items) structure fit. A combined sample of Portuguese (n = 268) and Brazilian (n = 318) workers. Latent loadings for each factor; residuals' correlations and factor loadings for each item are shown. ${\chi }_{(85)}^2$ = 514.098; p < 0.001; n = 586; CFI = 0.986; CFI_scaled = 0.937; NFI = 0.984; TLI = 0.983; SRMR = 0.064; RMSEA = 0.093; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.085; 0.101[.

The differences between the second-order model and the two-factor reduced model (ΔCFI_scaled = 0.000) and the bi-factor model (ΔCFI_scaled = 0.000) weren't meaningful based on the ΔCFI criteria (Cheung and Rensvold, 2002). After having in consideration all tested models (Table 5) the second-order model was selected, since it hadn't a worst fit than the bi-factor and the two-factor reduced model (based on the used criterion). This choice emerged as solution for the lack of evidence of discriminant validity of the two-factor reduced model. And also as a plausible option in theoretical terms as also suggested by other authors who have proposed a burnout second-order factor using MBI, CBI and OLBI (Marôco et al., 2008, 2014; Marôco and Campos, 2012). The bi-factor model presents equivalent fit, and some authors also proposed it as an alternative structure using MBI and the Job Burnout Scale (Wang and Gao, 2010; Mészáros et al., 2014; Morgan et al., 2014) although with problems in some cases (i.e., non-convergence, unsatisfactory unique proportion of variance explained of the observed scores). The second-order factor had very high structural weights, while the bi-factor model had only one factor loading above 0.50 on each for the two specific factors, pointing for clear insufficient proportion of the variance explained on the specific factors. Altogether, the obtained results seem to present evidence that favors the second-order model.

TABLE 5

Table 5. OLBI models' goodness-of-fit indices.

This structure also showed cross-validity evidence, since it presented a good fit to the data also when using the validation sample (${\chi }_{(85)}^2$ = 591.172; p < 0.001; n = 586; CFI = 0.985; CFI_scaled = 0.934; NFI = 0.983; TLI = 0.982; SRMR = 0.068; RMSEA = 0.101; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.093; 0.109[). The structural weights were high (γ = 0.93; p < 0.001).

Reliability of the Scores: Internal Consistency Evidence

To estimate the reliability of the scores on the various models, the complete sample was used. The internal consistency values of the two-factor first-order model reduced were high for the three coefficients (Table 6). They suggest very good validity evidence in terms of the reliability of the scores.

TABLE 6

Table 6. Internal consistency of OLBI dimensions (two-factor reduced version).

The same was observed on the unidimensional model internal consistency estimates (CR_burnout = 0.93; α_ordinal = 0.93; ω_ordinal = 0.91). Based on a bi-factor model, the hierarchical omega was high (ω_H = 0.85) and omega hierarchical subscale (ω_HS; Reise, 2012; Reise et al., 2013; Rodriguez et al., 2016b) were low (ω_{HS disengagement} = 0.08; ω_{HS exhaustion} = 0.20). The specific factors reliability score after controlling for the variance due to the general factor was clearly unsatisfactory, reinforcing the evidence in favor of the second-order model. The ω_H value was high (Rodriguez et al., 2016a), suggesting a strong influence of the latent variable common to the two factors.

The proportion of observed variance explained by the second-order factor after controlling for the uniqueness of the first-order factor (ω_{partial L1}) was 0.93. The proportion of the variance of the first-order factors explained by the second-order factor (ω_L2) was 0.91, and the proportion of the second-order factor explaining the total score (ω_L1) was 0.86. Thus, the internal consistency of the second-order construct was indicative of very good values.

Measurement Invariance

To verify if measurement invariance holds, the complete sample was used. The fit to the data of each individual group was globally acceptable. The Brazilian sample had an acceptable fit (${\chi }_{(85)}^2$ = 687.077; p < 0.001; n = 604; CFI = 0.982; CFI_scaled = 0.922; NFI = 0.980; TLI = 0.978; SRMR = 0.073; RMSEA = 0.109; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.102; 0.117[) as so did the Portuguese sample (${\chi }_{(85)}^2$ = 494.338; p < 0.001; n = 568; CFI = 0.988; CFI_scaled = 0.943; NFI = 0.985; TLI = 0.985; SRMR = 0.065; RMSEA = 0.092; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.084; 0.100[). The fit of the Females sample was slight better (${\chi }_{(85)}^2$ = 537.474; p < 0.001; n = 678; CFI = 0.989; CFI_scaled = 0.944; NFI = 0.987; TLI = 0.986; SRMR = 0.059; RMSEA = 0.089; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.082; 0.096[) whereas the Males sample fit was marginally worse (${\chi }_{(85)}^2$ = 492.213; p < 0.001; n = 364; CFI = 0.979; CFI_scaled = 0.922; NFI = 0.975; TLI = 0.974; SRMR = 0.081; RMSEA = 0.115; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.105; 0.125[).

To assess if the same second-order latent model holds in each sex and country, seven nested models with indications of equivalence were used (Marôco, 2014). Full uniqueness measurement invariance was supported for countries (Table 7) based on the Cheung and Rensvold (2002) criterion (absolute ΔCFI_scaled ≤ 0.010) and on the Chen (2007) criterion (absolute ΔRMSEA_scaled ≤ 0.015). The Δχ² criterion (Satorra and Bentler, 2001) demonstrated the second-order metric invariance. Since the Δχ² criterion is too restrictive, we opted for the ΔCFI_scaled criterion. Results supported the structural invariance of the OLBI between Portugal and Brazil. The measurement invariance for OLBI among sexes (Table 7) was obtained, since full uniqueness measurement invariance was observed with the support of the Cheung and Rensvold (2002) criterion (absolute ΔCFI_scaled ≤ 0.010) and of the Chen (2007) criterion (absolute ΔRMSEA_scaled ≤ 0.015).

TABLE 7

Table 7. OLBI second-order latent model measurement invariance.

Sex's and Country's Burnout Latent Means Comparisons and Dimensions' Quartiles

Following the existence of full uniqueness measurement invariance, latent means can be compared. The results of the chi-square difference test suggest that weren't significant differences in burnout (Δχ²_scaled (1) = 1.110; p = 0.292; d = 0.067) among countries. There weren't also statistically significant differences among countries in relation to the burnout dimension (Δχ²_scaled (1) = 1.066; p = 0.302; d = 0.066). The quartiles, means, and SDs (raw values) for each sex within each country are presented in Table 8, these values are presented with the intent of providing population norms values.

TABLE 8

Table 8. Quartiles, means, and SDs for OLBI's dimensions (raw values) for countries and sexes.

Validity Evidence Based on the Relations to Other Variables

Burnout can be conceptualized as being the opposite of work engagement (Halbesleben and Demerouti, 2005). The UWES-9 second-order latent factor model presented a good fit (${\chi }_{(24)}^2$ = 175.820; p < 0.001; n = 1,104; CFI = 0.999; CFI_scaled = 0.992; NFI = 0.999; TLI = 0.999; SRMR = 0.028; RMSEA = 0.076; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.065; 0.086[). All factor loadings were statistically significant as also one added residuals' correlation from item 1 and 2 (r = 0.76). The internal consistency reliability estimates were good both for the first-order model (ω_vigor = 0.84; ω_dedication = 0.92; ω_absorption = 0.88; ω_total = 0.96) and for the second-order (ω_{partial L1} = 0.96; ω_L1 = 0.94; ω_L2 = 0.97).

The first-order models of each instrument were used to establish correlations among the first-order latent variables (Table 9), and the second-order models of the instruments were used to analyze the correlation among the respective second-order latent variables (r_{burnout*work engagement} = −0.85). The obtained correlation between the latent OLBI and UWES dimensions was negative and moderate to high, demonstrating the divergent validity of the measures obtained with OLBI (burnout) and the UWES-9 (work engagement).

TABLE 9

Table 9. Correlations between OLBI's and UWES-9's latent variables.

Discussion

Regarding the psychometric properties of the OLBI, results of this study provide evidence of the two-factor structure of the original instrument, having convergent validity evidence and good goodness-of-fit indices except for RMSEA value, which can be indicative of moderate errors of approximation in the population. Nevertheless, this value had a narrow confidence interval, that reflects a good precision of the model fit in the population (MacCallum et al., 1996). However, the discriminant validity evidence was not satisfactory (H1), which led us to test a possible second-order latent factor. Item 13 was removed from the tested version. Problems with item 13 also have been reported by Chevrier (2009). Item 13 has also been deleted from the OLBI-S proposal for Portuguese and Brazilian students (Campos et al., 2012) and OLBI's Malay version for students (Mahadi et al., 2018). In the Brazilian version for workers, that item was removed from the proposed reduced 13-item version (Schuster and Dias, 2018). Its removal was also suggested from the Italian version (Estévez-Mujica and Quintane, 2018). Other authors found that item 13's removal increased the disengagement internal consistency (Baka and Basinska, 2016). In the OLBI reduced Russian version (Smirnova, 2017), that item was the only one that lacked statistically significant loading, also being removed. In the Swedish version, it was the item with the lowest loading (λ_{item 13} = 0.38) (Peterson et al., 2011). Item 13's content, “This is the only type of work that I can imagine myself doing,” did not seem to make sense within today's economic and professional context, where careers are so uncertain and the number of different employers across a career is increasing (Savickas, 2012). Furthermore, the sample is composed mainly of younger workers, most of whom have a higher education. This can contribute to a perception of more control over their career and desire for more professional experiences rather than maintaining the same employer for a long period. Additionally, four correlations within pairs of residuals' belonging to the same factor were added. Such modifications mean that some unwanted or unexplained source of variance exist outside of the original model, which can be due to various reasons (e.g., lack of comprehension of the items or an unwanted theoretical trait present in that factor) and as such is speculative (Cote et al., 2001). Nevertheless, it seems plausible that theory can be slightly imprecise, and since the model (second-order) passed through the cross-validation approach and reinforced the validity evidence obtained with the calibration subsample. The correlation between the two OLBI's dimensions was high; this finding is shared by other studies which found correlation values between OLBI's factors similar to ours (Khan et al., 2016). Those findings also pointed to unsatisfactory discriminant validity evidence between disengagement and exhaustion.

Our internal consistency estimates presented very good values supporting H2, in line with most of the studies which evaluated this kind of estimate (Table 1). Still, this study goes one step further, since it obtained estimates that gave evidence about the reliability of the second-order latent factor and the reliability of a potential bi-factor model (ω_H).

We admitted a possible second-order factor which was confirmed (H3) since the goodness-of-fit indices were not worse than the reduced version of the first-order model. The second-order model was also compared with a bi-factor model, which also didn't present meaningful better fit (i.e., ΔCFI_scaled < 0.01) to the data than the second-order model. The bi-factor model has various advantages (Reise et al., 2010), although—in this study- those potential advantages weren't confirmed by the obtained results. Since the unique variance explained by the specific factors of (after controlling for the variance due to the general factor) wasn't satisfactory (Rodriguez et al., 2016b). Additionally, empirical observations strongly suggest the second-order model presented high values of structural weights both on the calibration as on the validation subsamples. Regarding the existent theory, some authors proposed burnout as a second-order factor (Taris et al., 1999; Shirom and Melamed, 2006; Marôco et al., 2008) the same applies to the bi-factor model with some authors preferring it in relation to hierarchical model (Mészáros et al., 2014). However, if there wasn't prior knowledge from the field regarding burnout conceptualization, exploratory bi-factor analysis (Jennrich and Bentler, 2011) or bi-factor exploratory structural equation modeling (Morin et al., 2015) cloud be also suitable analyses. If one desires to analyze bi-factor models outside the standard CFA procedures (e.g., setting to zero paths from latent constructs to indicators that are not theoretically associated) can adopt the already referred bi-factor exploratory structural equation modeling or the bi-factor Bayesian structural equation modeling (Golay et al., 2013) analyses. Marôco and Campos (2012) have proposed the a second-order interpretation of burnout, although using a different instrument, the MBI. In other study, Marôco and Campos (2012) made the same suggestion using the MBI, CBI and OLBI. This suggests that, besides the two OLBI first-order factors, there is a higher-order more general burnout dimension. The obtained findings contribute to the study of OLBI and the burnout dimensionality, maintaining the two-order structure, but also suggesting a second-order latent, which brings a novelty to this study, since this was proposed for the first time for OLBI using a sample of workers.

The OLBI presented measurement invariance for sex and country supporting H4. To the best of our knowledge, this was the first time measurement invariance was assessed across sexes for the full OLBI instrument, since Foster (2015) only tested measurement invariance among sex on the separate factors. No study has found testing measurement invariance among countries, bringing a novelty to this study.

No statistically significant differences were found between sexes for burnout (H5), which shows that each sex experiences that dimension in the workplace similarly. Females are known for having higher levels of negative emotional states than males (Kessler et al., 2005); in consequence, females usually score higher than males on exhaustion (Purvanova and Muros, 2010; Kumar and Mellsop, 2013; Pu et al., 2017; Schadenhofer et al., 2018). However, a meta-analysis (Purvanova and Muros, 2010) found that males usually have higher depersonalization levels than females. Altogether, the differences reported on previous studies seem to be annulling each other in terms of the general second-order factor. The relationship between burnout dimensions and sex is not always clear. O'Connor et al. (2018) suggested in a meta-analysis on burnout in health professionals, that the burnout dimensions and sex have an inconsistent relationship. A similar finding was reported by Estévez-Mujica and Quintane (2018), which found no relationship between exhaustion, disengagement, and burnout and sex in a sample of research and development workers. A recent study with Portuguese health professionals at the national level, found no significant differences among sexes in terms of burnout (Marôco et al., 2016). For the country factor, no significant differences were observed for the burnout latent variable (H5). These results are in line with the findings of a comparative study among Portuguese and Brazilian health professionals using the MBI (Dias et al., 2010), no significant differences were found between the burnout dimensions. Western European countries seem to present lower average burnout scores than in other parts of the world (Golembiewski et al., 1996). These differences can be due to cultural differences (Golembiewski et al., 1993; Maslach et al., 2001), In some countries, as Brazil, such differences can also occur between regions of the same country as a result of being such a big and culturally-mixed country (Hofstede et al., 2010b). Differences between Portugal and Brazil seem to exist at the individualism level, with Portugal being more collectivist and with smaller power distance (Hofstede et al., 2010a). In Brazil, employers seem to be more risk-taking than their Portuguese counterparts (Silva et al., 2009). In other words, Portugal has larger avoidance to uncertainty values (Hofstede et al., 2010a). In Brazil, organizations operate through general rule as much as through personal relationships (Garibaldi de Hilal, 2009); whereas in Portugal, the work relations appear to be more impersonal and formal (Dias et al., 2010). Brazil presents larger indulgence values than Portugal (Hofstede et al., 2010a); the Brazilian culture also reflects ambiguity and double-edged ethics (Garibaldi de Hilal, 2006). Brazilian organizations seem to perceive responsibility toward employees as one of the less important business priorities (Hofstede et al., 2010a), which can lead to poorer attention to work conditions.

Regarding the last hypothesis, the validity evidence based on the relations with other variables was good (H6), since the presented correlations were moderate to high between OLBI and UWES factors. Other studies found Pearson's correlation value −0.55 between the work engagement (UWES) score and OLBI's disengagement factor, and −0.48 between work engagement (UWES) and OLBI's exhaustion (Bosman et al., 2005). In the present study, a higher work engagement correlation with disengagement than with exhaustion was also found. The produced results were in line with the findings of Petrović et al. (2017), who found similar correlation values between the instruments' dimensions and a higher correlation between the pairs of variables previously referred to González-Romá et al. (2006). In other words, the correlations between vigor and exhaustion were higher than with the other UWES variables, and the same for the disengagement-dedication pair. This study's results are in the same direction, reinforcing the higher association between disengagement and dedication than between disengagement and vigor or absorption, and a higher association between exhaustion and vigor than between exhaustion and dedication and absorption.

Our results confirmed the four of the five hypotheses and gave us, globally, good validity evidence regarding this OLBI version. However, since H1 had unsatisfactory validity evidence in terms of discriminant validity, this study brought a novelty (in terms of OLBI dimensionality studies); therefore, a second-order latent model was proposed, which is admissible from a theoretical and practical perspective. Thus, it is suggested that OLBI can be used to compare burnout levels among samples with different occupations and sexes from Portugal and Brazil. However, since the proposed version is a reduced version, it must be said that the reduced version must be tested in independent samples of the same populations (Marôco, 2014).

Theoretical Implications

Burnout seems to be a second-order factor that loads on two first-order factors, disengagement, and exhaustion—which are burnout's core dimensions. This underlines the expansion of the burnout domain beyond the exhaustion affective component (Halbesleben et al., 2004). Female and male workers experience similar levels of burnout, the same happens among Portuguese and Brazilian workers. The obtained results demonstrate that the absence of differences in burnout between sexes and countries suggest that the work experiences in terms of stress are similar in both samples. The observed differences between sexes reported on other studies are small (Purvanova and Muros, 2010) and might be related to other factors—namely family and workload (Bekker et al., 2005; Langballe et al., 2011).

Practical Implications

To have instruments with good validity evidence, it is mandatory to have confidence in the obtained measures. OLBI can help to establish comparisons between sexes, and countries. However, one should be aware of the different versions and of the quality of the evidence provided in each study. Also, OLBI can be useful and practical, since it is a freely available self-report psychometric instrument which can contribute to studies where the impact of companies' interventions are studied (Gíslason and Símonardóttir, 2018). In fact, there is evidence that occupational stress can be reduced with specific interventions (Ruotsalainen et al., 2015). The development of a family-friendly work environment should be approached, allowing one a focus on the importance of a balance between work and life (Lo, 2003; Rubino et al., 2013). Giving workers paid sick days, medical and family leave insurance programs, and greater control over their schedule (Appelbaum et al., 2014) are good suggestions to improve the balance between work and home activities. However, any change in organizational practices without a corresponding change in social attitudes will not be enough (Field and Bramwell, 1998). Prevention strategies should consider the social and individual level of those that will receive them (Maslach and Leiter, 2017).

Limitations

This study used two convenience samples. It had no other psychological measures besides burnout and work engagement, which would allow better assessment of the validity evidence based on the relationship between other variables—namely, predictive, concurrent, and discriminant evidence (American Educational Research Association, 2014). For example, a concurrent burnout measure which would allow verification of the concurrent validity evidence between different burnout instruments, as some studies have done (Demerouti et al., 2003; Marôco and Campos, 2012) would have been useful. Another limitation is that culture-specific aspects of stress were not assessed, which could explain some of the observed differences.

Future Research

Further studies using OLBI should test its concurrent validity evidence with other burnout instruments (American Educational Research Association, 2014), something that has been tested with success in the Portugal-Brazil version for students (Campos et al., 2012) and in the Greek version for workers (Demerouti et al., 2003). The Portugal-Brazil OLBI version should be tested in samples of specific occupational groups (e.g., Armed Forces Occupations, Craft and Related Trades Workers, Skilled Agricultural, Forestry and Fishery Workers, Plant and Machine Operators and Assemblers, and Elementary Occupations), of whom there was not a satisfactory number in the sample collected for this study. The same applies to lower academic levels, which showed unsatisfactory frequency in this study. Future research should also assess the family load together with the workload to understand the family-work interaction regarding burnout.

Conclusion

Initially, burnout research was linked to human services occupations: thus, sex was not a concern since most employees in this area were female. This study compared burnout levels between sexes in two different countries, and simultaneously adapting a specific instrument that allows establishing direct comparisons between countries.

OLBI offers various advantages over other instruments that can measure burnout, and the obtained findings focus on the utility of this inventory to compare burnout among sexes and countries using samples from Portugal and Brazil. The instrument showed validity evidence based on the internal structure and on the relation with other variables (work engagement and its first-order dimensions). Altogether, the proposed OLBI version appears to be a valid alternative to assess burnout and establish rigorous comparisons between Portuguese and Brazilian workers. The differences between sexes seem to non-existent. Burnout differences reported in other studies seem to be related to other factors, such as work and family load, as previous research suggested (Bekker et al., 2005; Langballe et al., 2011).

Author Contributions

All authors of this research paper have directly participated in the planning, execution, or analysis of this study. More specifically, JS wrote the first draft, and with JM performed all statistical analysis and its discussion. JS and SP discussed cross-cultural topics, and JS and CQ discussed theoretical framework.

Funding

The authors would like to thank the Portuguese National Occupational Health Program of the Directorate-General of Health (DGS) for the sponsorship. And to the University of São Paulo (USP) International grant conceded to JS (Grant agreement number: 2014.1.3766.1.8). This work was partially funded by the Center for Psychology at the University of Porto, Portuguese Science Foundation (FCT UID/PSI/00050/2013) and EU FEDER through COMPETE 2020 program (POCI-01-0145-FEDER-007294). This work was partially funded by the William James Center for Research, Portuguese Science Foundation (FCT UID/PSI/04810/2013).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

1. ^Bi-factor or bifactor model are both acceptable forms, we adopted the first one, originally used by Holzinger and Swineford (1937).

References

Adriaenssens, J., De Gucht, V., and Maes, S. (2015). Determinants and prevalence of burnout in emergency nurses: a systematic review of 25 years of research. Int. J. Nurs. Stud. 52, 649–661. doi: 10.1016/j.ijnurstu.2014.11.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Ahola, K., Toppinen-Tanner, S., and Seppänen, J. (2017). Interventions to alleviate burnout symptoms and to support return to work among employees with burnout: Systematic review and meta-analysis. Burn. Res. 4, 1–11. doi: 10.1016/j.burn.2017.02.001

CrossRef Full Text | Google Scholar

Al-Asadi, J., Khalaf, S., Al-Waaly, A., Abed, A., and Shami, S. (2018). Burnout among primary school teachers in Iraq: prevalence and risk factors. East. Mediterr. Heal. J. 24, 262–268. doi: 10.26719/2018.24.3.262

PubMed Abstract | CrossRef Full Text | Google Scholar

Alexandrova-Karamanova, A., Todorova, I., Montgomery, A., Panagopoulou, E., Costa, P., Baban, A., et al. (2016). Burnout and health behaviors in health professionals from seven European countries. Int. Arch. Occup. Environ. Health 89, 1059–1075. doi: 10.1007/s00420-016-1143-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Al-shuhail, A. S., Al-saleh, S., Al-saleh, S. S., Elhassan, I., and Wajid, S. (2017). Prevalence of burnout among National Guard Health Affairs physicians in Dammam, Saudi Arabia–a cross-sectional study. Asian J. Pharm. 11, S924–S929. doi: 10.22377/ajp.v11i04.1738

CrossRef Full Text | Google Scholar

Amagasa, T., Nakayama, T., and Takahashi, Y. (2005). Karojisatsu in Japan: characteristics of 22 cases of work-related suicide. J. Occup. Health 47, 157–164. doi: 10.1539/joh.47.157

PubMed Abstract | CrossRef Full Text | Google Scholar

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing. Washington, DC: American Psychological Association.

Ananthram, S., Teo, S. T. T., Connell, J., and Bish, A. (2017). Control and involvement HR practices in Indian call centres: still searching for answers. Asia Pacific J. Hum. Resour. 56, 196–215. doi: 10.1111/1744-7941.12153

CrossRef Full Text | Google Scholar

Appelbaum, E., Boushey, H., and Schmitt, J. (2014). The Economic Importance of Women's Rising Hours of Work: Time to Update Employment Standards. Center for American Progress and the Center for Economic and Policy Research. Available online at: https://www.americanprogress.org/wp-content/uploads/2014/04/WomensRisingWorkv2.pdf

Baka, Ł. (2011). Relationship between work – family conficts and burnout: mediating role of personal resources. Psychol. Społeczna 4, 367–374. Available online at: http://spbulletin.com/wp-content/uploads/2016/11/Baka_PS_4_2011.pdf

Baka, Ł., and Basinska, B. (2016). Psychometric properties of the Polish version of the Oldenburg Burnout Inventory (OLBI). Med. Pr. 67, 29–41. doi: 10.13075/mp.5893.00353

PubMed Abstract | CrossRef Full Text | Google Scholar

Baka, Ł., and Cieślak, R. (2010). Self-efficacy and social support and the effects of job stressors on job burnout and work engagement in teachers. Stud. Psychol. 48, 5–19. Available online at: https://www.researchgate.net/profile/Lukasz_Baka/publication/281209412_Zaleznosci_miedzy_stresorami_w_pracy_a_wypaleniem_zawodowym_i_zaangazowaniem_w_prace_w_grupie_nauczycieli/links/55db818608aec156b9afecc2/Zaleznosci-miedzy-stresorami-w-pracy-a-wypaleni

Bakker, A. B., and Demerouti, E. (2007). The job demands-resources model: state of the art. J. Manag. Psychol. 22, 309–328. doi: 10.1108/02683940710733115

CrossRef Full Text | Google Scholar

Bakker, A. B., Demerouti, E., and Sanz-Vergel, A. I. (2014). Burnout and work engagement: the JD–R approach. Annu. Rev. Organ. Psychol. Organ. Behav. 1, 389–411. doi: 10.1146/annurev-orgpsych-031413-091235

CrossRef Full Text | Google Scholar

Bakker, A. B., Demerouti, E., and Verbeke, W. (2004). Using the job demands-resources model to predict burnout and performance. Hum. Resour. Manage. 43, 83–104. doi: 10.1002/hrm.20004

CrossRef Full Text | Google Scholar

Barbier, M., Peters, S., and Hansez, I. (2009). Measuring positive and negative occupational states (PNOSI): structural confirmation of a new Belgian tool. Psychol. Belg. 49, 227–247. doi: 10.5334/pb-49-4-227

CrossRef Full Text | Google Scholar

Bekker, M. H. J., Croon, M. A., and Bressers, B. (2005). Childcare involvement, job characteristics, gender and work attitudes as predictors of emotional exhaustion and sickness absence. Work Stress 19, 221–237. doi: 10.1080/02678370500286095

CrossRef Full Text | Google Scholar

Bianchi, R. (2015). A reflection on the measurement of the burnout syndrome. Acad. Emerg. Med. 22:378. doi: 10.1111/acem.12611

PubMed Abstract | CrossRef Full Text | Google Scholar

Bianchi, R., Schonfeld, I. S., and Laurent, E. (2017). Burnout: moving beyond the status quo. Int. J. Stress Manag. 26, 36–45. doi: 10.1037/str0000088

CrossRef Full Text | Google Scholar

Bollen, K. A. (1980). Issues in the comparative measurement of political democracy. Am. Sociol. Rev. 45, 370–380. doi: 10.2307/2095172

CrossRef Full Text | Google Scholar

Boomsma, A. (2000). Reporting analyses of covariance structures. Struct. Equ. Model. A Multidiscip. J. 7, 461–483. doi: 10.1207/S15328007SEM0703_6

CrossRef Full Text | Google Scholar

Bosman, J., Rothmann, S., and Buitendach, J. H. (2005). Job insecurity, burnout and work engagement: the impact of positive and negative affectivity. SA J. Ind. Psychol. 31, 48–56. doi: 10.4102/sajip.v31i4.199

CrossRef Full Text | Google Scholar

Bresó, E., Salanova, M., and Schaufeli, W. B. (2007). In search of the “third dimension” of burnout: efficacy or inefficacy? Appl. Psychol. An Int. Rev. 56, 460–478. doi: 10.1111/j.1464-0597.2007.00290.x

CrossRef Full Text | Google Scholar

Brown, T. A. (2015). Confirmatory Factor Analysis for Applied Research. 2nd Edn. New York, NY: The Guildford Press.

Buitendach, J. H., Bobat, S., Muzvidziwa, R. F., and Kanengoni, H. (2016). Work engagement and its relationship with various dimensions of work-related well-being in the public transport industry. Psychol. Dev. Soc. J. 28, 50–72. doi: 10.1177/0971333615622895

CrossRef Full Text | Google Scholar

Byrne, B. M. (2010). Structural Equation Modeling with AMOS. 3rd Edn. New York, NY: Routledge.

Campos, J. A. D. B., Carlotto, M. S., and Marôco, J. (2012). Oldenburg Burnout Inventory–student version: cultural adaptation and validation into Portuguese. Psicol. Reflex. Crít. 25, 709–718. doi: 10.1590/S0102-79722012000400010

CrossRef Full Text | Google Scholar

Canivez, G. L. (2016). “Bifactor modeling in construct validation of multifactored tests: implications for understanding multidimensional constructs and test interpretation,” in Principles and Methods of Test Construction: Standards and Recent Advances, eds K. Schweizer, and C. DiStefano (Gottingen: Hogrefe), 247–271.

Google Scholar

Carod-Artal, F. J., and Vázquez-Cabrera, C. (2013). “Burnout syndrome in an international setting,” in Burnout for Experts: Prevention in the Context of Living and Working, ed S. Bährer-Kohler (Boston, MA: Springer), 15–35.

Google Scholar

Chen, F., Curran, P. J., Bollen, K. A., Kirby, J., and Paxton, P. (2008). An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models. Sociol. Methods Res. 36, 462–494. doi: 10.1177/0049124108314720

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct. Equ. Model. A Multidiscip. J. 14, 464–504. doi: 10.1080/10705510701301834

CrossRef Full Text | Google Scholar

Chen, F. F., Sousa, K. H., and West, S. G. (2005). Teacher's corner: testing measurement invariance of second-order factor models. Struct. Equ. Model. A Multidiscip. J. 12, 471–492. doi: 10.1207/s15328007sem1203_7

CrossRef Full Text | Google Scholar

Chen, F. F., and Zhang, Z. (2018). “Bifactor models in psychometric test development,” in The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development, eds P. Irwing, T. Booth, and D. Hughes (Hoboken, NJ: Wiley), 325–345.

Google Scholar

Chernoff, P., Adedokun, C., O'Sullivan, I., McManus, J., and Payne, A. (2018). Burnout in the emergency department hospital staff at Cork University Hospital. Ir. J. Med. Sci. 1–8. doi: 10.1007/s11845-018-1871-5. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheung, G. W., and Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Struct. Equ. Model. A Multidiscip. J. 9, 233–255. doi: 10.1207/S15328007SEM0902_5

CrossRef Full Text | Google Scholar

Chevrier, N. (2009). Adaptation Québécoise de l'Oldenberg Burnout Inventory (OLBI) [Quebecer adaptation of the Oldenburg Burnout Inventory (OLBI)] Montreal, QC: Université du Québec.

Chin, W. W., and Todd, P. A. (1995). On the use, usefulness, and ease or use of structural equation modeling in MIS research: a note of caution. MIS Q. 19, 237–246. doi: 10.2307/249690

CrossRef Full Text | Google Scholar

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. 2nd Edn. Hillsdale, NJ: Lawrence Erlbaum Associates.

Google Scholar

Cordes, C. L., and Dougherty, T. W. (1993). A review and an integration of research on job burnout. Acad. Manag. Rev. 18, 621–656. doi: 10.5465/AMR.1993.9402210153

CrossRef Full Text | Google Scholar

Cote, J., Netemeyer, R., and Bentler, P. M. (2001). Improving model Fit by correlating errors. J. Consum. Psychol. 10, 87–88. doi: 10.2307/1480490

CrossRef Full Text

Creedy, D. K., Sidebotham, M., Gamble, J., Pallant, J., and Fenwick, J. (2017). Prevalence of burnout, depression, anxiety and stress in Australian midwives: a cross-sectional survey. BMC Pregnancy Childbirth 17, 1–8. doi: 10.1186/s12884-016-1212-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Cudeck, R., and Browne, M. W. (1983). Cross-validation of covariance structures. Multivariate Behav. Res. 18, 147–167. doi: 10.1207/s15327906mbr1802_2

PubMed Abstract | CrossRef Full Text | Google Scholar

Dahlin, M., Joneborg, N., and Runeson, B. (2007). Performance-based self-esteem and burnout in a cross-sectional study of medical students. Med. Teach. 29, 43–48. doi: 10.1080/01421590601175309

PubMed Abstract | CrossRef Full Text | Google Scholar

Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., and Billiet, J. (2014). Measurement equivalence in cross-national research. Annu. Rev. Sociol. 40, 55–75. doi: 10.1146/annurev-soc-071913-043137

CrossRef Full Text | Google Scholar

Delgadillo, J., Saxon, D., and Barkham, M. (2018). Associations between therapists' occupational burnout and their patients' depression and anxiety treatment outcomes. Depress. Anxiety 35, 844–850. doi: 10.1002/da.22766

PubMed Abstract | CrossRef Full Text | Google Scholar

Demerouti, E., and Bakker, A. B. (2008). “The Oldenburg Burnout Inventory: a good alternative to measure burnout and engagement,” in Handbook of Stress and Burnout in Health Care, ed J. R. B. Halbesleben (New York, NY: Nova Science), 65–78. Available online at: https://pdfs.semanticscholar.org/7cb5/c694cb9ad8c38e63db5d6458e34cd4fef5ce.pdf

Google Scholar

Demerouti, E., Bakker, A. B., Nachreiner, F., and Ebbinghaus, M. (2002). From mental strain to burnout. Eur. J. Work Organ. Psychol. 11, 423–441. doi: 10.1080/13594320244000274

CrossRef Full Text | Google Scholar

Demerouti, E., Bakker, A. B., Nachreiner, F., and Schaufeli, W. B. (2000). A model of burnout and life satisfaction amongst nurses. J. Adv. Nurs. 32, 454–464. doi: 10.1046/j.1365-2648.2000.01496

PubMed Abstract | CrossRef Full Text | Google Scholar

Demerouti, E., Bakker, A. B., Nachreiner, F., and Schaufeli, W. B. (2001). The job demands-resources model of burnout. J. Appl. Psychol. 86, 499–512. doi: 10.1037/0021-9010.86.3.499

PubMed Abstract | CrossRef Full Text | Google Scholar

Demerouti, E., Bakker, A. B., Vardakou, I., and Kantas, A. (2003). The convergent validity of two burnout instruments: a multitrait-multimethod analysis. Eur. J. Psychol. Assess. 19, 12–23. doi: 10.1027//1015-5759.19.1.12

CrossRef Full Text | Google Scholar

Demerouti, E., Mostert, K., and Bakker, A. B. (2010). Burnout and work engagement: a thorough investigation of the independency of both constructs. J. Occup. Health Psychol. 15, 209–222. doi: 10.1037/a0019408

PubMed Abstract | CrossRef Full Text | Google Scholar

Demerouti, E., and Nachreiner, F. (1998). Zur spezifität von burnout für dienstleistungsberufe: Fakt oder artefakt? [The specificity of burnout in human services: Fact or artifact?]. Z. Arbeitswiss. 52, 82–89.

Dias, S., Queirós, C., and Carlotto, M. S. (2010). Burnout syndrome and associated factors among health professionals: a comparative study between Brazil and Portugal. Aletheia 32, 4–21. Available online at: http://www.redalyc.org/articulo.oa?id=115020838002

Google Scholar

Dimou, F. M., Eckelbarger, D., and Riall, T. S. (2016). Surgeon burnout: a systematic review. J. Am. Coll. Surg. 222, 1230–1239. doi: 10.1016/j.jamcollsurg.2016.03.022

PubMed Abstract | CrossRef Full Text | Google Scholar

Epstein, R. M., and Privitera, M. R. (2017). Physician burnout is better conceptualised as depression – Authors' reply. Lancet 389:1398. doi: 10.1016/S0140-6736(17)30898-X

PubMed Abstract | CrossRef Full Text | Google Scholar

Estévez-Mujica, C. P., and Quintane, E. (2018). E-mail communication patterns and job burnout. PLoS ONE 13:e0193966. doi: 10.1371/journal.pone.0193966

PubMed Abstract | CrossRef Full Text | Google Scholar

European Agency for Safety and Health at Work (2018). Management of Occupational Health and Safety in European Workplaces — Evidence From the Second European Survey of Enterprises on New and Emerging Risks (ESENER-2). Luxembourg: Publications Office of the European Union.

Field, S., and Bramwell, R. (1998). An investigation into the relationship between caring responsibilities and the levels of perceived pressure reported by female employees. J. Occup. Organ. Psychol. 71, 165–170. doi: 10.1111/j.2044-8325.1998.tb00670.x

CrossRef Full Text | Google Scholar

Finney, S. J., and DiStefano, C. (2013). “Non-normal and categorical data in structural equation modeling,” in Structural Equation Modeling: A Second Course, eds G. R. Hancock, and R. O. Mueller (Charlotte, NC: Information Age Publishing), 439–492.

Google Scholar

Ford, V. F., Swayze, S., and Burley, D. L. (2013). An exploratory investigation of the relationship between disengagement, exhaustion and turnover intention among IT professionals employed at a university. Inf. Resour. Manag. J. 26, 55–68. doi: 10.4018/irmj.2013070104

CrossRef Full Text | Google Scholar

Fornell, C., and Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. J. Mark. Res. 18, 39–50. doi: 10.2307/3151312

CrossRef Full Text | Google Scholar

Foster, G. C. (2015). Measurement Invariance of Burnout Inventories Across Sex. Available online at: https://etd.ohiolink.edu/!etd.send_file?accession=bgsu1428162452&disposition=inline

Freudenberger, H. J. (1974). Staff burn-out. J. Soc. Issues 30, 159–165. doi: 10.1111/j.1540-4560.1974.tb00706.x

CrossRef Full Text | Google Scholar

Garibaldi de Hilal, A. V. (2006). Brazilian National Culture, Organizational Culture and Cultural Agreement. Int. J. Cross Cult. Manag. 6, 139–167. doi: 10.1177/1470595806066325

CrossRef Full Text | Google Scholar

Garibaldi de Hilal, A. V. (2009). “Cultural mythology and global leadership in Brazil,” in Cultural mythology and Global Leadership, eds E. H. Kessler and D. J. Wong-MingJi (Cheltenham, UK: Edward Elgar), 93–107.

Google Scholar

Gignac, G. E. (2008). Higher-order models versus direct hierarchical models: g as superordinate or breadth factor? Psychol. Sci. Q. 50, 21–43. Available online at: http://www.psychologie-aktuell.com/fileadmin/download/PschologyScience/1-2008/04_Gignac.pdf

Google Scholar

Gil-Monte, P. R., Viotti, S., and Converso, D. (2017). Psychometric properties of the Spanish Burnout Inventory (SBI) in a sample of Italian health professionals: a gender perspective. Liberabit 23, 153–169. doi: 10.24265/liberabit.2017.v23n2.01

CrossRef Full Text | Google Scholar

Gíslason, I. V., and Símonardóttir, S. (2018). Mothering and gender equality in Iceland: irreconcilable opposites? Soc. Policy Soc. 17, 457–466. doi: 10.1017/S1474746417000525

CrossRef Full Text | Google Scholar

Goering, D. D., Shimazu, A., Zhou, F., Wada, T., and Sakai, R. (2017). Not if, but how they differ: a meta-analytic test of the nomological networks of burnout and engagement. Burn. Res. 5, 21–34. doi: 10.1016/j.burn.2017.05.003

CrossRef Full Text | Google Scholar

Golay, P., Reverte, I., Rossier, J., Favez, N., and Lecerf, T. (2013). Further insights on the French WISC–IV factor structure through Bayesian structural equation modeling. Psychol. Assess. 25, 496–508. doi: 10.1037/a0030676

PubMed Abstract | CrossRef Full Text | Google Scholar

Golembiewski, R. T., Boudreau, R. A., Munzenrider, R. F., and Luo, H. (1996). Global Burnout: A Worldwide Pandemic Explored by the Phase Model. Greenwich, CT: JAI Press.

Golembiewski, R. T., Scherb, K., and Boudreau, R. A. (1993). “Burnout in cross-national settings: Generic and model-specific perspectives,” in Professional Burnout: Recent Developments in Theory and Research, eds W. B. Schaufeli, C. Maslach, and T. Marek (London: Taylor & Francis), 217–236.

Gómez-Urquiza, J. L., De la Fuente-Solana, E. I., Albendín-García, L., Vargas-Pecino, C., Ortega-Campos, E. M., and Cañadas-De la Fuente, G. A. (2017). Prevalence of burnout syndrome in emergency nurses: a meta-analysis. Crit. Care Nurse 37, e1–e9. doi: 10.4037/ccn2017508

PubMed Abstract | CrossRef Full Text | Google Scholar

González-Romá, V., Schaufeli, W. B., Bakker, A. B., and Lloret, S. (2006). Burnout and work engagement: Independent factors or opposite poles? J. Vocat. Behav. 68, 165–174. doi: 10.1016/j.jvb.2005.01.003

CrossRef Full Text | Google Scholar

Green, S. B., and Yang, Y. (2009). Reliability of summed item scores using structural equation modeling: an alternative to coefficient alpha. Psychometrika 74, 155–167. doi: 10.1007/s11336-008-9099-3

CrossRef Full Text | Google Scholar

Guenette, J. P., and Smith, S. E. (2017). Burnout: Prevalence and associated factors among radiology residents in New England with comparison against United States resident physicians in other specialties. Am. J. Roentgenol. 209, 136–141. doi: 10.2214/AJR.16.17541

PubMed Abstract | CrossRef Full Text | Google Scholar

Gustafsson, J. E., and Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behav. Res. 28, 407–434. doi: 10.1207/s15327906mbr2804_2

PubMed Abstract | CrossRef Full Text | Google Scholar

Hair, J. F., Black, W. C., Babin, B. J., and Anderson, R. E. (2009). Multivariate Data Analysis. 7th Edn. Upper Saddle River, NJ: Prentice Hall.

Google Scholar

Halbesleben, J. R. B., Buckley, M. R., Hall, A., and Brooks, W. (2004). Burnout in organizational life. J. Manage. 30, 859–879. doi: 10.1016/j.jm.2004.06.004

CrossRef Full Text | Google Scholar

Halbesleben, J. R. B., and Demerouti, E. (2005). The construct validity of an alternative measure of burnout: investigating the English translation of the Oldenburg Burnout Inventory. Work Stress 19, 208–220. doi: 10.1080/02678370500340728

CrossRef Full Text | Google Scholar

Halliday, L., Walker, A., Vig, S., Hines, J., and Brecknell, J. (2017). Grit and burnout in UK doctors: a cross-sectional study across specialties and stages of training. Postgrad. Med. J. 93, 389–394. doi: 10.1136/postgradmedj-2015-133919

PubMed Abstract | CrossRef Full Text | Google Scholar

Henseler, J., Ringle, C. M., and Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. J. Acad. Mark. Sci. 43, 115–135. doi: 10.1007/s11747-014-0403-8

CrossRef Full Text | Google Scholar

Hofstede, G., Garibaldi de Hilal, A. V., Malvezzi, S., Tanure, B., and Vinken, H. (2010b). Comparing regional cultures within a country: lessons from Brazil. J. Cross. Cult. Psychol. 41, 336–352. doi: 10.1177/0022022109359696

CrossRef Full Text | Google Scholar

Hofstede, G. G. J., Hofstede, G. G. J., and Minkov, M. (2010a). Cultures and Organizations: Software of the Mind: Intercultural Cooperation and Its Importance for Survival. 3rd Edn. New York, NY: McGraw-Hill.

Højsgaard, S., and Halekoh, U. (2018). doBy: Groupwise Statistics, LSmeans, Linear Contrasts, Utilities (R package version 4.6-2) [Computer software]. Available online at: https://cran.r-project.org/package=doBy

Holzinger, K. J., and Harman, H. H. (1938). Comparison of two factorial analyses. Psychometrika 3, 45–60. doi: 10.1007/BF02287919

CrossRef Full Text | Google Scholar

Holzinger, K. J., and Swineford, F. (1937). The bi-factor method. Psychometrika 2, 41–54. doi: 10.1007/BF02287965

CrossRef Full Text | Google Scholar

Hoyle, R. H. and (Ed.). (1995). Structural Equation Modeling: Concepts, Issues and Applications. Thousand Oaks, CA: SAGE Publications.

Google Scholar

Innstrand, S. T. (2016). Occupational differences in work engagement: A longitudinal study among eight occupational groups in Norway. Scand. J. Psychol. 57, 338–349. doi: 10.1111/sjop.12298

PubMed Abstract | CrossRef Full Text | Google Scholar

Innstrand, S. T., Langballe, E. M., and Falkum, E. (2012). A longitudinal study of the relationship between work engagement and symptoms of anxiety and depression. Stress Heal. 28, 1–10. doi: 10.1002/smi.1395

PubMed Abstract | CrossRef Full Text | Google Scholar

Innstrand, S. T., Langballe, E. M., Falkum, E., and Aasland, O. G. (2011). Exploring within–and between-gender differences in burnout: 8 different occupational groups. Int. Arch. Occup. Environ. Health 84, 813–824. doi: 10.1007/s00420-011-0667-y

PubMed Abstract | CrossRef Full Text | Google Scholar

International Labour Office (2012). International Standard Classification of Occupations. Geneva: International Labour Organization.

International Labour Office (2016). Workplace Stress: A Collective Challenge. Geneva: International Labour Organization. Available online at: http://www.ilo.org/safework/info/publications/WCMS_466547/lang–en/index.htm (accessed July 1, 2017).

International Labour Organization (2013). Case Study: Karoshi: Death From Overwork. Available online at: http://www.ilo.org/safework/info/publications/WCMS_211571/lang–en/index.htm (accessed October 6, 2017).

International Test Commission (2018). ITC guidelines for translating and adapting tests (second edition). Int. J. Test. 18, 101–134. doi: 10.1080/15305058.2017.1398166

CrossRef Full Text

Irwing, P., and Hughes, D. J. (2018). “Test development,” in The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development, eds P. Irwing, T. Booth, and D. Hughes (Hoboken, NJ: John Wiley & Sons), 3–47.

Google Scholar

Jennrich, R. I., and Bentler, P. M. (2011). Exploratory bi-factor analysis. Psychometrika 76, 537–549. doi: 10.1007/s11336-011-9218-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., and Rosseel, Y. (2018). semTools: Useful Tools for Structural Equation Modeling (R package version 0.5-1) [Computer software]. Available online at: https://cran.r-project.org/package=semTools

Jovanović, N., Podlesek, A., Volpe, U., Barrett, E., Ferrari, S., Rojnic Kuzman, M., et al. (2016). Burnout syndrome among psychiatric trainees in 22 countries: Risk increased by long working hours, lack of supervision, and psychiatry not being first career choice. Eur. Psychiatry 32, 34–41. doi: 10.1016/j.eurpsy.2015.10.007

CrossRef Full Text | Google Scholar

Kessler, R. C., Chiu, W. T., Demler, O., and Walters, E. E. (2005). Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch. Gen. Psychiatry 62, 617–627. doi: 10.1001/archpsyc.62.6.617

PubMed Abstract | CrossRef Full Text | Google Scholar

Khan, A., and Yusoff, R. B. M. (2016). Psychometric testing of Oldenburg Burnout Inventory among academic staff in Pakistan. Int. Rev. Manag. Mark. 6, 683–687. Available online at: www.econjournals.com

Google Scholar

Khan, F., Khan, Q., and Naz, A. (2016). Female academicians are burnout in Pakistan universities? Gomal Univ. J. Res., 157–167. Available online at: http://www.gu.edu.pk/new/GUJR/Issues-2017/ONLINE_SpecialIssueI/16_FemaleAcademiciansareBurnoutinPakistanUniversities.pdf

Kline, R. B. (2016). Principles and Practice of Structural Equation Modeling. 4th Edn. New York, NY: The Guildford Press.

Google Scholar

Ko, C. (2017). Exploring how affectivity influence on job burnout through organizational politics perception. Int. J. Organ. Innov. 10, 86–94. Available online at: http://www.ijoi-online.org/attachments/article/54/FINAL%20ISSUE%20VOL%2010%20NUM%202%20OCTOBER%202017%20-%20SECTION%20A.pdf

Google Scholar

Koeske, G. F., and Koeske, R. D. (1989). Construct validity of the Maslach Burnout Inventory: a critical review and reconceptualization. J. Appl. Behav. Sci. 25, 131–144. doi: 10.1177/0021886389252004

CrossRef Full Text | Google Scholar

Kogoj, T. K., Cebašek-Travnik, Z., and Zaletel-Kragelj, L. (2014). Role of stress in burnout among students of medicine and dentistry–A study in Ljubljana, Slovenia, Faculty of Medicine. Coll. Antropol. 38, 879–887. Available online at: http://ezproxy.library.usyd.edu.au/login?url=http://ovidsp.ovid.com/ovidweb.cgi?T=JS&CSC=Y&NEWS=N&PAGE=fulltext&D=medl&AN=25420369%5Cnhttp://dd8gh5yx7k.search.serialssolutions.com/?sid=OVID:medline&id=pmid:25420369&id=doi:&issn=0350-6134&isbn=&volume=38&i

PubMed Abstract | Google Scholar

Kristensen, T. S., Borritz, M., Villadsen, E., and Christensen, K. B. (2005). The Copenhagen Burnout Inventory: a new tool for the assessment of burnout. Work Stress 19, 192–207. doi: 10.1080/02678370500297720

CrossRef Full Text | Google Scholar

Kumar, S., and Mellsop, G. (2013). “Burnout: Gender aspects,” in Burnout for Experts: Prevention in the Context of Living and Working, ed S. Bährer-Kohler (Boston, MA: Springer), 99–117.

Google Scholar

Langballe, E. M., Innstrand, S. T., Aasland, O. G., and Falkum, E. (2011). The predictive value of individual factors, work-related factors, and work-home interaction on burnout in female and male physicians: a longitudinal study. Stress Heal. 27, 73–87. doi: 10.1002/smi.1321

CrossRef Full Text | Google Scholar

Larsen, A. C., Ulleberg, P., and Rønnestad, M. H. (2017). Depersonalization reconsidered: an empirical analysis of the relation between depersonalization and cynicism in an extended version of the Maslach Burnout Inventory. Nord. Psychol. 69, 160–176. doi: 10.1080/19012276.2016.1227939

CrossRef Full Text | Google Scholar

le Roux, A. M. (2004). The Validation of Two Burnout Measures in the South African Earthmoving Equipment Industry. Master mini-dissertation, North-West Univ. Potchefstroom, South Africa. Available online at: http://dspace.nwu.ac.za/bitstream/handle/10394/674/leroux_am.pdf

Lee, R. T., and Ashforth, B. E. (1996). A meta-analytic examination of the correlates of the three dimensions of job burnout. J. Appl. Psychol. 81, 123–133. doi: 10.1037/0021-9010.81.2.123

PubMed Abstract | CrossRef Full Text | Google Scholar

Leiter, M. P. (1993). “Burnout as a developmental process: Consideration of models,” in Professional Burnout: Recent Developments in Theory and Research, eds W. B. Schaufeli, C. Maslach, and T. Marek (Washington, DC: Taylor & Francis), 237–250.

Google Scholar

Leiter, M. P., and Maslach, C. (2017). Editorial: Burnout and engagement: contributions to a new vision. Burn. Res. 5, 55–57. doi: 10.1016/j.burn.2017.04.003

CrossRef Full Text | Google Scholar

Lekutle, M., and Nel, J. A. (2012). Psychometric evaluation of the Utrecht Work Engagement Scale (UWES) and Oldenburg Burnout Inventory (OLBI) within a cement factory. J. Psychol. Africa 22, 641–647. doi: 10.1080/14330237.2012.10820580

CrossRef Full Text | Google Scholar

Lemon, J. (2006). Plotrix: a package in the red light district of R. R-News 6, 8–12. Available online at: https://www.r-project.org/doc/Rnews/Rnews_2006-4.pdf

LimeSurvey GmbH (2017). LimeSurvey: An Open Source Survey Tool [Computer software]. Available online at: http://www.limesurvey.org

Lindblom, K. M., Linton, S. J., Fedeli, C., and Bryngelsson, I.-L. (2006). Burnout in the working population: relations to psychosocial work factors. Int. J. Behav. Med. 13, 51–59. doi: 10.1207/s15327558ijbm1301_7

PubMed Abstract | CrossRef Full Text | Google Scholar

Lo, S. (2003). Perceptions of work-family conflict among married female professionals in Hong Kong. Pers. Rev. 32, 376–390. doi: 10.1108/00483480310467679

CrossRef Full Text | Google Scholar

Lüdecke, D. (2019). sjstats: Statistical Functions for Regression Models (R package version 0.17.3) [Computer software].

Lundkvist, E., Stenling, A., Gustafsson, H., and Hassmén, P. (2014). How to measure coach burnout: an evaluation of three burnout measures. Meas. Phys. Educ. Exerc. Sci. 18, 209–226. doi: 10.1080/1091367X.2014.925455

CrossRef Full Text | Google Scholar

MacCallum, R. C., Browne, M. W., and Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychol. Methods 1, 130–149. doi: 10.1037/1082-989X.1.2.130

CrossRef Full Text | Google Scholar

Mahadi, N. F., Chin, R. W. A., Chua, Y. Y., Chu, M. N., Wong, M. S., Yusoff, M. S. B., et al. (2018). Malay language translation and validation of the Oldenburg Burnout Inventory measuring burnout. Educ. Med. J. 10, 27–40. doi: 10.21315/eimj2018.10.2.4

CrossRef Full Text | Google Scholar

Malach-Pines, A. (2005). The Burnout Measure, short version. Int. J. Stress Manag. 12, 78–88. doi: 10.1037/1072-5245.12.1.78

CrossRef Full Text | Google Scholar

Mansolf, M., and Reise, S. P. (2017). When and why the second-order and bifactor models are distinguishable. Intelligence 61, 120–129. doi: 10.1016/j.intell.2017.01.012

CrossRef Full Text | Google Scholar

Marôco, J. (2014). Análise de equações estruturais: Fundamentos teóricos, software & aplicações. 2nd Edn. Pêro Pinheiro: ReportNumber.

Marôco, J., and Campos, J. A. D. B. (2012). Defining the student burnout construct: a structural analysis from three burnout inventories. Psychol. Rep. 111, 814–830. doi: 10.2466/14.10.20.PR0.111.6.814-830

PubMed Abstract | CrossRef Full Text | Google Scholar

Marôco, J., Marôco, A. L., and Campos, J. A. D. B. (2014). Student's academic efficacy or inefficacy? An example on how to evaluate the psychometric properties of a measuring instrument and evaluate the effects of item wording. Open J. Stat. 04, 484–493. doi: 10.4236/ojs.2014.46046

CrossRef Full Text | Google Scholar

Marôco, J., Marôco, A. L., Leite, E., Bastos, C., Vazao, M. J., and Campos, J. A. D. B. (2016). Burnout in Portuguese healthcare professionals: an analysis at the national level. Acta Med. Port. 29, 24–30. doi: 10.20344/acta%20med%20port.v29i1.6460

PubMed Abstract | CrossRef Full Text | Google Scholar

Marôco, J., Tecedeiro, M., Martins, P., and Meireles, A. (2008). O burnout como factor hierárquico de 2^a ordem da escala de burnout de Maslach. Análise Psicológica 4, 639–649. doi: 10.14417/ap.523

CrossRef Full Text | Google Scholar

Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika 57, 519–530. doi: 10.1093/biomet/57.3.519

CrossRef Full Text | Google Scholar

Marques-Pinto, A., Lima, M. L., Silva, A. L., and da (2008). “Delimitação do conceito de burnout,” in Burnout e engagement em contexto organizacional: Estudos com amostras portuguesas, eds A. Marques-Pinto, and M. J. Chambel (Lisbon: Livros Horizonte), 15–52.

Marques-Pinto, A., Lima, M. L., and Silva, A. L. da. (2003). Stress profissional em professores portugueses: Incidência, preditores e reacção de burnout. Psychologica 33, 181–194. Available online at: https://www.academia.edu/6286459/Stress_profissional_em_professores_portugueses_Incid%C3%AAncia_preditores_e_reac%C3%A7%C3%A3o_de_burnout

Maslach, C. (1976). Burned-out. Hum. Behav. 9, 16–22.

Maslach, C. (1998). “A multidimensional theory of burnout,” in Theories of Organizational Stress, ed C. L. Cooper (Oxford: Oxford University Press), 68–85.

Maslach, C. (2015). “Burnout, psychology of,” in International Encyclopedia of the Social and Behavioral Sciences, ed J. D. Wright (Amsterdam: Elsevier), 929–932.

Maslach, C., Jackson, S. E., and Leiter, M. P. (2016). Maslach Burnout Inventory Manual. 4th Edn. Menlo Park, CA: Mind Garden. Available online at: https://www.mindgarden.com

Maslach, C., and Leiter, M. P. (2017). “Understanding burnout: New models,” in The Handbook of Stress and Health: A Guide to Research and Practice, eds C. L. Cooper, and J. C. Quick (Chichester: John Wiley & Sons), 36–56.

Google Scholar

Maslach, C., Schaufeli, W. B., and Leiter, M. P. (2001). Job burnout. Annu. Rev. Psychol. 52, 397–422. doi: 10.1146/annurev.psych.52.1.397

PubMed Abstract | CrossRef Full Text | Google Scholar

Massey, D. S., and Tourangeau, R. (2013). Introduction: new challenges to social measurement. Ann. Am. Acad. Pol. Soc. Sci. 645, 6–22. doi: 10.1177/0002716212463314

CrossRef Full Text | Google Scholar

Maydeu-Olivares, A., Shi, D., and Rosseel, Y. (2018). Assessing fit in structural equation models: a Monte-Carlo evaluation of RMSEA versus SRMR confidence intervals and tests of close fit. Struct. Equ. Model. A Multidiscip. J. 25, 389–402. doi: 10.1080/10705511.2017.1389611

CrossRef Full Text | Google Scholar

Mbanga, C., Makebe, H., Tim, D., Fonkou, S., Toukam, L., and Njim, T. (2018). Determinants of burnout syndrome among nurses in Cameroon. BMC Res. Notes 11:893. doi: 10.1186/s13104-018-4004-3

PubMed Abstract | CrossRef Full Text | Google Scholar

McDonald, R. P. (1999). Test Theory: A Unified Treatment. Mahwah, NJ: Routledge.

Google Scholar

McDonald, R. P., and Ho, M. H. (2002). Principles and practice in reporting structural equation analyses. Psychol. Methods 7, 64–82. doi: 10.1037/1082-989X.7.1.64

PubMed Abstract | CrossRef Full Text | Google Scholar

McNamara, A., Arino de la Rubia, E., Zhu, H., Ellis, S., and Quinn, M. (2018). skimr: Compact and Flexible Summaries of Data (R package version 1.0.3) [Computer software]. Available online at: https://cran.r-project.org/package=skimr

Mészáros, V., Ádám, S., Szabó, M., Szigeti, R., and Urbán, R. (2014). The bifactor model of the Maslach Burnout Inventory-Human Services Survey (MBI-HSS): An alternative measurement model of burnout. Stress Heal. 30, 82–88. doi: 10.1002/smi.2481

PubMed Abstract | CrossRef Full Text | Google Scholar

Millsap, R. E., and Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behav. Res. 39, 479–515. doi: 10.1207/S15327906MBR3903_4

CrossRef Full Text | Google Scholar

Mion, G., Libert, N., and Journois, D. (2018). The prevalence of burnout. Intensive Care Med. 44, 1192–1194. doi: 10.1007/s00134-018-5200-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Mokgele, K. R. F., and Rothmann, S. (2014). A structural model of student well-being. South African J. Psychol. 44, 514–527. doi: 10.1177/0081246314541589

CrossRef Full Text | Google Scholar

Morgan, B., de Bruin, G. P., and de Bruin, K. (2014). Operationalizing burnout in the Maslach Burnout Inventory–Student Survey: personal efficacy versus personal inefficacy. South African J. Psychol. 44, 216–227. doi: 10.1177/0081246314528834

CrossRef Full Text | Google Scholar

Morin, A. J. S., Katrin Arens, A., and Marsh, H. W. (2015). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Struct. Equ. Model. 23, 116–139. doi: 10.1080/10705511.2014.961800

CrossRef Full Text | Google Scholar

Mulaik, S. A., and Quartetti, D. A. (1997). First order or higher order general factor? Struct. Equ. Model. A Multidiscip. J. 4, 193–211. doi: 10.1080/10705519709540071

CrossRef Full Text | Google Scholar

Murray, A. L., and Johnson, W. (2013). The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence 41, 407–422. doi: 10.1016/j.intell.2013.06.004

CrossRef Full Text | Google Scholar

Muthén, B. O. (1983). Latent variable structural equation modeling with categorical data. J. Econom. 22, 43–65. doi: 10.1016/0304-4076(83)90093-3

CrossRef Full Text | Google Scholar

O'Connor, K., Muller Neff, D., and Pitman, S. (2018). Burnout in mental health professionals: a systematic review and meta-analysis of prevalence and determinants. Eur. Psychiatry 53, 74–99. doi: 10.1016/j.eurpsy.2018.06.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Olinske, J. L., and Hellman, C. M. (2017). Leadership in the human service nonprofit organization: the Influence of the board of directors on executive director well-being and burnout. Hum. Serv. Organ. Manag. Leadersh. Gov. 41, 95–105. doi: 10.1080/23303131.2016.1222976

CrossRef Full Text | Google Scholar

Papenberg, M. (2018). minDiff: Minimize Differences Between Groups (R package version 0.01-3) [Computer software]. Available online at: https://github.com/m-Py/minDiff

Peterson, U., Bergström, G., Demerouti, E., Gustavsson, P., Åsberg, M., and Nygren, Å. (2011). Burnout levels and self-rated health prospectively predict future long-term sickness absence: a study among female health professionals. J. Occup. Environ. Med. 53, 788–793. doi: 10.1097/JOM.0b013e318222b1dc

PubMed Abstract | CrossRef Full Text | Google Scholar

Peterson, U., Bergström, G., Samuelsson, M., Asberg, M., and Nygren, Å. (2008). Reflecting peer-support groups in the prevention of stress and burnout: randomized controlled trial. J. Adv. Nurs. 63, 506–516. doi: 10.1111/j.1365-2648.2008.04743.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Petrović, I. B., Vukelić, M., and Cizmić, S. (2017). Work engagement in Serbia: psychometric properties of the Serbian version of the Utrecht Work Engagement Scale (UWES). Front. Psychol. 8:1799. doi: 10.3389/fpsyg.2017.01799

PubMed Abstract | CrossRef Full Text | Google Scholar

Pines, A. M., and Aronson, E. (1988). Career Burnout: Causes and Cures. New York, NY: Free Press.

Google Scholar

Poghosyan, L., Clarke, S. P., Finlayson, M., and Aiken, L. H. (2010). Nurse burnout and quality of care: cross-national investigation in six countries. Res. Nurs. Health 33, 288–298. doi: 10.1002/nur.20383

PubMed Abstract | CrossRef Full Text | Google Scholar

Price, J. L. (1997). Handbook of organizational measurement. Int. J. Manpow. 18, 305–558. doi: 10.1108/01437729710182260

CrossRef Full Text | Google Scholar

Pu, J., Zhou, X., Zhu, D., Zhong, X., Yang, L., Wang, H., et al. (2017). Gender differences in psychological morbidity, burnout, job stress and job satisfaction among Chinese neurologists: a national cross-sectional study. Psychol. Health Med. 22, 680–692. doi: 10.1080/13548506.2016.1211717

PubMed Abstract | CrossRef Full Text | Google Scholar

Purvanova, R. K., and Muros, J. P. (2010). Gender differences in burnout: a meta-analysis. J. Vocat. Behav. 77, 168–185. doi: 10.1016/j.jvb.2010.04.006

CrossRef Full Text | Google Scholar

Qiao, H., and Schaufeli, W. B. (2011). The convergent validity of four burnout measures in a Chinese sample: a confirmatory factor-analytic approach. Appl. Psychol. 60, 87–111. doi: 10.1111/j.1464-0597.2010.00428.x

CrossRef Full Text | Google Scholar

R Core Team (2018). R: A Language and Environment for Statistical Computing (version 3.5.1) [Computer software]. Available online at: https://www.r-project.org/

Rajeswari, H., and Sreelekha, B. (2015). Burnout among nurses. Int. J. Sci. Res. 4, 407–410. doi: 10.7748/en.22.9.17.s18

CrossRef Full Text

Raykov, T. (2001). Estimation of congeneric scale reliability using covariance structure analysis with nonlinear constraints. Br. J. Math. Stat. Psychol. 54, 315–323. doi: 10.1348/000711001159582

PubMed Abstract | CrossRef Full Text | Google Scholar

Reis, D., Xanthopoulou, D., and Tsaousis, I. (2015). Measuring job and academic burnout with the Oldenburg Burnout Inventory (OLBI): factorial invariance across samples and countries. Burn. Res. 2, 8–18. doi: 10.1016/j.burn.2014.11.001

CrossRef Full Text | Google Scholar

Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behav. Res. 47, 667–696. doi: 10.1080/00273171.2012.715555

PubMed Abstract | CrossRef Full Text | Google Scholar

Reise, S. P., Bonifay, W. E., and Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. J. Pers. Assess. 95, 129–140. doi: 10.1080/00223891.2012.725437

PubMed Abstract | CrossRef Full Text | Google Scholar

Reise, S. P., Moore, T. M., and Haviland, M. G. (2010). Bifactor models and rotations: exploring the extent to which multidimensional data yield univocal scale scores. J. Pers. Assess. 92, 544–559. doi: 10.1080/00223891.2010.496477

PubMed Abstract | CrossRef Full Text | Google Scholar

Revelle, W. (2018). psych: Procedures for Psychological, Psychometric, and Personality Research (R package version 1.8.12) [Computer software]. Available online at: https://cran.r-project.org/package=psych

Revelle, W., and Condon, D. M. (2018). “Reliability,” in The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development, eds P. Irwing, T. Booth, and D. Hughes (Hoboken, NJ: John Wiley & Sons), 709–749.

Google Scholar

Rios, J., and Wells, C. (2014). Validity evidence based on internal structure. Psicothema 26, 108–116. doi: 10.7334/psicothema2013.260

PubMed Abstract | CrossRef Full Text | Google Scholar

Rodriguez, A., Reise, S. P., and Haviland, M. G. (2016a). Applying bifactor statistical indices in the evaluation of psychological measures. J. Pers. Assess. 98, 223–237. doi: 10.1080/00223891.2015.1089249

PubMed Abstract | CrossRef Full Text | Google Scholar

Rodriguez, A., Reise, S. P., and Haviland, M. G. (2016b). Evaluating bifactor models: calculating and interpreting statistical indices. Psychol. Methods 21, 137–150. doi: 10.1037/met0000045

PubMed Abstract | CrossRef Full Text | Google Scholar

Rogala, A., Shoji, K., Luszczynska, A., Kuna, A., Yeager, C., Benight, C. C., et al. (2016). From exhaustion to disengagement via self-efficacy change: findings from two longitudinal studies among Human Services Workers. Front. Psychol. 6:2032. doi: 10.3389/fpsyg.2015.02032

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosnah, I., Idris, M. A., Azmi, M. T., and Noor Hassim, I. (2017). Job stress on abdominal obesity: the moderating effects of anger and overeating behaviour. Malays. J. Nutr. 23, 437–448. Available online at: https://www.scopus.com/record/display.uri?eid=2-s2.0-85040452397&origin=inward&txGid=21158a61fcf5cd5de411607df70ea1d0

Google Scholar

Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. J. Stat. Softw. 48, 1–21. doi: 10.18637/jss.v048.i02

CrossRef Full Text | Google Scholar

RStudio Team (2018). RStudio: Integrated Development for R (version 1.1.463) [Computer software]. Available online at: http://www.rstudio.com/

Rubino, C., Volpone, S. D., and Avery, D. R. (2013). Burnout on Mars and Venus: exploring gender differences in emotional exhaustion. Gend. Manag. An Int. J. 28, 74–93. doi: 10.1108/17542411311303220

CrossRef Full Text | Google Scholar

Rudman, A., Gustavsson, P., and Hultell, D. (2014). A prospective study of nurses' intentions to leave the profession during their first five years of practice in Sweden. Int. J. Nurs. Stud. 51, 612–624. doi: 10.1016/j.ijnurstu.2013.09.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Ruotsalainen, J. H., Verbeek, J. H., Mariné, A., and Serra, C. (2015). Preventing occupational stress in healthcare workers. Cochrane Database Syst. Rev. 18:CD002892. doi: 10.1002/14651858.CD002892.pub5

CrossRef Full Text | Google Scholar

Rzeszutek, M. (2013). Burnout syndrome in male and female gestalt and cognitive-behavioral psychotherapists. Rocz. Psychol. 16, 155–161. Available online at: https://www.ceeol.com/search/article-detail?id=110122

Google Scholar

Saboonchi, F., Perski, A., and Grossi, G. (2013). Validation of Karolinska Exhaustion Scale: psychometric properties of a measure of exhaustion syndrome. Scand. J. Caring Sci. 27, 1010–1017. doi: 10.1111/j.1471-6712.2012.01089.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Salazar, M. S. (2015). El dilema de combinar ítems positivos y negativos en escalas. Psicothema 27, 192–199. doi: 10.7334/psicothema2014.266

CrossRef Full Text

Sales, B., Macdonald, A., Scallan, S., and Crane, S. (2016). How can educators support general practice (GP) trainees to develop resilience to prevent burnout? Educ. Prim. Care 27, 487–493. doi: 10.1080/14739879.2016.1217170

PubMed Abstract | CrossRef Full Text | Google Scholar

Salmela-Aro, K., Rantanen, J., Hyvönen, K., Tilleman, K., and Feldt, T. (2011). Bergen Burnout Inventory: reliability and validity among Finnish and Estonian managers. Int. Arch. Occup. Environ. Health 84, 635–645. doi: 10.1007/s00420-010-0594-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Satorra, A., and Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika 66, 507–514. doi: 10.1007/BF02296192

CrossRef Full Text | Google Scholar

Savickas, M. L. (2012). Life design: a paradigm for career intervention in the 21st century. J. Couns. Dev. 90, 13–19. doi: 10.1111/j.1556-6676.2012.00002.x

CrossRef Full Text | Google Scholar

Scanlan, J. N., and Still, M. (2013). Job satisfaction, burnout and turnover intention in occupational therapists working in mental health. Aust. Occup. Ther. J. 60, n/a-n/a. doi: 10.1111/1440-1630.12067

PubMed Abstract | CrossRef Full Text | Google Scholar

Schadenhofer, P., Kundi, M., Abrahamian, H., Stummer, H., and Kautzky-Willer, A. (2018). Influence of gender, working field and psychosocial factors on the vulnerability for burnout in mental hospital staff: Results of an Austrian cross-sectional study. Scand. J. Caring Sci. 32, 335–345. doi: 10.1111/scs.12467

PubMed Abstract | CrossRef Full Text | Google Scholar

Schaufeli, W. B. (2017). “Burnout: A short socio-cultural history,” in Burnout, Fatigue, Exhaustion: An Interdisciplinary Perspective on a Modern Affliction, eds S. Neckel, A. K. Schaffner, and G. Wagner (Cham: Springer), 105–127.

Google Scholar

Schaufeli, W. B., and Bakker, A. B. (2003). Utrecht Work Engagement Scale. Utrecht: Occupational Health Psychology Unit, Utrecht University.

Schaufeli, W. B., and Bakker, A. B. (2010). “Defining and measuring work engagement: bringing clarity to the concept,” in Work Engagement: A Handbook of Essential Theory and Research, eds A. B. Bakker, and M. P. Leiter (New York, NY: Psychology Press), 10–24.

Google Scholar

Schaufeli, W. B., and de Witte, H. (2017). Work engagement in contrast to burnout: real or redundant? Burn. Res. 5, 1–2. doi: 10.1016/j.burn.2017.06.001

CrossRef Full Text

Schaufeli, W. B., Salanova, M., González-Romá, V., and Bakker, A. B. (2002). The measurement of engagement and burnout: a two sample confirmatory factor analytic approach. J. Vocat. Behav. 3, 71–92. doi: 10.1023/A:1015630930326

CrossRef Full Text | Google Scholar

Schuster, M. da S., Dias, V. da V., and Battistella, L. F. (2015). Burnout Inventory of Oldenburg: the influence of gender, occupation and supervision. Rev. Eletrôn. Mestr. Prof. Adm. Univ. Potiguar 8, 9–18. Available online at: Revista Eletrônica do Mestrado Profissional em Administração da Universidade Potiguar.

Schuster, M. D. S., and Dias, V. D. V. (2018). Oldenburg Burnout Inventory–validation of a new way to measure burnout in Brazil. Cien. Saude Colet. 23, 553–562. doi: 10.1590/1413-81232018232.27952015

PubMed Abstract | CrossRef Full Text

Schuster, M. d. S., Dias, V. d. V., Battistella, L. F., and Marquetto, M. F. (2013). “Oldenburg burnout inventory-validação de uma nova forma de mensurar burnout no Brasil,” in Paper Presented at the IV Encontro de Gestão de Pessoas e Relações de Trabalho (Brasília), 1–11.

Google Scholar

Sedlar, N., Šprah, L., Tement, S., and Sočan, G. (2015). Internal structure of an alternative measure of burnout: Study on the Slovenian adaptation of the Oldenburg Burnout Inventory (OLBI). Burn. Res. 2, 1–7. doi: 10.1016/j.burn.2015.02.001

CrossRef Full Text | Google Scholar

Seidler, A., Thinschmidt, M., Deckert, S., Then, F., Hegewald, J., Nieuwenhuijsen, K., et al. (2014). The role of psychosocial working conditions on burnout and its core component emotional exhaustion – a systematic review. J. Occup. Med. Toxicol. 9, 1–13. doi: 10.1186/1745-6673-9-10

PubMed Abstract | CrossRef Full Text | Google Scholar

Shanafelt, T. D., Hasan, O., Dyrbye, L. N., Sinsky, C., Satele, D., Sloan, J. A., et al. (2015). Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014. Mayo Clin. Proc. 90, 1600–1613. doi: 10.1016/j.mayocp.2015.08.023

PubMed Abstract | CrossRef Full Text | Google Scholar

Shirom, A. (2002). “Job related burnout: a review,” in Handbook of Occupational Health Psychology, eds J. C. Quick, and L. E. Tetrick (Washington, DC: American Psychological Association), 245–264.

Google Scholar

Shirom, A., and Melamed, S. (2006). A comparison of the construct validity of two burnout measures in two groups of professionals. Int. J. Stress Manag. 13, 176–200. doi: 10.1037/1072-5245.13.2.176

CrossRef Full Text | Google Scholar

Shupe, E. I., Wambaugh, S. K., and Bramble, R. J. (2015). Role-related stress experienced by academic librarians. J. Acad. Librariansh. 41, 264–269. doi: 10.1016/j.acalib.2015.03.016

CrossRef Full Text | Google Scholar

Silva, M. A. O. M., da Gomes, L. F. A. M., and Correia, M. F. (2009). Entrepreneurial culture: a comparative study of entrepreneurs in Brazil and Portugal. Rev. Adm. Contemp. 13, 57–71. doi: 10.1590/S1415-65552009000100005

CrossRef Full Text | Google Scholar

Simbula, S., and Guglielmi, D. (2010). Depersonalization or cynicism, efficacy or inefficacy: what are the dimensions of teacher burnout? Eur. J. Psychol. Educ. 25, 301–314. doi: 10.1007/s10212-010-0017-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Sinval, J., Pasian, S., Queirós, C., and Marôco, J. (2018). Brazil-Portugal transcultural adaptation of the UWES-9: internal consistency, dimensionality, and measurement invariance. Front. Psychol. 9:353. doi: 10.3389/fpsyg.2018.00353

PubMed Abstract | CrossRef Full Text | Google Scholar

Smirnova, A. Y. (2017). The Oldenburg Burnout Inventory: diagnostics of state of mind's change of the employ on a continuum: work engagement – professional burnout. Philos. Psychol. Pedagog. 17, 211–218. doi: 10.18500/1819-7671-2017-17-2-211-218

CrossRef Full Text

Staszczyk, S., and Tokarz, A. (2015). The relationship between indicators of workaholism and burnout in specialists and managers. Ann. Psychol. 18, 523–540. doi: 10.18290/rpsych.2015.18.4-2en

CrossRef Full Text | Google Scholar

Subburaj, A., and Vijayadurai, J. (2016). Translation, validation and psychometric properties of Tamil version of Oldenburg Burnout Inventory (OLBI). Proc. Soc. Behav. Sci. 219, 724–731. doi: 10.1016/j.sbspro.2016.05.067

CrossRef Full Text | Google Scholar

Suyi, Y., Meredith, P., and Khan, A. (2017). Effectiveness of mindfulness intervention in reducing stress and burnout for mental health professionals in Singapore. Explor. J. Sci. Heal. 13, 319–326. doi: 10.1016/j.explore.2017.06.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Taris, T. W., Schreurs, P. J. G., and Schaufeli, W. B. (1999). Construct validity of the Maslach Burnout Inventory-General Survey: a two-sample examination of its factor structure and correlates. Work Stress 13, 223–237. doi: 10.1080/026783799296039

CrossRef Full Text | Google Scholar

Tilakdharee, N., Ramidial, S., and Parumasur, S. B. (2010). The relationship between job insecurity and burnout. South African J. Econ. Manag. Sci. 13, 254–271. doi: 10.4102/sajems.v13i3.103

CrossRef Full Text | Google Scholar

Timms, C., Graham, D., and Cottrell, D. (2007). “I just want to teach”: Queensland independent school teachers and their workload. J. Educ. Adm. 45, 569–586. doi: 10.1108/09578230710778204

CrossRef Full Text | Google Scholar

Turtulla, S. (2017). Examining levels of job burnout among teachers working in Kosova in terms of different variables: proposing a psychological and counselling group program for managing. Eur. J. Multidiscip. Stud. 6, 285–297. doi: 10.26417/ejms.v6i1.p285-297

CrossRef Full Text

van de Vijver, F. J. R. (2016). “Test adaptations,” in The ITC International Handbook of Testing and Assessment, eds F. T. L. Leong, D. Bartram, F. M. Cheung, K. F. Geisinger, and D. Iliescu (New York, NY: Oxford University Press), 364–376.

Google Scholar

Wang, G., Liu, C., and Wu, X. (2003). Development of Educator Burnout Inventory. Psychol. Dev. Educ. 3, 82–86. Available online at: http://en.cnki.com.cn/Article_en/CJFDTOTAL-XLFZ200303014.htm

Wang, P., and Gao, F. (2010). Full-information item bifactor analysis of the Job Burnout Scale for Chinese college teachers. J. Converg. Inf. Technol. 5, 155–160. doi: 10.4156/jcit.vol5.issue7.20

CrossRef Full Text | Google Scholar

Westwood, S., Morison, L., Allt, J., and Holmes, N. (2017). Predictors of emotional exhaustion, disengagement and burnout among improving access to psychological therapies (IAPT) practitioners. J. Ment. Heal. 26, 172–179. doi: 10.1080/09638237.2016.1276540

PubMed Abstract | CrossRef Full Text | Google Scholar

Yanos, P. T., Vayshenker, B., DeLuca, J. S., and O'Connor, L. K. (2017). Development and validation of a scale assessing mental health clinicians' experiences of associative stigma. Psychiatr. Serv. 68, 1053–1060. doi: 10.1176/appi.ps.201600553

PubMed Abstract | CrossRef Full Text | Google Scholar

Zinbarg, R. E., Revelle, W., and Yovel, I. (2007). Estimating ωh for structures containing two group factors: perils and prospects. Appl. Psychol. Meas. 31, 135–157. doi: 10.1177/0146621606291558

CrossRef Full Text | Google Scholar

Zinbarg, R. E., Revelle, W., Yovel, I., and Li, W. (2005). Cronbach's α, Revelle's β, and Mcdonald's ω_H: their relations with each other and two alternative conceptualizations of reliability. Psychometrika 70, 123–133. doi: 10.1007/s11336-003-0974-7

CrossRef Full Text | Google Scholar

Zumbo, B. D., Gadermann, A. M., and Zeisser, C. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. J. Mod. Appl. Stat. Methods 6, 21–29. doi: 10.22237/jmasm/1177992180

CrossRef Full Text