School Burnout Inventory: Latent Profile and Item Response Theory Analyses in Undergraduate Samples

May, Ross W.; Rivera, Peter M.; Rogge, Ronald D.; Fincham, Frank D.

doi:10.3389/fpsyg.2020.00188

ORIGINAL RESEARCH article

Front. Psychol., 06 March 2020

Sec. Psychology for Clinical Settings

Volume 11 - 2020 | https://doi.org/10.3389/fpsyg.2020.00188

This article is part of the Research Topic Psychological Distress among University Students View all 14 articles

School Burnout Inventory: Latent Profile and Item Response Theory Analyses in Undergraduate Samples

$\r\nRoss W. May*$ Ross W. May^1*

Peter M. Rivera²

Ronald D. Rogge³

Frank D. Fincham¹

¹Family Institute, Florida State University, Tallahassee, FL, United States
²School of Psychology, Family, and Community, Seattle Pacific University, Seattle, WA, United States
³Department of Clinical and Social Sciences in Psychology, University of Rochester, Rochester, NY, United States

The current research reports both latent profile (person-oriented) and item response theory (IRT) analyses of the School Burnout Inventory (SBI) in United States undergraduate samples. Study 1 (n = 1,007) comprises a latent profile analysis (LPA) that identified four mutually exclusive subgroups based on patterns of school burnout responses. Covariate analyses of grade point average and negative affect suggested that school burnout profiles function similarly to variable-oriented approaches. Study 2 (n = 544) explored longitudinal patterns of school burnout among college students via use of a repeated measures LPA. Findings suggested that the profiles identified reflect a relatively stable school burnout trajectory over time. Covariate analysis of sleep quality and academic engagement demonstrated differences across profiles, but the patterns were similar to variable-oriented statistical approaches. Study 3 (n = 2,364) utilized an IRT analysis of the SBI to identify a short, efficient measure. Item information curves and graded response model item discrimination parameters identified a 4-item SBI scale (SBI-4) that offered reasonably high levels of information for assessing school burnout in comparison to the original nine-item SBI. Implications and future research are identified.

Introduction

The attainment of a baccalaureate degree offers a considerable advantage in job placement over those who have not acquired an undergraduate education (U.S. Department of Commerce, 2016). Not surprisingly, approximately 69.2% of high school graduates go on to enroll in college programs (U.S. Department of Commerce, 2016). Undergraduate enrollment is disproportionally female, with about 11.5 million females enrolled in the fall of 2017 as compared to 8.9 million males (U.S. Department of Education, 2017). Reflecting the importance of higher education, the number of American undergraduate students is increasing. Enrollment of traditional college-aged students (18- to 24-year-olds) is up more than 5.1 million in 2017 (20.4 million) from that of 2000 (U.S. Department of Education, 2017). Given the prevalence of college education and its resulting employment advantages, investigations into factors that may jeopardize retention in United States colleges are warranted. One such emerging factor is school burnout.

Burnout is a well-researched, multidimensional, affective response to occupational stress. Although burnout has traditionally been investigated within the workplace, it is becoming more common to extend burnout investigations into academic populations (Salmela-Aro et al., 2008, 2009; Parker and Salmela-Aro, 2011). Within an educational context, school burnout has been characterized by chronic exhaustion from school-related work, cynicism toward the meaning of school, and feelings of inadequacy toward school-related accomplishments (Salmela-Aro et al., 2008, 2009). A host of negative conditions have been associated with school burnout, including suboptimal physiological functioning (May et al., 2014a, b, 2016) and diminished cognitive and academic performance (Salmela-Aro et al., 2008, 2009; May et al., 2015).

Although school burnout is typically studied using a variable-oriented approach, valuable contributions stemming from a person-oriented approach have improved the school burnout literature. Contrary to research treating school burnout symptoms as the unit of analysis, person-oriented analyses focus on groups of individuals as the unit of analysis. As thoroughly discussed in Mäkikangas and Kinnunen (2016), person- and variable-oriented statistical approaches diverge both theoretically and methodologically. These approaches differ statistically in that person-oriented analyses examine intra-individual variation in school burnout symptomology (e.g., class or cluster analyses), whereas variable-oriented analyses examine inter-individual variation (e.g., linear association between variables through either correlational or mean level analyses). Person-oriented statistical analyses assume that there is heterogeneity in burnout symptoms in the population (Laursen and Hoff, 2006).

Typically, person-oriented analyses begin with an unknown number of classes or clusters and proceed with the development of different class solutions. Class solutions are then compared and evaluated on the basis of statistical and theoretical criteria (Bergman et al., 2003; Mäkikangas and Kinnunen, 2016). The advantage of a person-oriented approach is its ability to identify types or classes of symptomology and the trajectory of development and change in symptomology. Types or classes are not identified by any predefined scores or cutoff values and, in the context of school burnout, may help to provide a unique perspective on how individuals are similar (or different) from each other in burnout symptomology. Thus, person-oriented statistical approaches help identify how many groups of individuals are needed to largely describe between-person differences (either longitudinally or cross-sectionally), assuming that they are drawn from several subpopulations with between-person variability.

The systematic review conducted by Mäkikangas and Kinnunen (2016) identified only 24 publications (out of 470) that reported findings pertaining to person-oriented approaches to modeling burnout between the years 2004 and 2015. Their findings reveal that person-oriented analyses of burnout types and trajectories largely parallel burnout findings from variable-oriented approaches: burnout typically develops with the appearance of exhaustion and cynicism occurring simultaneously and then the development of reduced professional efficacy followed by symptoms that are largely stable over time. However, person-oriented analyses provided unique information regarding the identification of atypical burnout types and developmental trajectories (e.g., cynicism occurring alone or together with reduced professional efficacy).

Importantly, only 4 of the 24 studies identified by Mäkikangas and Kinnunen (2016) pertained to school burnout (the remainder focused on occupational burnout). Furthermore, none of the four publications utilized American samples (one publication utilized Korean students, one utilized Chinese students, and two utilized Finnish students; Lee et al., 2010; Zhang et al., 2013; Salmela-Aro and Upadaya, 2014; Tuominen-Soini and Salmela-Aro, 2014, respectively). In addition, only one of those publications reported on a collegiate sample (see Tuominen-Soini and Salmela-Aro, 2014, modeling burnout and engagement profiles, although see the more recent work of Salmela-Aro and Read, 2017). Differences in tuition price, enrollment rates, university degree curriculum structure, and typical class size are just a few of many factors (for additional factors, see Taylor, 2012; Chang and Wang, 2017; Chatlani, 2018) that may lead to differential student stress across various countries and across different educational settings within countries (e.g., university, technical college, community college).

Person-oriented analyses provide a worthwhile supplement to variable-oriented approaches to examining school burnout. However, given their limited use in the school burnout literature and the lack of representation of American collegiate samples in the person-oriented literature, the current research reports findings of latent profile analyses (LPA) of school burnout in American collegiate samples. Study 1 reports on a LPA examining school burnout in United States undergraduates. Study 2 then explores longitudinal patterns of school burnout among United States college students using repeated measures LPA (RMLPA). Finally, supplementing the person-oriented approaches of Studies 1–3 utilizes an item response theory (IRT) analysis to identify an efficient, psychometrically optimized 4-item School Burnout Inventory (SBI) scale (SBI-4). All three studies focus on the measurement of school burnout using the SBI (Salmela-Aro et al., 2009).

Study 1

Person-oriented approaches to burnout complement traditional variable-oriented approaches by helping to reveal intra-individual heterogeneity within the burnout syndrome. This allows for the identification of types or patterns of burnout symptomology within individuals. Study 1 explores school burnout via a person-oriented approach, specifically LPA, using the SBI (Salmela-Aro et al., 2009). LPA is a latent variable modeling technique that classifies individuals into mutually exclusive subgroups based on patterns of responses to either continuous or a combination of categorical and continuous observations (Roesch et al., 2010). Extending LPA, this study also examines the associations between school burnout profile membership, related negative affect symptomology (depression and anxiety symptoms), and an indicator of academic achievement (grade point average, GPA). Prior variable-oriented research has documented relationships between the aforementioned constructs indicating that even when controlling for affective symptoms, increased school burnout predicts lower GPA (May et al., 2015).

Analytical Approach

To explore heterogeneity within school burnout among United States college students, a LPA was conducted using Mplus (Version 7.4). The LPA focused on participants’ responses to the nine items from the SBI (Salmela-Aro et al., 2009). Independent models were estimated in an iterative manner that began with a one-profile solution, and profiles were added until the best-fitting model was identified. For each model estimated, random parameter start values were used to verify the replication of the log-likelihood value and to ensure that model convergence was due to a global rather than local maximum (Hipp and Bauer, 2006). Any missing data were handled using full-information-maximum-likelihood.

The final model was chosen based on a number of statistical indicators, model parameters, and interpretative meaning of the final solution. Specifically, absolute model fit was assessed by the following fit criterion: Akaike information criterion (AIC), Bayesian information criterion (BIC), and sample size adjusted BIC (A-BIC). Lower AIC, BIC, and A-BIC indicate better model data fit (Roesch et al., 2010). Bootstrap likelihood ratio tests (BLRTs) also informed this process, and provided a p-value that, when significant, indicated a model with k number of profiles to be preferred over a model with k - 1 profiles. Two parameters estimated in latent profile models also informed the selection of the final model: latent class probabilities (i.e., likelihood of belonging to a particular class) and conditional response means (i.e., mean of an item within a particular profile). Higher latent class probabilities and distinguished conditional response means are preferred. Entropy was considered in this process, which is an indicator of profile enumeration and ranges from 0 to 1, where closer to 1 indicates how well profiles have been distinguished (Roesch et al., 2010). Models that contained profiles with less than 5% of the sample were considered to be less parsimonious and less likely to replicate in future samples, and were therefore considered undesirable solutions independent of the other fit indices.

A three-step approach was used to examine the associations between profile membership and GPA and depressive and anxiety symptomology. This approach is preferred to the commonly used classify–analyze approach when determining latent profile differences on auxiliary variables because it makes use of the latent measurement model by accounting for measurement error and because of its advanced performance in examining associations between latent classes and external variables (Bolck et al., 2004; Vermunt, 2010; Asparouhov and Muthén, 2014). Following identification of the best-fitting latent profile model, the next steps involved using the posterior distribution to create an indicator of most likely latent class membership before estimating the final model with auxiliary variables (i.e., indicators of emerging adult GPA, anxiety, and depression), while accounting for the measurement error associated with latent profile classification (Asparouhov and Muthén, 2014). This process utilized Wald chi-square significance tests to determine mean differences for each outcome across identified profiles.

Method

Participants

Participants were 1,007 undergraduate students (67% females, M_age = 20.06 years, SD = 1.26). Students who completed at least one full academic semester were eligible for study participation. Sample demographics include: 66% Caucasian, 16% African American, 1.5% Asian, 8% Hispanic, and 8.5% endorsing either biracial or non-disclosed ethnicity, with 14% freshmen, 26% sophomores, 35% juniors, and 25% seniors.

Measures

School burnout

School burnout was measured using the SBI (Salmela-Aro et al., 2009). The SBI consists of nine items measuring three first-order factors of school burnout: (a) exhaustion at school (four items), (b) cynicism toward the meaning of school (three items), and (c) sense of inadequacy at school (two items). Summed scores from the first-order factors comprise a second-order overall school burnout score. Higher composite scores indicate higher burnout. Participants responded to items (e.g., “I feel overwhelmed by my schoolwork” as an example exhaustion item, “I feel a lack motivation in schoolwork and often think of giving up” as an example cynicism item, and “I often have feelings of inadequacy in my schoolwork” as an example inadequacy item) on a scale from 1 (completely disagree) to 6 (completely agree). Internal consistency for the present sample was α = 0.89.

Depressive symptoms

Depressive symptomology was assessed via the 10-item Center for Epidemiological Studies Depression Scale (CES-D; Radloff, 1977; Santor and Coyne, 1997). The CES-D has been widely used as a measure of depressive symptoms in non-clinical samples. It asks participants to respond to a list of ways they may have felt or behaved during the previous week. Sample items include “I could not ‘get going”’ and “I felt hopeful about the future.” Participants responded 0 (rarely) to 3 (most/all of the time) on items, such as feelings of loneliness, hopelessness, and restless sleep. Some items were reverse-coded, such that higher responses indicate more depressive symptoms. Internal consistency for the present sample was α = 0.89.

Anxiety symptoms

Anxiety was measured using the 20-item State–Trait Anxiety Inventory (STAI; Spielberger et al., 1970). Participants were asked to respond to anxiety items such as “upset,” “calm,” “secure,” “at ease,” and “nervous.” Responses were scored on a four-point Likert scale from 1 (not at all) to 4 (very much so). Half of the items were reverse-coded so that higher scores indicated greater anxiety. Items were summed to create a composite anxiety score with a possible range of 20–80. Internal consistency for the sample was α = 0.90.

Academic achievement

Academic achievement was assessed through self-reported cumulative GPA ranging from 1.50 to 4.00.

Procedure

Data from all eligible participants was collected via online survey questionnaires. Questionnaires contained demographic questions and the measurement scales described. All participants were recruited from undergraduate classrooms at a large southern university in the United States as an option for voluntary class credit. Extra credit was generally less than 1% of the final grade. Data were collected in the middle (weeks 3–9) of the spring academic semester. All participants gave their written consent prior to study participation, and approval was obtained from the institutional review board before any data were collected.

Results and Discussion

Selection of Optimal Latent Profile Solution

Statistical indicators used to determine the final profile solution are presented in Table 1. The analysis of the nine school burnout items suggested that a four-profile solution fit the data best. AIC, BIC, and A-BIC all decreased for every solution over one, while BLRT indicated that a five-profile solution was preferred over a four-profile solution. When comparing four- and five-profile models, however, entropy was stronger for a four-profile solution, and latent class probabilities were consistently higher for a four-profile solution. Moreover, the latent class probabilities of subgroup membership assignment were strong for the four-class solution: 91% for profile 1, 87% for profile 2, 93% for profile 3, and 93% for profile 4. For these reasons, we considered the four-profile solution to be the best-fitting model.

TABLE 1

Table 1. Comparison of latent profile models (N = 1,007).

Description of Identified Profiles

The conditional response means for each indicator of school burnout were used to interpret each subgroup and are presented in Figure 1. Profile 1 participants (approximately 13% of the overall sample) reported the lowest levels of school burnout across all indicators and were labeled low burnout. Participants in profile 2 (approximately 25% of the overall sample) reported higher levels of burnout than their counterparts in profile 1 but lower-than-average responses overall. As such, this profile was labeled below-average burnout, reflecting their close yet lower-than-average responses across all indicators. Profile 3 was the largest profile, including close to half of the overall sample (48%). These participants reported above-average scores of school burnout across all nine indicators; therefore, profile 3 was labeled above-average burnout. Lastly, profile 4 (approximately 14% of the overall sample) was labeled high burnout because it comprised those with the highest reported levels of school burnout across all indicators.

FIGURE 1

Figure 1. Conditional response means for a four-profile model.

Differences in GPA as a Function of Profile Membership

Results from Wald chi-square significance tests determining whether differences existed across profile mean scores of GPA are presented in Figure 2, with Table 2 showing all post hoc comparisons. Findings showed that levels of GPA varied across profiles. As expected, participants comprising the low burnout profile reported the second highest GPA (M = 3.38), which was significantly higher than that of those classified into the high burnout profile. Those in the below-average burnout profile recorded the highest GPA (M = 3.45), which was significantly higher than that of those in the above-average burnout (M = 3.30) and high burnout (M = 3.17) profiles. Notably, participants in the high burnout profile reported the lowest GPA, which was statistically significantly different from each of the other profiles.

FIGURE 2

Figure 2. Study 1 differences on grade point average (GPA), depressive symptoms [Center for Epidemiological Studies Depression Scale (CES-D)], and anxiety [State–Trait Anxiety Inventory (STAI)] symptoms across the four burnout profiles.

TABLE 2

Table 2. Profile differences in grade point average, depression, and anxiety (N = 1,007).

Differences in Depressive Symptomology as a Function of Profile Membership

Findings determining profile differences in depressive symptomology are reported in Figure 2, with post hoc comparisons in Table 2. Participants comprising the low burnout profile reported the lowest levels of depressive symptomology (M = 4.75), followed by those in the below-average burnout profile (M = 6.98) and above-average burnout profile (M = 8.90), with individuals in the high burnout profile reporting the highest levels of depressive symptomology (M = 14.00). All profile comparisons yielded statistically significant differences except for that between participants in the below-average burnout and above-average burnout profiles.

Differences in Anxiety Symptomology as a Function of Profile Membership

Figure 2 also shows results from the Wald chi-square significance tests for profile differences on reported levels of anxiety symptomology, with post hoc comparisons in Table 2. Levels of anxiety symptomology significantly varied across all profiles. Participants in the low burnout profile reported the lowest levels of anxiety symptomology (M = 28.32), followed by those in the below-average burnout profile (M = 34.96) and then the above-average burnout profile (M = 39.13), with individuals in the high burnout profile reporting the highest levels of anxiety symptomology (M = 46.39).

These findings are novel as they reflect the first person-oriented approach to understanding intra-individual heterogeneity in burnout symptomology in a United States collegiate sample. Four unique categories of school burnout symptomology were established with a four-class profile solution: categories labeled low burnout (13% of the overall sample), below-average burnout (25% of the overall sample), above-average burnout (48% of the overall sample), and high burnout (14% of the overall sample). Similarly, Ahola et al. (2014), albeit in an older sample of employed individuals, identified a similar clustering of burnout and depressive symptoms, identifying a three-factor solution of “low level of symptoms” (low level of burnout and depressive symptoms), “intermediate level of symptoms” (intermediate levels of burnout and depressive symptoms), and “high level of symptoms” (high levels of burnout and depressive symptoms).

Interestingly, covariate analyses suggest that profiles function similarly to a variable-oriented approach. This is evidenced by a reduction in GPA and increase in affective symptoms corresponding to the increase in burnout symptomology found within the established profiles. However, as this is the first LPA using the SBI in an American collegiate sample, supplemental profile analyses would be advantageous. Toward this objective, Study 2 provides an investigation of developmental trajectories of school burnout profiles.

Study 2

To date, consensus regarding a school burnout trajectory remains elusive. Earlier work regarding development change in burnout in European students seemed to suggest that school burnout is surprisingly consistent over time (Parker and Salmela-Aro, 2011; Salmela-Aro and Upadaya, 2014). More recent work in latent profiles, however, has suggested that while a stable, zero-slope burnout trajectory may be found within a majority of students sampled (Salmela-Aro and Upadaya, 2014), substantial trajectory heterogeneity can be identified within subclasses (see Salmela-Aro and Read, 2017). Although informative, such school burnout trajectory analyses have not yet been examined within any United States student population. Thus, to improve upon the cross-sectional limitation of Study 1 and begin to develop an idea of the developmental trajectory of school burnout over time in the United States, this study explores longitudinal patterns in school burnout profiles across three academic semesters.

Additionally, associations between school burnout profiles and student’ levels of sleep quality and academic engagement were examined. Both diminished sleep quality and poor job engagement are well-documented correlates of increased occupational burnout (for a review of sleep and occupational burnout findings, see Akerstedt et al., 2017; for a review of job engagement and occupational burnout findings, see Leon et al., 2015 or Taris et al., 2017). However, considerably less is known of these relationships in regard to school burnout, and then even less in non-European academic populations, demonstrating a need for research regarding sleep and engagement relationships with school burnout in American collegiate samples.

Sleep is a vital component of health and has been shown to be important in occupational burnout (Alvarez and Ayas, 2007; Akerstedt et al., 2017). However, research linking sleep and school burnout is largely absent in primary to post-secondary student populations. What little research there is primarily focuses on sleep–burnout relationships in medical students (for a review, see Dyrbye et al., 2006). As an example of a typical finding, a survey study of medical students in India found that poorer sleep as measured with the Pittsburgh Sleep Quality Index (PSQI) corresponded to increased subscale school burnout scores (e.g., burnout and disengagement) using the Oldenburg Burnout Inventory (Shad et al., 2015).

Engagement is important within the burnout literature as it represents motivational, cognitive, and behavioral components that may prevent burnout’s occurrence and continued development (although the uniqueness of burnout and engagement as constructs is admittedly controversial; see Taris et al., 2017). In higher education, academic engagement is conceptualized as a three-dimensional construct, comprising energy for, dedication toward, and absorption in schoolwork. Specifically, academic engagement is thought of as a positive approach to schoolwork (energy), dedication to a positive attitude while perceiving schoolwork as meaningful (dedication), and absorption in concentration on schoolwork (absorption; Salmela-Aro and Upadaya, 2014, p. 60; see also Schaufeli et al., 2002a). Attempts to link school burnout and school engagement have been limited and have primarily involved European samples, with no findings pertaining to United States students (to the best of our knowledge). However, the research that does exist indicates that decreased school engagement corresponds to a variety of negative outcomes, including increased depression and increased school burnout (Salmela-Aro et al., 2009). Demonstrating similar relationships, a longitudinal profile approach by Salmela-Aro and Read (2017) evaluating school burnout and academic engagement indicated that as academic engagement decreases, school burnout increases over time.

Therefore, given the importance of sleep and engagement and their absence in the school burnout literature, we examined their relationships with profiles of school burnout. As suggested by prior research on medical students, we expect profiles with increased school burnout symptomology to align with poorer sleep quality. Furthermore, as suggested by the findings in Salmela-Aro and Read’s (2017) study, we expected school burnout and academic engagement to be inversely related so that increased engagement corresponds to lower school burnout.

Analytical Approach

To explore longitudinal patterns of school burnout among college students, an RMLPA was conducted in Mplus (Version 7.4). RMLPA is an application of LPA to repeated measures that aims to identify a categorical latent variable that underlies the heterogeneity in a population’s responses to questions over time (Collins and Lanza, 2010). We again focused on participants’ responses to the nine items from the SBI obtained across three time points. Independent RMLCA models were estimated in the same iterative manner that the latent profile models were estimated in the previous study, and the same statistical indicators (i.e., AIC, BIC, A-BIC, BLRT, and entropy) and model parameters (i.e., latent class probabilities and conditional response means) were used to select the optimal fitting model. Once the final model was identified, a three-step approach as conducted in Study 1 (Asparouhov and Muthén, 2014) was used to examine the associations between profile membership and student’ levels of sleep and academic engagement.