A commentary has been posted on this article:
Response: Commentary: Early Risk Detection of Burnout: Development of the Burnout Prevention Questionnaire for Coaches
Erik Lundkvist1*
Henrik Gustafsson2,3
Markus Gerber4
Carolina Lundqvist5
Andreas Ivarsson6
Daniel J. Madigan7- 1Performance and Training Unit, The Swedish School of Sport and Health Sciences, Stockholm, Sweden
- 2Faculty of Arts and Social Sciences, Karlstad University, Karlstad, Sweden
- 3Department of Coaching and Psychology, Norwegian School of Sport Sciences, Oslo, Norway
- 4Sport Science Section, Department of Sport, Exercise and Health, University of Basel, Basel, Switzerland
- 5Division of Psychology, Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden
- 6Centre of Research on Welfare, Health and Sport, Halmstad University, Halmstad, Sweden
- 7School of Sport, York St John University, York, United Kingdom
A Commentary on
Early Risk Detection of Burnout: Development of the Burnout Prevention Questionnaire for Coaches
by Schaffran, P., Kleinert, J., Altfeld, S., Zepp, C., Kallus, K. W., and Kellmann, M. (2019). Front. Psychol. 10:714. doi: 10.3389/fpsyg.2019.00714
Introduction
In a recent volume of this journal, Schaffran et al. (2019) introduced the Burnout Prevention Questionnaire for Coaches (BPQ-C). Although we recognize the worthwhile efforts of Schaffran et al., we believe that there are several issues associated with this instrument. This commentary aims to expand on why we think the BPQ-C should not currently be used by practitioners and researchers to screen for burnout.
Lack of Theoretical Foundations
With regards to theory, we have four concerns. First, Schaffran et al.'s definition (and subsequent measurement) of burnout is inadequate. Although several definitions of burnout exist, most researchers agree that burnout is a multidimensional construct and models share—at a minimum—an exhaustion component (Maslach et al., 1996; Raedeke and Smith, 2001; Shirom, 2005). This was recently highlighted by the World Health Organization (2018). While not everyone agrees with one exact definition, no well-recognized definition includes general stress, social stress, and amotivation (Maslach et al., 1996; Raedeke and Smith, 2001; World Health Organization, 2018).
Second, Schaffran et al. (2019) seem to conflate antecedents, outcomes, and the construct of burnout itself. This point is exemplified by the fact that chronic job stress has historically been defined as an essential burnout antecedent and not a defining characteristic of burnout (Maslach et al., 1996; Shirom, 2005). Therefore, using stress as a symptom of burnout and not as an antecedent is conceptually incorrect. Similarly, the dimension labeled “pre-burnout” includes fatigue. This is confusing because of the evident conceptual overlap with exhaustion, which is known as a defining characteristic of burnout (Grossi et al., 2015).
Third, an underlying assumption for the BPQ-C seems to be that there exist sequential events in the burnout process. It is therefore peculiar that the authors relied exclusively on cross-sectional data when they created and tested their instrument (Leiter and Maslach, 1999, 2003; Hendrix et al., 2000). Especially since the complexity regarding temporality between burnout dimensions has been discussed before both in sport and more generally (Shirom and Melamed, 2006; Martinent et al., 2016).
Our final conceptual concern is the reliance on statistical considerations (from analysis of one sample), instead of also considering theoretical aspects, when deciding on the inclusion/exclusion of certain BPQ-C dimensions (Study 2). This exploratory approach has been criticized in the burnout literature previously and heightens risks for conceptual overlap with other constructs (see Shirom and Melamed, 2006), as well as misunderstanding of temporal relations with emotional exhaustion (Moore, 2000). The problem with this procedure becomes particularly apparent in relation to the resources dimension which defines sleep as a resource that may buffer potential burnout. However, with the same logic, poor sleep quality could be seen as a part of “pre-burnout” since poor sleep quality is a well-known antecedent of burnout symptoms (Söderström et al., 2012). Thus, although the authors briefly acknowledge the complexity of sleep in relation to burnout in their discussion, they do not provide any guidelines based on sound empirical work and theory to explain how sleep scores in the BPQ-C should be interpreted.
Utility as a Screening Tool
The authors claim that the BPQ-C has the potential to be used as a screening tool in practical settings to detect early signs of burnout, and to guide prevention strategies before a coach develops more serious burnout symptoms. Although a commendable goal, one well-known problem is that there is often no clear distinction between normal functioning and psychological illness. Evaluations of symptoms should be related to the context in which they appear as well as to contemporary values and economic forces present in society and politics (Kinderman et al., 2017). Furthermore, Schaffran and colleagues do not provide evidence that the BPQ-C can be used to adequately screen for clinical burnout. To do so, analyses such as receiver operating characteristic analysis (ROC), which tests the ability of measures to discriminate between individuals with and without a certain clinical characteristic, is necessary (Kraemer, 1992). Comparisons between a clinical and a healthy sample were not a part of the validation of the BPQ-C. In fact, no descriptive statistics of burnout are reported. This makes it even harder to determine variation and representability of the samples that were used. In addition, even when a clinical sample is compared with a healthy sample, discriminating between healthy and ill individuals is challenging as they show substantial overlap (Lundgren-Nilsson et al., 2012). Therefore, using screening tools without clinical interviews creates substantial risks of both false positives and negatives, a matter that has been neglected in the sport psychology literature and is not considered by Schaffran et al. in their article.
Questionnaire Development
Our last point of criticism concerns the development and validation of the BPQ-C. Several guides on how to develop a psychological instrument are available. Although they differ, there are some common grounds when it comes to item development, scale development, and scale evaluation (MacKenzie et al., 2011; Boateng et al., 2018). Schaffran et al., however, only communicate a few of these suggested steps. When such a step is communicated, like confirmatory factor analysis (CFA), the high degree of similarity in subscale content suggests a redundancy of these factors. This issue is compounded by the fact that parallel analysis suggested a one-factor model (see Sample 1). A stepwise evaluation would have easily eliminated this in the item developing phase. Lastly, in the three-factor CFA, several residuals are correlated (see Figure 1 in Schaffran et al., 2019). The reasons for this needs to be communicated since correlating residuals comes with several risks (with capitalization of chance being the most prominent one), making replication in another sample unlikely (Landis et al., 2009).
Discussion
This commentary provides several arguments as to why, in our opinion, the BPQ-C fails to meet several theoretical, clinical, and psychometric standards. Substantial work is therefore needed before the BPQ-C can be considered a valid measure. Instead, we recommend the use of existing validated measures of burnout. One example of such a measure is the Shirom Melamed Burnout Questionnaire (SMBQ; Kushnir and Melamed, 1992), which is based on Hobfoll's Conservation of Resources (COR) theory (Hobfoll, 1989). The SMBQ has been validated in both clinical and healthy samples (Lundgren-Nilsson et al., 2012). Another alternative would be the Maslach Burnout Inventory (Maslach et al., 1996), which follows the theoretical underpinnings provided by Maslach and colleagues (Maslach et al., 1996). The MBI has shown reasonable psychometric properties in previous research, and for which at least some evidence exists that the instrument is able to discriminate between clinical and healthy individuals. However, the latter only applies to the exhaustion subscale (Kleijweg et al., 2013). Here, we note, however, that using self-report measures as the only source for clinical screening may not be successful because of the overlap between clinical and healthy individuals. We therefore recommend that, wherever possible, self-report screening should be combined with clinical interviews (Lundgren-Nilsson et al., 2012; Kleijweg et al., 2013) and that the interpretation of self-reported burnout measures with clinical cut-offs should be made with caution.
Author Contributions
EL is the main author who initiated the commentary, wrote the first draft, and has been editing all other contributors' drafts and comments. HG was involved in the design of the paper and has been commenting and editing the paper throughout the whole process. MG has been commenting on the paper and has contributed uniquely to the lack of theoretical foundations part. CL has been commenting on the paper and has contributed uniquely to the clinical screening tool part. AI has been commenting on the paper and has contributed uniquely on the part called questionnaire development. DM was involved in the initial scoping of the paper, has done English editing, and contributed uniquely on the lack of theoretical foundations part.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., and Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front. Public Health 6:149. doi: 10.3389/fpubh.2018.00149
Grossi, G., Perski, A., Osika, W., and Savic, I. (2015). Stress-related exhaustion disorder–clinical manifestation of burnout? A review of assessment methods, sleep impairments, cognitive disturbances, and neuro-biological and physiological changes in clinical burnout. Scand. J. Psychol. 56, 626–636. doi: 10.1111/sjop.12251
Hendrix, A. E., Acevedo, E. O., and Hebert, E. (2000). An examination of stress and burnout in certified athletic trainers at Division IA universities. J Athletic Train. 35:139.
Hobfoll, S. E. (1989). Conservation of resources: a new attempt at conceptualizing stress. Am. Psychol. 44:513.
Kinderman, P., Allsopp, K., and Cooke, A. (2017). Responses to the publication of the American Psychiatric Association's DSM-5. J. Hum. Psychol. 57, 625–649. doi: 10.1177/0022167817698262
Kleijweg, J. H., Verbraak, M. J., and Van Dijk, M. K. (2013). The clinical utility of the Maslach Burnout Inventory in a clinical population. Psychol. Assess. 25:435. doi: 10.1037/a0031334
Kraemer, H. C. (1992). Evaluating Medical Tests: Objective and Quantitative Guidelines, Vol. 26. Newbury Park, CA: Sage publications.
Kushnir, T., and Melamed, S. (1992). The Gulf War and its impact on burnout and well-being of working civilians. Psychol. Med. 22, 987–995.
Landis, R., Edwards, B., and Cortina, J. (2009). “Correlated residuals among items in the estimation of measurement models,” in Statistical and Methodological Myths and Urban Legends: Doctrine, Verity, and Fable in the Organizational and Social Sciences, eds C. E. Lance and R. J. Vanderberg (Abingdon: Taylor & Francis), 195–214.
Leiter, M. P., and Maslach, C. (1999). Six areas of worklife: a model of the organizational context of burnout. J. Health Hum. Serv. Admin. 21, 472–489.
Leiter, M. P., and Maslach, C. (2003). “Areas of worklife: a structured approach to organizational predictors of job burnout,” in Emotional and Physiological Processes and Positive Intervention Strategies, eds D. C. Ganster and P. L. Perrewé (Bingley: Emerald Group Publishing Limited), 91–134.
Lundgren-Nilsson, Å., Jonsdottir, I. H., Pallant, J., and Ahlborg, G. (2012). Internal construct validity of the Shirom-Melamed Burnout questionnaire (SMBQ). BMC Public Health 12:1. doi: 10.1186/1471-2458-12-1
MacKenzie, S. B., Podsakoff, P. M., and Podsakoff, N. P. (2011). Constructmeasurement and validation procedures in MIS and behavioral research: integrating new and existing techniques. MIS Q. 35, 293–334. doi: 10.2307/23044045
Martinent, G., Louvet, B., and Decret, J.-C. (2016). Longitudinal trajectories of athlete burnout among young table tennis players: a 3-wave study. J. Sport Health Sci. doi: 10.1016/j.jshs.2016.09.003. [Epub ahead of print].
Maslach, C., Jackson, S. E., and Leiter, M. P. (1996). Maslach Burnout Inventory Manual, Vol. 4. Palo Alto, CA: Consulting Psychologists Press.
Moore, J. E. (2000). Why is this happening? A causal attribution approach to work exhaustion consequences. Acad. Manage. Rev. 25, 335–349. doi: 10.5465/amr.2000.3312920
Raedeke, T. D., and Smith, A. L. (2001). Development and preliminary validation of an athlete burnout measure. J. Sport Exerc. Psychol. 23, 281–306. doi: 10.1123/jsep.23.4.281
Schaffran, P., Kleinert, J., Altfeld, S., Zepp, C., Kallus, K. W., and Kellmann, M. (2019). Early risk detection of burnout: development of the burnout prevention questionnaire for coaches. Front. Psychol. 10:714. doi: 10.3389/fpsyg.2019.00714
Shirom, A. (2005). Reflections on the study of burnout. Work Stress 19, 263–270. doi: 10.1080/02678370500376649
Shirom, A., and Melamed, S. (2006). A comparison of the construct validity of two burnout measures in two groups of professionals. Int. J. Stress Manage. 13:176. doi: 10.1037/1072-5245.13.2.176
Söderström, M., Jeding, K., Ekstedt, M., Perski, A., and Åkerstedt, T. (2012). Insufficient sleep predicts clinical burnout. J. Occup. Health Psychol. 17:175. doi: 10.1037/a0027518
World Health Organization (2018). ICD-11 International Classification of Diseases for Mortality and Morbidity Statistics (11th Revision). Available online at: https://icd.who.int/browse11/l-m/en
Keywords: burnout, fatigue, exhaustion disorder, clinical burnout, burnout measurement
Citation: Lundkvist E, Gustafsson H, Gerber M, Lundqvist C, Ivarsson A and Madigan DJ (2019) Commentary: Early Risk Detection of Burnout: Development of the Burnout Prevention Questionnaire for Coaches. Front. Psychol. 10:2721. doi: 10.3389/fpsyg.2019.02721
Received: 10 October 2019; Accepted: 18 November 2019;
Published: 04 December 2019.
Edited by:
Maicon Rodrigues Albuquerque, Federal University of Minas Gerais, BrazilReviewed by:
Daniel Alvarez Pires, Federal University of Pará, BrazilCopyright © 2019 Lundkvist, Gustafsson, Gerber, Lundqvist, Ivarsson and Madigan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Erik Lundkvist, erik.lundkvist@gih.se