Andreas Gegenfurtner- Methods in Learning Research, University of Augsburg, Augsburg, Germany
Multivariate behavioral research often focuses on latent constructs—such as motivation, self-concept, or wellbeing—that cannot be directly observed. Typically, these latent constructs are measured with items in standardized instruments. To test the factorial structure and multidimensionality of latent constructs in educational and psychological research, Morin et al. (2016a) proposed bifactor exploratory structural equation modeling (B-ESEM). This meta-analytic review (158 studies, k = 308, N = 778,624) aimed to estimate the extent to which B-ESEM model fit differs from other model representations, including confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), hierarchical CFA, hierarchical ESEM, and bifactor-CFA. The study domains included learning and instruction, motivation and emotion, self and identity, depression and wellbeing, and interpersonal relations. The meta-analyzed fit indices were the χ2/df ratio, the comparative fit index (CFI), the Tucker-Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardized root mean squared residual (SRMR). The findings of this meta-analytic review indicate that the B-ESEM model fit is superior to the fit of reference models. Furthermore, the results suggest that model fit is sensitive to sample size, item number, and the number of specific and general factors in a model.
Introduction
To examine the factorial structure and multidimensionality of latent constructs in educational and psychological research, Morin et al. (2016a) proposed bifactor exploratory structural equation modeling (B-ESEM) as a methodological synergy that integrates bifactor modeling and exploratory structural equation modeling. Since their seminal paper, a number of studies have applied B-ESEM in research on learning and instruction, motivation and emotion, self and identity, wellbeing, and other areas. The present systematic review and meta-analysis aimed to collect and describe these studies, meta-analyze their reported model fit, and estimate the extent to which the fit of the B-ESEM model differs from that of the other tested model representations. A secondary aim was to analyze how sensitive model fit was to sample size, item number, and the number of specific and general factors in a model.
Exploring the factor structure of multidimensional constructs
Researchers interested in exploring the factorial structure of a multidimensional construct can choose among analytical options including confirmatory factor analysis (CFA; Jöreskog, 1969), exploratory structural equation modeling (ESEM; Asparouhov and Muthén, 2009), and hierarchical and bifactor representations of CFA and ESEM (Rindskopf and Rose, 1988; Reise, 2012; Morin et al., 2013, 2016a). A number of excellent review papers and pedagogical illustrations of these factor analytic techniques exist (Marsh, 2007; Marsh et al., 2009, 2014; Reise, 2012; Morin et al., 2013, 2016a, 2020; Bandalos and Finney, 2018; Sellbom and Tellegen, 2019). Interested readers can consult these excellent resources for more detail, but let us briefly review how the factor structure of multidimensional latent constructs can be represented and analyzed. Figure 1 offers an overview of schematic model representations.
Figure 1. Schematic model representations. Ovals represent latent factors; squares represent observed variables; full unidirectional arrows between ovals and squares represent factor loadings; dotted unidirectional arrows between ovals and squares represent cross-loadings; full unidirectional arrows linked to a single oval represent factor disturbance; full unidirectional arrows linked to a single square represent item uniqueness; bidirectional full arrows between ovals represent factor covariances/correlations; bidirectional dashed arrows linked to a single oval represent factor variance; CFA, confirmatory factor analysis; ESEM, exploratory structural equation modeling; H Factor, higher-order factor in a hierarchical model; L Factor 1–3, lower-order factors in a hierarchical model; S Factor 1–3, specific factors in a bifactor model; G Factor, general factor in a bifactor model.
The simplest form of representation and analysis, arguably, is a one-factor first-order model in which all items are loaded on a single factor. More complex are CFA and ESEM that can be applied to test a hypothesized factor structure: in CFA, items load on a single factor with zero cross-loadings; in ESEM, too, items load a single factor but without CFA's strict requirement of zero cross-loadings. When exploring hierarchically ordered constructs, CFA and ESEM can be extended to models in which items are loaded on specific lower-order factors (L-factors), which are in turn loaded on a single higher-order factor (the H-factor). Finally, in the case of bifactor modeling, CFA and ESEM can be extended to a model in which items are loaded on one general factor (the G-factor) representing a global overarching construct as well as on multiple specific factors (S-factors) representing their subscales—in bifactor CFA (B-CFA) without and in bifactor ESEM (B-ESEM) with free estimation of cross-loadings between items and non-target factors. B-ESEM in particular has been shown to result in a better model fit for a number of multidimensional hierarchical constructs that are frequently explored in educational and psychological research1.
B-ESEM in multivariate behavioral research
A growing number of studies test the multidimensionality of latent constructs in research on learning and instruction, motivation and emotion, self and identity, wellbeing and depression, and interpersonal relationships. In the domain of learning and instruction, for example, Scherer et al. (2016) identified the presence of a general factor of student-perceived instructional quality and three distinct sub-dimensions: teacher support, cognitive activation, and classroom management. Fernandez-Rio and colleagues explored the factor structure of a scale measuring dimensions of cooperative learning. In the domain of motivation and emotion, Howard et al. (2020) showed that motivation scales based on the self-determination theory were best represented as a B-ESEM model in which all items defined specific motivation regulatory qualities on a continuum (from amotivation to intrinsic motivation) and were also used to define a global self-determination factor. Perera et al. (2018b) tested the structure of teacher engagement, identifying a general factor and specific cognitive-physical, emotional, and social engagement factors. In the domain of self and identity, for example, Arens et al. (2021) explored the structure of academic self-concept and reported that a B-ESEM representation including one general self-concept factor and multiple specific self-concept factors (for German, English, math, physics, chemistry, biology, and history) provided the best model fit compared with reference models. In the domain of wellbeing and depression, Morin et al. (2017) tested the dimensionality of the Index of Psychological WellBeing at Work that can also be used to wellbeing of teachers. In the domain of interpersonal relations, Ratelle et al. (2018) showed that the factor structure of a scale measuring parental structure, including parents' rules, predictability, feedback, opportunities, rationale, and authority, was best represented as a B-ESEM model.
These individual studies support the assumption that B-ESEM is best used to deal with multidimensional hierarchical constructs, yet the extent to which model fit of B-ESEM differs from other model representations has not yet been investigated across studies, domains, and scales. A meta-analysis of published studies could help to close this research gap. Among the frequently used parameters used in the studies reviewed to test model fit are the χ2/df ratio (Pearson, 1900), the comparative fit index (CFI; Bentler, 1990), the Tucker-Lewis index (TLI; Bentler and Bonett, 1990), the root mean square error of approximation (RMSEA; Steiger, 1990), and the standardized root mean squared residual (SRMR; Bentler, 1995). Table 1 provides an overview of goodness-of-fit indices used in B-ESEM studies2. Furthermore, if the evidence that model fit indices are sensitive to sample size, item number, and factor number from simulation studies is valid (Hu and Bentler, 1999; Marsh et al., 2004; Shi et al., 2019), it would be interesting to replicate these findings through a research synthesis using real data rather than simulated data. Such a synthesis would also be useful to describe the status quo of research adopting a B-ESEM framework, the mapping of tested constructs (such as academic self-concept; Arens et al., 2021), scales (such as the Multidimensional Work Motivation Scale; Howard et al., 2020), and domains (such as learning and instruction).
Table 1. Factor analytic techniques and model fit indices.
The present study
Based on a systematic literature review and meta-analysis of previous research applying the B-ESEM framework, the present study compares goodness-of-fit indices of B-ESEM with reference models. The study sought to answer three research questions: (1) Which domains, constructs, and scales are targeted in studies adopting a B-ESEM framework? (2) What is the model fit of B-ESEM representations compared to CFA, ESEM, H-CFA, H-ESEM, and B-CFA representations? (3) How sensitive is model fit to sample size, item number, and the number of specific and general factors in a model?
Methods
Steps in conducting and reporting this meta-analytic review of B-ESEM included defining inclusion and exclusion criteria, searching the literature, coding information from the retrieved studies, calculating intercoder reliability, and meta-analyzing the fit indices (Page et al., 2021). The following sections describe these steps in more detail.
Defining inclusion and exclusion criteria
We identified studies that reported goodness-of-fit indices of B-ESEM in educational and psychological research. Table 2 presents the criteria for inclusion and exclusion of these studies. To be included, a study had to report a χ2 (df), CFI, TLI, RMSEA, or SRMR estimate of a bifactor ESEM model and compare it with a CFA, ESEM, H-CFA, H-ESEM, or B-CFA reference model. Studies were omitted if they did not examine a B-ESEM representation, did not compare B-ESEM to a reference model, or relied on simulated data. To minimize publication bias, our literature search was deliberately broad and included reports in journal articles, book chapters, monographs, conference papers, and unpublished theses or dissertations, regardless of participant population, publication language, publication year, or domain.
Table 2. Criteria for inclusion and exclusion in the meta-analytic review.
To illustrate how we applied the criteria, examples of both excluded and included articles may help. On the one hand, an excellent study by Zhu et al. (2020) that reports model fit evaluation of a B-ESEM representation of mindfulness was excluded because B-ESEM was not compared with other models; rather, the B-ESEM factor scores were used to perform latent profile analyses. On the other hand, a study by Arens et al. (2021) was included because the authors compared goodness-of-fit indices obtained from higher-order and bifactor ESEM representations of the structure of academic self-concept.
Searching the literature
Grounded in the inclusion criteria presented in Table 2, a two-phase literature search was performed. First, we conducted an electronic search of five databases—ERIC, PsycINFO, PubMed, Scopus, and Web of Science—for all publication types in any language published up to December 31, 2021 and with relevant keywords in the title or abstract. The keywords were bifactor exploratory structural equation modeling, bifactor ESEM, and B-ESEM, along with their spelling variants. This database search resulted in a total of 525 hits: 25 from ERIC, 106 from PsycINFO, 34 from PubMed, 182 from Scopus, and 178 from Web of Science. Two trained raters eliminated 318 duplicates because they were listed in more than one database, after which 207 articles remained. Next, both raters independently and in duplicate screened a random subset of the 207 identified articles (10 percent; n = 21 articles). As interrater agreement was high, with Cohen's κ = 0.881 (95% CI = 0.659–1.00), a single rater continued to screen the remaining studies for eligibility by reading titles and abstracts, after which 186 articles remained. Both raters then read the full texts of these articles independently to check for eligibility, and 60 studies were excluded because they did not examine a B-ESEM representation (37 removals) or did not compare B-ESEM to a reference model (23 removals), resulting in 126 articles for inclusion in the meta-analytic review.
Second, we performed a forward cross-referencing search, using Google Scholar to identify studies that cited the influential article of Morin et al. (2016a) up until December 31, 2021. We read the titles and abstracts of the 723 citing studies and retrieved the full texts of 169 of these reports. After the full texts were read, 137 records were excluded because the studies did not examine a B-ESEM representation (129 removals), did not include a reference model (4 removals), or used simulated data (4 removals). The remaining 32 studies met all inclusion criteria and were thus included in the review.
In summary, the literature search resulted in a total of 158 studies: 126 from the database search and 32 from the forward search. Figure 2 presents the PRISMA study selection flow diagram (Page et al., 2021). The included studies reported 308 comparisons of B-ESEM representations to reference models, which were subsequently coded.
Figure 2. PRISMA study selection flow chart.
Coding information from the retrieved studies
Once the studies for inclusion were selected, two trained raters used the coding scheme presented in Table 3 to code a random subset of the 308 identified reports (10 percent; n = 31 reports) independently and in duplicate. Coded information included publication characteristics, measurement characteristics, and goodness-of-fit indices. Publication characteristics were coded as the first author and publication year. Measurement characteristics were coded as sample size, target construct, and scale name and abbreviation, as well as the number of items, specific factors, and general factors. Goodness-of-fit indices were coded separately for one-factor, CFA, ESEM, H-CFA, H-ESEM, B-CFA, and B-ESEM model representations; indices included CFI, TLI, RMSEA, SRMR, and the χ2/df ratio.
Table 3. Coding scheme.
To illustrate our coding decisions, examples how we applied the coding scheme may help. For instance, Rodrigues et al. (2020) reported the fit of model representations with (a) one general and two specific factors, (b) one general and six specific factors, and (c) two general and six specific factors for the Behavioral Regulation in Sport Questionnaire and the Behavioral Regulation in Exercise Questionnaire. Because this meta-analytic review aimed to estimate the influence of factor number on model fit, we coded each model separately. Similarly, Tóth-Király et al. (2019) reported the model fit of representations with one or two general factors of the Basic Psychological Need Satisfaction and Frustration Scale. We coded each representation separately to allow for comparison. Generally, model fit was coded separately for studies that tested models with varying numbers of specific and/or general factors (e.g., Arias et al., 2016; Rodenacker et al., 2017; Gu et al., 2020; Frutos, 2021; Yi et al., 2021) and with varying item numbers. For example, Bianchi and Verkuilen (2021) examined the Green et al. Paranoid Thoughts Scale (GPTS) in its original 32-item version, a revised 18-item version, and an 8-item version. Model fit for each version was coded independently to allow for estimation of the influence of item number on the goodness-of-fit indices. Model fit was also coded separately for studies that used more than one sample (e.g., Neff et al., 2019; Howard et al.,2020; Longo et al., 2020; Vaughan et al., 2020) or measurement time (e.g., Stenling et al., 2018; Cece et al., 2019; Garn et al., 2019; Neff et al., 2021b). Section Research question 1: Description of B-ESEM studies provides a complete description of all coded study information.
Calculating interrater reliability
To calculate interrater reliability and agreement of the literature search and the literature coding, we used an intraclass correlation coefficient (ICC) for continuous scales and Cohen's kappa coefficient (κ) for nominal scales. Table 4 presents all estimates. ICC and κ estimates tend to be robust when there are exactly two raters, as in our case, who searched and coded the studies independently and in duplicate. Consensus was reached via discussion when coding conflicts emerged. Following the recommendations of Koo and Li (2016), ICC estimates were calculated together with their 95% confidence intervals based on a mean-rating (k = 2), absolute-agreement, two-way mixed-effects model which can generally be judged as moderate (0.50–0.75), good (0.75–0.90), and excellent (>0.90) reliability. Following recommendations of Landis and Koch (1977), standard errors of Cohen's κ were calculated to compute the 95% confidence intervals around κ which can generally be judged as moderate (0.41–0.60), substantial (0.61–0.80), and almost perfect (>0.80) agreement.
Table 4. Interrater reliability and agreement of the literature search and coding.
Analyses for the literature search indicated substantial to almost perfect agreement for identification, κ = 0.990 (95% CI = 0.981; 0.999), screening, κ = 0.881 (95% CI = 0.659; 1.000), and eligibility, κ = 1.000 (95% CI = 1.000; 1.000). Analyses for the literature coding were performed separately for the continuous and nominal variables: the continuous scales were publication year, number of items, number of specific factors, number of general factors, and all goodness-of-fit indices, with an ICC = 0.923 (95% CI = 0.901; 0.932); the remaining scales were nominal, with κ = 0.976 (95% CI = 0.727; 1.000). In summary, these estimates indicate almost perfect agreement (Landis and Koch, 1977) and excellent reliability (Koo and Li, 2016).
Meta-analyzing the fit indices
We meta-analyzed the fit indices in two steps. First, we computed average estimates of the goodness-of-fit indices of each model. Mean differences between models were calculated using one-way analyses of variance. Missing values were deleted casewise. Second, we estimated the effects of sample size, item number, number of specific factors, and number of general factors on the fit indices using within-study and between-study analyses. Within-study analyses included (a) descriptive analyses of two-tailed Pearson correlation coefficients and (b) aggregated changes of model fit as a result of changes in B-ESEM model structure. Between-study analysis included (a) an outlier detection analysis using the median absolute deviation approach with the formulae reported in Miller's (1991) and Leys et al. (2013) very conservative threshold 3 and (b) an unrestricted weighted least squares meta-regression analysis (Stanley and Doucouliagos, 2017) to estimate the relative influence of sample size, item number, and factor number on model fit.
Results and discussion
Research question 1: Description of B-ESEM studies
Research question 1 asked, “Which domains, constructs, and scales are targeted in studies adopting a B-ESEM framework?” The 158 studies included in the present analysis offered a total of 308 reports of B-ESEM model fit. Total sample size was 778,624 participants, with a mean sample size of 2,528.00 (SD = 25,492.06). On average, B-ESEM models were composed of 24.62 items (SD = 17.23), 4.62 specific factors (2.96), and 1.05 general factors (0.26). B-ESEM representations were compared to representations from B-CFA in 221 reports, H-ESEM in 25 reports, H-CFA in 57 reports, ESEM in 266 reports, CFA in 265 reports, and a one-factor model in 123 reports. Concerning publication language, 151 of the 158 included studies were written in English, 4 in Spanish, and 1 each in French, German, and Hungarian. Concerning publication type, 154 of the 158 included studies were journal articles, 3 were doctoral theses, and 1 was an unpublished manuscript. Table 5 presents a description of all included studies clustered into six domains.
Table 5. Study description.
The first domain of learning and instruction includes the constructs of cooperative learning (Fernandez-Rio et al., 2022) and instructional quality (Scherer et al., 2016).
The second domain of motivation and emotion includes the constructs academic motivation (Litalien et al., 2017; Guay and Bureau, 2018; Gordeeva et al., 2020; Howard et al., 2020; Kartal, 2020; Guay et al., 2021; Dierendonck et al., 2022), sport motivation (Appleton et al., 2016; Milton et al., 2018; Stenling et al., 2018; Cece et al., 2019; Méndez-Giménez et al., 2020; Rodrigues et al., 2020), work motivation (Burk and Wiese, 2018; Calkins, 2018; Howard et al., 2018, 2020, 2021; Gegenfurtner and Quesada-Pallarès, 2022), job satisfaction (Sutherland, 2020), affective commitment (Perreira et al., 2018), work engagement (Gillet et al., 2019; Huyghebaert-Zouaghi et al., 2021a, 2022), teacher engagement (Perera et al., 2018b), student engagement (Hoi and Hang, 2021; Dierendonck et al., 2022; Tomás et al., 2022), basic psychological needs (Stenling et al., 2015; Sánchez-Oliva et al., 2017; Tóth-Király et al., 2018, 2019; Bhavsar et al., 2019, 2020; Garn et al., 2019; Burgueño et al., 2020a,b; Cromhout, 2020; Gillet et al., 2020; Gucciardi et al., 2020; Huyghebaert-Zouaghi et al., 2021b; Rodrigues et al., 2021), subjective task value (Fadda et al., 2020b; Part et al., 2020), flow (Kyriazos et al., 2018b; Gu et al., 2020), interest (Garn, 2017), locus of causality (Howard et al., 2020), attitudes (Deemer et al., 2014), purpose (Summers and Falco, 2020), self-efficacy (Barbaranelli et al., 2018; Dominguez-Lara et al., 2019), emotion regulation (Clifton et al., 2020; Hanley et al., 2020; Lauriola et al., 2021), happiness (Appiah et al., 2020), hope (Krafft et al., 2020), anxiety (Lohbeck and Petermann, 2019), fear (Jastrzebski et al., 2020), and loneliness (Grygiel et al., 2019).
The third domain of self and identity includes the constructs of self-compassion (Tóth-Király et al., 2017; Benda, 2018; Neff et al., 2018, 2019, 2021a,b), self-concept (Morin et al., 2016a; Arens et al., 2021), self-perception (Chung et al., 2016; Arens and Morin, 2017; Giotsa and Kyriazos, 2019), body checking (Maïano et al., 2021), big five personality traits (Lee et al., 2017; Arias et al., 2018), multicultural personality (Korol et al., 2018), psychopath personality (Gu et al., 2017; McLarnon and Tarraf, 2017, 2021; Somma et al., 2019; Gomez et al., 2020b; Vaughan et al., 2020), conspiracy beliefs (García-Garzón et al., 2020), paranoid thoughts (Bianchi and Verkuilen, 2021), character strength (Ng et al., 2017; Wang et al., 2021), consideration of future consequences (McKay et al., 2016), morningness (Morin et al., 2016b; Díaz-Morales and Parra-Robledo, 2018), compassion (Halamová et al., 2020; Pommier et al., 2020), emotional intelligence (Esnaola et al., 2018; Pirsoul et al., 2022), intelligence (Lecerf and Canivez, 2018), life skills (Cronin and Allen, 2017; Jaotombo, 2019; Choisay et al., 2021), mental toughness (Bédard-Thom and Guay, 2018; Schmid et al., 2018; Kawabata et al., 2021), and resilience (Decroos et al., 2017; Perera and Ganguly, 2018; Dai et al., 2019).
The fourth dimension of depression and wellbeing includes the constructs of depression (Borges et al., 2017; Volmer et al., 2019; Bianchi, 2020; Bianchi and Schonfeld, 2020; Cano-García et al., 2020; Gomez et al., 2020a; Nixon et al., 2020; Vaughan et al., 2020; Høstmælingen et al., 2021; Jovanovic et al., 2021), burnout (Isoard-Gautheur et al., 2018; Bianchi, 2020; Sakakibara et al., 2020; Trógolo et al., 2020; Doherty et al., 2021; Tóth-Király et al., 2021), positive thoughts (Appiah et al., 2020), wellbeing (Myers et al., 2016; Fadda et al., 2017, 2020a; Morin et al., 2017; Schutte and Wissing, 2017; Espinoza et al., 2018; Kyriazos et al., 2018a; Lamborn et al., 2018; Perera et al., 2018a; Rogoza et al., 2018; Silverman et al., 2018; Ferentinos et al., 2019; Cromhout, 2020; Longo et al., 2020; Reinhardt et al., 2020a,b), workaholism (Huyghebaert-Zouaghi et al., 2022), stress (Morin et al., 2016c; Portoghese et al., 2020), stress disorder (Fresno et al., 2020), anxiety disorder (Deller et al., 2020; Styck et al., 2021), problem behavior (Lahey et al., 2018; Hukkelberg, 2019; Hukkelberg and Ogden, 2020; Gomez et al., 2021), and attention deficit hyperactivity disorder (Arias et al., 2016; Rodenacker et al., 2017; Frutos, 2019, 2021; Gomez and Stavropoulos, 2021; Yi et al., 2021).
The fifth dimension of interpersonal relations includes the constructs of romantic relationships (Vajda et al., 2019; Goodboy et al., 2021), parenting (Ratelle et al., 2018), group cohesion (Haugen et al., 2021), working alliance (Hukkelberg and Ogden, 2019), work teams (Leo et al., 2017), supervision (Junga et al., 2019), and perceived social support (Perera, 2016).
The sixth and last category includes two constructs that did not fit the aforementioned five domains: motoric skills (Garn and Webster, 2021) and scar evaluation (Sen et al., 2015).
The 158 included studies psychometrically tested a total of 135 different scales. Among the most often-investigated constructs are wellbeing (n = 17), basic psychological needs (n = 14 studies), depression (n = 10), academic motivation (n = 7), psychopathic personality (n = 7), sport motivation (n = 7), attention deficit hyperactivity disorder (n = 6), burnout (n = 6), self-compassion (n = 6), and work motivation (n = 6). Scales whose factorial structures were frequently examined using a B-ESEM approach include the Mental Health Continuum—Short Form (MHC-SF), addressed in eight studies, the Academic Motivation Scale (AMS), addressed in six studies, and the Self-Compassion Scale (SCS), addressed in six studies.
Research question 2: Model fit
Research question 2 asked, “What is the model fit of B-ESEM representations in comparison to CFA, ESEM, H-CFA, H-ESEM, and B-CFA representations?” To answer this research question, a primary meta-analysis was performed to cumulate the reported goodness-of-fit-indices across studies and then to generate average estimates of model fit for each representation. Table 6 reports the results of this meta-analysis. Analyses of variance suggest that model fit differed significantly between representations. The findings of comparing the model representations, as shown in Figure 1, indicate that the B-ESEM representation showed the best model fit, with the lowest χ2/df ratio, the highest CFI, the lowest RMSEA, and the lowest SRMR. TLI was minimally higher for H-ESEM (0.950) than for B-ESEM (0.948) [ΔTLI = 0.002, t(23) = 1.321, p = 0.200]. These results demonstrate the superior model fit of B-ESEM solutions over other factorial representations across a range of scales and domains. Figure 3 portrays the strength of the B-ESEM goodness-of-fit indices relative to the other model representations.
Table 6. Mean (and standard deviation) estimates of the goodness-of-fit indices per model.
Figure 3. Goodness-of-fit indices per model representation.
Table 7 presents a correlation matrix of the goodness-of-fit indices of all model representations. Across models, correlations were highly positive between CFI and TLI. Correlations also tended to be substantial between the same fit indices of different models, for example between ESEM RMSEA, H-ESEM RMSEA, and B-ESEM RMSEA or between one-factor χ2/df, CFA χ2/df, and ESEM χ2/df . Correlation coefficients between the incremental (CFI, TLI) and absolute (RMSEA, SRMR) fit indices tended to be negative.
Table 7. Correlation matrix.
Research question 3: Influence of sample size, item number, and factor number on model fit
Research question 3 asked, “How sensitive is model fit to sample size, item number, and the number of specific and general factors in a model?” To answer this research question, we performed within-study and between-study analyses.
First, within-study analyses aggregated the change in model fit as a result of changes in B-ESEM model structure. A number of studies reported model fit for different model specifications. For example, Yi et al. (2021) examined the fit of a B-ESEM model with two or three specific factors; their findings demonstrate better model fit for a three-factor representation. A variety of different representations were explored among the included studies, such as one and three general factors (Bhavsar et al., 2019), as well as three and nine specific factors (Bhavsar et al., 2019), four and five specific factors (Choisay et al., 2021), four and six specific factors (Part et al., 2020), seven and eight specific factors (Bédard-Thom and Guay, 2018), and four and twenty specific factors (Sutherland, 2020). Table 8 presents within-study analyses of changes in B-ESEM model fit by factor number. Average estimates across reports suggest that, when the number of specific factors changed from two to three, two to six, three to four, or three to six, CFI and TLI increased, while the χ2/df ratio, RMSEA, and SRMR decreased. Similarly, when the number of general factors changed from one to two, CFI and TLI increased, while the χ2/df ratio, RMSEA, and SRMR decreased.
Table 8. Within-Study analysis of changes in B-ESEM model fit by factor number.
Second, between-study analyses were performed to estimate the extent to which model fit is influenced by sample size, item number, and the number of specific and general factors. Between-study analyses included correlation analyses and meta-regression analysis. The correlation analysis shown in Table 7 suggests high correlation coefficients between sample size and the χ2/df ratio. Prior to the meta-regression, an outlier detection analysis was conducted to identify studies with outlying values. Analyses were performed separately for sample size, item number, number of specific factors, and number of general factors. Table 9 presents the outcomes of the outlier analysis. For sample size, 40 reports with more than 1,763 participants were identified. For item number, 12 reports examined scales with 44 items or more. For the number of specific factors, 19 reports used B-ESEM representations with nine specific factors or more. For the number of general factors, 13 reports used B-ESEM representations with more than one general factor. These reports were removed prior to the meta-regression analysis.
Table 9. Outlier detection analysis.
Table 10 presents the results of the meta-regression. First, the findings indicate that sample size significantly influenced the χ2/df ratio in all models except H-ESEM. This finding supports simulation study results on the sample-size sensitivity of the χ2/df ratio. Second, item number influenced (a) CFI and TLI in all models except ESEM, (b) RMSEA in CFA, H-CFA, and H-ESEM, and (c) SRMR in all models except ESEM and H-ESEM. Third, the number of specific factors influenced (a) CFI and TLI in all models except CFA, H-ESEM, and B-ESEM, (b) RMSEA in the CFA, ESEM, H-ESEM, B-CFA, and B-ESEM representations, and (c) SRMR in the ESEM, H-CFA, B-CFA, and B-ESEM representations.
Table 10. Between-Study analysis of the influence of sample size, item number, and specific factor number on model fit.
As a cautionary note, goodness-of-fit is a critical component in evaluating support for a model, but it is not the only one. Particularly when fitting a more parsimonious a priori model with a less parsimonious ex-post-facto model, there is danger in simply comparing fit. Similarly, set-ESEM (Marsh et al., 2020) allows for greater parsimony by keeping theoretically independent dimensions from having cross-loadings. There needs to be some focus on why the bi-factor model is theoretically and substantively more relevant as well as providing a better fit. A saturated model, for example, will necessarily fit better than any of the models, but it would provide an appropriate test of the underlying theoretical model or a substantively relevant model. If the theoretical model is a CFA or ESEM factor model, then strong support for a Bi-ESEM reflects a failure of the a priori prediction. From this perspective, it is important to classify results in relation to the a priori model. Similarly, when comparing nested and non-nested models, the less parsimonious model necessarily fits better for models that do not take into account parsimony. Hence, the size of the difference is relevant, but not necessarily the direction. More interesting are comparisons between non-nested models. These cautionary notes can be useful to avoid over-interpreting goodness-of-fit: even if a Bi-ESEM shows superior fit, it is not necessarily the preferred model which is contingent on theoretical considerations, particularly when bifactor modeling was applied to a single-level rather than a two-level sampling approach (Eid et al., 2017).
Conclusion
This systematic review and meta-analysis aimed to collect and describe studies using a B-ESEM framework in multivariate behavioral research, aggregate their reported model fit, and analyze model fit differences in comparison to reference models and as a function of sample size, item number, and factor number.
This meta-analysis is the first to replicate findings from simulation studies with real data on the superiority of B-ESEM models and examine the relative influence of sample size, item number, and factor number on model fit (Hu and Bentler, 1999; Marsh et al., 2004; Shi et al., 2019). While meta-analyses of structural equation models have been performed (MASEM; e.g., Cheung and Chan, 2005; Reinhold et al., 2018), this meta-analysis is also among the first to synthesize B-ESEM model fit and compare fit between models meta-analytically. This meta-analysis also documented how widespread B-ESEM has been used within the past 6 years since the seminal paper of Morin et al. (2016a) has been published: B-ESEM is now used in multiple domains to explore the multidimensional structure of numerous constructs in educational and psychological research.
Because the review included original empirical reports only—and excluded simulation studies—our examination of the influence of item number and factor number was contingent on and limited to what had been published in original empirical research. Another limitation concerned the domains within which B-ESEM has been applied. Although research in the domains of learning and instruction, motivation and emotion, self and identity, depression and wellbeing, and interpersonal relations covers large areas of educational and psychological research, the findings of this meta-analytic review are limited to these fields and cannot easily be generalized to other domains within which B-ESEM may be applied in the future. Still, we are confident that the present research synthesis—focusing on five fit indices and six reference models, cumulating 158 studies with 308 reports of B-ESEM model fit from a total sample size of 778,624 participants, and examining the relative influence of sample size, item number, and factor number within and between studies—can inform further applications of B-ESEM to identify construct-relevant psychometric multidimensionality. Future research is encouraged to examine how particular constructs and scales in educational and psychological research can be represented with B-ESEM (Morin et al., 2016a).
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Author contributions
The author confirms being the sole contributor of this work and has approved it for publication.
Conflict of interest
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Footnotes
1. ^For reasons of completeness, it is worth mentioning that researchers can also use Bayesian SEM (BSEM; Muthén and Asparouhov, 2012) and set-ESEM (Marsh et al., 2020).
2. ^Despite the widespread use of goodness-of-fit indices to evaluate model fit, many scholars warn against the use of fixed cutoff values (Marsh et al., 2004). McNeish and Wolf (2022) propose dynamic fit index cutoffs as an alternative.
*^Studies preceded by an asterisk were included in the meta-analysis.
References
*Appiah, R., Schutte, L., Fadiji, A. W., Wissing, M. P., and Cromhout, A. (2020). Factorial validity of the Twi versions of five measures of mental health and well-being in Ghana. PLoS ONE 15, e0236707. doi: 10.1371/journal.pone.0236707
*Appleton, P. R., Ntoumanis, N., Quested, E., Viladrich, C., and Duda, J. L. (2016). Initial validation of the coach-created empowering and disempowering motivational climate questionnaire (EDMCQ-C). Psychol. Sport Exerc. 22, 53–65. doi: 10.1016/j.psychsport.2015.05.008
*Arens, A. K., Jansen, M., Preckel, F., Schmidt, I., and Brunner, M. (2021). The structure of academic self-concept: a methodological review and empirical illustration of central models. Rev. Educ. Res. 91, 34–72. doi: 10.3102/0034654320972186
*Arens, A. K., and Morin, A. J. S. (2017). Improved representation of the self-perception profile for children through bifactor exploratory structural equation modeling. Am. Educ. Res. J. 54, 59–87. doi: 10.3102/0002831216666490
*Arias, V. B., Jenaro, C., and Ponce, F. P. (2018). Testing the generality of the general factor of personality: an exploratory bifactor approach. Pers. Individ. Dif. 129, 17–23. doi: 10.1016/j.paid.2018.02.042
*Arias, V. B., Ponce, F. P., Martinez-Molina, A., Arias, B., and Nunez, D. (2016). General and specific attention-deficit/hyperactivity disorder factors of children 4 to 6 years of age: an exploratory structural equation modeling approach to assessing symptom multidimensionality. J. Abnorm. Psychol. 125, 125–137. doi: 10.1037/abn0000115
Asparouhov, T., and Muthén, B. (2009). Exploratory structural equation modeling. Struct. Equ. Model. 16, 397–438. doi: 10.1080/10705510903008204
Bandalos, D. L., and Finney, S. J. (2018). “Exploratory and confirmatory factor analysis,” in Quantitative Methods in the Social and Behavioral Sciences: A Guide for Researchers and Reviewers, 2nd Edn, eds G. R. Hancock, and R. O. Mueller (New York, NY: Routledge), 93–114.
*Barbaranelli, C., Fida, R., Paciello, M., and Tramontano, C. (2018). ‘Possunt, quia posse videntur': THEY can because they think they can. Development and validation of the work self-efficacy scale: evidence from two studies. J. Voc. Behav. 106, 249–269. doi: 10.1016/j.jvb.2018.01.006
*Bédard-Thom, C., and Guay, F. (2018). Mental toughness among high school students: a test of its multidimensionality and nomological validity with academic achievement and preference for difficult tasks. Soc. Psychol. Educ. 21, 827–848. doi: 10.1007/s11218-018-9437-y
*and Benda, J. (2018). Alternative Models of the Czech Version of the Self-Compassion Scale (SCS-26-CZ) (Prague: University of Prague). Unpublished manuscript.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychol. Bull. 107, 238–246. doi: 10.1037/0033-2909.107.2.238
Bentler, P. M. (1995). EQS Structural Equations Program Manual. Encino, CA: Multivariate Software Inc.
Bentler, P. M., and Bonett, D. G. (1990). Significance tests and goodness of fit in the analysis of covariance structures. Psychol. Bull. 88, 588–606. doi: 10.1037/0033-2909.88.3.588
*Bhavsar, N., Bartholomew, K. J., Quested, E., Gucciardi, D. F., Thøgersen-Ntoumani, C., Reeve, J., et al. (2020). Measuring psychological need states in sport: theoretical considerations and a new measure. Psychol. Sport Exerc. 47, 101617. doi: 10.1016/j.psychsport.2019.101617
*Bhavsar, N., Ntoumanis, N., Quested, E., Gucciardi, D. F., Thøgersen-Ntoumani, C., Ryan, R. M., et al. (2019). Conceptualizing and testing a new tripartite measure of coach interpersonal behaviors. Psychol. Sport Exerc. 44, 107–120. doi: 10.1016/j.psychsport.2019.05.006
*and Bianchi, R. (2020). Do burnout and depressive symptoms form a single syndrome? Confirmatory factor analysis and exploratory structural equation modeling bifactor analysis. J. Psychosom. Res. 131, 109954. doi: 10.1016/j.jpsychores.2020.109954
*Bianchi, R., and Schonfeld, I. S. (2020). The occupational depression inventory: a new tool for clinicians and epidemiologists. J. Psychosom. Res. 138, 110249. doi: 10.1016/j.jpsychores.2020.110249
*Bianchi, R., and Verkuilen, J. (2021). “Green et al. paranoid thoughts scale”: French validation and development of a brief version. Pers. Indiv. Diff. 171, 110554. doi: 10.1016/j.paid.2020.110554
*Borges, L., Nunes Baptista, M., and de Oliveira Serpa, A. L. (2017). Structural analysis of depression indicators scale-children and adolescents (BAID-IJ): a bifactor-ESEM approach. Trends Psychol. 25, 545–552. doi: 10.9788/TP2017.2-08
*Burgueño, R., González-Cutre, D., Sevil-Serrano, J., Herrador-Colmenero, M., SeguraDíaz, J. M., Medina-Casaubón, J., et al. (2020a). Validation of the basic psychological need satisfaction in active commuting to and from school (BPNS-ACS) scale in Spanish young people. J. Transp. Health 16, 100825. doi: 10.1016/j.jth.2020.100825
*Burgueño, R., Macarro-Moreno, J., and Medina-Casaubón, J. (2020b). Psychometry of the multidimensional perceived autonomy support scale in physical education with Spanish secondary school students. SAGE Open 10, 1–12. doi: 10.1177/2158244019901253
*Burk, C. L., and Wiese, B. S. (2018). Professor or manager? A model of motivational orientations applied to preferred career paths. J. Res. Pers. 75, 113–132. doi: 10.1016/j.jrp.2018.06.002
*and Calkins, C. M. (2018). Developing and measuring faculty motivation to teach in higher education. (Doctoral thesis), University of Nevada, Reno (Nevada).
*Cano-García, F. J., Muñoz-Navarro, R., Abad, A. S., Moretti, L. S., Medrano, L. A., Ruiz-Rodríguez, P., et al. (2020). Latent structure and factor invariance of somatic symptoms in the patient health questionnaire (PHQ−15). J. Affect. Disord. 261, 21–29. doi: 10.1016/j.jad.2019.09.077
*Cece, V., Lienhart, N., Nicaise, V., Guillet-Descas, E., and Martinent, G. (2019). Longitudinal sport motivation among young athletes in intensive training settings: using methodological advances to explore temporal structure of youth behavioral regulation in sport questionnaire scores. J. Sport Exerc. Psychol. 41, 24–35. doi: 10.1123/jsep.2017-0194
Cheung, M. W.-L., and Chan, W. (2005). Meta-analytic structural equation modeling: a two-stage approach. Psychol. Methods 10, 40–64. doi: 10.1037/1082-989X.10.1.40
*Choisay, F., Fouquereau, E., Coillot, H., and Chevalier, S. (2021). Validation of the french psychological capital questionnaire (F–PCQ−24) and its measurement invariance using bifactor exploratory structural equation modeling framework. Mil. Psychol. 33, 50–65. doi: 10.1080/08995605.2020.1852873
*Chung, C. M., Liao, X. L., Song, H. R., and Lee, T. H. (2016). Bifactor approach to modeling multidimensionality of physical self-perception profile. Meas. Phys. Educ. Exerc. Sci. 20, 1–15. doi: 10.1080/1091367X.2015.1081594
*Clifton, J., Seehuus, M., Parent, J., Pichler, E., and Fondacaro, K. (2020). Emotional responding: integration of multiple constructs and association with psychological health. J. Clin. Psychol. 76, 699–715. doi: 10.1002/jclp.22885
*and Cromhout, A. (2020). Measuring and understanding eudaimonic well-being: A bifactor exploratory structural equation modelling approach. (Doctoral thesis), North-West University, Boloka. Available online at: http://hdl.handle.net/10394/36666 (accessed June 29, 2021).
*Cronin, L. D., and Allen, J. (2017). Development and initial validation of the life skills scale for sport. Psychol. Sport Exerc. 28, 105–119. doi: 10.1016/j.psychsport.2016.11.001
Dai, S. S., Xu, H. G., and Chen, F. F. (2019). A hierarchical measurement model of perceived resilience of urban tourism destination. Soc. Indic. Res. 145, 777–804. doi: 10.1007/s11205-019-02117-9
*Decroos, S., Lines, R., Morgan, P. B. C., Fletcher, D., Sarkar, M., Fransen, K., et al. (2017). Development and validation of the characteristics of resilience in sports teams inventory. Sport Exerc. Perform. Psychol. 6, 158–178. doi: 10.1037/spy0000089
*Deemer, E. D., Smith, J. L., Thoman, D. B., and Chase, J. P. (2014). Precision in career motivation assessment: testing the subjective science attitude change measures. J. Car. Assess. 22, 489–504. doi: 10.1177/1069072713498683
*Deller, J., Perrotte, J., Wainwright, K., Brunsman, J., and Osman, A. (2020). Dimensionality, reliability, invariance, and validity of the multidimensional social anxiety response inventory−21 (MSARI−21). J. Pers. Assess. 102, 527–537. doi: 10.1080/00223891.2019.1569529
*Díaz-Morales, J. F., and Parra-Robledo, Z. (2018). Age and sex differences in morningness/eveningness along the life span: a cross-sectional study in Spain. J. Genet. Psychol. 179, 71–84. doi: 10.1080/00221325.2018.1424706
*Dierendonck, C., Tóth-Király, I., Morin, A. J. S., Kerger, S., Milmeister, P, and Poncelet, D. (2022). Testing associations between global and specific levels of student academic motivation and engagement in the classroom. J. Exp. Educ. doi: 10.1080/00220973.2021.1913979
*Doherty, A. S., Mallett, J., Leiter, M. P., and McFadden, P. (2021). Measuring burnout in social work factorial validity of the Maslach burnout inventory – human Services Survey. Euro. J. Psychol. Assess. 37, 6–14. doi: 10.1027/1015-5759/a000568
*Dominguez-Lara, S., Fernández-Arata, M., Cesar, M. S., Navarro-Loli, J. S., and Calderón-De la Cruz, G. (2019). Escala de autoeficacia docente: análisis estructural e invarianza de medición en docentes peruanos de escuelas públicas [Teacher's self-efficacy scale: Structural analysis and measurement invariance in Peruvian teachers of public schools]. Rev. Argent. Cienc. Comport. 11, 61–72. doi: 10.32348/1852.4206.v11.n3.24624
Eid, M., Geiser, C., Koch, T., and Heene, M. (2017). Anomalous results in G-factor models: explanations and alternatives. Psychol. Methods 22, 541–562. doi: 10.1037/met0000083
*Esnaola, I., Arias, V. B., Freeman, J., Wang, Y., and Arias, B. (2018). Validity evidence based on internal structure of scores of the emotional quotient inventory: youth version short (EQ-i: YV-S) in a Chinese sample. J. Psychoeduc. Assess. 36, 576–587. doi: 10.1177/0734282916689439
*Espinoza, J. A., Meyer, J. P., Anderson, B. K., Vaters, C., and Politis, C. (2018). Evidence for a bifactor structure of the scales of psychological well-being using exploratory structural equation modeling. J. Well Being Assess. 2, 21–40. doi: 10.1007/s41543-018-0008-y
*Fadda, D., Quevedo-Aguado, M. P., Cuesta, M. H. B., and Scalas, L. F. (2020a). The multidimensional and hierarchical nature of the questionnaire for eudaimonic wellbeing: a bifactor–ESEM representation in a Spanish Sample. Front. Psychol. 11, 422. doi: 10.3389/fpsyg.2020.00422
*Fadda, D., Scalas, L. F., Meleddu, M., and Morin, A. J. S. (2017). A bifactor–ESEM representation of the questionnaire for eudaimonic wellbeing. Pers. Individ. Dif. 116, 216–222. doi: 10.1016/j.paid.2017.04.062
*Fadda, D., Scalas, L. F., Morin, A. J. S., Marsh, H. W., and Gaspard, H. (2020b). Value beliefs about math: a bifactor–ESEM representation. Euro. J. Psychol. Assess. 36, 259–268. doi: 10.1027/1015-5759/a000513
*Ferentinos, P., Yotsidi, V., Porichi, E., Douzenis, A., Papageorgiou, C., and Stalikas, A. (2019). Well-being in patients with affective disorders compared to nonclinical participants: a multi-model evaluation of the mental health continuum–short form. J. Clin. Psychol. 75, 1585–1612. doi: 10.1002/jclp.22780
*Fernandez-Rio, J., Cecchini, J. A., Morgan, K., Méndez-Giménez, A., and Lloyd, R. (2022). Validation of the cooperative learning scale and cooperation global factor using bifactor structural equation modelling. Psicol. Educ. 28, 91–97. doi: 10.5093/psed2021a2
*Fresno, A., Arias, V., Nunez, D., Spencer, R., Ramos, N., Espinoza, C., et al. (2020). Using exploratory structural equation modeling (ESEM) to examine the internal structure of posttraumatic stress disorder symptoms. Span. J. Psychol. 23, e48. doi: 10.1017/SJP.2020.46
*and Frutos, J. (2019). Evaluación multidimensional de los factores generales y específicos del TDAH en población infantial mediante el enfoque bifactor-ESEM [multidimensional evaluation of the general and specific facors of ADHD in children population with the bifactor-ESEM approach]. Rev. Argent. Clín. Psicol. 28, 967–980. doi: 10.51668/bp.8321105s
*and Frutos, J. (2021). Análisis multidimensional del trastorno por déficit de atención e hiperactividad (TDAH) mediante el método bifactor-ESEM [multidimensional analysis of attention deficit hyperactivity disorder (ADHD) using the bifactor–ESEM method]. Behav. Psychol. 29, 95–110.
*García-Garzón, E., Nieto, M. D., Garrido, L. E., and Abad, F. J. (2020). Bi-factor exploratory structural equation modeling done right: Using the SLiDapp application. Psicothema 32, 607–614. doi: 10.7334/psicothema2020.179
*and Garn, A. C. (2017). Multidimensional measurement of situational interest in physical education: application of bifactor exploratory structural equation modeling. J. Teach. Phys. Educ. 36, 323–339. doi: 10.1123/jtpe.2017-0035
*Garn, A. C., Morin, A. J. S., and Lonsdale, C. (2019). Basic psychological need satisfaction toward learning: a longitudinal test of mediation using bifactor exploratory structural equation modeling. J. Educ. Psychol. 111, 354–372. doi: 10.1037/edu0000283
*Garn, A. C., and Webster, E. K. (2021). Bifactor structure and model reliability of the test of gross motor development−3rd edition. J. Sci. Med. Sport 24, 67–73. doi: 10.1016/j.jsams.2020.08.009
*Gegenfurtner, A., and Quesada-Pallarès, C. (2022). Toward a multidimensional conceptualization of motivation to transfer training: validation of the transfer motivation questionnaire from a self-determination theory perspective using bifactor-ESEM. Stud. Educ. Eval. 73, 101116. doi: 10.1016/j.stueduc.2021.101116
*Gillet, N., Caesens, G., Morin, A. J. S., and Stinglhamber, F. (2019). Complementary variable– and person-centred approaches to the dimensionality of work engagement: a longitudinal investigation. Euro. J. Work Organ. Psychol. 28, 239–258. doi: 10.1080/1359432X.2019.1575364
*Gillet, N., Morin, A. J. S., Huart, I., Colombat, P., and Fouquereau, E. (2020). The forest and the trees: investigating the globality and specificity of employees' basic need satisfaction at work. J. Pers. Assess. 102, 702–713. doi: 10.1080/00223891.2019.1591426
*Giotsa, A., and Kyriazos, T. A. (2019). Early childhood acceptance rejection questionnaire: psychometric properties of the Greek version. Psychology 10, 722–741. doi: 10.4236/psych.2019.105047
*Gomez, R., and Stavropoulos, V. (2021). Confirmatory factor analysis and exploratory structural equation modeling of the structure of attention-deficit/hyperactivity disorder symptoms in adults. Assessment 28, 1570–1582. doi: 10.1177/1073191120905892
*Gomez, R., Stavropoulos, V., and Griffiths, M. D. (2020a). Confirmatory factor analysis and exploratory structural equation modelling of the factor structure of the depression anxiety and stress scales−21. PLoS ONE 15, e0233998. doi: 10.1371/journal.pone.0233998
*Gomez, R., Stavropoulos, V., Zarate, D., and Palikara, O. (2021). Symptom checklist-90–revised: a structural examination in relation to family functioning. PLoS ONE 16, e0247902. doi: 10.1371/journal.pone.0247902
*Gomez, R., Watson, S., and Stavropoulos, V. (2020b). Personality inventory for DSM−5, brief form: factor structure, reliability, and coefficient of congruence. Pers. Disord. Theory Res. Treat. 11, 69–77. doi: 10.1037/per0000364
*Goodboy, A. K., Bolkan, S., Brisini, K., and Solomon, D. H. (2021). Relational uncertainty within relational turbulence theory: the bifactor exploratory structural equation model. J. Commun. 71, 403–430.. doi: 10.1093/joc/jqab009
*Gordeeva, T. O., Sychev, O. A., and Lynch, M. F. (2020). The construct validity of the Russian version of the modified academic self-regulation questionnaire (SRQ–A) among elementary and middle school children. Psychol. Russia 13, 16–34. doi: 10.11621/pir.2020.0308
*Grygiel, P., Humenny, G., and Rebisz, S. (2019). Using the De Jong Gierveld loneliness scale with early adolescents: factor structure, reliability, stability, and external validity. Assessment 26, 151–165. doi: 10.1177/1073191116682298
*Gu, H. L., Wen, Z. L., and Fan, X. T. (2017). Structural validity of the Machiavellian personality scale: a bifactor exploratory structural equation modeling approach. Pers. Individ. Dif. 105, 116–123. doi: 10.1016/j.paid.2016.09.042
*Gu, H. L., Wen, Z. L., and Fan, X. T. (2020). Investigating the multidimensionality of the work-related flow inventory (WOLF): a bifactor exploratory structural equation modeling framework. Front. Psychol. 11, 740. doi: 10.3389/fpsyg.2020.00740
*Guay, F., and Bureau, J. S. (2018). Motivation at school: differentiation between and within school subjects matters in the prediction of academic achievement. Contemp. Educ. Psychol. 54, 42–54. doi: 10.1016/j.cedpsych.2018.05.004
*Guay, F., Morin, A. J. S., Litalien, D., Howard, J. L., and Gilbert, W. (2021). Trajectories of self-determined motivation during the secondary school: a growth mixture analysis. J. Educ. Psychol. 113, 390–410. doi: 10.1037/edu0000482
*Gucciardi, D. F., Weixian, J. C., Gibson, W., Ntoumanis, N., and Ng, L. (2020). Motivational climate in the classroom factorial and convergent validity: evidence of the need-supportive behaviors scale with health science students. Euro. J. Psychol. Assess. 36, 324–335. doi: 10.1027/1015-5759/a000524
*Halamová, J., Kanovský, M., and Pacúchová, M. (2020). Psychometric analysis of the Slovak version of the compassionate engagement and action scales. J. Psychol. Educ. Res. 28, 64–80.
*Hanley, A. W., Bernstein, A., Nakamura, Y., Hadash, Y., Rojas, J., Tennant, K. E., et al. (2020). The metacognitive processes of decentering scale: development and initial validation of trait and state versions. Psychol. Assess. 32, 956–971. doi: 10.1037/pas0000931
*Haugen, T., Peters, D. M., Ommundsen, Y., Martin, L. J., Stenling, A., and Høigaard, R. (2021). Psychometric evaluation of the Norwegian versions of the modified group environment questionnaire and the youth sport environment questionnaire. Meas. Phys. Educ. Exerc. Sci. 25, 365–378. doi: 10.1080/1091367X.2021.1917414
*Hoi, V. N., and Hang, H. L. (2021). The structure of student engagement in online learning: a bi-factor exploratory structural equation modelling approach. J. Comput. Assist. Learn. 37,1141–1153. doi: 10.1111/jcal.12551
*Høstmælingen, A., Ulvenes, P., Nissen-Lie, H. A., Eielsen, M., and Wampold, B. E. (2021). Do self-criticism and somatic symptoms play a key role in chronic depression? Exploring the factor structure of beck depression inventory–II in a sample of chronically depressed inpatients. J. Affect. Disord. 283, 317–324. doi: 10.1016/j.jad.2021.01.066
*Howard, J. L., Gagn,é, M., Van den Broeck, A., Guay, F., Chatzisarantis, N., Ntoumanis, N., et al. (2020). A review and empirical comparison of motivation scoring methods: an application to self-determination theory. Motiv. Emot. 44, 534–548. doi: 10.1007/s11031-020-09831-9
*Howard, J. L., Gagné, M., Morin, A. J. S., and Forest, J. (2018). Using bifactor exploratory structural equation modeling to test for a continuum structure of motivation. J. Manage. 44, 2638–2664. doi: 10.1177/0149206316645653
*Howard, J. L., Morin, A. J. S., and Gagné, M. (2021). A longitudinal analysis of motivation profiles at work. Motiv. Emot. 45, 39–59. doi: 10.1007/s11031-020-09852-4
Hu, L. T., and Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Model. 6, 1–55. doi: 10.1080/10705519909540118
*Hukkelberg, S., and Ogden, T. (2019). General and specific factors of working alliance in parent training: a bifactor exploratory structural equation modelling approach. Psychother. Res. 29, 267–276. doi: 10.1080/10503307.2017.1330574
*Hukkelberg, S., and Ogden, T. (2020). What is social competence? An investigation into the concept among children with antisocial behaviours. Emot. Behav. Diff. 25, 80–93. doi: 10.1080/13632752.2019.1687168
*and Hukkelberg, S. S. (2019). A reexamination of child problem behaviors as measured by ECBI: factor structure and measurement invariance across two parent training interventions. Assessment 26, 1270–1281. doi: 10.1177/1073191117706022
*Huyghebaert-Zouaghi, T., Berjot, S., Cougot, B., and Gillet, N. (2021a). Psychological and relational conditions for job crafting to occur. Stress Health 37, 516–527. doi: 10.1002/smi.3014
*Huyghebaert-Zouaghi, T., Caesens, G., Sandrin, É., and Gillet, N. (2022). Workaholism and work engagement: an examination of their psychometric multidimensionality and relations with employees' functioning. Curr. Psychol. doi: 10.1007/s12144-021-01820-6
*Huyghebaert-Zouaghi, T., Ntoumanis, N., Berjot, S., and Gillet, N. (2021b). Advancing the conceptualization and measurement of psychological need states: a 3 x 3 model based on self-determination theory. J. Car. Assess. 29, 396–421. doi: 10.1177/1069072720978792
*Isoard-Gautheur, S., Martinent, G., Guillet-Descas, E., Trouilloud, D., Cece, V., and Mette, A. (2018). Development and evaluation of the psychometric properties of a new measure of athlete burnout: the athlete burnout scale. Int. J. Stress Manag. 25, 108–123. doi: 10.1037/str0000083
*and Jaotombo, F. F. (2019). Le fonctionnement optimal psychologique: apports conceptuels et méthodologiques [Optimal psychological functioning: conceptual and methodological contributions]. Psychol. Trav. Organ. 25, 281–300. doi: 10.1016/j.pto.2019.06.001
*Jastrzebski, J., Rogoza, R., and Slaski, S. (2020). The hierarchical structure of fear of personal death: from the general factor to specific forms. Psicol. Reflexao Crit. 33, 16. doi: 10.1186/s41155-020-00152-x
Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika 34, 183–202. doi: 10.1007/BF02289343
*Jovanovic, V., Gavrilov-Jerkovic, V., and Lazic, M. (2021). Can adolescents differentiate between depression, anxiety and stress? Testing competing models of the depression anxiety stress scales (DASS−21). Curr. Psychol. 40, 6045–6056. doi: 10.1007/s12144-019-00540-2
*Junga, Y. M., Witthoft, M., and Weck, F. (2019). Assessing therapist development: reliability and validity of the supervisee levels questionnaire (SLQ–R). J. Clin. Psychol. 75, 1658–1672. doi: 10.1002/jclp.22794
*and Kartal, S. K. (2020). Examining factors for the academic motivation based on the confirmatory, the exploratory, and the bifactor exploratory structural equation modeling. Int. J. Prog. Educ. 16, 192–204. doi: 10.29329/ijpe.2020.228.14
*Kawabata, M., Pavey, T. G., and Coulter, T. J. (2021). Evolving the validity of a mental toughness measure: refined versions of the mental toughness questionnaire-48. Stress Health 37, 378–391. doi: 10.1002/smi.3004
Koo, T. K., and Li, M. Y. (2016). A guideline for selecting and reporting intraclass correlation coefficients for reliability research. J. Chiropr. Med. 15, 155–163. doi: 10.1016/j.jcm.2016.02.012
*Korol, L., Fietzer, A. W., and Ponterotto, J. G. (2018). The relationship between multicultural personality, intergroup contact, and positive outgroup attitudes toward Asian Americans. Asian Am. J. Psychol. 9, 200–210. doi: 10.1037/aap0000107
*Krafft, A., Guse, T., and Maree, D. (2020). Distinguishing perceived hope and dispositional optimism: theoretical foundations and empirical findings beyond future expectancies and cognition. J. Well Being Assess. 4, 217–243. doi: 10.1007/s41543-020-00030-4
*Kyriazos, T. A., Stalikas, A., Prassa, K., Galanakis, M., Flora, K., and Chatzilia, V. (2018b). The flow short scale (FSS) dimensionality and what MIMIC shows on heterogeneity and invariance. Psychology 9, 1357–1382. doi: 10.4236/psych.2018.96083
*Kyriazos, T. A., Stalikas, A., Prassa, K., and Yotsidi, V. (2018a). A 3-faced construct validation and a bifactor subjective well-being model using the scale of positive and negative experience, Greek version. Psychology 9, 1143–1175. doi: 10.4236/psych.2018.95071
*Lahey, B. B., Zald, D. H., Perkins, S. F., Villalta-Gil, V., Werts, K. B., Van Hulle, C. A., et al. (2018). Measuring the hierarchical general factor model of psychopathology in young adults. Int. J. Methods Psychiatr. Res. 27, e1593. doi: 10.1002/mpr.1593
*Lamborn, P., Cramer, K. M., and Riberdy, A. (2018). The structural validity and measurement invariance of the mental health continuum–short form (MHC–SF) in a large Canadian same. J. Well Being Assess 2, 1–19. doi: 10.1007/s41543-018-0007-z
Landis, J. R., and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics 33, 159–174. doi: 10.2307/2529310
*Lauriola, M., Donati, M. A., Trentini, C., Tomai, M., Pontone, S., and Baker, R. (2021). The structure of the Emotional processing scale (EPS-25): an exploratory structural equation modeling analysis using medical and community samples. Euro. J. Psychol. Assess. 37, 423–432. doi: 10.1027/1015-5759/a000632
*Lecerf, T., and Canivez, G. L. (2018). Complementary exploratory and confirmatory factor analyses of the French WISC–V: analyses based on the standardization sample. Psychol. Assess. 30, 793–808. doi: 10.1037/pas0000526
*Lee, P., Mahoney, K. T., and Lee, S. (2017). An application of the exploratory structural equation modeling framework to the study of personality faking. Pers. Individ. Dif. 119, 220–226. doi: 10.1016/j.paid.2017.07.029
*Leo, F. M., González-Ponce, I., Sánchez-Oliva, D., Pulido, J. J., and García-Calvo, T. (2017). Role ambiguity: translation to Spanish and analysis of scale structure. Small Group Res. 48, 365–385. doi: 10.1177/1046496417706554
Leys, C., Ley, C., Klein, O., Bernard, P., and Licata, L. (2013). Detecting outliers: do not use standard deviation around the mean, use absolute deviation around the median. J. Exp. Soc. Psychol. 49, 764–766. doi: 10.1016/j.jesp.2013.03.013
*Litalien, D., Morin, A. J. S., Gagn,é, M., Vallerand, R. J., Losier, G. F., and Ryan, R. M. (2017). Evidence of a continuum structure of academic self–determination: a two-study test using a bifactor-ESEM representation of academic motivation. Contemp. Educ. Psychol. 51, 67–82. doi: 10.1016/j.cedpsych.2017.06.010
*Lohbeck, A., and Petermann, F. (2019). Factorial validity of the anxiety questionnaire for students (AFS): bifactor modeling and measurement invariance. J. Psychoeduc. Assess. 37, 770–781. doi: 10.1177/0734282918794834
*Longo, Y., Jovanovic, V., de Carvalho, J. S., and Karas, D. (2020). The general factor of well-being: multinational evidence using bifactor ESEM on the mental health continuum–short form. Assessment 27, 596–606. doi: 10.1177/1073191117748394
*Maïano, C., Morin, A. J. S., Aimé, A., Lepage, G., and Bouchard, S. (2021). Psychometric properties of the body checking questionnaire (BCQ) and of the body checking cognitions scale (BCCS): a bifactor-exploratory structural equation modeling approach. Assessment 28, 632–646. doi: 10.1177/1073191119858411
Marsh, H. W. (2007). “Application of confirmatory factor analysis and structural equation modeling in sport/exercise psychology,” in Handbook of Sport Psychology, eds G. Tenenbaum, and R. C. Eklund (New York, NY: Wiley), 774–798. doi: 10.1002/9781118270011.ch35
Marsh, H. W., Guo, J., Dicke, T., Parker, P. D., and Craven, R. G. (2020). Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and set-ESEM: optimal balance between goodness of fit and parsimony. Multivariate Behav. Res. 55, 102–119. doi: 10.1080/00273171.2019.1602503
Marsh, H. W., Hau, K.-T., and Wen, Z. (2004). In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Struct. Equ. Model. 11, 320–341. doi: 10.1207/s15328007sem1103_2
Marsh, H. W., Morin, A. J. S., Parker, P. D., and Kaur, G. (2014). Exploratory structural equation modeling: an integration of the best features of exploratory and confirmatory factor analysis. Annu. Rev. Clin. Psychol. 10, 85–110. doi: 10.1146/annurev-clinpsy-032813-153700
Marsh, H. W., Muthén, B., Asparouhov, A., Lüdtke, O., Robitzsch, A., Morin, A. J. S., et al. (2009). Exploratory structural equation modeling, integrating CFA and EFA: application to students' evaluations of university teaching. Struct. Equ. Model. 16, 439–476. doi: 10.1080/10705510903008220
*McKay, M. T., Perry, J. L., Percy, A., and Cole, J. C. (2016). Evidence for the reliability and validity, and some support for the practical utility of the two-factor consideration of future consequences scale−14. Pers. Individ. Dif. 98, 133–136. doi: 10.1016/j.paid.2016.03.097
*McLarnon, M. J. W., and Tarraf, R. C. (2017). The dark triad: specific or general sources of variance? A bifactor exploratory structural equation modeling approach. Pers. Indiv. Diff. 112, 67–73. doi: 10.1016/j.paid.2017.02.049
*McLarnon, M. J. W., and Tarraf, R. C. (2021). Getting to the core: how “(dis)honest” is the core of the dark triad? Pers. Individ. Dif. 171, 110545. doi: 10.1016/j.paid.2020.110545
McNeish, D., and Wolf, M. G. (2022). Dynamic fit index cutoffs for confirmatory factor analysis models. Psychol. Methods. doi: 10.1037/met0000425
*Méndez-Giménez, A., Cecchini-Estrada, J. A., and Rodríguez-González, P. (2020). Competencia percibida (tridimensional), regulaciones motivacionales y autoeficacia en educación física [perceived competence (three-dimensional), motivational regulations and self-efficacy in physical education]. Rev. Latinoam. Psicol. 52, 51–62. doi: 10.14349/rlp.2020.v52.6
Miller, J. (1991). Reaction time analysis with outlier exclusion: bias varies with sample size. Q. J. Exp. Psychol. 43, 907–912. doi: 10.1080/14640749108400962
*Milton, D., Appleton, P. R., Bryant, A., and Duda, J. L. (2018). Initial validation of the teacher-created empowering and disempowering motivational climate questionnaire in physical education. J. Teach. Phys. Educ. 37, 340–351. doi: 10.1123/jtpe.2018-0119
*Morin, A. J. S., Arens, A. K., and Marsh, H. W. (2016a). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Struct. Equ. Model. 23, 116–139. doi: 10.1080/10705511.2014.961800
*Morin, A. J. S., Arens, A. K., Tran, A., and Caci, H. (2016b). Exploring sources of construct-relevant multidimensionality in psychiatric measurement: a tutorial and illustration using the composite scale of morningness. Int. J. Methods Psychiatr. Res. 25, 277–288. doi: 10.1002/mpr.1485
*Morin, A. J. S., Boudrias, J. S., Marsh, H. W., Madore, I., and Desrumaux, P. (2016c). Further reflections on disentangling shape and level effects in person-centered analyses: an illustration exploring the dimensionality of psychological health. Struct. Equ. Model. 23, 438–454. doi: 10.1080/10705511.2015.1116077
*Morin, A. J. S., Boudrias, J. S., Marsh, H. W., McInerney, D. M., Dagenais–Desmarais, V., Madore, I., et al. (2017). Complementary variable- and person-centered approaches to the dimensionality of psychometric constructs: application to psychological wellbeing at work. J. Bus. Psychol. 32, 395–419. doi: 10.1007/s10869-016-9448-7
Morin, A. J. S., Marsh, H. W., and Nagengast, B. (2013). “Exploratory structural equation modeling,” in Structural Equation Modeling: A Second Course, 2nd Edn, eds G. R. Hancock, and R. O. Mueller (Charlotte, NC), 395–436.
Morin, A. J. S., Myers, N. D., and Lee, S. (2020). “Modern factor analytic techniques: bifactor models, exploratory structural equation modeling (ESEM) and bifactor-ESEM,” in Handbook of Sport Psychology, Vol. 2, 4th Edn, eds G. Tenenbaum, and R. C. Eklund (London: Wiley), 1044–1073. doi: 10.1002/9781119568124.ch51
Muthén, B., and Asparouhov, T. (2012). Bayesian structural equation modeling: a more flexible representation of substantive theory. Psychol. Methods 17, 313–335. doi: 10.1037/a0026802
*Myers, N. D., Park, S. E., Lefevor, G. T., Dietz, S., Prilleltensky, I., and Prado, G. J. (2016). Measuring multidimensional subjective well-being with the I COPPE scale in a hispanic sample. Meas. Phys. Educ. Exerc. Sci. 20, 230–243. doi: 10.1080/1091367X.2016.1226836
*Neff, K. D., Bluth, K., Tóth-Király, I., Davidson, O., Knox, M. C., Williamson, Z., et al. (2021a). Development and validation of the self-compassion scale for youth. J. Pers. Assess. 103, 92–105. doi: 10.1080/00223891.2020.1729774
*Neff, K. D., Tóth-Király, I., and Colosimo, K. (2018). Self-compassion is best measured as a global construct and is overlapping with but distinct from neuroticism: a response to Pfattheicher, Geiger, Hartung, Weiss, and Schindler (2017). Eur. J. Pers. 32, 371–392. doi: 10.1002/per.2148
*Neff, K. D., Tóth-Király, I., Knox, M. C., Kuchar, A., and Davidson, O. (2021b). The development and validation of the state self–compassion scale (long- and short form). Mindfulness 12, 121–140. doi: 10.1007/s12671-020-01505-4
*Neff, K. D., Tóth-Király, I., Yarnell, L. M., Arimitsu, K., Castilho, P., Ghorbani, N., et al. (2019). Examining the factor structure of the self-compassion scale in 20 diverse samples: support for use of a total score and six subscale scores. Psychol. Assess. 31, 27–45. doi: 10.1037/pas0000629
*Ng, V., Cao, M. Y., Marsh, H. W., Tay, L., and Seligman, M. E. P. (2017). The factor structure of the values in action inventory of strengths (VIA–IS): an item–level exploratory structural equation modeling (ESEM) bifactor analysis. Psychol. Assess. 29, 1053–1058. doi: 10.1037/pas0000396
*Nixon, N., Guo, B. L., Garland, A., Kaylor-Hughes, C., Nixon, E., and Morriss, R. (2020). The bi-factor structure of the 17-item hamilton depression rating scale in persistent major depression; dimensional measurement of outcome. PLoS ONE 15, e0241370. doi: 10.1371/journal.pone.0241370
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372, 71. doi: 10.1136/bmj.n71
*Part, R., Perera, H. N., Marchand, G. C., and Bernacki, M. L. (2020). Revisiting the dimensionality of subjective task value: towards clarification of competing perspectives. Contemp. Educ. Psychol. 62, 101875. doi: 10.1016/j.cedpsych.2020.101875
Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Lond. Edinburgh Dublin Philos. Mag. J. Sci. 50, 157–175. doi: 10.1080/14786440009463897
*and Perera, H. N. (2016). Construct validity of the social provisions scale: a bifactor exploratory structural equation modeling approach. Assessment 23, 720–733. doi: 10.1177/1073191115589344
*Perera, H. N., and Ganguly, R. (2018). Construct validity of scores from the Connor–Davidson resilience scale in a sample of postsecondary students with disabilities. Assessment 25, 193–205. doi: 10.1177/1073191116646444
*Perera, H. N., Izadikhah, Z., O'Connor, P., and McIlveen, P. (2018a). Resolving dimensionality problems with WHOQOL–BREF item responses. Assessment 25, 1014–1025. doi: 10.1177/1073191116678925
*Perera, H. N., Vosicka, L., Granziera, H., and McIlveen, P. (2018b). Towards an integrative perspective on the structure of teacher work engagement. J. Vocat. Behav. 108, 28–41. doi: 10.1016/j.jvb.2018.05.006
*Perreira, T. A., Morin, A. J. S., Hebert, M., Gillet, N., Houle, S. A., and Berta, W. (2018). The short form of the workplace affective commitment multidimensional questionnaire (WACMQ–S): a bifactor–ESEM approach among healthcare professionals. J. Vocat. Behav. 106, 62–83. doi: 10.1016/j.jvb.2017.12.004
*Pirsoul, T., Parmentier, M., and Nils, F. (2022). One step beyond emotional intelligence measurement in the career development of adult learners: a bifactor exploratory structural equation modeling framework. Curr. Psychol. doi: 10.1007/s12144-021-01772-x
*Pommier, E., Neff, K. D., and Tóth-Király, I. (2020). The development and validation of the compassion scale. Assessment 27, 21–39. doi: 10.1177/1073191119874108
*Portoghese, I., Porru, F., Galletta, M., Campagna, M., and Burdorf, A. (2020). Stress among medical students: factor structure of the university stress scale among Italian students. BMJ Open 10, e035255. doi: 10.1136/bmjopen-2019-035255
*Ratelle, C. F., Duchesne, S., Guay, F., and Châteauvert, G. B. (2018). Comparing the contribution of overall structure and its specific dimensions for competence–related constructs: a bifactor model. Contemp. Educ. Psychol. 54, 89–98. doi: 10.1016/j.cedpsych.2018.05.005
*Reinhardt, M., Horváth, Z., Morgan, A., and Kökönyei, G. (2020a). Well-being profiles in adolescence: psychometric properties and latent profile analysis of the mental health continuum model—a methodological study. Health Qual. Life Outcomes 18, 95. doi: 10.1186/s12955-020-01332-0
*Reinhardt, M., Horváth, Z., Tóth, L., and Kökönyei, G. (2020b). A mentális egészség kontinuum skála rövid változatának hazai validációja. Magyar Pszichol. Szemle 75, 217–246. doi: 10.1556/0016.2020.00014
Reinhold, S., Gegenfurtner, A., and Lewalter, D. (2018). Social support and motivation to transfer as predictors of training transfer: testing full and partial mediation using meta-analytic structural equation modeling. Int. J. Train. Dev. 22, 1–14. doi: 10.1111/ijtd.12115
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behav. Res. 47, 667–669. doi: 10.1080/00273171.2012.715555
Rindskopf, D., and Rose, T. (1988). Some theory and applications of confirmatory second-order factor analyses. Multivariate Behav. Res. 23, 51–67. doi: 10.1207/s15327906mbr2301_3
*Rodenacker, K., Hautmann, C., Gortz–Dorten, A., and Dopfner, M. (2017). The factor structure of ADHD—different models, analyses and informants in a bifactor framework. J. Psychopathol. Behav. Assess. 39, 92–102. doi: 10.1007/s10862-016-9565-7
*Rodrigues, F., Cid, L., Teixeira, D., and Monteiro, D. (2021). Re-applying the basic psychological needs in exercise scale to various Portuguese exercise groups: an analysis of bifactor models and contextual invariance. Percept. Mot. Skills 128, 1660–1683. doi: 10.1177/00315125211016803
*Rodrigues, F., Macedo, R., Teixeira, D. S., Cid, L., and Monteiro, D. (2020). Motivation in sport and exercise: a comparison between the BRSQ and BREQ. Qual. Quant. 54, 1335–1350. doi: 10.1007/s11135-020-00988-6
*Rogoza, R., Thi, K. H. T., Rózycka-Tran, J., Piotrowski, J., and Zemojtel-Piotrowska, M. (2018). Psychometric properties of the MHC–SF: an integration of the existing measurement approaches. J. Clin. Psychol. 74, 1742–1758. doi: 10.1002/jclp.22626
*Sakakibara, K., Shimazu, A., Toyama, H., and Schaufeli, W. B. (2020). Validation of the Japanese version of the burnout assessment tool. Front. Psychol. 11, 1819. doi: 10.3389/fpsyg.2020.01819
*Sánchez-Oliva, D., Morin, A. J. S., Teixeira, P. J., Carraça, E. V., Palmeira, A. L., and Silva, M. N. (2017). A bifactor exploratory structural equation modeling representation of the structure of the basic psychological needs at work scale. J. Vocat. Behav. 98, 173–187. doi: 10.1016/j.jvb.2016.12.001
*Scherer, R., Nilsen, T., and Jansen, M. (2016). Evaluating individual students' perceptions of instructional quality: an investigation of their factor structure, measurement invariance, and relations to educational outcomes. Front. Psychol. 7, 110. doi: 10.3389/fpsyg.2016.00110
*Schmid, J., Steiner, S., Rensch, M., Middleton, C., and Seiler, R. (2018). Psychometrische eigenschaften einer deutschsprachigen übersetzung des mental toughness inventory (MTI-D) [psychometric properties of a German translation of the mental toughness inventory (MIT-D)]. Diagnostica 64, 61–73. doi: 10.1026/0012-1924/a000192
*Schutte, L., and Wissing, M. P. (2017). Clarifying the factor structure of the mental health continuum short form in three languages: a bifactor exploratory structural equation modeling approach. Soc. Ment. Health 7, 142–158. doi: 10.1177/2156869317707793
*Sellbom, M., and Tellegen, A. (2019). Factor analysis in psychological assessment research: common pitfalls and recommendations. Psychol. Assess. 31, 1428–1441. doi: 10.1037/pas0000623
*Sen, R., Pleil, A. M., Coon, C., and Shields, A. L. (2015). Evaluating the dimensionality of complex pros using bifactor analysis within an exploratory structural equation modeling (ESEM) framework: An example using the patient-reported scar evaluation questionnaire (PR–SEQ). Quality of Life Research 24, 26.
Shi, D., Lee, T., and Maydeu-Olivares, A. (2019). Understanding the model size effect on SEM fit indices. Educ. Psychol. Meas. 79, 310–334. doi: 10.1177/0013164418783530
*Silverman, A. L., Foregard, M., Beard, C., and Björgvinsson, T. (2018). Psychometric properties of the mental health continuum—short form in a psychiatric sample. Journal of Well Being Assess. 2, 57–73. doi: 10.1007/s41543-018-0011-3
*Somma, A., Borroni, S., Drislane, L. E., Patrick, C. J., and Fossati, A. (2019). Modeling the structure of the triarchic psychopathy measure: comceptual, empirical, and analytical considerations. J. Pers. Disord. 33, 470–496. doi: 10.1521/pedi_2018_32_354
Stanley, T. D., and Doucouliagos, H. (2017). Neither fixed nor random: weighted least squares meta-regression. Res. Synth. Methods 8, 19–42. doi: 10.1002/jrsm.1211
Steiger, J. H. (1990). Structural model evaluation and modification: an interval estimation approach. Multivariate Behav. Res. 25, 173–180. doi: 10.1207/s15327906mbr2502_4
*Stenling, A., Ivarsson, A., Hassmén, P., and Lindwall, M. (2015). Using bifactor exploratory structural equation modeling to examine global and specific factors in measures of sports coaches' interpersonal styles. Front. Psychol. 6, 1303. doi: 10.3389/fpsyg.2015.01303
*Stenling, A., Ivarsson, A., Lindwall, M., and Gucciardi, D. F. (2018). Exploring longitudinal measurement invariance and the continuum hypothesis in the Swedish version of the behavioral regulation in sport questionnaire (BRSQ): an exploratory structural equation modeling approach. Psychol. Sport Exerc. 36, 187–196. doi: 10.1016/j.psychsport.2018.03.002
*Styck, K. M., Rodriguez, M. C., and Yi, E. H. (2021). Dimensionality of the state-trait inventory of cognitive and somatic anxiety. Assessment 29, 103–127. doi: 10.1177/1073191120953628
*Summers, J. J., and Falco, L. D. (2020). The development and validation of a new measure of adolescent purpose. J. Exp. Educ. 88, 47–71. doi: 10.1080/00220973.2019.1575178
*and Sutherland, E. A. (2020). The measurement of job satisfaction among workplace leaders: Scale development and validation of the leader satisfaction assessment. (Doctoral thesis), Western University. Available online at: https://ir.lib.uwo.ca/etd/7006 (accessed June 29, 2021).
*Tomás, J. M., Gutiérrez, M., Alberola, S., and Georgieva, S. (2022). Psychometric properties of two major approaches to measure school engagement in university students. Curr. Psychol. 41, 2654–2667. doi: 10.1007/s12144-020-00769-2
*Tóth-Király, I., Bothe, B., and Orosz, G. (2017). Exploratory structural equation modeling analysis of the self–compassion scale. Mindfulness 8, 881–892. doi: 10.1007/s12671-016-0662-1
*Tóth-Király, I., Böthe, B., Orosz, G., and Rigo, A. (2019). A new look on the representation and criterion validity of need fulfillment: application of the bifactor exploratory structural equation modeling framework. J. Happiness Stud. 20, 1609–1626. doi: 10.1007/s10902-018-0015-y
*Tóth-Király, I., Morin, A. J. S., Böthe, B., Orosz, G., and Rigó, A. (2018). Investigating the multidimensionality of need fulfillment: a bifactor exploratory structural equation modeling representation. Struct. Equ. Model. 25, 267–286. doi: 10.1080/10705511.2017.1374867
*Tóth-Király, I., Morin, A. J. S., and Salmela-Aro, K. (2021). Reciprocal associations between burnout and depression: An 8-year longitudinal study. Appl. Psychol. Int. Rev. 70, 1691–1727. doi: 10.1111/apps.12295
*Trógolo, M. A., Morera, L. P., Castellano, E., Spontón, C., and Medrano, L. A. (2020). Work engagement and burnout: real, redundant, or both? A further examination using a bifactor modelling approach. Euro. J. Work Organ. Psychol. 29, 922–937. doi: 10.1080/1359432X.2020.1801642
*Vajda, D., Thege, B. K., and Rózsa, S. (2019). Factor structure of the dyadic adjustment scale: a bifactor exploratory structural equation modeling approach. Euro. J. Psychol. Assess. 35, 326–334. doi: 10.1027/1015-5759/a000405
*Van Zyl, L. E., Olckers, C., and Roll, L. C. (2020). The psychometric properties of the Grit–O scale within the twente region in Netherlands: an ICM–CFA vs. ESEM approach. Front. Psychol. 11, 796. doi: 10.3389/fpsyg.2020.00796
*Vaughan, R., Madigan, D. J., Carter, G. L., and Nicholls, A. R. (2019). The dark triad in male and female athletes and non-athletes: group differences and psychometric properties of the short dark triad (SD3). Psychol. Sport Exerc. 43, 64–72. doi: 10.1016/j.psychsport.2019.01.002
*Vaughan, R. S., Edwards, E. J., and MacIntyre, T. E. (2020). Mental health measurement in a post Covid-19 world: psychometric properties and invariance of the DASS−21 in athletes and non-athletes. Front. Psychol. 11, 590559. doi: 10.3389/fpsyg.2020.590559
*Volmer, J., Koch, I. K., and Wolff, C. (2019). Illuminating the 'dark core': mapping global versus specific sources of variance across multiple measures of the dark triad. Pers. Individ. Dif. 145, 97–102. doi: 10.1016/j.paid.2019.03.024
*Wang, T., Zhang, T., Mu, W. L., and Li, X. (2021). Psychometric evaluation of the encouragement character strength scale in a Chinese sample. Curr. Psychol. 40, 4741–4749. doi: 10.1007/s12144-020-01161-w
*Yi, Z. Y., Wang, Y., and Tan, T. X. (2021). Evidence for a higher-order ESEM structure of ADHD in a sample of Chinese children. J. Psychopathol. Behav. Assess. 43, 376–387. doi: 10.1007/s10862-020-09837-0
Keywords: factor analysis, meta-analysis, multidimensionality, goodness-of-fit, exploratory structural equation modeling, bifactor ESEM
Citation: Gegenfurtner A (2022) Bifactor exploratory structural equation modeling: A meta-analytic review of model fit. Front. Psychol. 13:1037111. doi: 10.3389/fpsyg.2022.1037111
Received: 05 September 2022; Accepted: 06 October 2022;
Published: 26 October 2022.
Edited by:
Cesar Merino-Soto, Universidad de San Martin de Porres, PeruReviewed by:
Conrad Stanisław Zygmont, Helderberg College, South AfricaPanagiotis Ferentinos, National and Kapodistrian University of Athens, Greece
Copyright © 2022 Gegenfurtner. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Andreas Gegenfurtner, YW5kcmVhcy5nZWdlbmZ1cnRuZXImI3gwMDA0MDtwaGlsLnVuaS1hdWdzYnVyZy5kZQ==