Skip to main content

SYSTEMATIC REVIEW article

Front. Psychol., 26 October 2022
Sec. Quantitative Psychology and Measurement

Bifactor exploratory structural equation modeling: A meta-analytic review of model fit

  • Methods in Learning Research, University of Augsburg, Augsburg, Germany

Multivariate behavioral research often focuses on latent constructs—such as motivation, self-concept, or wellbeing—that cannot be directly observed. Typically, these latent constructs are measured with items in standardized instruments. To test the factorial structure and multidimensionality of latent constructs in educational and psychological research, Morin et al. (2016a) proposed bifactor exploratory structural equation modeling (B-ESEM). This meta-analytic review (158 studies, k = 308, N = 778,624) aimed to estimate the extent to which B-ESEM model fit differs from other model representations, including confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), hierarchical CFA, hierarchical ESEM, and bifactor-CFA. The study domains included learning and instruction, motivation and emotion, self and identity, depression and wellbeing, and interpersonal relations. The meta-analyzed fit indices were the χ2/df ratio, the comparative fit index (CFI), the Tucker-Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardized root mean squared residual (SRMR). The findings of this meta-analytic review indicate that the B-ESEM model fit is superior to the fit of reference models. Furthermore, the results suggest that model fit is sensitive to sample size, item number, and the number of specific and general factors in a model.

Introduction

To examine the factorial structure and multidimensionality of latent constructs in educational and psychological research, Morin et al. (2016a) proposed bifactor exploratory structural equation modeling (B-ESEM) as a methodological synergy that integrates bifactor modeling and exploratory structural equation modeling. Since their seminal paper, a number of studies have applied B-ESEM in research on learning and instruction, motivation and emotion, self and identity, wellbeing, and other areas. The present systematic review and meta-analysis aimed to collect and describe these studies, meta-analyze their reported model fit, and estimate the extent to which the fit of the B-ESEM model differs from that of the other tested model representations. A secondary aim was to analyze how sensitive model fit was to sample size, item number, and the number of specific and general factors in a model.

Exploring the factor structure of multidimensional constructs

Researchers interested in exploring the factorial structure of a multidimensional construct can choose among analytical options including confirmatory factor analysis (CFA; Jöreskog, 1969), exploratory structural equation modeling (ESEM; Asparouhov and Muthén, 2009), and hierarchical and bifactor representations of CFA and ESEM (Rindskopf and Rose, 1988; Reise, 2012; Morin et al., 2013, 2016a). A number of excellent review papers and pedagogical illustrations of these factor analytic techniques exist (Marsh, 2007; Marsh et al., 2009, 2014; Reise, 2012; Morin et al., 2013, 2016a, 2020; Bandalos and Finney, 2018; Sellbom and Tellegen, 2019). Interested readers can consult these excellent resources for more detail, but let us briefly review how the factor structure of multidimensional latent constructs can be represented and analyzed. Figure 1 offers an overview of schematic model representations.

FIGURE 1
www.frontiersin.org

Figure 1. Schematic model representations. Ovals represent latent factors; squares represent observed variables; full unidirectional arrows between ovals and squares represent factor loadings; dotted unidirectional arrows between ovals and squares represent cross-loadings; full unidirectional arrows linked to a single oval represent factor disturbance; full unidirectional arrows linked to a single square represent item uniqueness; bidirectional full arrows between ovals represent factor covariances/correlations; bidirectional dashed arrows linked to a single oval represent factor variance; CFA, confirmatory factor analysis; ESEM, exploratory structural equation modeling; H Factor, higher-order factor in a hierarchical model; L Factor 1–3, lower-order factors in a hierarchical model; S Factor 1–3, specific factors in a bifactor model; G Factor, general factor in a bifactor model.

The simplest form of representation and analysis, arguably, is a one-factor first-order model in which all items are loaded on a single factor. More complex are CFA and ESEM that can be applied to test a hypothesized factor structure: in CFA, items load on a single factor with zero cross-loadings; in ESEM, too, items load a single factor but without CFA's strict requirement of zero cross-loadings. When exploring hierarchically ordered constructs, CFA and ESEM can be extended to models in which items are loaded on specific lower-order factors (L-factors), which are in turn loaded on a single higher-order factor (the H-factor). Finally, in the case of bifactor modeling, CFA and ESEM can be extended to a model in which items are loaded on one general factor (the G-factor) representing a global overarching construct as well as on multiple specific factors (S-factors) representing their subscales—in bifactor CFA (B-CFA) without and in bifactor ESEM (B-ESEM) with free estimation of cross-loadings between items and non-target factors. B-ESEM in particular has been shown to result in a better model fit for a number of multidimensional hierarchical constructs that are frequently explored in educational and psychological research1.

B-ESEM in multivariate behavioral research

A growing number of studies test the multidimensionality of latent constructs in research on learning and instruction, motivation and emotion, self and identity, wellbeing and depression, and interpersonal relationships. In the domain of learning and instruction, for example, Scherer et al. (2016) identified the presence of a general factor of student-perceived instructional quality and three distinct sub-dimensions: teacher support, cognitive activation, and classroom management. Fernandez-Rio and colleagues explored the factor structure of a scale measuring dimensions of cooperative learning. In the domain of motivation and emotion, Howard et al. (2020) showed that motivation scales based on the self-determination theory were best represented as a B-ESEM model in which all items defined specific motivation regulatory qualities on a continuum (from amotivation to intrinsic motivation) and were also used to define a global self-determination factor. Perera et al. (2018b) tested the structure of teacher engagement, identifying a general factor and specific cognitive-physical, emotional, and social engagement factors. In the domain of self and identity, for example, Arens et al. (2021) explored the structure of academic self-concept and reported that a B-ESEM representation including one general self-concept factor and multiple specific self-concept factors (for German, English, math, physics, chemistry, biology, and history) provided the best model fit compared with reference models. In the domain of wellbeing and depression, Morin et al. (2017) tested the dimensionality of the Index of Psychological WellBeing at Work that can also be used to wellbeing of teachers. In the domain of interpersonal relations, Ratelle et al. (2018) showed that the factor structure of a scale measuring parental structure, including parents' rules, predictability, feedback, opportunities, rationale, and authority, was best represented as a B-ESEM model.

These individual studies support the assumption that B-ESEM is best used to deal with multidimensional hierarchical constructs, yet the extent to which model fit of B-ESEM differs from other model representations has not yet been investigated across studies, domains, and scales. A meta-analysis of published studies could help to close this research gap. Among the frequently used parameters used in the studies reviewed to test model fit are the χ2/df ratio (Pearson, 1900), the comparative fit index (CFI; Bentler, 1990), the Tucker-Lewis index (TLI; Bentler and Bonett, 1990), the root mean square error of approximation (RMSEA; Steiger, 1990), and the standardized root mean squared residual (SRMR; Bentler, 1995). Table 1 provides an overview of goodness-of-fit indices used in B-ESEM studies2. Furthermore, if the evidence that model fit indices are sensitive to sample size, item number, and factor number from simulation studies is valid (Hu and Bentler, 1999; Marsh et al., 2004; Shi et al., 2019), it would be interesting to replicate these findings through a research synthesis using real data rather than simulated data. Such a synthesis would also be useful to describe the status quo of research adopting a B-ESEM framework, the mapping of tested constructs (such as academic self-concept; Arens et al., 2021), scales (such as the Multidimensional Work Motivation Scale; Howard et al., 2020), and domains (such as learning and instruction).

TABLE 1
www.frontiersin.org

Table 1. Factor analytic techniques and model fit indices.

The present study

Based on a systematic literature review and meta-analysis of previous research applying the B-ESEM framework, the present study compares goodness-of-fit indices of B-ESEM with reference models. The study sought to answer three research questions: (1) Which domains, constructs, and scales are targeted in studies adopting a B-ESEM framework? (2) What is the model fit of B-ESEM representations compared to CFA, ESEM, H-CFA, H-ESEM, and B-CFA representations? (3) How sensitive is model fit to sample size, item number, and the number of specific and general factors in a model?

Methods

Steps in conducting and reporting this meta-analytic review of B-ESEM included defining inclusion and exclusion criteria, searching the literature, coding information from the retrieved studies, calculating intercoder reliability, and meta-analyzing the fit indices (Page et al., 2021). The following sections describe these steps in more detail.

Defining inclusion and exclusion criteria

We identified studies that reported goodness-of-fit indices of B-ESEM in educational and psychological research. Table 2 presents the criteria for inclusion and exclusion of these studies. To be included, a study had to report a χ2 (df), CFI, TLI, RMSEA, or SRMR estimate of a bifactor ESEM model and compare it with a CFA, ESEM, H-CFA, H-ESEM, or B-CFA reference model. Studies were omitted if they did not examine a B-ESEM representation, did not compare B-ESEM to a reference model, or relied on simulated data. To minimize publication bias, our literature search was deliberately broad and included reports in journal articles, book chapters, monographs, conference papers, and unpublished theses or dissertations, regardless of participant population, publication language, publication year, or domain.

TABLE 2
www.frontiersin.org

Table 2. Criteria for inclusion and exclusion in the meta-analytic review.

To illustrate how we applied the criteria, examples of both excluded and included articles may help. On the one hand, an excellent study by Zhu et al. (2020) that reports model fit evaluation of a B-ESEM representation of mindfulness was excluded because B-ESEM was not compared with other models; rather, the B-ESEM factor scores were used to perform latent profile analyses. On the other hand, a study by Arens et al. (2021) was included because the authors compared goodness-of-fit indices obtained from higher-order and bifactor ESEM representations of the structure of academic self-concept.

Searching the literature

Grounded in the inclusion criteria presented in Table 2, a two-phase literature search was performed. First, we conducted an electronic search of five databases—ERIC, PsycINFO, PubMed, Scopus, and Web of Science—for all publication types in any language published up to December 31, 2021 and with relevant keywords in the title or abstract. The keywords were bifactor exploratory structural equation modeling, bifactor ESEM, and B-ESEM, along with their spelling variants. This database search resulted in a total of 525 hits: 25 from ERIC, 106 from PsycINFO, 34 from PubMed, 182 from Scopus, and 178 from Web of Science. Two trained raters eliminated 318 duplicates because they were listed in more than one database, after which 207 articles remained. Next, both raters independently and in duplicate screened a random subset of the 207 identified articles (10 percent; n = 21 articles). As interrater agreement was high, with Cohen's κ = 0.881 (95% CI = 0.659–1.00), a single rater continued to screen the remaining studies for eligibility by reading titles and abstracts, after which 186 articles remained. Both raters then read the full texts of these articles independently to check for eligibility, and 60 studies were excluded because they did not examine a B-ESEM representation (37 removals) or did not compare B-ESEM to a reference model (23 removals), resulting in 126 articles for inclusion in the meta-analytic review.

Second, we performed a forward cross-referencing search, using Google Scholar to identify studies that cited the influential article of Morin et al. (2016a) up until December 31, 2021. We read the titles and abstracts of the 723 citing studies and retrieved the full texts of 169 of these reports. After the full texts were read, 137 records were excluded because the studies did not examine a B-ESEM representation (129 removals), did not include a reference model (4 removals), or used simulated data (4 removals). The remaining 32 studies met all inclusion criteria and were thus included in the review.

In summary, the literature search resulted in a total of 158 studies: 126 from the database search and 32 from the forward search. Figure 2 presents the PRISMA study selection flow diagram (Page et al., 2021). The included studies reported 308 comparisons of B-ESEM representations to reference models, which were subsequently coded.

FIGURE 2
www.frontiersin.org

Figure 2. PRISMA study selection flow chart.

Coding information from the retrieved studies

Once the studies for inclusion were selected, two trained raters used the coding scheme presented in Table 3 to code a random subset of the 308 identified reports (10 percent; n = 31 reports) independently and in duplicate. Coded information included publication characteristics, measurement characteristics, and goodness-of-fit indices. Publication characteristics were coded as the first author and publication year. Measurement characteristics were coded as sample size, target construct, and scale name and abbreviation, as well as the number of items, specific factors, and general factors. Goodness-of-fit indices were coded separately for one-factor, CFA, ESEM, H-CFA, H-ESEM, B-CFA, and B-ESEM model representations; indices included CFI, TLI, RMSEA, SRMR, and the χ2/df ratio.

TABLE 3
www.frontiersin.org

Table 3. Coding scheme.

To illustrate our coding decisions, examples how we applied the coding scheme may help. For instance, Rodrigues et al. (2020) reported the fit of model representations with (a) one general and two specific factors, (b) one general and six specific factors, and (c) two general and six specific factors for the Behavioral Regulation in Sport Questionnaire and the Behavioral Regulation in Exercise Questionnaire. Because this meta-analytic review aimed to estimate the influence of factor number on model fit, we coded each model separately. Similarly, Tóth-Király et al. (2019) reported the model fit of representations with one or two general factors of the Basic Psychological Need Satisfaction and Frustration Scale. We coded each representation separately to allow for comparison. Generally, model fit was coded separately for studies that tested models with varying numbers of specific and/or general factors (e.g., Arias et al., 2016; Rodenacker et al., 2017; Gu et al., 2020; Frutos, 2021; Yi et al., 2021) and with varying item numbers. For example, Bianchi and Verkuilen (2021) examined the Green et al. Paranoid Thoughts Scale (GPTS) in its original 32-item version, a revised 18-item version, and an 8-item version. Model fit for each version was coded independently to allow for estimation of the influence of item number on the goodness-of-fit indices. Model fit was also coded separately for studies that used more than one sample (e.g., Neff et al., 2019; Howard et al.,2020; Longo et al., 2020; Vaughan et al., 2020) or measurement time (e.g., Stenling et al., 2018; Cece et al., 2019; Garn et al., 2019; Neff et al., 2021b). Section Research question 1: Description of B-ESEM studies provides a complete description of all coded study information.

Calculating interrater reliability

To calculate interrater reliability and agreement of the literature search and the literature coding, we used an intraclass correlation coefficient (ICC) for continuous scales and Cohen's kappa coefficient (κ) for nominal scales. Table 4 presents all estimates. ICC and κ estimates tend to be robust when there are exactly two raters, as in our case, who searched and coded the studies independently and in duplicate. Consensus was reached via discussion when coding conflicts emerged. Following the recommendations of Koo and Li (2016), ICC estimates were calculated together with their 95% confidence intervals based on a mean-rating (k = 2), absolute-agreement, two-way mixed-effects model which can generally be judged as moderate (0.50–0.75), good (0.75–0.90), and excellent (>0.90) reliability. Following recommendations of Landis and Koch (1977), standard errors of Cohen's κ were calculated to compute the 95% confidence intervals around κ which can generally be judged as moderate (0.41–0.60), substantial (0.61–0.80), and almost perfect (>0.80) agreement.

TABLE 4
www.frontiersin.org

Table 4. Interrater reliability and agreement of the literature search and coding.

Analyses for the literature search indicated substantial to almost perfect agreement for identification, κ = 0.990 (95% CI = 0.981; 0.999), screening, κ = 0.881 (95% CI = 0.659; 1.000), and eligibility, κ = 1.000 (95% CI = 1.000; 1.000). Analyses for the literature coding were performed separately for the continuous and nominal variables: the continuous scales were publication year, number of items, number of specific factors, number of general factors, and all goodness-of-fit indices, with an ICC = 0.923 (95% CI = 0.901; 0.932); the remaining scales were nominal, with κ = 0.976 (95% CI = 0.727; 1.000). In summary, these estimates indicate almost perfect agreement (Landis and Koch, 1977) and excellent reliability (Koo and Li, 2016).

Meta-analyzing the fit indices

We meta-analyzed the fit indices in two steps. First, we computed average estimates of the goodness-of-fit indices of each model. Mean differences between models were calculated using one-way analyses of variance. Missing values were deleted casewise. Second, we estimated the effects of sample size, item number, number of specific factors, and number of general factors on the fit indices using within-study and between-study analyses. Within-study analyses included (a) descriptive analyses of two-tailed Pearson correlation coefficients and (b) aggregated changes of model fit as a result of changes in B-ESEM model structure. Between-study analysis included (a) an outlier detection analysis using the median absolute deviation approach with the formulae reported in Miller's (1991) and Leys et al. (2013) very conservative threshold 3 and (b) an unrestricted weighted least squares meta-regression analysis (Stanley and Doucouliagos, 2017) to estimate the relative influence of sample size, item number, and factor number on model fit.

Results and discussion

Research question 1: Description of B-ESEM studies

Research question 1 asked, “Which domains, constructs, and scales are targeted in studies adopting a B-ESEM framework?” The 158 studies included in the present analysis offered a total of 308 reports of B-ESEM model fit. Total sample size was 778,624 participants, with a mean sample size of 2,528.00 (SD = 25,492.06). On average, B-ESEM models were composed of 24.62 items (SD = 17.23), 4.62 specific factors (2.96), and 1.05 general factors (0.26). B-ESEM representations were compared to representations from B-CFA in 221 reports, H-ESEM in 25 reports, H-CFA in 57 reports, ESEM in 266 reports, CFA in 265 reports, and a one-factor model in 123 reports. Concerning publication language, 151 of the 158 included studies were written in English, 4 in Spanish, and 1 each in French, German, and Hungarian. Concerning publication type, 154 of the 158 included studies were journal articles, 3 were doctoral theses, and 1 was an unpublished manuscript. Table 5 presents a description of all included studies clustered into six domains.

TABLE 5
www.frontiersin.org

Table 5. Study description.

The first domain of learning and instruction includes the constructs of cooperative learning (Fernandez-Rio et al., 2022) and instructional quality (Scherer et al., 2016).

The second domain of motivation and emotion includes the constructs academic motivation (Litalien et al., 2017; Guay and Bureau, 2018; Gordeeva et al., 2020; Howard et al., 2020; Kartal, 2020; Guay et al., 2021; Dierendonck et al., 2022), sport motivation (Appleton et al., 2016; Milton et al., 2018; Stenling et al., 2018; Cece et al., 2019; Méndez-Giménez et al., 2020; Rodrigues et al., 2020), work motivation (Burk and Wiese, 2018; Calkins, 2018; Howard et al., 2018, 2020, 2021; Gegenfurtner and Quesada-Pallarès, 2022), job satisfaction (Sutherland, 2020), affective commitment (Perreira et al., 2018), work engagement (Gillet et al., 2019; Huyghebaert-Zouaghi et al., 2021a, 2022), teacher engagement (Perera et al., 2018b), student engagement (Hoi and Hang, 2021; Dierendonck et al., 2022; Tomás et al., 2022), basic psychological needs (Stenling et al., 2015; Sánchez-Oliva et al., 2017; Tóth-Király et al., 2018, 2019; Bhavsar et al., 2019, 2020; Garn et al., 2019; Burgueño et al., 2020a,b; Cromhout, 2020; Gillet et al., 2020; Gucciardi et al., 2020; Huyghebaert-Zouaghi et al., 2021b; Rodrigues et al., 2021), subjective task value (Fadda et al., 2020b; Part et al., 2020), flow (Kyriazos et al., 2018b; Gu et al., 2020), interest (Garn, 2017), locus of causality (Howard et al., 2020), attitudes (Deemer et al., 2014), purpose (Summers and Falco, 2020), self-efficacy (Barbaranelli et al., 2018; Dominguez-Lara et al., 2019), emotion regulation (Clifton et al., 2020; Hanley et al., 2020; Lauriola et al., 2021), happiness (Appiah et al., 2020), hope (Krafft et al., 2020), anxiety (Lohbeck and Petermann, 2019), fear (Jastrzebski et al., 2020), and loneliness (Grygiel et al., 2019).

The third domain of self and identity includes the constructs of self-compassion (Tóth-Király et al., 2017; Benda, 2018; Neff et al., 2018, 2019, 2021a,b), self-concept (Morin et al., 2016a; Arens et al., 2021), self-perception (Chung et al., 2016; Arens and Morin, 2017; Giotsa and Kyriazos, 2019), body checking (Maïano et al., 2021), big five personality traits (Lee et al., 2017; Arias et al., 2018), multicultural personality (Korol et al., 2018), psychopath personality (Gu et al., 2017; McLarnon and Tarraf, 2017, 2021; Somma et al., 2019; Gomez et al., 2020b; Vaughan et al., 2020), conspiracy beliefs (García-Garzón et al., 2020), paranoid thoughts (Bianchi and Verkuilen, 2021), character strength (Ng et al., 2017; Wang et al., 2021), consideration of future consequences (McKay et al., 2016), morningness (Morin et al., 2016b; Díaz-Morales and Parra-Robledo, 2018), compassion (Halamová et al., 2020; Pommier et al., 2020), emotional intelligence (Esnaola et al., 2018; Pirsoul et al., 2022), intelligence (Lecerf and Canivez, 2018), life skills (Cronin and Allen, 2017; Jaotombo, 2019; Choisay et al., 2021), mental toughness (Bédard-Thom and Guay, 2018; Schmid et al., 2018; Kawabata et al., 2021), and resilience (Decroos et al., 2017; Perera and Ganguly, 2018; Dai et al., 2019).

The fourth dimension of depression and wellbeing includes the constructs of depression (Borges et al., 2017; Volmer et al., 2019; Bianchi, 2020; Bianchi and Schonfeld, 2020; Cano-García et al., 2020; Gomez et al., 2020a; Nixon et al., 2020; Vaughan et al., 2020; Høstmælingen et al., 2021; Jovanovic et al., 2021), burnout (Isoard-Gautheur et al., 2018; Bianchi, 2020; Sakakibara et al., 2020; Trógolo et al., 2020; Doherty et al., 2021; Tóth-Király et al., 2021), positive thoughts (Appiah et al., 2020), wellbeing (Myers et al., 2016; Fadda et al., 2017, 2020a; Morin et al., 2017; Schutte and Wissing, 2017; Espinoza et al., 2018; Kyriazos et al., 2018a; Lamborn et al., 2018; Perera et al., 2018a; Rogoza et al., 2018; Silverman et al., 2018; Ferentinos et al., 2019; Cromhout, 2020; Longo et al., 2020; Reinhardt et al., 2020a,b), workaholism (Huyghebaert-Zouaghi et al., 2022), stress (Morin et al., 2016c; Portoghese et al., 2020), stress disorder (Fresno et al., 2020), anxiety disorder (Deller et al., 2020; Styck et al., 2021), problem behavior (Lahey et al., 2018; Hukkelberg, 2019; Hukkelberg and Ogden, 2020; Gomez et al., 2021), and attention deficit hyperactivity disorder (Arias et al., 2016; Rodenacker et al., 2017; Frutos, 2019, 2021; Gomez and Stavropoulos, 2021; Yi et al., 2021).

The fifth dimension of interpersonal relations includes the constructs of romantic relationships (Vajda et al., 2019; Goodboy et al., 2021), parenting (Ratelle et al., 2018), group cohesion (Haugen et al., 2021), working alliance (Hukkelberg and Ogden, 2019), work teams (Leo et al., 2017), supervision (Junga et al., 2019), and perceived social support (Perera, 2016).

The sixth and last category includes two constructs that did not fit the aforementioned five domains: motoric skills (Garn and Webster, 2021) and scar evaluation (Sen et al., 2015).

The 158 included studies psychometrically tested a total of 135 different scales. Among the most often-investigated constructs are wellbeing (n = 17), basic psychological needs (n = 14 studies), depression (n = 10), academic motivation (n = 7), psychopathic personality (n = 7), sport motivation (n = 7), attention deficit hyperactivity disorder (n = 6), burnout (n = 6), self-compassion (n = 6), and work motivation (n = 6). Scales whose factorial structures were frequently examined using a B-ESEM approach include the Mental Health Continuum—Short Form (MHC-SF), addressed in eight studies, the Academic Motivation Scale (AMS), addressed in six studies, and the Self-Compassion Scale (SCS), addressed in six studies.

Research question 2: Model fit

Research question 2 asked, “What is the model fit of B-ESEM representations in comparison to CFA, ESEM, H-CFA, H-ESEM, and B-CFA representations?” To answer this research question, a primary meta-analysis was performed to cumulate the reported goodness-of-fit-indices across studies and then to generate average estimates of model fit for each representation. Table 6 reports the results of this meta-analysis. Analyses of variance suggest that model fit differed significantly between representations. The findings of comparing the model representations, as shown in Figure 1, indicate that the B-ESEM representation showed the best model fit, with the lowest χ2/df ratio, the highest CFI, the lowest RMSEA, and the lowest SRMR. TLI was minimally higher for H-ESEM (0.950) than for B-ESEM (0.948) [ΔTLI = 0.002, t(23) = 1.321, p = 0.200]. These results demonstrate the superior model fit of B-ESEM solutions over other factorial representations across a range of scales and domains. Figure 3 portrays the strength of the B-ESEM goodness-of-fit indices relative to the other model representations.

TABLE 6
www.frontiersin.org

Table 6. Mean (and standard deviation) estimates of the goodness-of-fit indices per model.

FIGURE 3
www.frontiersin.org

Figure 3. Goodness-of-fit indices per model representation.

Table 7 presents a correlation matrix of the goodness-of-fit indices of all model representations. Across models, correlations were highly positive between CFI and TLI. Correlations also tended to be substantial between the same fit indices of different models, for example between ESEM RMSEA, H-ESEM RMSEA, and B-ESEM RMSEA or between one-factor χ2/df, CFA χ2/df, and ESEM χ2/df . Correlation coefficients between the incremental (CFI, TLI) and absolute (RMSEA, SRMR) fit indices tended to be negative.

TABLE 7
www.frontiersin.org

Table 7. Correlation matrix.

Research question 3: Influence of sample size, item number, and factor number on model fit

Research question 3 asked, “How sensitive is model fit to sample size, item number, and the number of specific and general factors in a model?” To answer this research question, we performed within-study and between-study analyses.

First, within-study analyses aggregated the change in model fit as a result of changes in B-ESEM model structure. A number of studies reported model fit for different model specifications. For example, Yi et al. (2021) examined the fit of a B-ESEM model with two or three specific factors; their findings demonstrate better model fit for a three-factor representation. A variety of different representations were explored among the included studies, such as one and three general factors (Bhavsar et al., 2019), as well as three and nine specific factors (Bhavsar et al., 2019), four and five specific factors (Choisay et al., 2021), four and six specific factors (Part et al., 2020), seven and eight specific factors (Bédard-Thom and Guay, 2018), and four and twenty specific factors (Sutherland, 2020). Table 8 presents within-study analyses of changes in B-ESEM model fit by factor number. Average estimates across reports suggest that, when the number of specific factors changed from two to three, two to six, three to four, or three to six, CFI and TLI increased, while the χ2/df ratio, RMSEA, and SRMR decreased. Similarly, when the number of general factors changed from one to two, CFI and TLI increased, while the χ2/df ratio, RMSEA, and SRMR decreased.

TABLE 8
www.frontiersin.org

Table 8. Within-Study analysis of changes in B-ESEM model fit by factor number.

Second, between-study analyses were performed to estimate the extent to which model fit is influenced by sample size, item number, and the number of specific and general factors. Between-study analyses included correlation analyses and meta-regression analysis. The correlation analysis shown in Table 7 suggests high correlation coefficients between sample size and the χ2/df ratio. Prior to the meta-regression, an outlier detection analysis was conducted to identify studies with outlying values. Analyses were performed separately for sample size, item number, number of specific factors, and number of general factors. Table 9 presents the outcomes of the outlier analysis. For sample size, 40 reports with more than 1,763 participants were identified. For item number, 12 reports examined scales with 44 items or more. For the number of specific factors, 19 reports used B-ESEM representations with nine specific factors or more. For the number of general factors, 13 reports used B-ESEM representations with more than one general factor. These reports were removed prior to the meta-regression analysis.

TABLE 9
www.frontiersin.org

Table 9. Outlier detection analysis.

Table 10 presents the results of the meta-regression. First, the findings indicate that sample size significantly influenced the χ2/df ratio in all models except H-ESEM. This finding supports simulation study results on the sample-size sensitivity of the χ2/df ratio. Second, item number influenced (a) CFI and TLI in all models except ESEM, (b) RMSEA in CFA, H-CFA, and H-ESEM, and (c) SRMR in all models except ESEM and H-ESEM. Third, the number of specific factors influenced (a) CFI and TLI in all models except CFA, H-ESEM, and B-ESEM, (b) RMSEA in the CFA, ESEM, H-ESEM, B-CFA, and B-ESEM representations, and (c) SRMR in the ESEM, H-CFA, B-CFA, and B-ESEM representations.

TABLE 10
www.frontiersin.org

Table 10. Between-Study analysis of the influence of sample size, item number, and specific factor number on model fit.

As a cautionary note, goodness-of-fit is a critical component in evaluating support for a model, but it is not the only one. Particularly when fitting a more parsimonious a priori model with a less parsimonious ex-post-facto model, there is danger in simply comparing fit. Similarly, set-ESEM (Marsh et al., 2020) allows for greater parsimony by keeping theoretically independent dimensions from having cross-loadings. There needs to be some focus on why the bi-factor model is theoretically and substantively more relevant as well as providing a better fit. A saturated model, for example, will necessarily fit better than any of the models, but it would provide an appropriate test of the underlying theoretical model or a substantively relevant model. If the theoretical model is a CFA or ESEM factor model, then strong support for a Bi-ESEM reflects a failure of the a priori prediction. From this perspective, it is important to classify results in relation to the a priori model. Similarly, when comparing nested and non-nested models, the less parsimonious model necessarily fits better for models that do not take into account parsimony. Hence, the size of the difference is relevant, but not necessarily the direction. More interesting are comparisons between non-nested models. These cautionary notes can be useful to avoid over-interpreting goodness-of-fit: even if a Bi-ESEM shows superior fit, it is not necessarily the preferred model which is contingent on theoretical considerations, particularly when bifactor modeling was applied to a single-level rather than a two-level sampling approach (Eid et al., 2017).

Conclusion

This systematic review and meta-analysis aimed to collect and describe studies using a B-ESEM framework in multivariate behavioral research, aggregate their reported model fit, and analyze model fit differences in comparison to reference models and as a function of sample size, item number, and factor number.

This meta-analysis is the first to replicate findings from simulation studies with real data on the superiority of B-ESEM models and examine the relative influence of sample size, item number, and factor number on model fit (Hu and Bentler, 1999; Marsh et al., 2004; Shi et al., 2019). While meta-analyses of structural equation models have been performed (MASEM; e.g., Cheung and Chan, 2005; Reinhold et al., 2018), this meta-analysis is also among the first to synthesize B-ESEM model fit and compare fit between models meta-analytically. This meta-analysis also documented how widespread B-ESEM has been used within the past 6 years since the seminal paper of Morin et al. (2016a) has been published: B-ESEM is now used in multiple domains to explore the multidimensional structure of numerous constructs in educational and psychological research.

Because the review included original empirical reports only—and excluded simulation studies—our examination of the influence of item number and factor number was contingent on and limited to what had been published in original empirical research. Another limitation concerned the domains within which B-ESEM has been applied. Although research in the domains of learning and instruction, motivation and emotion, self and identity, depression and wellbeing, and interpersonal relations covers large areas of educational and psychological research, the findings of this meta-analytic review are limited to these fields and cannot easily be generalized to other domains within which B-ESEM may be applied in the future. Still, we are confident that the present research synthesis—focusing on five fit indices and six reference models, cumulating 158 studies with 308 reports of B-ESEM model fit from a total sample size of 778,624 participants, and examining the relative influence of sample size, item number, and factor number within and between studies—can inform further applications of B-ESEM to identify construct-relevant psychometric multidimensionality. Future research is encouraged to examine how particular constructs and scales in educational and psychological research can be represented with B-ESEM (Morin et al., 2016a).

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Conflict of interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^For reasons of completeness, it is worth mentioning that researchers can also use Bayesian SEM (BSEM; Muthén and Asparouhov, 2012) and set-ESEM (Marsh et al., 2020).

2. ^Despite the widespread use of goodness-of-fit indices to evaluate model fit, many scholars warn against the use of fixed cutoff values (Marsh et al., 2004). McNeish and Wolf (2022) propose dynamic fit index cutoffs as an alternative.

*^Studies preceded by an asterisk were included in the meta-analysis.

References

*Appiah, R., Schutte, L., Fadiji, A. W., Wissing, M. P., and Cromhout, A. (2020). Factorial validity of the Twi versions of five measures of mental health and well-being in Ghana. PLoS ONE 15, e0236707. doi: 10.1371/journal.pone.0236707

PubMed Abstract | CrossRef Full Text | Google Scholar

*Appleton, P. R., Ntoumanis, N., Quested, E., Viladrich, C., and Duda, J. L. (2016). Initial validation of the coach-created empowering and disempowering motivational climate questionnaire (EDMCQ-C). Psychol. Sport Exerc. 22, 53–65. doi: 10.1016/j.psychsport.2015.05.008

CrossRef Full Text | Google Scholar

*Arens, A. K., Jansen, M., Preckel, F., Schmidt, I., and Brunner, M. (2021). The structure of academic self-concept: a methodological review and empirical illustration of central models. Rev. Educ. Res. 91, 34–72. doi: 10.3102/0034654320972186

CrossRef Full Text | Google Scholar

*Arens, A. K., and Morin, A. J. S. (2017). Improved representation of the self-perception profile for children through bifactor exploratory structural equation modeling. Am. Educ. Res. J. 54, 59–87. doi: 10.3102/0002831216666490

CrossRef Full Text | Google Scholar

*Arias, V. B., Jenaro, C., and Ponce, F. P. (2018). Testing the generality of the general factor of personality: an exploratory bifactor approach. Pers. Individ. Dif. 129, 17–23. doi: 10.1016/j.paid.2018.02.042

CrossRef Full Text | Google Scholar

*Arias, V. B., Ponce, F. P., Martinez-Molina, A., Arias, B., and Nunez, D. (2016). General and specific attention-deficit/hyperactivity disorder factors of children 4 to 6 years of age: an exploratory structural equation modeling approach to assessing symptom multidimensionality. J. Abnorm. Psychol. 125, 125–137. doi: 10.1037/abn0000115

PubMed Abstract | CrossRef Full Text | Google Scholar

Asparouhov, T., and Muthén, B. (2009). Exploratory structural equation modeling. Struct. Equ. Model. 16, 397–438. doi: 10.1080/10705510903008204

CrossRef Full Text | Google Scholar

Bandalos, D. L., and Finney, S. J. (2018). “Exploratory and confirmatory factor analysis,” in Quantitative Methods in the Social and Behavioral Sciences: A Guide for Researchers and Reviewers, 2nd Edn, eds G. R. Hancock, and R. O. Mueller (New York, NY: Routledge), 93–114.

*Barbaranelli, C., Fida, R., Paciello, M., and Tramontano, C. (2018). ‘Possunt, quia posse videntur': THEY can because they think they can. Development and validation of the work self-efficacy scale: evidence from two studies. J. Voc. Behav. 106, 249–269. doi: 10.1016/j.jvb.2018.01.006

CrossRef Full Text | Google Scholar

*Bédard-Thom, C., and Guay, F. (2018). Mental toughness among high school students: a test of its multidimensionality and nomological validity with academic achievement and preference for difficult tasks. Soc. Psychol. Educ. 21, 827–848. doi: 10.1007/s11218-018-9437-y

CrossRef Full Text | Google Scholar

*and Benda, J. (2018). Alternative Models of the Czech Version of the Self-Compassion Scale (SCS-26-CZ) (Prague: University of Prague). Unpublished manuscript.

Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychol. Bull. 107, 238–246. doi: 10.1037/0033-2909.107.2.238

PubMed Abstract | CrossRef Full Text | Google Scholar

Bentler, P. M. (1995). EQS Structural Equations Program Manual. Encino, CA: Multivariate Software Inc.

Bentler, P. M., and Bonett, D. G. (1990). Significance tests and goodness of fit in the analysis of covariance structures. Psychol. Bull. 88, 588–606. doi: 10.1037/0033-2909.88.3.588

CrossRef Full Text | Google Scholar

*Bhavsar, N., Bartholomew, K. J., Quested, E., Gucciardi, D. F., Thøgersen-Ntoumani, C., Reeve, J., et al. (2020). Measuring psychological need states in sport: theoretical considerations and a new measure. Psychol. Sport Exerc. 47, 101617. doi: 10.1016/j.psychsport.2019.101617

CrossRef Full Text | Google Scholar

*Bhavsar, N., Ntoumanis, N., Quested, E., Gucciardi, D. F., Thøgersen-Ntoumani, C., Ryan, R. M., et al. (2019). Conceptualizing and testing a new tripartite measure of coach interpersonal behaviors. Psychol. Sport Exerc. 44, 107–120. doi: 10.1016/j.psychsport.2019.05.006

CrossRef Full Text | Google Scholar

*and Bianchi, R. (2020). Do burnout and depressive symptoms form a single syndrome? Confirmatory factor analysis and exploratory structural equation modeling bifactor analysis. J. Psychosom. Res. 131, 109954. doi: 10.1016/j.jpsychores.2020.109954

PubMed Abstract | CrossRef Full Text | Google Scholar

*Bianchi, R., and Schonfeld, I. S. (2020). The occupational depression inventory: a new tool for clinicians and epidemiologists. J. Psychosom. Res. 138, 110249. doi: 10.1016/j.jpsychores.2020.110249

PubMed Abstract | CrossRef Full Text | Google Scholar

*Bianchi, R., and Verkuilen, J. (2021). “Green et al. paranoid thoughts scale”: French validation and development of a brief version. Pers. Indiv. Diff. 171, 110554. doi: 10.1016/j.paid.2020.110554

CrossRef Full Text | Google Scholar

*Borges, L., Nunes Baptista, M., and de Oliveira Serpa, A. L. (2017). Structural analysis of depression indicators scale-children and adolescents (BAID-IJ): a bifactor-ESEM approach. Trends Psychol. 25, 545–552. doi: 10.9788/TP2017.2-08

CrossRef Full Text | Google Scholar

*Burgueño, R., González-Cutre, D., Sevil-Serrano, J., Herrador-Colmenero, M., SeguraDíaz, J. M., Medina-Casaubón, J., et al. (2020a). Validation of the basic psychological need satisfaction in active commuting to and from school (BPNS-ACS) scale in Spanish young people. J. Transp. Health 16, 100825. doi: 10.1016/j.jth.2020.100825

CrossRef Full Text | Google Scholar

*Burgueño, R., Macarro-Moreno, J., and Medina-Casaubón, J. (2020b). Psychometry of the multidimensional perceived autonomy support scale in physical education with Spanish secondary school students. SAGE Open 10, 1–12. doi: 10.1177/2158244019901253

CrossRef Full Text | Google Scholar

*Burk, C. L., and Wiese, B. S. (2018). Professor or manager? A model of motivational orientations applied to preferred career paths. J. Res. Pers. 75, 113–132. doi: 10.1016/j.jrp.2018.06.002

CrossRef Full Text | Google Scholar

*and Calkins, C. M. (2018). Developing and measuring faculty motivation to teach in higher education. (Doctoral thesis), University of Nevada, Reno (Nevada).

*Cano-García, F. J., Muñoz-Navarro, R., Abad, A. S., Moretti, L. S., Medrano, L. A., Ruiz-Rodríguez, P., et al. (2020). Latent structure and factor invariance of somatic symptoms in the patient health questionnaire (PHQ−15). J. Affect. Disord. 261, 21–29. doi: 10.1016/j.jad.2019.09.077

PubMed Abstract | CrossRef Full Text | Google Scholar

*Cece, V., Lienhart, N., Nicaise, V., Guillet-Descas, E., and Martinent, G. (2019). Longitudinal sport motivation among young athletes in intensive training settings: using methodological advances to explore temporal structure of youth behavioral regulation in sport questionnaire scores. J. Sport Exerc. Psychol. 41, 24–35. doi: 10.1123/jsep.2017-0194

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheung, M. W.-L., and Chan, W. (2005). Meta-analytic structural equation modeling: a two-stage approach. Psychol. Methods 10, 40–64. doi: 10.1037/1082-989X.10.1.40

PubMed Abstract | CrossRef Full Text | Google Scholar

*Choisay, F., Fouquereau, E., Coillot, H., and Chevalier, S. (2021). Validation of the french psychological capital questionnaire (F–PCQ−24) and its measurement invariance using bifactor exploratory structural equation modeling framework. Mil. Psychol. 33, 50–65. doi: 10.1080/08995605.2020.1852873

CrossRef Full Text | Google Scholar

*Chung, C. M., Liao, X. L., Song, H. R., and Lee, T. H. (2016). Bifactor approach to modeling multidimensionality of physical self-perception profile. Meas. Phys. Educ. Exerc. Sci. 20, 1–15. doi: 10.1080/1091367X.2015.1081594

CrossRef Full Text | Google Scholar

*Clifton, J., Seehuus, M., Parent, J., Pichler, E., and Fondacaro, K. (2020). Emotional responding: integration of multiple constructs and association with psychological health. J. Clin. Psychol. 76, 699–715. doi: 10.1002/jclp.22885

PubMed Abstract | CrossRef Full Text | Google Scholar

*and Cromhout, A. (2020). Measuring and understanding eudaimonic well-being: A bifactor exploratory structural equation modelling approach. (Doctoral thesis), North-West University, Boloka. Available online at: http://hdl.handle.net/10394/36666 (accessed June 29, 2021).

*Cronin, L. D., and Allen, J. (2017). Development and initial validation of the life skills scale for sport. Psychol. Sport Exerc. 28, 105–119. doi: 10.1016/j.psychsport.2016.11.001

CrossRef Full Text | Google Scholar

Dai, S. S., Xu, H. G., and Chen, F. F. (2019). A hierarchical measurement model of perceived resilience of urban tourism destination. Soc. Indic. Res. 145, 777–804. doi: 10.1007/s11205-019-02117-9

CrossRef Full Text | Google Scholar

*Decroos, S., Lines, R., Morgan, P. B. C., Fletcher, D., Sarkar, M., Fransen, K., et al. (2017). Development and validation of the characteristics of resilience in sports teams inventory. Sport Exerc. Perform. Psychol. 6, 158–178. doi: 10.1037/spy0000089

CrossRef Full Text | Google Scholar

*Deemer, E. D., Smith, J. L., Thoman, D. B., and Chase, J. P. (2014). Precision in career motivation assessment: testing the subjective science attitude change measures. J. Car. Assess. 22, 489–504. doi: 10.1177/1069072713498683

CrossRef Full Text | Google Scholar

*Deller, J., Perrotte, J., Wainwright, K., Brunsman, J., and Osman, A. (2020). Dimensionality, reliability, invariance, and validity of the multidimensional social anxiety response inventory−21 (MSARI−21). J. Pers. Assess. 102, 527–537. doi: 10.1080/00223891.2019.1569529

PubMed Abstract | CrossRef Full Text | Google Scholar

*Díaz-Morales, J. F., and Parra-Robledo, Z. (2018). Age and sex differences in morningness/eveningness along the life span: a cross-sectional study in Spain. J. Genet. Psychol. 179, 71–84. doi: 10.1080/00221325.2018.1424706

PubMed Abstract | CrossRef Full Text | Google Scholar

*Dierendonck, C., Tóth-Király, I., Morin, A. J. S., Kerger, S., Milmeister, P, and Poncelet, D. (2022). Testing associations between global and specific levels of student academic motivation and engagement in the classroom. J. Exp. Educ. doi: 10.1080/00220973.2021.1913979

CrossRef Full Text | Google Scholar

*Doherty, A. S., Mallett, J., Leiter, M. P., and McFadden, P. (2021). Measuring burnout in social work factorial validity of the Maslach burnout inventory – human Services Survey. Euro. J. Psychol. Assess. 37, 6–14. doi: 10.1027/1015-5759/a000568

PubMed Abstract | CrossRef Full Text | Google Scholar

*Dominguez-Lara, S., Fernández-Arata, M., Cesar, M. S., Navarro-Loli, J. S., and Calderón-De la Cruz, G. (2019). Escala de autoeficacia docente: análisis estructural e invarianza de medición en docentes peruanos de escuelas públicas [Teacher's self-efficacy scale: Structural analysis and measurement invariance in Peruvian teachers of public schools]. Rev. Argent. Cienc. Comport. 11, 61–72. doi: 10.32348/1852.4206.v11.n3.24624

CrossRef Full Text | Google Scholar

Eid, M., Geiser, C., Koch, T., and Heene, M. (2017). Anomalous results in G-factor models: explanations and alternatives. Psychol. Methods 22, 541–562. doi: 10.1037/met0000083

PubMed Abstract | CrossRef Full Text | Google Scholar

*Esnaola, I., Arias, V. B., Freeman, J., Wang, Y., and Arias, B. (2018). Validity evidence based on internal structure of scores of the emotional quotient inventory: youth version short (EQ-i: YV-S) in a Chinese sample. J. Psychoeduc. Assess. 36, 576–587. doi: 10.1177/0734282916689439

CrossRef Full Text | Google Scholar

*Espinoza, J. A., Meyer, J. P., Anderson, B. K., Vaters, C., and Politis, C. (2018). Evidence for a bifactor structure of the scales of psychological well-being using exploratory structural equation modeling. J. Well Being Assess. 2, 21–40. doi: 10.1007/s41543-018-0008-y

CrossRef Full Text | Google Scholar

*Fadda, D., Quevedo-Aguado, M. P., Cuesta, M. H. B., and Scalas, L. F. (2020a). The multidimensional and hierarchical nature of the questionnaire for eudaimonic wellbeing: a bifactor–ESEM representation in a Spanish Sample. Front. Psychol. 11, 422. doi: 10.3389/fpsyg.2020.00422

PubMed Abstract | CrossRef Full Text | Google Scholar

*Fadda, D., Scalas, L. F., Meleddu, M., and Morin, A. J. S. (2017). A bifactor–ESEM representation of the questionnaire for eudaimonic wellbeing. Pers. Individ. Dif. 116, 216–222. doi: 10.1016/j.paid.2017.04.062

PubMed Abstract | CrossRef Full Text | Google Scholar

*Fadda, D., Scalas, L. F., Morin, A. J. S., Marsh, H. W., and Gaspard, H. (2020b). Value beliefs about math: a bifactor–ESEM representation. Euro. J. Psychol. Assess. 36, 259–268. doi: 10.1027/1015-5759/a000513

CrossRef Full Text | Google Scholar

*Ferentinos, P., Yotsidi, V., Porichi, E., Douzenis, A., Papageorgiou, C., and Stalikas, A. (2019). Well-being in patients with affective disorders compared to nonclinical participants: a multi-model evaluation of the mental health continuum–short form. J. Clin. Psychol. 75, 1585–1612. doi: 10.1002/jclp.22780

PubMed Abstract | CrossRef Full Text | Google Scholar

*Fernandez-Rio, J., Cecchini, J. A., Morgan, K., Méndez-Giménez, A., and Lloyd, R. (2022). Validation of the cooperative learning scale and cooperation global factor using bifactor structural equation modelling. Psicol. Educ. 28, 91–97. doi: 10.5093/psed2021a2

CrossRef Full Text | Google Scholar

*Fresno, A., Arias, V., Nunez, D., Spencer, R., Ramos, N., Espinoza, C., et al. (2020). Using exploratory structural equation modeling (ESEM) to examine the internal structure of posttraumatic stress disorder symptoms. Span. J. Psychol. 23, e48. doi: 10.1017/SJP.2020.46

PubMed Abstract | CrossRef Full Text | Google Scholar

*and Frutos, J. (2019). Evaluación multidimensional de los factores generales y específicos del TDAH en población infantial mediante el enfoque bifactor-ESEM [multidimensional evaluation of the general and specific facors of ADHD in children population with the bifactor-ESEM approach]. Rev. Argent. Clín. Psicol. 28, 967–980. doi: 10.51668/bp.8321105s

CrossRef Full Text | Google Scholar

*and Frutos, J. (2021). Análisis multidimensional del trastorno por déficit de atención e hiperactividad (TDAH) mediante el método bifactor-ESEM [multidimensional analysis of attention deficit hyperactivity disorder (ADHD) using the bifactor–ESEM method]. Behav. Psychol. 29, 95–110.

*García-Garzón, E., Nieto, M. D., Garrido, L. E., and Abad, F. J. (2020). Bi-factor exploratory structural equation modeling done right: Using the SLiDapp application. Psicothema 32, 607–614. doi: 10.7334/psicothema2020.179

PubMed Abstract | CrossRef Full Text | Google Scholar

*and Garn, A. C. (2017). Multidimensional measurement of situational interest in physical education: application of bifactor exploratory structural equation modeling. J. Teach. Phys. Educ. 36, 323–339. doi: 10.1123/jtpe.2017-0035

CrossRef Full Text | Google Scholar

*Garn, A. C., Morin, A. J. S., and Lonsdale, C. (2019). Basic psychological need satisfaction toward learning: a longitudinal test of mediation using bifactor exploratory structural equation modeling. J. Educ. Psychol. 111, 354–372. doi: 10.1037/edu0000283

CrossRef Full Text | Google Scholar

*Garn, A. C., and Webster, E. K. (2021). Bifactor structure and model reliability of the test of gross motor development−3rd edition. J. Sci. Med. Sport 24, 67–73. doi: 10.1016/j.jsams.2020.08.009

PubMed Abstract | CrossRef Full Text | Google Scholar

*Gegenfurtner, A., and Quesada-Pallarès, C. (2022). Toward a multidimensional conceptualization of motivation to transfer training: validation of the transfer motivation questionnaire from a self-determination theory perspective using bifactor-ESEM. Stud. Educ. Eval. 73, 101116. doi: 10.1016/j.stueduc.2021.101116

CrossRef Full Text | Google Scholar

*Gillet, N., Caesens, G., Morin, A. J. S., and Stinglhamber, F. (2019). Complementary variable– and person-centred approaches to the dimensionality of work engagement: a longitudinal investigation. Euro. J. Work Organ. Psychol. 28, 239–258. doi: 10.1080/1359432X.2019.1575364

CrossRef Full Text | Google Scholar

*Gillet, N., Morin, A. J. S., Huart, I., Colombat, P., and Fouquereau, E. (2020). The forest and the trees: investigating the globality and specificity of employees' basic need satisfaction at work. J. Pers. Assess. 102, 702–713. doi: 10.1080/00223891.2019.1591426

PubMed Abstract | CrossRef Full Text | Google Scholar

*Giotsa, A., and Kyriazos, T. A. (2019). Early childhood acceptance rejection questionnaire: psychometric properties of the Greek version. Psychology 10, 722–741. doi: 10.4236/psych.2019.105047

CrossRef Full Text | Google Scholar

*Gomez, R., and Stavropoulos, V. (2021). Confirmatory factor analysis and exploratory structural equation modeling of the structure of attention-deficit/hyperactivity disorder symptoms in adults. Assessment 28, 1570–1582. doi: 10.1177/1073191120905892

PubMed Abstract | CrossRef Full Text | Google Scholar

*Gomez, R., Stavropoulos, V., and Griffiths, M. D. (2020a). Confirmatory factor analysis and exploratory structural equation modelling of the factor structure of the depression anxiety and stress scales−21. PLoS ONE 15, e0233998. doi: 10.1371/journal.pone.0233998

PubMed Abstract | CrossRef Full Text | Google Scholar

*Gomez, R., Stavropoulos, V., Zarate, D., and Palikara, O. (2021). Symptom checklist-90–revised: a structural examination in relation to family functioning. PLoS ONE 16, e0247902. doi: 10.1371/journal.pone.0247902

PubMed Abstract | CrossRef Full Text | Google Scholar

*Gomez, R., Watson, S., and Stavropoulos, V. (2020b). Personality inventory for DSM−5, brief form: factor structure, reliability, and coefficient of congruence. Pers. Disord. Theory Res. Treat. 11, 69–77. doi: 10.1037/per0000364

PubMed Abstract | CrossRef Full Text | Google Scholar

*Goodboy, A. K., Bolkan, S., Brisini, K., and Solomon, D. H. (2021). Relational uncertainty within relational turbulence theory: the bifactor exploratory structural equation model. J. Commun. 71, 403–430.. doi: 10.1093/joc/jqab009

CrossRef Full Text | Google Scholar

*Gordeeva, T. O., Sychev, O. A., and Lynch, M. F. (2020). The construct validity of the Russian version of the modified academic self-regulation questionnaire (SRQ–A) among elementary and middle school children. Psychol. Russia 13, 16–34. doi: 10.11621/pir.2020.0308

CrossRef Full Text | Google Scholar

*Grygiel, P., Humenny, G., and Rebisz, S. (2019). Using the De Jong Gierveld loneliness scale with early adolescents: factor structure, reliability, stability, and external validity. Assessment 26, 151–165. doi: 10.1177/1073191116682298

PubMed Abstract | CrossRef Full Text | Google Scholar

*Gu, H. L., Wen, Z. L., and Fan, X. T. (2017). Structural validity of the Machiavellian personality scale: a bifactor exploratory structural equation modeling approach. Pers. Individ. Dif. 105, 116–123. doi: 10.1016/j.paid.2016.09.042

CrossRef Full Text | Google Scholar

*Gu, H. L., Wen, Z. L., and Fan, X. T. (2020). Investigating the multidimensionality of the work-related flow inventory (WOLF): a bifactor exploratory structural equation modeling framework. Front. Psychol. 11, 740. doi: 10.3389/fpsyg.2020.00740

PubMed Abstract | CrossRef Full Text | Google Scholar

*Guay, F., and Bureau, J. S. (2018). Motivation at school: differentiation between and within school subjects matters in the prediction of academic achievement. Contemp. Educ. Psychol. 54, 42–54. doi: 10.1016/j.cedpsych.2018.05.004

CrossRef Full Text | Google Scholar

*Guay, F., Morin, A. J. S., Litalien, D., Howard, J. L., and Gilbert, W. (2021). Trajectories of self-determined motivation during the secondary school: a growth mixture analysis. J. Educ. Psychol. 113, 390–410. doi: 10.1037/edu0000482

CrossRef Full Text | Google Scholar

*Gucciardi, D. F., Weixian, J. C., Gibson, W., Ntoumanis, N., and Ng, L. (2020). Motivational climate in the classroom factorial and convergent validity: evidence of the need-supportive behaviors scale with health science students. Euro. J. Psychol. Assess. 36, 324–335. doi: 10.1027/1015-5759/a000524

CrossRef Full Text | Google Scholar

*Halamová, J., Kanovský, M., and Pacúchová, M. (2020). Psychometric analysis of the Slovak version of the compassionate engagement and action scales. J. Psychol. Educ. Res. 28, 64–80.

*Hanley, A. W., Bernstein, A., Nakamura, Y., Hadash, Y., Rojas, J., Tennant, K. E., et al. (2020). The metacognitive processes of decentering scale: development and initial validation of trait and state versions. Psychol. Assess. 32, 956–971. doi: 10.1037/pas0000931

PubMed Abstract | CrossRef Full Text | Google Scholar

*Haugen, T., Peters, D. M., Ommundsen, Y., Martin, L. J., Stenling, A., and Høigaard, R. (2021). Psychometric evaluation of the Norwegian versions of the modified group environment questionnaire and the youth sport environment questionnaire. Meas. Phys. Educ. Exerc. Sci. 25, 365–378. doi: 10.1080/1091367X.2021.1917414

CrossRef Full Text | Google Scholar

*Hoi, V. N., and Hang, H. L. (2021). The structure of student engagement in online learning: a bi-factor exploratory structural equation modelling approach. J. Comput. Assist. Learn. 37,1141–1153. doi: 10.1111/jcal.12551

CrossRef Full Text | Google Scholar

*Høstmælingen, A., Ulvenes, P., Nissen-Lie, H. A., Eielsen, M., and Wampold, B. E. (2021). Do self-criticism and somatic symptoms play a key role in chronic depression? Exploring the factor structure of beck depression inventory–II in a sample of chronically depressed inpatients. J. Affect. Disord. 283, 317–324. doi: 10.1016/j.jad.2021.01.066

PubMed Abstract | CrossRef Full Text | Google Scholar

*Howard, J. L., Gagn,é, M., Van den Broeck, A., Guay, F., Chatzisarantis, N., Ntoumanis, N., et al. (2020). A review and empirical comparison of motivation scoring methods: an application to self-determination theory. Motiv. Emot. 44, 534–548. doi: 10.1007/s11031-020-09831-9

CrossRef Full Text | Google Scholar

*Howard, J. L., Gagné, M., Morin, A. J. S., and Forest, J. (2018). Using bifactor exploratory structural equation modeling to test for a continuum structure of motivation. J. Manage. 44, 2638–2664. doi: 10.1177/0149206316645653

PubMed Abstract | CrossRef Full Text | Google Scholar

*Howard, J. L., Morin, A. J. S., and Gagné, M. (2021). A longitudinal analysis of motivation profiles at work. Motiv. Emot. 45, 39–59. doi: 10.1007/s11031-020-09852-4

CrossRef Full Text | Google Scholar

Hu, L. T., and Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Model. 6, 1–55. doi: 10.1080/10705519909540118

CrossRef Full Text | Google Scholar

*Hukkelberg, S., and Ogden, T. (2019). General and specific factors of working alliance in parent training: a bifactor exploratory structural equation modelling approach. Psychother. Res. 29, 267–276. doi: 10.1080/10503307.2017.1330574

PubMed Abstract | CrossRef Full Text | Google Scholar

*Hukkelberg, S., and Ogden, T. (2020). What is social competence? An investigation into the concept among children with antisocial behaviours. Emot. Behav. Diff. 25, 80–93. doi: 10.1080/13632752.2019.1687168

CrossRef Full Text | Google Scholar

*and Hukkelberg, S. S. (2019). A reexamination of child problem behaviors as measured by ECBI: factor structure and measurement invariance across two parent training interventions. Assessment 26, 1270–1281. doi: 10.1177/1073191117706022

PubMed Abstract | CrossRef Full Text | Google Scholar

*Huyghebaert-Zouaghi, T., Berjot, S., Cougot, B., and Gillet, N. (2021a). Psychological and relational conditions for job crafting to occur. Stress Health 37, 516–527. doi: 10.1002/smi.3014

PubMed Abstract | CrossRef Full Text | Google Scholar

*Huyghebaert-Zouaghi, T., Caesens, G., Sandrin, É., and Gillet, N. (2022). Workaholism and work engagement: an examination of their psychometric multidimensionality and relations with employees' functioning. Curr. Psychol. doi: 10.1007/s12144-021-01820-6

CrossRef Full Text | Google Scholar

*Huyghebaert-Zouaghi, T., Ntoumanis, N., Berjot, S., and Gillet, N. (2021b). Advancing the conceptualization and measurement of psychological need states: a 3 x 3 model based on self-determination theory. J. Car. Assess. 29, 396–421. doi: 10.1177/1069072720978792

CrossRef Full Text | Google Scholar

*Isoard-Gautheur, S., Martinent, G., Guillet-Descas, E., Trouilloud, D., Cece, V., and Mette, A. (2018). Development and evaluation of the psychometric properties of a new measure of athlete burnout: the athlete burnout scale. Int. J. Stress Manag. 25, 108–123. doi: 10.1037/str0000083

CrossRef Full Text | Google Scholar

*and Jaotombo, F. F. (2019). Le fonctionnement optimal psychologique: apports conceptuels et méthodologiques [Optimal psychological functioning: conceptual and methodological contributions]. Psychol. Trav. Organ. 25, 281–300. doi: 10.1016/j.pto.2019.06.001

CrossRef Full Text | Google Scholar

*Jastrzebski, J., Rogoza, R., and Slaski, S. (2020). The hierarchical structure of fear of personal death: from the general factor to specific forms. Psicol. Reflexao Crit. 33, 16. doi: 10.1186/s41155-020-00152-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika 34, 183–202. doi: 10.1007/BF02289343

CrossRef Full Text | Google Scholar

*Jovanovic, V., Gavrilov-Jerkovic, V., and Lazic, M. (2021). Can adolescents differentiate between depression, anxiety and stress? Testing competing models of the depression anxiety stress scales (DASS−21). Curr. Psychol. 40, 6045–6056. doi: 10.1007/s12144-019-00540-2

CrossRef Full Text | Google Scholar

*Junga, Y. M., Witthoft, M., and Weck, F. (2019). Assessing therapist development: reliability and validity of the supervisee levels questionnaire (SLQ–R). J. Clin. Psychol. 75, 1658–1672. doi: 10.1002/jclp.22794

PubMed Abstract | CrossRef Full Text | Google Scholar

*and Kartal, S. K. (2020). Examining factors for the academic motivation based on the confirmatory, the exploratory, and the bifactor exploratory structural equation modeling. Int. J. Prog. Educ. 16, 192–204. doi: 10.29329/ijpe.2020.228.14

CrossRef Full Text | Google Scholar

*Kawabata, M., Pavey, T. G., and Coulter, T. J. (2021). Evolving the validity of a mental toughness measure: refined versions of the mental toughness questionnaire-48. Stress Health 37, 378–391. doi: 10.1002/smi.3004

PubMed Abstract | CrossRef Full Text | Google Scholar

Koo, T. K., and Li, M. Y. (2016). A guideline for selecting and reporting intraclass correlation coefficients for reliability research. J. Chiropr. Med. 15, 155–163. doi: 10.1016/j.jcm.2016.02.012

PubMed Abstract | CrossRef Full Text | Google Scholar

*Korol, L., Fietzer, A. W., and Ponterotto, J. G. (2018). The relationship between multicultural personality, intergroup contact, and positive outgroup attitudes toward Asian Americans. Asian Am. J. Psychol. 9, 200–210. doi: 10.1037/aap0000107

CrossRef Full Text | Google Scholar

*Krafft, A., Guse, T., and Maree, D. (2020). Distinguishing perceived hope and dispositional optimism: theoretical foundations and empirical findings beyond future expectancies and cognition. J. Well Being Assess. 4, 217–243. doi: 10.1007/s41543-020-00030-4

CrossRef Full Text | Google Scholar

*Kyriazos, T. A., Stalikas, A., Prassa, K., Galanakis, M., Flora, K., and Chatzilia, V. (2018b). The flow short scale (FSS) dimensionality and what MIMIC shows on heterogeneity and invariance. Psychology 9, 1357–1382. doi: 10.4236/psych.2018.96083

CrossRef Full Text | Google Scholar

*Kyriazos, T. A., Stalikas, A., Prassa, K., and Yotsidi, V. (2018a). A 3-faced construct validation and a bifactor subjective well-being model using the scale of positive and negative experience, Greek version. Psychology 9, 1143–1175. doi: 10.4236/psych.2018.95071

CrossRef Full Text | Google Scholar

*Lahey, B. B., Zald, D. H., Perkins, S. F., Villalta-Gil, V., Werts, K. B., Van Hulle, C. A., et al. (2018). Measuring the hierarchical general factor model of psychopathology in young adults. Int. J. Methods Psychiatr. Res. 27, e1593. doi: 10.1002/mpr.1593

PubMed Abstract | CrossRef Full Text | Google Scholar

*Lamborn, P., Cramer, K. M., and Riberdy, A. (2018). The structural validity and measurement invariance of the mental health continuum–short form (MHC–SF) in a large Canadian same. J. Well Being Assess 2, 1–19. doi: 10.1007/s41543-018-0007-z

CrossRef Full Text | Google Scholar

Landis, J. R., and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics 33, 159–174. doi: 10.2307/2529310

PubMed Abstract | CrossRef Full Text | Google Scholar

*Lauriola, M., Donati, M. A., Trentini, C., Tomai, M., Pontone, S., and Baker, R. (2021). The structure of the Emotional processing scale (EPS-25): an exploratory structural equation modeling analysis using medical and community samples. Euro. J. Psychol. Assess. 37, 423–432. doi: 10.1027/1015-5759/a000632

CrossRef Full Text | Google Scholar

*Lecerf, T., and Canivez, G. L. (2018). Complementary exploratory and confirmatory factor analyses of the French WISC–V: analyses based on the standardization sample. Psychol. Assess. 30, 793–808. doi: 10.1037/pas0000526

PubMed Abstract | CrossRef Full Text | Google Scholar

*Lee, P., Mahoney, K. T., and Lee, S. (2017). An application of the exploratory structural equation modeling framework to the study of personality faking. Pers. Individ. Dif. 119, 220–226. doi: 10.1016/j.paid.2017.07.029

CrossRef Full Text | Google Scholar

*Leo, F. M., González-Ponce, I., Sánchez-Oliva, D., Pulido, J. J., and García-Calvo, T. (2017). Role ambiguity: translation to Spanish and analysis of scale structure. Small Group Res. 48, 365–385. doi: 10.1177/1046496417706554

CrossRef Full Text | Google Scholar

Leys, C., Ley, C., Klein, O., Bernard, P., and Licata, L. (2013). Detecting outliers: do not use standard deviation around the mean, use absolute deviation around the median. J. Exp. Soc. Psychol. 49, 764–766. doi: 10.1016/j.jesp.2013.03.013

CrossRef Full Text | Google Scholar

*Litalien, D., Morin, A. J. S., Gagn,é, M., Vallerand, R. J., Losier, G. F., and Ryan, R. M. (2017). Evidence of a continuum structure of academic self–determination: a two-study test using a bifactor-ESEM representation of academic motivation. Contemp. Educ. Psychol. 51, 67–82. doi: 10.1016/j.cedpsych.2017.06.010

CrossRef Full Text | Google Scholar

*Lohbeck, A., and Petermann, F. (2019). Factorial validity of the anxiety questionnaire for students (AFS): bifactor modeling and measurement invariance. J. Psychoeduc. Assess. 37, 770–781. doi: 10.1177/0734282918794834

CrossRef Full Text | Google Scholar

*Longo, Y., Jovanovic, V., de Carvalho, J. S., and Karas, D. (2020). The general factor of well-being: multinational evidence using bifactor ESEM on the mental health continuum–short form. Assessment 27, 596–606. doi: 10.1177/1073191117748394

PubMed Abstract | CrossRef Full Text | Google Scholar

*Maïano, C., Morin, A. J. S., Aimé, A., Lepage, G., and Bouchard, S. (2021). Psychometric properties of the body checking questionnaire (BCQ) and of the body checking cognitions scale (BCCS): a bifactor-exploratory structural equation modeling approach. Assessment 28, 632–646. doi: 10.1177/1073191119858411

PubMed Abstract | CrossRef Full Text | Google Scholar

Marsh, H. W. (2007). “Application of confirmatory factor analysis and structural equation modeling in sport/exercise psychology,” in Handbook of Sport Psychology, eds G. Tenenbaum, and R. C. Eklund (New York, NY: Wiley), 774–798. doi: 10.1002/9781118270011.ch35

CrossRef Full Text | Google Scholar

Marsh, H. W., Guo, J., Dicke, T., Parker, P. D., and Craven, R. G. (2020). Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and set-ESEM: optimal balance between goodness of fit and parsimony. Multivariate Behav. Res. 55, 102–119. doi: 10.1080/00273171.2019.1602503

PubMed Abstract | CrossRef Full Text | Google Scholar

Marsh, H. W., Hau, K.-T., and Wen, Z. (2004). In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Struct. Equ. Model. 11, 320–341. doi: 10.1207/s15328007sem1103_2

CrossRef Full Text | Google Scholar

Marsh, H. W., Morin, A. J. S., Parker, P. D., and Kaur, G. (2014). Exploratory structural equation modeling: an integration of the best features of exploratory and confirmatory factor analysis. Annu. Rev. Clin. Psychol. 10, 85–110. doi: 10.1146/annurev-clinpsy-032813-153700

PubMed Abstract | CrossRef Full Text | Google Scholar

Marsh, H. W., Muthén, B., Asparouhov, A., Lüdtke, O., Robitzsch, A., Morin, A. J. S., et al. (2009). Exploratory structural equation modeling, integrating CFA and EFA: application to students' evaluations of university teaching. Struct. Equ. Model. 16, 439–476. doi: 10.1080/10705510903008220

CrossRef Full Text | Google Scholar

*McKay, M. T., Perry, J. L., Percy, A., and Cole, J. C. (2016). Evidence for the reliability and validity, and some support for the practical utility of the two-factor consideration of future consequences scale−14. Pers. Individ. Dif. 98, 133–136. doi: 10.1016/j.paid.2016.03.097

CrossRef Full Text | Google Scholar

*McLarnon, M. J. W., and Tarraf, R. C. (2017). The dark triad: specific or general sources of variance? A bifactor exploratory structural equation modeling approach. Pers. Indiv. Diff. 112, 67–73. doi: 10.1016/j.paid.2017.02.049

CrossRef Full Text | Google Scholar

*McLarnon, M. J. W., and Tarraf, R. C. (2021). Getting to the core: how “(dis)honest” is the core of the dark triad? Pers. Individ. Dif. 171, 110545. doi: 10.1016/j.paid.2020.110545

CrossRef Full Text | Google Scholar

McNeish, D., and Wolf, M. G. (2022). Dynamic fit index cutoffs for confirmatory factor analysis models. Psychol. Methods. doi: 10.1037/met0000425

PubMed Abstract | CrossRef Full Text | Google Scholar

*Méndez-Giménez, A., Cecchini-Estrada, J. A., and Rodríguez-González, P. (2020). Competencia percibida (tridimensional), regulaciones motivacionales y autoeficacia en educación física [perceived competence (three-dimensional), motivational regulations and self-efficacy in physical education]. Rev. Latinoam. Psicol. 52, 51–62. doi: 10.14349/rlp.2020.v52.6

CrossRef Full Text | Google Scholar

Miller, J. (1991). Reaction time analysis with outlier exclusion: bias varies with sample size. Q. J. Exp. Psychol. 43, 907–912. doi: 10.1080/14640749108400962

PubMed Abstract | CrossRef Full Text | Google Scholar

*Milton, D., Appleton, P. R., Bryant, A., and Duda, J. L. (2018). Initial validation of the teacher-created empowering and disempowering motivational climate questionnaire in physical education. J. Teach. Phys. Educ. 37, 340–351. doi: 10.1123/jtpe.2018-0119

CrossRef Full Text | Google Scholar

*Morin, A. J. S., Arens, A. K., and Marsh, H. W. (2016a). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Struct. Equ. Model. 23, 116–139. doi: 10.1080/10705511.2014.961800

CrossRef Full Text | Google Scholar

*Morin, A. J. S., Arens, A. K., Tran, A., and Caci, H. (2016b). Exploring sources of construct-relevant multidimensionality in psychiatric measurement: a tutorial and illustration using the composite scale of morningness. Int. J. Methods Psychiatr. Res. 25, 277–288. doi: 10.1002/mpr.1485

PubMed Abstract | CrossRef Full Text | Google Scholar

*Morin, A. J. S., Boudrias, J. S., Marsh, H. W., Madore, I., and Desrumaux, P. (2016c). Further reflections on disentangling shape and level effects in person-centered analyses: an illustration exploring the dimensionality of psychological health. Struct. Equ. Model. 23, 438–454. doi: 10.1080/10705511.2015.1116077

CrossRef Full Text | Google Scholar

*Morin, A. J. S., Boudrias, J. S., Marsh, H. W., McInerney, D. M., Dagenais–Desmarais, V., Madore, I., et al. (2017). Complementary variable- and person-centered approaches to the dimensionality of psychometric constructs: application to psychological wellbeing at work. J. Bus. Psychol. 32, 395–419. doi: 10.1007/s10869-016-9448-7

CrossRef Full Text | Google Scholar

Morin, A. J. S., Marsh, H. W., and Nagengast, B. (2013). “Exploratory structural equation modeling,” in Structural Equation Modeling: A Second Course, 2nd Edn, eds G. R. Hancock, and R. O. Mueller (Charlotte, NC), 395–436.

Google Scholar

Morin, A. J. S., Myers, N. D., and Lee, S. (2020). “Modern factor analytic techniques: bifactor models, exploratory structural equation modeling (ESEM) and bifactor-ESEM,” in Handbook of Sport Psychology, Vol. 2, 4th Edn, eds G. Tenenbaum, and R. C. Eklund (London: Wiley), 1044–1073. doi: 10.1002/9781119568124.ch51

CrossRef Full Text | Google Scholar

Muthén, B., and Asparouhov, T. (2012). Bayesian structural equation modeling: a more flexible representation of substantive theory. Psychol. Methods 17, 313–335. doi: 10.1037/a0026802

PubMed Abstract | CrossRef Full Text | Google Scholar

*Myers, N. D., Park, S. E., Lefevor, G. T., Dietz, S., Prilleltensky, I., and Prado, G. J. (2016). Measuring multidimensional subjective well-being with the I COPPE scale in a hispanic sample. Meas. Phys. Educ. Exerc. Sci. 20, 230–243. doi: 10.1080/1091367X.2016.1226836

PubMed Abstract | CrossRef Full Text | Google Scholar

*Neff, K. D., Bluth, K., Tóth-Király, I., Davidson, O., Knox, M. C., Williamson, Z., et al. (2021a). Development and validation of the self-compassion scale for youth. J. Pers. Assess. 103, 92–105. doi: 10.1080/00223891.2020.1729774

PubMed Abstract | CrossRef Full Text | Google Scholar

*Neff, K. D., Tóth-Király, I., and Colosimo, K. (2018). Self-compassion is best measured as a global construct and is overlapping with but distinct from neuroticism: a response to Pfattheicher, Geiger, Hartung, Weiss, and Schindler (2017). Eur. J. Pers. 32, 371–392. doi: 10.1002/per.2148

CrossRef Full Text | Google Scholar

*Neff, K. D., Tóth-Király, I., Knox, M. C., Kuchar, A., and Davidson, O. (2021b). The development and validation of the state self–compassion scale (long- and short form). Mindfulness 12, 121–140. doi: 10.1007/s12671-020-01505-4

CrossRef Full Text | Google Scholar

*Neff, K. D., Tóth-Király, I., Yarnell, L. M., Arimitsu, K., Castilho, P., Ghorbani, N., et al. (2019). Examining the factor structure of the self-compassion scale in 20 diverse samples: support for use of a total score and six subscale scores. Psychol. Assess. 31, 27–45. doi: 10.1037/pas0000629

PubMed Abstract | CrossRef Full Text | Google Scholar

*Ng, V., Cao, M. Y., Marsh, H. W., Tay, L., and Seligman, M. E. P. (2017). The factor structure of the values in action inventory of strengths (VIA–IS): an item–level exploratory structural equation modeling (ESEM) bifactor analysis. Psychol. Assess. 29, 1053–1058. doi: 10.1037/pas0000396

PubMed Abstract | CrossRef Full Text | Google Scholar

*Nixon, N., Guo, B. L., Garland, A., Kaylor-Hughes, C., Nixon, E., and Morriss, R. (2020). The bi-factor structure of the 17-item hamilton depression rating scale in persistent major depression; dimensional measurement of outcome. PLoS ONE 15, e0241370. doi: 10.1371/journal.pone.0241370

PubMed Abstract | CrossRef Full Text | Google Scholar

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372, 71. doi: 10.1136/bmj.n71

PubMed Abstract | CrossRef Full Text | Google Scholar

*Part, R., Perera, H. N., Marchand, G. C., and Bernacki, M. L. (2020). Revisiting the dimensionality of subjective task value: towards clarification of competing perspectives. Contemp. Educ. Psychol. 62, 101875. doi: 10.1016/j.cedpsych.2020.101875

CrossRef Full Text | Google Scholar

Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Lond. Edinburgh Dublin Philos. Mag. J. Sci. 50, 157–175. doi: 10.1080/14786440009463897

CrossRef Full Text | Google Scholar

*and Perera, H. N. (2016). Construct validity of the social provisions scale: a bifactor exploratory structural equation modeling approach. Assessment 23, 720–733. doi: 10.1177/1073191115589344

PubMed Abstract | CrossRef Full Text | Google Scholar

*Perera, H. N., and Ganguly, R. (2018). Construct validity of scores from the Connor–Davidson resilience scale in a sample of postsecondary students with disabilities. Assessment 25, 193–205. doi: 10.1177/1073191116646444

PubMed Abstract | CrossRef Full Text | Google Scholar

*Perera, H. N., Izadikhah, Z., O'Connor, P., and McIlveen, P. (2018a). Resolving dimensionality problems with WHOQOL–BREF item responses. Assessment 25, 1014–1025. doi: 10.1177/1073191116678925

PubMed Abstract | CrossRef Full Text | Google Scholar

*Perera, H. N., Vosicka, L., Granziera, H., and McIlveen, P. (2018b). Towards an integrative perspective on the structure of teacher work engagement. J. Vocat. Behav. 108, 28–41. doi: 10.1016/j.jvb.2018.05.006

CrossRef Full Text | Google Scholar

*Perreira, T. A., Morin, A. J. S., Hebert, M., Gillet, N., Houle, S. A., and Berta, W. (2018). The short form of the workplace affective commitment multidimensional questionnaire (WACMQ–S): a bifactor–ESEM approach among healthcare professionals. J. Vocat. Behav. 106, 62–83. doi: 10.1016/j.jvb.2017.12.004

CrossRef Full Text | Google Scholar

*Pirsoul, T., Parmentier, M., and Nils, F. (2022). One step beyond emotional intelligence measurement in the career development of adult learners: a bifactor exploratory structural equation modeling framework. Curr. Psychol. doi: 10.1007/s12144-021-01772-x

CrossRef Full Text | Google Scholar

*Pommier, E., Neff, K. D., and Tóth-Király, I. (2020). The development and validation of the compassion scale. Assessment 27, 21–39. doi: 10.1177/1073191119874108

PubMed Abstract | CrossRef Full Text | Google Scholar

*Portoghese, I., Porru, F., Galletta, M., Campagna, M., and Burdorf, A. (2020). Stress among medical students: factor structure of the university stress scale among Italian students. BMJ Open 10, e035255. doi: 10.1136/bmjopen-2019-035255

PubMed Abstract | CrossRef Full Text | Google Scholar

*Ratelle, C. F., Duchesne, S., Guay, F., and Châteauvert, G. B. (2018). Comparing the contribution of overall structure and its specific dimensions for competence–related constructs: a bifactor model. Contemp. Educ. Psychol. 54, 89–98. doi: 10.1016/j.cedpsych.2018.05.005

CrossRef Full Text | Google Scholar

*Reinhardt, M., Horváth, Z., Morgan, A., and Kökönyei, G. (2020a). Well-being profiles in adolescence: psychometric properties and latent profile analysis of the mental health continuum model—a methodological study. Health Qual. Life Outcomes 18, 95. doi: 10.1186/s12955-020-01332-0

PubMed Abstract | CrossRef Full Text | Google Scholar

*Reinhardt, M., Horváth, Z., Tóth, L., and Kökönyei, G. (2020b). A mentális egészség kontinuum skála rövid változatának hazai validációja. Magyar Pszichol. Szemle 75, 217–246. doi: 10.1556/0016.2020.00014

CrossRef Full Text | Google Scholar

Reinhold, S., Gegenfurtner, A., and Lewalter, D. (2018). Social support and motivation to transfer as predictors of training transfer: testing full and partial mediation using meta-analytic structural equation modeling. Int. J. Train. Dev. 22, 1–14. doi: 10.1111/ijtd.12115

CrossRef Full Text | Google Scholar

Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behav. Res. 47, 667–669. doi: 10.1080/00273171.2012.715555

PubMed Abstract | CrossRef Full Text | Google Scholar

Rindskopf, D., and Rose, T. (1988). Some theory and applications of confirmatory second-order factor analyses. Multivariate Behav. Res. 23, 51–67. doi: 10.1207/s15327906mbr2301_3

PubMed Abstract | CrossRef Full Text | Google Scholar

*Rodenacker, K., Hautmann, C., Gortz–Dorten, A., and Dopfner, M. (2017). The factor structure of ADHD—different models, analyses and informants in a bifactor framework. J. Psychopathol. Behav. Assess. 39, 92–102. doi: 10.1007/s10862-016-9565-7

CrossRef Full Text | Google Scholar

*Rodrigues, F., Cid, L., Teixeira, D., and Monteiro, D. (2021). Re-applying the basic psychological needs in exercise scale to various Portuguese exercise groups: an analysis of bifactor models and contextual invariance. Percept. Mot. Skills 128, 1660–1683. doi: 10.1177/00315125211016803

PubMed Abstract | CrossRef Full Text | Google Scholar

*Rodrigues, F., Macedo, R., Teixeira, D. S., Cid, L., and Monteiro, D. (2020). Motivation in sport and exercise: a comparison between the BRSQ and BREQ. Qual. Quant. 54, 1335–1350. doi: 10.1007/s11135-020-00988-6

CrossRef Full Text | Google Scholar

*Rogoza, R., Thi, K. H. T., Rózycka-Tran, J., Piotrowski, J., and Zemojtel-Piotrowska, M. (2018). Psychometric properties of the MHC–SF: an integration of the existing measurement approaches. J. Clin. Psychol. 74, 1742–1758. doi: 10.1002/jclp.22626

PubMed Abstract | CrossRef Full Text | Google Scholar

*Sakakibara, K., Shimazu, A., Toyama, H., and Schaufeli, W. B. (2020). Validation of the Japanese version of the burnout assessment tool. Front. Psychol. 11, 1819. doi: 10.3389/fpsyg.2020.01819

PubMed Abstract | CrossRef Full Text | Google Scholar

*Sánchez-Oliva, D., Morin, A. J. S., Teixeira, P. J., Carraça, E. V., Palmeira, A. L., and Silva, M. N. (2017). A bifactor exploratory structural equation modeling representation of the structure of the basic psychological needs at work scale. J. Vocat. Behav. 98, 173–187. doi: 10.1016/j.jvb.2016.12.001

CrossRef Full Text | Google Scholar

*Scherer, R., Nilsen, T., and Jansen, M. (2016). Evaluating individual students' perceptions of instructional quality: an investigation of their factor structure, measurement invariance, and relations to educational outcomes. Front. Psychol. 7, 110. doi: 10.3389/fpsyg.2016.00110

PubMed Abstract | CrossRef Full Text | Google Scholar

*Schmid, J., Steiner, S., Rensch, M., Middleton, C., and Seiler, R. (2018). Psychometrische eigenschaften einer deutschsprachigen übersetzung des mental toughness inventory (MTI-D) [psychometric properties of a German translation of the mental toughness inventory (MIT-D)]. Diagnostica 64, 61–73. doi: 10.1026/0012-1924/a000192

CrossRef Full Text | Google Scholar

*Schutte, L., and Wissing, M. P. (2017). Clarifying the factor structure of the mental health continuum short form in three languages: a bifactor exploratory structural equation modeling approach. Soc. Ment. Health 7, 142–158. doi: 10.1177/2156869317707793

CrossRef Full Text | Google Scholar

*Sellbom, M., and Tellegen, A. (2019). Factor analysis in psychological assessment research: common pitfalls and recommendations. Psychol. Assess. 31, 1428–1441. doi: 10.1037/pas0000623

PubMed Abstract | CrossRef Full Text | Google Scholar

*Sen, R., Pleil, A. M., Coon, C., and Shields, A. L. (2015). Evaluating the dimensionality of complex pros using bifactor analysis within an exploratory structural equation modeling (ESEM) framework: An example using the patient-reported scar evaluation questionnaire (PR–SEQ). Quality of Life Research 24, 26.

Google Scholar

Shi, D., Lee, T., and Maydeu-Olivares, A. (2019). Understanding the model size effect on SEM fit indices. Educ. Psychol. Meas. 79, 310–334. doi: 10.1177/0013164418783530

PubMed Abstract | CrossRef Full Text | Google Scholar

*Silverman, A. L., Foregard, M., Beard, C., and Björgvinsson, T. (2018). Psychometric properties of the mental health continuum—short form in a psychiatric sample. Journal of Well Being Assess. 2, 57–73. doi: 10.1007/s41543-018-0011-3

CrossRef Full Text | Google Scholar

*Somma, A., Borroni, S., Drislane, L. E., Patrick, C. J., and Fossati, A. (2019). Modeling the structure of the triarchic psychopathy measure: comceptual, empirical, and analytical considerations. J. Pers. Disord. 33, 470–496. doi: 10.1521/pedi_2018_32_354

PubMed Abstract | CrossRef Full Text | Google Scholar

Stanley, T. D., and Doucouliagos, H. (2017). Neither fixed nor random: weighted least squares meta-regression. Res. Synth. Methods 8, 19–42. doi: 10.1002/jrsm.1211

PubMed Abstract | CrossRef Full Text | Google Scholar

Steiger, J. H. (1990). Structural model evaluation and modification: an interval estimation approach. Multivariate Behav. Res. 25, 173–180. doi: 10.1207/s15327906mbr2502_4

PubMed Abstract | CrossRef Full Text | Google Scholar

*Stenling, A., Ivarsson, A., Hassmén, P., and Lindwall, M. (2015). Using bifactor exploratory structural equation modeling to examine global and specific factors in measures of sports coaches' interpersonal styles. Front. Psychol. 6, 1303. doi: 10.3389/fpsyg.2015.01303

PubMed Abstract | CrossRef Full Text | Google Scholar

*Stenling, A., Ivarsson, A., Lindwall, M., and Gucciardi, D. F. (2018). Exploring longitudinal measurement invariance and the continuum hypothesis in the Swedish version of the behavioral regulation in sport questionnaire (BRSQ): an exploratory structural equation modeling approach. Psychol. Sport Exerc. 36, 187–196. doi: 10.1016/j.psychsport.2018.03.002

CrossRef Full Text | Google Scholar

*Styck, K. M., Rodriguez, M. C., and Yi, E. H. (2021). Dimensionality of the state-trait inventory of cognitive and somatic anxiety. Assessment 29, 103–127. doi: 10.1177/1073191120953628

PubMed Abstract | CrossRef Full Text | Google Scholar

*Summers, J. J., and Falco, L. D. (2020). The development and validation of a new measure of adolescent purpose. J. Exp. Educ. 88, 47–71. doi: 10.1080/00220973.2019.1575178

CrossRef Full Text | Google Scholar

*and Sutherland, E. A. (2020). The measurement of job satisfaction among workplace leaders: Scale development and validation of the leader satisfaction assessment. (Doctoral thesis), Western University. Available online at: https://ir.lib.uwo.ca/etd/7006 (accessed June 29, 2021).

Google Scholar

*Tomás, J. M., Gutiérrez, M., Alberola, S., and Georgieva, S. (2022). Psychometric properties of two major approaches to measure school engagement in university students. Curr. Psychol. 41, 2654–2667. doi: 10.1007/s12144-020-00769-2

CrossRef Full Text | Google Scholar

*Tóth-Király, I., Bothe, B., and Orosz, G. (2017). Exploratory structural equation modeling analysis of the self–compassion scale. Mindfulness 8, 881–892. doi: 10.1007/s12671-016-0662-1

CrossRef Full Text | Google Scholar

*Tóth-Király, I., Böthe, B., Orosz, G., and Rigo, A. (2019). A new look on the representation and criterion validity of need fulfillment: application of the bifactor exploratory structural equation modeling framework. J. Happiness Stud. 20, 1609–1626. doi: 10.1007/s10902-018-0015-y

CrossRef Full Text | Google Scholar

*Tóth-Király, I., Morin, A. J. S., Böthe, B., Orosz, G., and Rigó, A. (2018). Investigating the multidimensionality of need fulfillment: a bifactor exploratory structural equation modeling representation. Struct. Equ. Model. 25, 267–286. doi: 10.1080/10705511.2017.1374867

PubMed Abstract | CrossRef Full Text | Google Scholar

*Tóth-Király, I., Morin, A. J. S., and Salmela-Aro, K. (2021). Reciprocal associations between burnout and depression: An 8-year longitudinal study. Appl. Psychol. Int. Rev. 70, 1691–1727. doi: 10.1111/apps.12295

CrossRef Full Text | Google Scholar

*Trógolo, M. A., Morera, L. P., Castellano, E., Spontón, C., and Medrano, L. A. (2020). Work engagement and burnout: real, redundant, or both? A further examination using a bifactor modelling approach. Euro. J. Work Organ. Psychol. 29, 922–937. doi: 10.1080/1359432X.2020.1801642

CrossRef Full Text | Google Scholar

*Vajda, D., Thege, B. K., and Rózsa, S. (2019). Factor structure of the dyadic adjustment scale: a bifactor exploratory structural equation modeling approach. Euro. J. Psychol. Assess. 35, 326–334. doi: 10.1027/1015-5759/a000405

CrossRef Full Text | Google Scholar

*Van Zyl, L. E., Olckers, C., and Roll, L. C. (2020). The psychometric properties of the Grit–O scale within the twente region in Netherlands: an ICM–CFA vs. ESEM approach. Front. Psychol. 11, 796. doi: 10.3389/fpsyg.2020.00796

PubMed Abstract | CrossRef Full Text | Google Scholar

*Vaughan, R., Madigan, D. J., Carter, G. L., and Nicholls, A. R. (2019). The dark triad in male and female athletes and non-athletes: group differences and psychometric properties of the short dark triad (SD3). Psychol. Sport Exerc. 43, 64–72. doi: 10.1016/j.psychsport.2019.01.002

CrossRef Full Text | Google Scholar

*Vaughan, R. S., Edwards, E. J., and MacIntyre, T. E. (2020). Mental health measurement in a post Covid-19 world: psychometric properties and invariance of the DASS−21 in athletes and non-athletes. Front. Psychol. 11, 590559. doi: 10.3389/fpsyg.2020.590559

PubMed Abstract | CrossRef Full Text | Google Scholar

*Volmer, J., Koch, I. K., and Wolff, C. (2019). Illuminating the 'dark core': mapping global versus specific sources of variance across multiple measures of the dark triad. Pers. Individ. Dif. 145, 97–102. doi: 10.1016/j.paid.2019.03.024

CrossRef Full Text | Google Scholar

*Wang, T., Zhang, T., Mu, W. L., and Li, X. (2021). Psychometric evaluation of the encouragement character strength scale in a Chinese sample. Curr. Psychol. 40, 4741–4749. doi: 10.1007/s12144-020-01161-w

CrossRef Full Text | Google Scholar

*Yi, Z. Y., Wang, Y., and Tan, T. X. (2021). Evidence for a higher-order ESEM structure of ADHD in a sample of Chinese children. J. Psychopathol. Behav. Assess. 43, 376–387. doi: 10.1007/s10862-020-09837-0

CrossRef Full Text | Google Scholar

Zhu, L., Wang, J., and Schroevers, M. J. (2020). Looking beyond the value of individual facets of mindfulness: a person-centered examination of mindfulness. Mindfulness 11, 2349–2359. doi: 10.1007/s12671-020-01452-0

CrossRef Full Text | Google Scholar

Keywords: factor analysis, meta-analysis, multidimensionality, goodness-of-fit, exploratory structural equation modeling, bifactor ESEM

Citation: Gegenfurtner A (2022) Bifactor exploratory structural equation modeling: A meta-analytic review of model fit. Front. Psychol. 13:1037111. doi: 10.3389/fpsyg.2022.1037111

Received: 05 September 2022; Accepted: 06 October 2022;
Published: 26 October 2022.

Edited by:

Cesar Merino-Soto, Universidad de San Martin de Porres, Peru

Reviewed by:

Conrad Stanisław Zygmont, Helderberg College, South Africa
Panagiotis Ferentinos, National and Kapodistrian University of Athens, Greece

Copyright © 2022 Gegenfurtner. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Andreas Gegenfurtner, andreas.gegenfurtner@phil.uni-augsburg.de

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.