Guo-Dong Sun1†
Hua-Ke Chen1†Wei-Xing Sun2Éva Szabó3Enikő Tóth3
Jin-Chuan Hu4Balázs Jagodics3*
Jing-Dong Liu1*- 1Department of Physical Education, Sun Yat-sen University, Guangzhou, China
- 2Hebei Sport University, Shijiazhuang, China
- 3Department of Social and Developmental Psychology, University of Szeged, Szeged, Hungary
- 4Institute of Physical Education, Kunming University of Science and Technology, Kunming, China
Purpose: The present study aimed to further examine the factor structure and measurement invariance of the UDRQ among a sample of Hungarian university students.
Methods: Firstly, the factor structure of the UDRQ was examined among 837 Hungarian university students. Specifically, two measurement models (first-order model and second-order model) were constructed and compared. Secondly, the internal consistency reliability of the UDRQ was examined. Thirdly, measurement invariance of the UDRQ was evaluated across genders. Finally, measurement invariance of the UDRQ was evaluated across two different samples.
Results: It was found that the first-order model outperformed the second-order model and better represented the factor structure of the UDRQ subscales. Results of Cronbach’s alpha and Composite Reliability suggested that the internal consistency reliabilities of the two UDRQ subscales were satisfactory. Measurement invariance analysis revealed that the UDRQ measurement model was strict invariant across genders and samples.
Conclusion: The findings of the present study indicated that the UDRQ displayed satisfactory reliability and validity and could be used to assess demands and resources of Hungarian university students.
Introduction
Student burnout has been an important issue and a major challenge at different levels of education. Although burnout is often discussed as a work-related problem, research has shown the presence of symptoms among students (Salmela-Aro et al., 2009, 2022). For example, in a recent study, 39.2% university students reported significant burnout symptoms (Obregon et al., 2020). Previous research has revealed that student burnout was associated with lower academic achievement (Madigan and Curran, 2021), decreased motivation (Chang et al., 2016) and higher dropout (Alves et al., 2022). The problems associated with burnout highlight the necessity of exploring its underlying causes. The Job Demands-Resources model (JD-R; Demerouti et al., 2001) is a widely used framework to reveal factors contributing to the development of burnout. It distinguishes psychological, physical, social, and organizational factors that facilitate or inhibit the emergence of burnout. According to the JD-R model, any work-related factors that increase stress and exhaustion contribute to burnout. These factors are collectively referred to as demands, which include workload, time pressure or interpersonal conflicts. However, the development of burnout is mitigated by resources, which facilitate the achievement of work-related goals and enable personal growth (Bakker and Demerouti, 2007, 2017).
Although the JD-R model was originally developed in a workplace context, several studies have successfully applied it in educational environments recently (Ouweneel et al., 2011; Salmela-Aro and Upadyaya, 2014; Wolff et al., 2014; Akkermans et al., 2018; Lesener et al., 2020; Oger et al., 2022; Wei et al., 2022; Jagodics et al., 2023). For example, previous research has revealed that resources were negatively related to students’ burnout symptoms (e.g., Cilliers et al., 2018; Lesener et al., 2020; Lee et al., 2022; Salmela-Aro et al., 2022; Jagodics and Szabó, 2023; Jagodics et al., 2023) and positively related to school engagement (Lee et al., 2022), while demands were positively associated with anxiety and depression of students (Wörfel et al., 2016). However, it is noteworthy that while an increasing number of studies applying JD-R framework in educational contexts, there was no unified methodological approach to operationalizing it until recently. Both the number and the nature of the factors used to measure demands and resources under the JD-R model vary between studies. This diversity originates from the flexibility of the JD-R model, as its creators argued that demands and resources should be defined according to the specific context (Schaufeli, 2017). For example, previous research has conceptualized demands using cognitive and mental demands (Cilliers et al., 2018; Hodge et al., 2019; Lesener et al., 2020; Oger et al., 2022; Salmela-Aro et al., 2022), time pressure (Lesener et al., 2020; Wei et al., 2022), task difficulty and subjective work pressure (Salmela-Aro and Upadyaya, 2014; Wolff et al., 2014), lack of learning environment (Salmela-Aro et al., 2022) and emotional demands (Hodge et al., 2019). For the school related resources, previous research has conceptualized it using personal resources (such as academic self-efficacy, hope and optimism; Ouweneel et al., 2011), resources in the social environment in form of feedback and social support of peers, family and teachers (Mokgele and Rothmann, 2014; Lesener et al., 2020; Oger et al., 2022; Salmela-Aro et al., 2022; Wei et al., 2022), possibility of personal development and growth (Salmela-Aro and Upadyaya, 2014; Cilliers et al., 2018; Hodge et al., 2019), perceived control (Wolff et al., 2014) as well as career adaptability (Akkermans et al., 2018).
In higher education context, previous research has mainly employed self-constructed scales without going through stringent psychometric examinations or relied on modifying measures that developed in other contexts to measure demands and resources of university students. For example, Cilliers et al. (2018) measured demands and resources of South African university students using an adapted version of the questionnaire on the experience and assessment of work (VBBA; Van Veldhoven et al., 1997) in their study. Study demands includes two subscales of pace and amount of work in studies (five items) and cognitive demands (six items), while study resources includes four subscales of support from family (three items), support from lecturers (three items), support from friends (four items) and opportunities for growth and development. Lesener et al. (2020) operationalized study demands of German university students using challenging demands and time pressure and used a 4-item adapted version of Bakker’s (2014) job demands-resources Questionnaire to measure demands and a 3-item self-constructed scale to measure time pressure. For study resources, they operationalized it using student support, teach support and developmental opportunities, which were measured using three 3-item self-constructed scales, respectively (Lesener et al., 2020). On contrary, Luruli et al. (2020) conceptualized study demands of South African university students using academic demands, personal relationship demands, personal relationship problems and lecturers’ demands, which were measured using the Student-Stress Questionnaire (SSQ; Burge, 2009). They conceptualized study resources using family support, lecturer’s support and autonomy, which were measured using an adapted version of the Questionnaire on the Experience and Assessment of Work (QEAW; Van Veldhoven et al., 2002). From a methodological perspective, the diversity of subdimensions of demands and resources across studies makes it challenging to replicate studies and to compare results across studies using different approaches, while it also complicates the standardization of research methodologies. Therefore, Jagodics and Szabó (2023) developed the University Demand-Resource Questionnaire (UDRQ) by taking various perspectives into consideration to measure the perceived demands encountered and resources received during the study process of Hungarian university students. The development of the UDRQ is part of a bigger project that aims to create reliable tools to measure demands and resources of students from different educational contexts (primary schools, high schools and universities), which was inspired by previous work on the development of questionnaire measuring job demands and resources of Hungarian teachers (Jagodics and Szabó, 2014). Specifically, three questionnaires were developed among Hungarian students in this project. The first one was developed and validated among primary school students (Jagodics et al., 2020), while the second one was created to measure demands and resources of high school students (Jagodics et al., 2023). The UDRQ was the third one (Jagodics and Szabó, 2023), in which adjustments were made on wording of the items of the first two questionnaires as the educational contexts of primary schools, high schools and universities are significantly different from each other. The detailed development process of the UDRQ was described in a previous work (Jagodics and Szabó, 2023). The key difference between the UDRQ and other measures aiming to tap demands and resources in educational context is that the UDRQ grasps wider ranges of factors that considered to be important in the original JD-R framework (Demerouti et al., 2001; Bakker and Demerouti, 2017). To the best of our knowledge, the UDRQ was the first one that specifically developed to measure demands and resources of university students. The UDRQ distinguishes five subdimensions each for the Demands and Resources subscales and includes 34 items with 17 items measuring Demands (work style: 4 items; mental demands: 3 items; emotional demands: 4 items; conflict with lectures: 3 items; and career choice anxiety: 3 items) and 17 items measuring Resources (support from lecturers: 3 items; possibility of personal development: 3 items; information: 4 items; feedback: 3 items; and perceived control: 3 items) respectively (see Appendix A). Previous research has demonstrated that the UDRQ displayed satisfactory reliability and validity among university students from Hungary (Jagodics and Szabó, 2023). The development and initial validation of the UDRQ makes it possible for future researchers to compare the university related demands and resources of university students from different universities regionally in Hungary and potentially from different countries internationally.
Previous research has treated the two subscales of the UDRQ as independent structures and examined the factor structure of each subscale separately as five-correlated subdimensions construct in a first-order CFA model (see Figure 1). According to the definition of the JD-R model, it is also reasonable to conceptualize both subscales as a hierarchical structure (second-order CFA model) in which the Demands and Resources treated as the second-order variables and the five-subdimensions of each subscale as the first-order variables, respectively (see Figure 2). Taking Demands subscale as example, all items of Demands subscale could be represented by a second-order variable of Demands, which causes the first-order variables of Work Style, Mental Demands, Emotional Demands, Conflict with Lecturers and Career Choice Anxiety, which in turn, cause the perceived demands taped by all Demands items. One of the advantages of the second-order model is that it can distinguish residual error associated with prediction of the first-order factors by the second-order factor and measurement error associated with the observed variables (Byrne, 2005). Investigation on the question that whether the first- or the second-order CFA model may better represent the factor structure of the UDRQ will provide further psychometric evidence for the UDRQ and inform future practitioners in terms of how to apply it in their practical works.
Figure 1. First-order CFA models of university recourses (left) and university demands (right).
Figure 2. Second-order CFA models of university recourses (left) and university demands (right).
Measurement invariance has been considered as one of the important psychometric properties of the psychometric sound instruments. Measurement invariance assesses the psychometric equivalence of a construct across groups (or measurement occasions) and evaluates whether a construct has the same meaning to different groups or not (Putnick and Bornstein, 2016). Therefore, measurement invariance is a prerequisite to compare group means. The four main steps for testing measurement invariance (configural invariance, metric invariance, scalar invariance and residual invariance) described by Widaman and Reise (1997) has been widely used in previous research (Schellenberg et al., 2014; Putnick and Bornstein, 2016). For configural invariance, which is the least stringent invariance that measures whether the construct has the same pattern of free and fixed loadings across groups. It is tested by constraining the factor structure to be same across groups and investigates whether participants from different groups take the same conceptual framework as reference to answer the items (Wu et al., 2007; Milfont and Fischer, 2010). If the configural invariance was supported, the next step is to test the metric invariance, which is considered as weak invariance. It is tested by constraining all factor loadings to be equivalent across groups and investigates whether each item contributes to the latent construct to a similar degree across groups (Putnick and Bornstein, 2016). In other words, this model tests whether participants from different groups respond to the items in the same way. If metric invariance was supported, the next step is to test for the scalar invariance, which is considered as strong invariance. It is tested by constraining factor loadings and item intercepts to be equivalent across groups and investigates whether the mean differences in the latent constructs capture all mean differences in the shared variance of the items (Putnick and Bornstein, 2016). The fourth step is to test for the residual invariance if scale invariance was supported. It is tested by constraining factor loadings, intercepts and uniqueness to be equivalent across groups and examines whether the sum of specific variance and error variance is similar across groups, which is considered as the strict invariance. It should be noted that residual invariance is not a prerequisite for testing latent mean differences, as they are not part of the latent factor. To our best knowledge, no previous research has shed light on this issue yet by examining whether the UDRQ measurement model would be invariant across genders and samples or not.
Collectively, the purpose of the current study was to provide further psychometric evidence for the UDRQ by comparing two measurement models (first-order CFA vs. second-order CFA) of the UDRQ and evaluating its measurement invariance across genders and samples. Specifically, factor structure, reliability and measurement invariance across genders of the UDRQ were examined in an independent sample of Hungarian university students (sample 1). Furthermore, measurement invariance of the UDRQ across the current sample (sample 1) and another sample (sample 2) used in previous research (Jagodics et al., 2024) were evaluated.
Methods
Participants
In sample 1, a total of 868 Hungarian university students at the University of Szeged were invited to participate in this study by answering the UDRQ via an online survey. Excluding invalid and incomplete data, data from 837 participants (652 females, 181 males, 4 missing; age: M = 21.89, SD = 1.964, ranging from 18 to 25) were identified as valid and used for data analysis. The data of 688 Hungarian university students (507 females and 181 males; age: M = 22.3 years, SD = 1.99 years, ranging from 18 to 27) were randomly selected from the dataset obtained in previous research (Jagodics et al., 2024) were treated as sample 2 in this study to evaluate the measurement invariance of UDRQ across two samples (sample 1 and 2). At the moment of data collection, 66.4% of students were pursuing their Bachelor degree and 33.6% of students were pursuing their Master degree. Information about specialization was not collected.
Procedure
The data collection procedure for sample 1 and sample 2 were the same, in which convenient sampling method was used. Qualified university students at the University of Szeged were contacted and invited to participate in this study. Participants who returned their informed written consent forms were asked to answer the questionnaire. All participants were informed that the survey was voluntary and that they had the right to withdraw at any time from the study. They were also told that it was an anonymous survey, and that all the information they provided would be confidentially kept and no third parties including their teachers could be able to access their responses. Data of sample 2 were collected between March and April 2022 and data of sample 1 were collected between October and November 2022. All participants voluntarily participated in the study.
Measure
The University Demand-Resource Questionnaire (UDRQ; Jagodics and Szabó, 2023) was developed based on the Job Demand-Resource Model (JD-R Model; Demerouti et al., 2001) and measures the demands and resources of university students. The UDRQ includes 34 items with 17 items measuring Demands (work style: 4 items; mental demands: 3 items; emotional demands: 4 items; conflict with lectures: 3 items; and career choice anxiety: 3 items) and 17 items measuring Resources (support from lectures: 3 items; possibility of personal development: 3 items; information: 4 items; feedback: 3 items; and perceived control: 3 items) respectively. Responses were provided on a 6-point Likert scale ranging from completely disagree (1) to strongly agree (6). Previous research has demonstrated that the UDRQ displayed satisfactory reliability and validity among university students from Hungary (Jagodics and Szabó, 2023).
Data analysis
SPSS (Version 23.0, Armonk, NY, United States: IBM Corp.) was used for data processing. Firstly, two measurement models of the UDRQ (first-order CFA and second-order CFA) were estimated using Mplus 8.0 (Muthén and Muthén, 1998–2014) based on the robust maximum likelihood (MLR) estimator. Specifically, in the first-order CFA model, for each subscale, each item was allowed to load only on the factor it was assumed to measure and was not allowed to cross-load on other factors. This model includes five interrelated subdimensions representing each subscale of the UDRQ. In the second-order CFA model, for each subscale, the five first-order factors were specified to be associated with a single higher-order CFA factor, and no residual correlations were specified between the five first-order factors. For model evaluation, given the chi-square differences tests were sensitive to the sample size (e.g., Marsh et al., 2005), multiple common goodness-of-fit indices and information criteria were used to evaluate the fit of models including the Comparative Fit Index (CFI), the Tucker-Lewis index (TLI), the Root Mean Square Error of Approximation (RMSEA) with its 90% confidence intervals (CI), the Standardized Root Mean Square Residual (SRMR). For the CFI and TLI, values >0.95 indicate a good model fit, but values around 0.90 are acceptable. For RMSEA and SRMR, values <0.08 or 0.06 indicate acceptable or good model fits, respectively (Hu and Bentler, 1999). For model comparison, the guidelines for nested model comparisons proposed by Chen et al. (2012) was followed in this study. When the sample size was larger than 300 (n = 837 in this study), a change in CFI (ΔCFI) ≥ 0.005 accompanied by a change in RMSEA (ΔRMSEA) of ≥0.015 would suggest the simpler model to be better than the more complex model.
Secondly, the Cronbach’s alpha coefficient (α) and Composite Reliability (CR) were used to evaluate the internal consistency reliability. The cut-off values of 0.70 (α) and 0.60 (CR) were used in this study to indicate satisfactory internal consistency reliability. Average variance extracted values were computed to evaluate the convergent validity of the UDRQ.
Thirdly, once optimal measurement model was evidenced (first-order CFA or second-order CFA), measurement invariance of the UDRQ across genders and samples would be evaluated using multiple-group CFA (MGCFA). Four models were evaluated: configural (M1), metric invariance (M2: weak invariance), scalar invariance (M3: strong invariance), and residual invariance (M4: strict invariance). Measurement invariance analysis involves comparing nested models to one another, recommendations of Cheung and Rensvold (2002) and Chen (2007) on changes in CFI and RMSEA (ΔCFI and ΔRMSEA) were employed in this study. Specifically, changes in CFI of ≤0.01 and in RMSEA of ≤0.015 from less constrained to more constrained models were considered as evidence of measurement invariance.
Results
Factor structure and internal consistency reliability
The model fit indices for the two models of each subscale of UDRQ are presented in Table 1. For Resources subscale, both the models demonstrated acceptable fit to the data and the model fit of the first-order CFA model and the second-order CFA model were similar (ΔCFI = −0.003; ΔRMSEA = 0). Following the parsimonious principle, the first-order CFA was simpler and preferred in comparison with the second-order CFA model and therefore was retained. For Demands subscale, the first-order CFA outperformed the second-order CFA model (ΔCFI = −0.028; ΔRMSEA = 0.010), which indicates that the first-order CFA model better represents the factor structure of the Demands subscale. Factor loadings of different models were found satisfactory ranging from 0.502 to 0.901 (see Table 2), which suggest that all items function well in the current sample. Regarding the inter-factor correlations, for the Resources subscale, the five subdimensions were moderately associated with each other with correlation coefficients ranging from 0.493 to 0.782 (Table 3). For the Demands subscale, associations amongst the five subdimensions ranged from 0.210 to 0.814 (Table 3). Collectively, these results suggest that the first-order CFA models are more parsimonious and interpretable than the second-order CFA models and better represent the factor structure of the two UDRQ subscales.
Table 1. Goodness of fit statistics.
Table 2. Standardized factor loadings of the 1st order CFA and 2nd-order CFA models.
Table 3. Inter-factor correlations, Cronbach’s alpha coefficients, composite reliability and average variance extracted (first-order CFA).
For the Resources subscale, the Cronbach’s alpha coefficients of the five subdimensions ranged from 0.739 to 0.895 and the composite reliability (CR) values ranged from 0.742 to 0.897. The Average Variance Extracted (AVE) values ranged from 0.491 to 0.686 (see Table 3). For the Demands subscale, the Cronbach’s alpha coefficients of the five subdimensions ranged from 0.751 to 0.869 and the composite reliability (CR) values ranged from 0.760 to 0.870. The Average Variance Extracted (AVE) values ranged from 0.451 to 0.691 (see Table 3). These results suggest that the reliability of the subscales of the UDRQ are satisfactory.
Measurement invariance across genders
Given the first-order CFA model of each subscale of the UDRQ demonstrated better represented the factor structure of the two UDRQ subscales, measurement invariance was examined by progressively adding invariance constraints across genders. Table 4 presents the goodness-of-fit indices for independent and invariance models. All models displayed acceptable fit to the data. Comparing the more constrained models with the less constrained models across genders, no decreases in model fit (ΔCFI ≤0.001 and ΔRMSEA ≤0.002) exceeded the recommended cutoff values for the fit indexes. These results provide support for weak, strong and strict measurement invariance of the first-order CFA model of the two UDRQ subscales across genders.
Table 4. Measurement invariance across genders (female = 652; male = 181).
Measurement invariance across samples
The measurement invariance of the first-order CFA model of each subscale of the UDRQ was examined by progressively adding invariance constraints across the two samples. Table 5 presents the goodness-of-fit indices for independent and invariance models. All models displayed acceptable fit to the data. Comparing the more constrained models with the less constrained models across samples, no decreases in model fit (ΔCFI ≤0.001 and ΔRMSEA ≤0.002) exceeded the recommended cutoff values for the fit indexes. These results provide support for weak, strong and strict measurement invariance of the first-order CFA model of the two UDRQ subscales across samples.
Table 5. Measurement invariance across samples (sample 1 = 837; sample 2 = 688).
Discussion
Previous research has provided initial support for the first-order five-dimensional structure of the two subscales of the UDRQ among Hungarian university students (Jagodics and Szabó, 2023) and similar factor structure of the demands and resources measure in school setting (School Demands and Resources Questionnaire; SDRQ) were also evidenced among secondary school students (Jagodics et al., 2023). According to the definition of the JD-R model, it is also reasonable to consider both Demands and Resources subscales as a hierarchical structure in which the demands and resources are considered as the second-order variables represented by five subdimensions of each subscale as the first-order variables, respectively. However, no previous research has shed light on this research question. This study contributes to the growing literature on the application of JD-R model among university students by comparing whether the first-order model or the second-order model better represents the underlying multidimensional structure of the UDRQ among an independent sample of Hungarian university students. It is also a positive response to the replication crisis in psychology field (Open Science Collaboration, 2015; Tackett et al., 2019; Wu et al., 2023). Furthermore, measurement invariance, as one of the important psychometric properties of a psychometric sound instrument, is a prerequisite to compare group means. Researchers cannot assume that the instrument may measure the same psychological constructs across groups without testing for the measurement invariance of the instrument. Therefore, another important contribution of the current study to the literature is to examine the measurement invariance of UDRQ across genders and groups, which will inform the researchers and practitioners about the application of the UDRQ in their researches and practical works. More importantly, further psychometric properties of the UDRQ evidenced by the current study will ensure that future researchers from Hungary could use it and interpret their results with confidence and researchers from other countries or cultures who are interested in the topic could translate it into other languages to make cross-cultural comparisons possible.
According to Byrne (2005), four criteria could be considered for choosing the second-order model over the first-order model in model comparisons. Firstly, there should be reasonable theoretical justifications on the second-order construct; secondly, the second-order model demonstrates good model fit; thirdly, the discrepancy in model fit indices of the first-and second-order models should be small; fourthly, the second-order model should be more parsimonious compared to the first-order model. Results of the current study revealed that although the second-order CFA model of Resources subscale of the UDRQ demonstrated acceptable model fit to the data, the first-order CFA model was more parsimonious. For the Demands subscale, the second-order CFA model demonstrated a marginal acceptable model fit to the data, but were much worse than that of the first-order CFA model. Meanwhile, the latter one was more parsimonious. In addition, results of the second-order CFA model in this study revealed that regression coefficients of two subdimensions (Career Choice Anxiety and Conflict with Lecturers) on the second-order variable of Demands were relatively low (0.392 and 0.384) compared with that of the other three subdimensions (0.972, 0.797 and 0.774), which may indicate that the contributions of the five subdimensions to the second-order variable of Demands vary a lot. Meanwhile, previous research revealed that some subdimensions of the Demands and Resources subscales of the UDRQ were not significantly associated with burnout symptoms of university students (Jagodics and Szabó, 2023). The results obtained in this study together with evidence from previous research imply that some important information would be missing if the second-order model of the UDRQ subscales were used to explore its associations with other variables. Therefore, we contend that the first-order models better represent the factor structures of the two UDRQ subscales in the current Hungarian university student sample.
It is noteworthy that errors of some items were constrained to be correlated with each other based on modification indices in the initial validation study of UDRQ (Jagodics and Szabó, 2023). For the Resources subscale, errors of two items belonging to the same construct were correlated with each other in measurement model, which resulted in a good model fit, χ2 = 334, p < 0.001; df = 107, CFI = 0.962, TLI = 0.950, RMSEA = 0.054 (90% CI: 0.048/0.061), SRMR = 0.037. For the Demands subscale, errors of five items belonging to the same construct were correlated with each other in the measurement model, which resulted in a good model fit, χ2 = 334, p < 0.001, df = 104, CFI = 0.962, TLI = 0.950, RMSEA = 0.054 (90% CI: 0.048/0.061), SRMR = 0.037. In the present study, no errors were constrained to be correlated with each other as the two models displayed acceptable model fit to the data, therefore, we did not compare our findings with that in previous research directly. The inter-factor correlations of the five-subdimensions of each UDRQ subscale (Demands subscale: 0.200–0.814; Resources subscale: 0.493–0.782) in our study were comparable to the findings in previous study (Demands subscale: 0.111–0.652; Resources subscale: 0.395–0.696; Jagodics and Szabó, 2023), which suggest that the subdimensions of each UDRQ subscale were associated with but distinctive from each other. In addition, the internal consistency reliabilities of subdimensions of each UDRQ subscale were satisfactory (Cronbach α: 0.739–0.895; Composite reliability: 0.741–0.897) in the present study, which were consistent with findings in previous research (Cronbach α: 0.741–0.875; Jagodics and Szabó, 2023).
Measurement invariance is one of the fundamental psychometric properties of psychometrically sound instruments (Widaman and Reise, 1997; Putnick and Bornstein, 2016; Marsh et al., 2019), which is especially crucial for group comparisons. No previous research has shed light on this issue regarding the UDRQ yet. The results of the current study provided evidence for the weak, strong and strict invariance of the first-order CFA models of the two UDRQ subscales across genders and samples. Configural invariance is the foundation for the measurement invariance, which was evidenced in the current study. It implies that university students from different groups (male vs. female; sample 1 vs. sample 2) conceptualized the constructs of UDRQ subscales in the same way (Wu et al., 2007; Milfont and Fischer, 2010). With the evidence of configural invariance, three more constraining invariance models (metric invariance, scalar invariance and residual invariance) were evaluated. Specifically, metric invariance investigates whether each item contributes to the latent construct to a similar degree across groups (Putnick and Bornstein, 2016). In other words, evidence of metric invariance in this study indicates that university students from different groups responded to the UDRQ items in the same way, which were reflected of revealing that the strengths of the correlations between specific UDRQ items and their corresponding underlying construct (subdimensions of each UDRQ subscale) were same across groups (Milfont and Fischer, 2010). Scalar invariance is the basis for group comparisons, which investigates whether the mean differences in the latent constructs capture all mean differences in the shared variance of the items (Putnick and Bornstein, 2016). Therefore, in the present study, evidence of scalar invariance suggests that university students from different groups who reported same scores on latent constructs would obtain same scores on the observed variables (e.g., subdimensions of the UDRQ subscales). In other words, with this evidence support, researchers and practitioners could be confident to compare scores obtained using the UDRQ across genders and samples directly. Finally, although residual invariance is not a prerequisite for group comparisons directly as they are not part of the latent factor, it examines whether the sum of specific variance and error variance is similar across groups. Therefore, with the evidence of residual invariance in this study, we can conclude that if there were any differences on scores measured using the UDRQ across university students from different groups (e.g., male vs. female; different samples), the differences were true and meaningful because measurement artifacts having been taken into considerations (Wu et al., 2007; Schellenberg et al., 2014). It is noteworthy that although some researchers argued that residual invariance was overly rigorous and should not be considered for establishing measurement invariance across groups (Vandenberg and Lance, 2000), other researchers have demonstrated its necessities for group comparisons (see Wu et al., 2007; Schellenberg et al., 2014).
Although the present study provides further evidence for the psychometric properties of the UDRQ among Hungarian university students and contributes to the growing literature on UDRQ, several limitations should be acknowledged. Firstly, convenient sampling method was used and university students from one university were invited in the current study. Researchers are suggested to enlarge the sample size and invite more university students from various kind of universities to further examine the psychometric properties of the UDRQ. Secondly, gender proportion in the sample 1 was uneven with about 21% were males. Although the sample size of male students is enough for MGCFA analysis (according to the participant to item ratio > 10:1), future studies are suggested to recruit more male university students. Thirdly, information about specialization of students was not collected in this study. Future studies should explore whether there are significant differences on the university resources received and demands encountered by students majoring in different specializations or not, which will contribute to our understanding on the topic. Fourthly, given the focus of the current study was to further examine the factor structure of the UDRQ, other validity properties (e.g., nomological validity) were not examined in this study. Researchers are suggested to further investigate other validity properties of the UDRQ. Fifthly, the current study only examined the measurement invariance of the UDRQ across genders and samples, future studies are suggested to further examine the longitudinal invariance of the UDRQ measurement models, which will be informative for studies using longitudinal and interventional designs.
Conclusion
Collectively, this study provided further support for the psychometric properties of the UDRQ among an independent sample of Hungarian university students. It was found that the first-order CFA model of the two UDRQ subscales should be retained, which is consistent with previous research. Furthermore, the internal consistency reliability of the subscales of the UDRQ were found satisfactory. Finally, it was found that the first-order CFA measurement model was invariant across genders and samples. Collectively, the results of the study suggested that the UDRQ demonstrated satisfactory validity and reliability and could be used to assess demands and resources of Hungarian university students.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving humans were approved by Hungarian United Ethical Review Committee for Research in Psychology. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.
Author contributions
G-DS: Writing – original draft, Writing – review & editing, Data curation, Formal analysis, Methodology. H-KC: Validation, Writing – original draft, Writing – review & editing, Data curation, Formal analysis, Methodology. W-XS: Data curation, Formal analysis, Methodology, Writing – review & editing, Writing – original draft, Validation. ÉS: Methodology, Project administration, Resources, Validation, Writing – review & editing, Writing – original draft, Investigation. ET: Investigation, Project administration, Resources, Validation, Writing – review & editing, Writing – original draft. J-CH: Data curation, Formal analysis, Methodology, Writing – review & editing, Writing – original draft, Validation. BJ: Conceptualization, Investigation, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing, Methodology, Project administration. J-DL: Conceptualization, Formal analysis, Supervision, Validation, Writing – original draft, Writing – review & editing, Methodology.
Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2024.1433331/full#supplementary-material
References
Akkermans, J., Paradniké, K., Van der Heijden, B. I. J. M., and De Vos, A. (2018). The best of both worlds: the role of career adaptability and career competencies in students’ well-being and performance. Front. Psychol. 9:1678. doi: 10.3389/fpsyg.2018.01678
Alves, S. A., Sinval, J., Lucas Neto, L., Marôco, J., Gonçalves Ferreira, A., and Oliveira, P. (2022). Burnout and dropout intention in medical students: the protective role of academic engagement. BMC Med. Educ. 22:83. doi: 10.1186/s12909-021-03094-9
Bakker, A. B., and Demerouti, E. (2007). The job demands-resources model: state of the art. J. Manag. Psychol. 22, 309–328. doi: 10.1108/02683940710733115
Bakker, A. B., and Demerouti, E. (2017). Job demands–resources theory: taking stock and looking forward. J. Occup. Health Psychol. 22, 273–285. doi: 10.1037/ocp0000056
Burge, J. (2009). Coping frequency, coping effectiveness, and personality factors in university students [Unpublished honours thesis].: University of Canberra.
Byrne, B. M. (2005). Factor analytic models: viewing the structure of an assessment instrument from three perspectives. J. Pers. Assess. 85, 17–32. doi: 10.1207/s15327752jpa8501_02
Chang, E., Lee, A., Byeon, E., Seong, H., and Lee, S. M. (2016). The mediating effect of motivational types in the relationship between perfectionism and academic burnout. Personal. Individ. Differ. 89, 202–210. doi: 10.1016/j.paid.2015.10.010
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct. Equ. Model. Multidiscip. J. 14, 464–504. doi: 10.1080/10705510701301834
Chen, F. F., Hayes, A., Carver, C. S., Laurenceau, J. P., and Zhang, Z. (2012). Modeling general and specific variance in multifaceted constructs: a comparison of the bifactor model to other approaches. J. Pers. 80, 219–251. doi: 10.1111/j.1467-6494.2011.00739.x
Cheung, G. W., and Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Struct. Equ. Model. 9, 233–255. doi: 10.1207/S15328007SEM0902_5
Cilliers, J. R., Mostert, K., and Nel, J. A. (2018). Study demands, study resources and the role of personality characteristics in predicting the engagement of fist-year university students. S. Afr. J. High. Educ. 32, 49–70. doi: 10.20853/32-1-1575
Demerouti, E., Bakker, A. B., Nachreiner, F., and Schaufeli, W. B. (2001). The job demands-resources model of burnout. J. Appl. Psychol. 86, 499–512. doi: 10.1037/0021-9010.86.3.499
Hodge, B., Wright, B., and Bennett, P. (2019). Increasing student engagement and reducing exhaustion through the provision of demanding but well-resourced training. J. Furth. High. Educ. 43, 406–417. doi: 10.1080/0309877X.2017.1363385
Hu, L., and Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Model. Multidiscip. J. 6, 1–55. doi: 10.1080/10705519909540118
Jagodics, B., Nagy, K., Szénási, S., Varga, R., and Szabó, É. (2020). Az iskolai leterheltség és kiégés vizsgálata a Követelmény–Erőforrás Modell segítségével magyar általános iskolások körében. [Exploring the link between workload and burnout among primary school students using the Demand-Resource Model]. Appl. Psychol. 20, 57–81. doi: 10.17627/ALKPSZICH.2020.1.57
Jagodics, B., Nagy, K., Szénási, S., Varga, R., and Szabó, É. (2023). School demands and resources as predictors of student burnout among high school students. Sch. Ment. Heal. 15, 90–104. doi: 10.1007/s12310-022-09534-1
Jagodics, B., and Szabó, É. (2014). Job demands versus resources: workplace factors related to teacher burnout. Pract. Theory Syst. Educ. 9, 377–390.
Jagodics, B., and Szabó, É. (2023). Student burnout in higher education: a demand-resource model approach. Trend Psychol. 31, 757–776. doi: 10.1007/s43076-021-00137-4
Jagodics, B., Tóth, E., and Szabó, É. (2024). In search of the missing link: The mediating role of motivation between student burnout and the demand-resource framework [unpublished manuscript]. Szeged, Hungary: Department of Social and Developmental Psychology, University of Szeged.
Lee, T., Lee, S., Ko, H., and Lee, S. M. (2022). Self-compassion among university students as a personal resource in the job demand-resources model. Educ. Psychol. 42, 1160–1179. doi: 10.1080/01443410.2022.2120600
Lesener, T., Pleiss, L. S., Gusy, B., and Wolter, C. (2020). The study demands-resources framework: an empirical introduction. Int. J. Environ. Res. Public Health 17:5183. doi: 10.3390/ijerph17145183
Luruli, K., Mostert, K., and Jacobs, M. (2020). Testing a structural model for study demands and resources, study engagement and well-being of first-year university students. J. Psychol. Afr. 30, 179–186. doi: 10.1080/14330237.2020.1767925
Madigan, D. J., and Curran, T. (2021). Does burnout affect academic achievement? A Meta-analysis of over 100,000 students. Educ. Psychol. Rev. 33, 387–405. doi: 10.1007/s10648-020-09533-1
Marsh, H. W., Guo, J., Dicke, T., Parker, P. D., and Craven, R. G. (2019). Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and set-ESEM: optimal balance between goodness of fit and parsimony. Multivar. Behav. Res. 55, 102–119. doi: 10.1080/00273171.2019.1602503
Marsh, H. W., Hau, K.-T., and Grayson, D. (2005). “Goodness of fit evaluation in structural equation modeling” in Contemporary psychometrics. A festschrift for Roderick P. McDonald. eds. A. Maydeu-Olivares and J. McArdle (Mahwah NJ: Erlbaum).
Milfont, T. L., and Fischer, R. (2010). Testing measurement invariance across groups: application in cross-cultural research. Int. J. Psychol. Res. 3, 111–130. doi: 10.21500/20112084.857
Mokgele, K. R., and Rothmann, S. (2014). A structural model of student well-being. S. Afr. J. Psychol. 44, 514–527. doi: 10.1177/0081246314541589
Muthén, L. K., and Muthén, B. O. (1998–2014). Mplus user's guide. 7th Edn. Los Angeles, CA: Muthén & Muthén.
Obregon, M., Luo, J., Shelton, J., Blevins, T., and MacDowell, M. (2020). Assessment of burnout in medical students using the Maslach burnout inventory-student survey: a cross-sectional data analysis. BMC Med. Educ. 20:376. doi: 10.1186/s12909-020-02274-3
Oger, M., Broc, G., Martin-Krumm, C., Le Roux, F., Muller, A., and Tarquinio, C. (2022). Pupils’ adaptability at school, a balance between demands and resources? Front. Educ. 7:814376. doi: 10.3389/feduc.2022.814376
Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science 349:aac4716. doi: 10.1126/science.aac4716
Ouweneel, E., Le Blanc, P. M., and Schaufeli, W. B. (2011). Flourishing students: a longitudinal study on positive emotions, personal resources, and study engagement. J. Posit. Psychol. 6, 142–153. doi: 10.1080/17439760.2011.558847
Putnick, D. L., and Bornstein, M. H. (2016). Measurement invariance conventions and reporting: the state of the art and future directions for psychological research. Dev. Rev. 41, 71–90. doi: 10.1016/j.dr.2016.06.004
Salmela-Aro, K., Kiuru, N., Leskinen, E., and Nurmi, J.-E. (2009). School burnout inventory (SBI): reliability and validity. Eur. J. Psychol. Assess. 25, 48–57. doi: 10.1027/1015-5759.25.1.48
Salmela-Aro, K., and Upadyaya, K. (2014). School burnout and engagement in the context of demands-resources model. Br. J. Educ. Psychol. 84, 137–151. doi: 10.1111/bjep.12018
Salmela-Aro, K., Upadyaya, K., Ronkainen, I., and Hietajärvi, L. (2022). Study burnout and engagement during COVID-19 among university students: the role of demands, resources, and psychological needs. J. Happiness Stud. 23, 2685–2702. doi: 10.1007/s10902-022-00518-1
Schaufeli, W. B. (2017). Applying the job demands-resources model: a ‘how to’ guide to measuring and tackling work engagement and burnout. Organ. Dyn. 46, 120–132. doi: 10.1016/j.orgdyn.2017.04.008
Schellenberg, B. J. I., Gunnell, K. E., Mosewich, A. D., and Bailis, D. S. (2014). Measurement invariance of the passion scale across three samples: an ESEM approach. Meas. Phys. Educ. Exerc. Sci. 18, 242–258. doi: 10.1080/1091367X.2014.942453
Tackett, J. L., Brandes, C. M., King, K. M., and Markon, K. E. (2019). Psychology’s replication crisis and clinical psychological science. Annu. Rev. Clin. Psychol. 15, 579–604. doi: 10.1146/annurev-clinpsy-050718-095710
Van Veldhoven, M., De Jonge, J., Broersen, S., Kompier, M., and Meijman, T. (2002). Specific relationships between psychosocial job conditions and job-related stress: a three-level analytic approach. Work Stress. 16, 207–228. doi: 10.1080/02678370210166399
Van Veldhoven, M., Meijman, T. F., Broersen, J. P. J., and Fortuin, R. J. (1997). Handleiding VBBA: Onderzoek naar de beleving van psychosociale arbeidsbelasting en werkstress met behulp van de vragenlijst beleving en beoordeling van de arbeid [manual VBBA: Research on the experience of psychosocial workload and job stress by means of the questionnaire on the experience and evaluation of work]. Amsterdam, The Netherlands: SKB.
Vandenberg, R. J., and Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ. Res. Methods 3, 4–70. doi: 10.1177/109442810031002
Wei, C., Ma, Y., Ye, J.-H., and Nong, L. (2022). First-year college students’ mental health in the post-COVID-19 era in Guangxi, China: a study demands-resources model perspective. Front. Public Health 10:906788. doi: 10.3389/fpubh.2022.906788
Widaman, K. F., and Reise, S. P. (1997). “Exploring the measurement invariance of psychological instruments: applications in the substance use domain” in The science of prevention: Methodological advances from alcohol and substance abuse research. eds. K. J. Bryant, M. E. Windle, and S. G. West (Washington, DC: American Psychological Association), 281–324.
Wolff, W., Brand, R., Baumgarten, F., Lösel, J., and Ziegler, M. (2014). Modeling students’ instrumental (mis-) use of substances to enhance cognitive performance: Neuroenhancement in the light of job demands-resources theory. Biopsychosoc. Med. 8:12. doi: 10.1186/1751-0759-8-12
Wörfel, F., Gusy, B., Lohmann, K., Töpritz, K., and Kleiber, D. (2016). Mental health problems among university students and the impact of structural conditions. J. Public Health 24, 125–133. doi: 10.1007/s10389-015-0703-6
Wu, A. D., Li, Z., and Zumbo, B. D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: a demonstration with TIMSS data. Pract. Assess. Res. Eval. 12, 1–26. doi: 10.7275/mhqa-cd89
Keywords: job demands-resources model, university students, factor structure, reliability, measurement invariance
Citation: Sun G-D, Chen H-K, Sun W-X, Szabó É, Tóth E, Hu J-C, Jagodics B and Liu J-D (2024) Factor structure and measurement invariance of the University Demand-Resource Questionnaire: further evidence from Hungarian university students. Front. Psychol. 15:1433331. doi: 10.3389/fpsyg.2024.1433331
Edited by:
Laurent Sovet, Université de Paris, FranceReviewed by:
Toni Babarovic, Institute of Social Sciences Ivo Pilar (IPI), CroatiaEsther Lopez-Zafra, University of Jaén, Spain
Copyright © 2024 Sun, Chen, Sun, Szabó, Tóth, Hu, Jagodics and Liu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jing-Dong Liu, liujd7@mail.sysu.edu.cn; Balázs Jagodics, balazs.jagodics@gmail.com
†These authors have contributed equally to this work and share first authorship