Huiwen Xiao
Rongmao Lin
Qiaoling Wu1- 1School of Psychology, Fujian Normal University, Fuzhou, China
- 2Guilin University of Aerospace Technology, Guilin, China
The Negative Problem Orientation Questionnaire (NPOQ) is a widely used tool for assessing negative problem orientation (NPO). However, its construct and measurement invariance has not been adequately tested in adolescents. The present study explored the possible construct of the NPOQ and its measurement invariance in a sample of 754 Chinese adolescents (51.6% girls, all 12–18 years old). The results supported a bifactor model of the NPOQ that consists of a general factor NPO and three domain-specific factors including perceived threat, self-inefficacy, and negative outcome expectancy. A multiple-group CFA indicated that the bifactor model showed strict invariance across gender and age. The general and domain factors showed unique variance in indexes of worry, depression, anxiety, and stress, which supported well incremental validity of them. This study confirms for a bifactor conceptualization of the NPOQ and its measurement invariance across gender and age in Chinese adolescents. Additionally, it is recommended that the total score should be used to assess NPO in Chinese adolescents.
Introduction
Negative problem orientation (NPO) is an important construct in our understanding of deficits in problem-solving ability as manifested in daily life. It refers to a set dysfunctional attitudes regarding social problem solving, which increases worry about social problems and inhibits effective resolution (Nezu and D’Zurilla, 2006; Bell and D’Zurilla, 2009). Indeed, NPO not only results in poor social problem solving, but also uniquely links to many mental health disorders, for example, generalized anxiety disorder (Dupuy and Ladouceur, 2008), social anxiety disorder (Hearn et al., 2017; Romano et al., 2019), depressive disorder (Becker-Weidman et al., 2010; Anderson et al., 2011; Balsamo et al., 2013), and eating disorders (Konstantellou et al., 2011).
Considered its important role in social and health problems, how to measure NPO is vital in health and clinical context. Since its publication in 2005, the Negative Problem Orientation Questionnaire (NPOQ) has emerged as the most widely used self-report measure of NPO. It uses 12 items to assess negative beliefs regarding social problems on a 5-point scale (ranging from 1 to 5). It shows excellent internal consistency (α = 0.92), good test-retest reliability (r = 0.80, over interval 5 weeks), and demonstrates both convergent and discriminant validity (Robichaud and Dugas, 2005a,b). Over the past 15 years, the NPOQ has been used in most psychosocial studies of NPO including clinical evaluation and diagnosis of anxiety and mood disorders (Dupuy and Ladouceur, 2008; Fergus and Wu, 2010), treatment outcome trails (Humphrey, 2016; Erdley-Kass et al., 2018), and also assessment of the social problem solving-ability in adolescent population (Barahmand, 2008; Zumberg et al., 2008). Despite the popularity and empirical utility of the NPOQ, its construct validity and measurement invariance across age and gender have not been sufficiently tested in adolescents.
Construct Validity: From a Bifactor Perspective
The construct validity of NPOQ is relatively few supported. Robichaud and Dugas (2005a,b), which relied on two samples of university students (N = 201, 148), found only one factor in NPOQ by principal components factor analysis and maximum likelihood factor analysis, accounting for 54.8 and 52% of the variance, respectively. So far, the unitary factor model only tested in adult sample, but has yet been validated in adolescents. Previous studies have showed that adolescents experience an increasingly NPO trajectory, differing from the relatively stable level in adults (Ciarrochi et al., 2009). Furthermore, the one factor structure may not fully reflect the concept of NPO. As the social problem-solving theory described, NPO is a general negative disposition toward problems that can lead to negative emotions, avoidance, increased worry, and a reduction in effort to resolve complications (D’Zurilla et al., 2004). Specifically, it refers to a disruptive cognitive-emotional set or attitude toward social problems that involves three components: perceived threats to wellbeing, self-inefficacy or doubt regarding one’s problem-solving ability, and a tendency to be pessimistic about outcomes (D’Zurilla et al., 1998; Hawkins et al., 2009). From the social problem-solving model, a three-factor construct may be better described the NPOQ.
Whether NPOQ shows as a unitary or three-factor construct in adolescents? To answer the question, this study firstly compares the unitary and three-factor models via traditional correlated-factor models in confirmatory factor analysis, and further determines from a bifactor perspective. The bifactor model, which is also named as general-specific factor model, describes that there is a general factor representing the common variance of all the items and there are some group factors that account for additional common variance of all items after controlling the general factor (Rodriguez et al., 2016a,b). In the bifactor model, the general factor represents the conceptually broad “target” construct an instrument was designed to measure, and the group factors represent more conceptually narrow subdomain constructs (Reise, 2012). In contrast to the traditional correlated-factors model that must compare the unitary and multidimensional constructs in different models, the bifactor model can simultaneously estimate the different role of general factor and group factors in a bifactor model, it further determines the assumption that the group factors capture unique variances after accounting for the common variances or alternatively that the general factor mainly captures the common variance (Reise, 2012; Reise et al., 2016; Rodriguez et al., 2016b). The factor loadings on the general factor and group factors also can be considered to evaluate the validity of the bifactor model (Nadort et al., 2009; Saggino et al., 2018). In the study, performing a bifactor model is helpful to validate whether three group factors of NPOQ capture unique variances after accounting for the common variances or that the NPOQ manifests in a single construct.
Measurement Invariance Across Age and Gender
Another potential weakness in common practices of using NPOQ is the lack of accounting adequately for the measurement error. Prior studies indicated that adolescent NPO may differ profoundly by gender and age (Barahmand, 2008; Ciarrochi et al., 2009). For example, Barahmand (2008) found that boys reported greater NPO than girls. Also, the findings revealed certain age trends that the NPO may be under the influence of developmental tasks (Barahmand, 2008). Similarly, with a latent growth analysis, Ciarrochi et al. (2009) showed that adolescents experienced an increasingly NPO trajectory as their age. However, these researches have not considered the implications of measurement invariance when making group comparisons.
Measurement invariance generally refers to the extent to which the content of each item is being perceived and interpreted in the same way across samples (Vandenberg and Lance, 2000). In fact, if measures of NPO operate differently across gender and age groups and these variations are not taken into account in the measurement, it is inappropriate to compare levels of NPO across groups (Cheung, 2008; Schmitt and Kuljanin, 2008). Thus, measurement invariance is precondition for group comparisons. Testing for measurement invariance includes a series of hierarchical steps, beginning with the establishment of a baseline model in each group, followed by tests for equivalence across groups at several increasingly more restricted levels including but not limited at configural, factor loadings, indicator thresholds and indicator residual errors (Vandenberg and Lance, 2000; Schmitt and Kuljanin, 2008; Carlucci et al., 2018).
The Current Study
In sum, though NPOQ has been widely applied to assessment of adolescent’s NPO, its construct validity and measurement invariance have not been sufficiently supported. The main purpose of the present work is to explore the factor structure of the NPOQ and test its measurement invariance with Chinese adolescents. In addition, this research also tests the incremental validity of the NPOQ for Chinese adolescents. An increasing number of studies have shown that NPO is an important predictive factor for worry and anxiety (Kertz and Woodruff-Borden, 2012; for a review, see Clarke et al., 2017). In the cognitive model of anxiety disorder, Dugas et al. (2005) proposed that NPO is the key cognitive factor in chronic anxiety and worry. Also, patients with depressive symptoms tend to use more NPO (Bell and D’Zurilla, 2009). Thus, the present study uses worry, anxiety, depression, and related stress as criteria.
Materials and Methods
Participants
A sample of 855 adolescents was recruited via a stratified cluster sampling, which each class was saw as a cluster, and the grade was saw as level (included 4 grades). In this survey, four classes were sampled in each grade, and a total of 16 classes were recruited from four public middle schools in Fuzhou, Fujian Province. The total number of participants for each school were 188 (25%), 171 (22.7%), 203 (27%), and 192 (25.5%), respectively. All participants completed the questionnaire in class, which was administrated by their head teacher and a postgraduate majoring in psychology. Approval was obtained from participants’ head teachers and parents. A primary check was conducted when the responses were collected; participants who responded to the same option for more than half of the questionnaire or blank were excluded. Finally, an available sample of 754 adolescents was retained (the rate of availability was 88.2%). Participants ranged in age from 12 to 18 years (M = 14.9, SD = 1.6). In the available sample, 365 were boys (48.4%) and 389 were girls (51.6%). The numbers of students in the 7th, 8th, 10th, and 11th grades were 188 (25%), 171 (22.7%), 203 (27%), and 192 (25.5%), respectively. This study was approved by the Academic Committee of Fujian Normal University.
Measures
Negative Problem Orientation Questionnaire (NPOQ)
NPOQ is a 12-item measure that assesses a dysfunctional cognitive set that includes the tendency to see problems as a threat, to doubt one’s own problem-solving ability, and to be pessimistic about the outcome (Robichaud and Dugas, 2005a,b). It was shown to have excellent internal consistency (α = 0.92), good test-retest reliability (r = 0.80, interval 5 weeks), and demonstrated convergent (r = 0.83, p < 0.001) and discriminant validity (r = 0.40–0.42, ps< 0.001). After obtaining permission from the authors, the original version of the NPOQ was translated into Chinese. The work was conducted via four independent bilingual translators, two whose native language was Chinese and who majored in psychology translated the English version into Chinese, and two bilingual translators whose native language was English translated the Chinese back into English. The translation process was repeated until discrepancies between the original and reverse-translated versions were negligible. Finally, the Chinese version of the NPOQ was proposed and used to test the psychometric properties of it in adolescents.
Depression-Anxiety-Stress Scale
The Depression-Anxiety-Stress Scale (DASS-21) includes three subscales: depression, anxiety and stress. Each subscale incorporates seven items. Participants respond to each by rating the frequency and severity of symptoms experienced in the previous week on a 4-point Likert scale ranging from 0 (“did not apply to me at all”) to 3 (“applied to me very much or most of the time”). Higher scores are indicative of higher levels of negative emotionality. In the current study, the Cronbach’s alpha coefficient of the DASS-21 was 0.94; those of the depression, anxiety, and stress subscales were 0.88, 0.85, and 0.85, respectively.
Penn State Worry Questionnaire
PSWQ is a 16-item questionnaire developed by Meyer et al. (1990). PSWQ measures both frequency and intensity of worry, with participant responding on 5-point Likert scale (from 1 to 5, indicates “not typical at all,” “very typical,” respectively). The total score ranges from 16 to 80, and higher scores indicate a greater tendency to feel worry. In the current study, the Cronbach’s alpha coefficient for the PSWQ was 0.84, and the coefficients of the subscales of “engaging in worry” and “absence of worry” were 0.92 and 0.70, respectively.
Data Analytic Strategy
Measurement Model
Three models were compared by CFA: unitary factor, correlated three-factor, and bifactor. In the unitary factor model, all items were loaded onto one factor. The second model was a correlated three-factor model consisting of three items (1, 8, and 12) with primary loading on Factor 1 (perceived threat), four items (2, 3, 5, and 9) with primary loading on Factor 2 (self-inefficacy), and five items (4, 6, 7, 10, and 11) with primary loading on Factor 3 (negative outcome expectancy). The third model was bifactor model: all 12 items were freely loaded onto a general factor and respective domain-specific factors. The dimensions of the group factor were the same as the classifications for the three-factor model. The covariance of all factors was fixed to zero in the bifactor model (Rodriguez et al., 2016b).
Measurement Invariance
Measurement invariance testing was accomplished via a multiple-group CFA. The best fitting model was tested for different genders and age to ensure that the same measurement model was supported for each group. Next, restrictive models were used to test for: (a) configural invariance (equal forms), (b) metric invariance (equal factor loadings), (c) scalar invariance (equal indicator thresholds), and (d) error invariance (equal indicator residual errors).
Structural Regression Model
A structural regression model was used to test if the factors of the NPOQ were related to the PSWQ and DASS. The structural regression model consisted of simultaneously modeling the selected model for the NPOQ described above, a bifactor model of the PSWQ (Zhong et al., 2009), and a three-factor model of the DASS (Henry and Crawford, 2005). Pathway coefficients were freely estimated from factors of the NPOQ for the PSWQ and DASS.
Model Estimation and Evaluation
All models were tested with Mplus version 7.4 for windows, and were estimated by a maximum likelihood estimation procedure (ML) and principal axis factoring (PAF). When evaluated individual model fit, the chi-square test (χ2), the normed chi-square test (χ2/df), and multiple complementary fit indices were considered. The chi-square test and normed chi-square test are significantly influenced by sample sizes (Hu and Bentler, 1998; Wen et al., 2004). However, results less than 5 for χ2/df usually indicate acceptability (Wen et al., 2004). Multiple complementary fit indices including root mean square error of approximation (RMSEA), Tucker-Lewis Index (TLI), Comparative Fit Index (CFI), and Standardized Root Mean Square Residual (SRMR) were used to evaluate model fit (Hu and Bentler, 1999). The following criteria were used to evaluate fit: CFI and TLI should be equal to or larger than 0.90, RMSEA and SRMR should be equal to or smaller than 0.08 (Hu and Bentler, 1999). When compared the nested models in multiple-group CFA, The indicators mentioned above will also be used. And the difference in CFI is also recommended. A value of △CFI smaller than or equal to 0.01 indicates rationally trivial differences in parameter estimates among models (Cheung and Rensvold, 2002; Meade et al., 2008).
Results
NPOQ Measurement Models
Data-screening analysis was conducted to detect items with skewed and kurtosis distribution. As shown in Table 1, the means of each item ranged from 1.98 to 2.86, with standard deviations from 1.06 to 1.42. The amount of skewness ranged from 0.00 to 0.97, and Kurtosis was from −1.37 to 0.184, suggesting that there was no outlier with extreme non-moral distribution.
Table 1. Descriptive statistics of negative problem orientation questionnaire in Chinese adolescents (N = 754).
Goodness-of-fit statistics for the measurement models appear in Table 2. Both the unitary factor model (Model 1) and correlated three-factor model (Model 2) did not achieve the specified guidelines. For the bifactor model (Model 3), χ2 was significant, but χ2/df was less than 5 (Wen et al., 2004); CFI, TLI, RMSEA, and SRMR achieved the specified guidelines. The bifactor model produced the most favorable fit statistics than others. The factor loadings are presented in Table 3. Almost 12 items on the NPOQ continued to robustly load onto their respective domain-specific factors after controlling for the general factor. The loading on the general factor was nearly larger than the loadings on the domain-specific factors. Moreover, the factor loadings for items 4 and 6 for Factor 3 (negative outcome expectancy) were negative, while the remaining items loaded positively.
Table 2. Goodness-of-fit statistics for tested models.
Table 3. Standardized factor loadings for the bifactor model of NPOQ.
Evaluation of the Bifactor Model
Seven indices including coefficient omega (ω), coefficient omega hierarchical (ωH), coefficient omega hierarchical subscale (ωHs), explained common variance (ECV), item for ECV (I-ECV), and percent uncontaminated correlations (PUC) were used to further evaluate the bifactor model (Rodriguez et al., 2016a,b). The coefficient omega (ω) estimates the proportion of variance in the total score attributable to all “modeled” sources of common variance. The ω for the general factor was 0.93, and the three domain-specific factors were 0.77, 0.86, and 0.85. Thus, both the total scale and three domain-specific factors had good consistent reliability. The coefficient hierarchical omega (ωH) reflects the proportion of variance in the NPOQ scores attributable to the general factor. The ωH for the general factor was 0.87, which was larger than the criterion (ωH > 0.50; Reise, 2012; Gu et al., 2017), indicating that 87% of the variance in the composite NPOQ scores was attributable to the general factor and the homogeneity of the NPOQ was high. Thus, synthesizing the total scores for the NPOQ was meaningful. The coefficient omega hierarchical subscale (ωHS) reflects that the proportion of variance in the scores was attributable to each domain-specific factor, independent of the general factor. The ωHS values were as follows: Factor 1 (perceived threat) = 0.33, Factor 2 (self-inefficacy) = 0.15, and Factor 3 (negative outcome expectancy) = 0.01. Further calculating the ratio of ωHS divided by ω showed that the variation contributed by group factors were 42.9, 17.4, and 1.2%. These results indicated that the homogeneity of Factor 3 (negative outcome expectancy) was relatively low.
ECV quantifies the amount of common variance attributable to general and domain-specific factors. In this study, the general factor accounted for 78% and domain-specific factor explained 22% of the common variance, indicating that the common variance was largely spread across the general NPO. Similarity, The average I-ECV value was 0.76 (values ranged from 0.44 to 0.98, see Table 2), with only four items (items 3, 6, 7, and 9) evidencing I-ECV values larger than 0.80. PUC characterizes the percentage of NPOQ item correlations contaminated by variance attributable to the general and domain-specific factors. The PUC value for the NPOQ was 0.71. The high I-ECV and PUC indicate that many item correlations were attributable to the general factor.
Measurement Invariance of the NPOQ
The results of the measurement invariance test and the fits for all models are summarized in Table 2. We initially separately examined the fits of the bifactor model to the sub-groups (Schmitt and Kuljanin, 2008; Chen et al., 2012). And the bifactor model showed an adequate model fit for boys and girls (see Models 4 and 5), which was a prerequisite for testing the measure of invariance across gender. The model testing for configural invariance (Model 6) achieved an acceptable fit, indicating that both girls and boys possessed the same bifactor structure for the NPOQ. Following this, all factor loadings were constrained to be equal across genders (Model 7). Comparing Model 7 with Model 6 showed that χ2 significantly increased (p = 0.038) but χ2/df was less than 5 (1.569); CFI decreased less than 0.01 (0.004), which supported metric invariance across girls and boys. Model 8 added constraints to intercepts, resulting in a significant χ2 of 23.125 for 8 df (p = 0.003). However, χ2/df was smaller than 5 (2.891) and CFI decreased less than 0.01 (0.003), supporting scalar invariance across girls and boys. Model 9 constrained the error variances to be equivalent across gender, resulting in a significant χ2 of 23.124 for 12 df (p = 0.027) but small χ2/df (1.927 < 5) and CFI (0.002 < 0.01), supporting error invariance across girls and boys. Thus, a strict measurement invariance across genders was supported.
Measurement invariance across age was also tested. As showed in Table 2. The bifactor model showed well model fit for age at 12–13, 14–15, and 16–18 (see Models 10, 12, and 13), which was a prerequisite for testing the measurement invariance across age. The model testing for configural invariance (Model 14) achieved an acceptable fit, indicating that three age groups possessed the same bifactor structure for the NPOQ. Following this, all factor loadings were constrained to be equal across age (Model 15). Comparing Model 14 with Model 15 showed that χ2 significantly increased (p = 0.007) but χ2/df was less than 5 (1.540); CFI decreased less than 0.01 (0.009), which supported metric invariance across age. Model 16 added constraints to intercepts, resulting in a significant χ2 of 7.268 for 10 df (p = 0.700). And χ2/df was smaller than 5 (0.727) and CFI decreased less than 0.01 (0.000), supporting scalar invariance across age. Model 17 constrained the error variances to be equivalent across age, resulting in a significant χ2 of 35.672 for 24 df (p = 0.059) and small χ2/df (1.486 < 5) and CFI (0.003 < 0.01), supporting error invariance across age. Thus, a strict measurement invariance across age was supported.
Structural Regression Model
The incremental validity of the NPOQ’s domain-specific factors over the general factor was examined using a structural regression model that simultaneously modeled the bifactor model of the NPOQ, bifactor model of the PSWQ, and three-factor model of the DASS. The structural regression model provided an adequate fit to the data (χ2 = 2621.425, df = 1065, χ2/df = 2.461; RMSEA = 0.046, CFI = 0.915; TLI = 0.906; SRMR = 0.056). The standardized beta weights from the structural regression model are presented in Table 4. The general factor NPO accounted for the main variance in engagement with worry, depression, anxiety, and stress. Moreover, Factor 1 (perceived threat) was positively correlated with engagement with worry and stress, while Factor 2 (self-inefficiency) and Factor 3 (negative outcome expectancy) accounted for unique variances in the general factor of the PSWQ.
Table 4. Standardized beta weights from structural regression model.
Discussion
The present study supported a bifactor model of the NPOQ in Chinese adolescents, which consists of one general and three domain-specific NPO factors including perceived threat, self-inefficacy, and negative outcome expectancy. Then multiple-group SEM showed that the bifactor model demonstrated strict invariance across gender and age. The results also showed that the general factor NPO accounted for the main variance in engagement with worry, depression, anxiety, and stress. Perceived threat was positively correlated with engagement with worry and stress, and both self-inefficiency and negative outcome expectancy accounted for unique variances in the general factor of the PSWQ. These results indicate that the NPOQ shows good construct and incremental validity for Chinese adolescents.
This study supported a bifactor model of the NPOQ for Chinese adolescents, which is better for our understanding the NPO. In the bifactor model of NPOQ, the general factor NPO captures the whole disruptive cognitive-emotional set for social problems, and the group factors describe special cognitive and emotional characters related to NPO. Of those, perceived threat involves a negative belief in social problem being a threat to health and wellbeing; self-inefficacy is a comprehensive cognitive-emotional set that includes self-doubt over one’s problem-solving ability and related negative emotional reactions such as low self-efficacy; and negative outcome expectancy mainly includes a negative emotional reaction of being pessimistic about outcome, and a cognitive threshold that is a low tolerance for frustration (Ladouceur et al., 1998; Robichaud and Dugas, 2005a). The bifactor model solves the contradiction between the unitary factor model and the concept of NPO in Western samples, and thus more fully reflects NPO’s connotations. From the bifactor model, NPO is not only defined as an entirely disruptive cognitive-emotional set or attitude toward social problems (i.e., a general factor), it is also described via three aspects: perceived threat, self-inefficacy, and negative outcome expectancy (i.e., group factors).
The bifactor model is also significantly useful when using the NPOQ in practice. First, it should consider the general NPO factor, for that accounts for the overwhelming amount of variance in the NPOQ. For this study, the substantial amount of reliable and common variances in the NPOQ was attributable to the general factor NPO (ωH = 0.85, ECV = 0.78); loadings on the general factor were nearly larger relative to the loadings on the group factors (see Table 4). The results indicated that the general NPO factor is a reliable variable and the NPOQ can be represented as a unidimensional model with little parameter bias. Second, the group factors should be considered, for they show special significance in the factor construct of the NPOQ. Both the perceived threat and self-inefficacy factors explained a certain proportion of reliable variance in the NPOQ (42.9 and 17.4%, respectively), and the two factors uniquely predicted engagement with worry, stress, or general worry (see Table 3). Thus, they can be considered separate from the general factor, and using the two group factors is warranted. Though it significantly predicted the worry symptoms, the negative outcome expectancy factor should not be independently interpreted from the general NPO factor, for its omega HS value was low (0.01).
This study also tested the measurement invariance of bifactor model of NPOQ. This study supports configural, metric, scalar, and error measurement invariance across gender and age. The results further evidence the construct validity of NPOQ, and suggest that the NPOQ is most accurately operationalized as consisting of a general factor and three domain-specific factors among both boys and girls, different age groups. Additionally, higher or lower scores on the NPOQ (both general and domain-specific factors) can be considered to reflect true group differences in the construct rather than the NPOQ having items that preclude sub-groups from responding in comparable ways (Vandenberg and Lance, 2000; Schmitt and Kuljanin, 2008). The results of this study provide the enough evidence to show the different NPO levels in boys and girls, different age group reflect the true group difference, which suggested that the NPOQ has the same construct across gender and age.
The incremental validity of the NPOQ was also tested with a structural regression model. The results mainly support the general factor NPO being positively correlated with depression, anxiety, and stress. This finding is similar to those of previous clinical studies that supported the important role of NPO in the development of anxiety and depressive disorders (Fergus et al., 2015; Koerner et al., 2015; Hearn et al., 2017). Moreover, this study also showed that the domain-specific factor of perceived threat was positively related to engagement with worry, and both self-inefficiency and negative outcome expectancy were positively associated with the general factor of worry. These results support the role of domain-specific factors of NPO in adolescent worry. They can be explained via the cognitive model of worry (Dugas et al., 2005; Dugas and Robichaud, 2007), which holds that negative beliefs regarding problems have salient effects on developing and maintaining worry. Specifically, the cognitive model of worry supposes that people with higher levels of NPO tend to consider problems as threats to their wellbeing and believe that social problems will result in negative outcomes; thus, they are usually less effective at problem solving (Robichaud and Dugas, 2005b). These negative cognitive and emotional sets lead people to avoid real problems and engage in worry (Dugas et al., 2005).
Several limitations in the study should be acknowledged. Firstly, this study only used few and limited external criterion variables. It is possible that the domain-specific factors in NPOQ may uniquely contribute to criterion variables not assessed in the study. Secondly, the criterion variables in the study were self-report measures that were fulfilled concurrently with NPOQ. Moreover, only the non-clinical adolescents sample was considered in this study. Future studies should explore it in clinical adolescent samples and survey in a longitudinal time to precisely test the predictive effects of general factor and domain-specific factors in NPOQ. Although existed outstanding limitations, this study provide support for a bifactor model of NPOQ with a general NPO factor and three domain factors in Chinese adolescents and measurement invariance across gender. The bifactor model not only provides adequate construct for NPOQ but also benefit to our understanding the concept of NPO in Chinese adolescents. This research also suggests that when using the NPOQ, the total score should be first considered, and its two sub-scales, perceived threat and self-inefficiency, should be analyzed subsequently because they have unique effects on the measure.
Data Availability Statement
All datasets generated for this study are included in the article/supplementary material, further inquiries can be directed to the corresponding authors.
Ethics Statement
The studies involving human participants were reviewed and approved by the Academic Committee of Fujian Normal University. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.
Author Contributions
RL and HX conceived the original idea for the study and wrote the study manuscript. QW and SS managed the day to carry out the data collection. RL and YY supervised this study. All the authors participated in the conception and design of the work, read and approved the final manuscript, and believe that the manuscript represents valid work.
Funding
This research was supported by the four middle schools in Fujian province. This research was supported by a grant from the Program of Cultivating Outstanding Youth Scientific Research Talents in Colleges and Universities of Fujian Province, PRC and the Selected Topics of Outstanding Doctoral Dissertation of Philosophy and Social Science under the guidance of Marxism, funding by Fujian Provincial Party Committee Publicity Department, Fujian Provincial Education Department, and Fujian Provincial Philosophy and Social Science Committee, PRC.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
We would like to thank all of the participants in this study.
References
Anderson, R. J., Goddard, L., and Powell, J. H. (2011). Social problem-solving and depressive symptom vulnerability: the importance of real-life problem-solving performance. Cognit. Ther. Res. 35, 48–56. doi: 10.1007/s10608-009-9286-2
Balsamo, M., Imperatori, C., Sergi, M. R., Murri, M. B., Continisio, M., Tamburello, A., et al. (2013). Cognitive vulnerabilities and depression in young adults: an roc curves analysis. Depress. Res. Treat. 2013:407602. doi: 10.1155/2013/407602
Barahmand, U. (2008). Age and gender differences in adolescent worry. Pers. Individ. Dif. 45, 778–783. doi: 10.1016/j.paid.2008.08.006
Becker-Weidman, E. G., Jacobs, R. H., Reinecke, M. A., Silva, S. G., and March, J. S. (2010). Social problem-solving among adolescents treated for depression. Behav. Res. Ther. 48, 11–18. doi: 10.1016/j.brat.2009.08.006
Bell, A. C., and D’Zurilla, T. J. (2009). Problem-solving therapy for depression: a meta-analysis. Clin. Psychol. Rev. 29, 348–353. doi: 10.1016/j.cpr.2009.02.003
Bell, A. C., and D’Zurilla, T. (2009). The influence of social problem-solving ability on the relationship between daily stress and adjustment. Cognit. Ther. Res. 33, 439–448. doi: 10.1007/s10608-009-9256-8
Carlucci, L., Watkins, M. W., Sergi, M. R., Cataldi, F., Saggino, A., and Balsamo, M. (2018). Dimensions of anxiety, age, and gender: assessing dimensionality and measurement invariance of the state-trait for cognitive and somatic anxiety (STICSA) in an Italian sample. Front. Psychol. 9:2345. doi: 10.3389/fpsyg.2018.02345
Chen, F. F., Hayes, A., Carver, C. S., Laurenceau, J.-P., and Zhang, Z. (2012). Modeling general and specific variance in multifaceted constructs: a comparison of the bifactor model to other approaches. J. Pers. 80, 219–251. doi: 10.1111/j.1467-6494.2011.00739.x
Cheung, G. W. (2008). Testing equivalence in the structure, means, and variances of higher-order constructs with structural equation modeling. Organ. Res. Methods 11, 593–613. doi: 10.1177/1094428106298973
Cheung, G. W., and Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Struct. Equ. Modeling 9, 233–255. doi: 10.1207/S15328007SEM0902_5
Ciarrochi, J., Leeson, P., and Heaven, P. C. L. (2009). A longitudinal study into the interplay between problem orientation and adolescent well-being. J. Couns. Psychol. 56, 441–449. doi: 10.1037/a0015765
Clarke, J. B., Ford, M., Heary, S., Rodgers, J., and Freeston, M. H. (2017). The relationship between negative problem orientation and worry: a meta-analytic review. Psychopathol. Rev. 4, 319–340. doi: 10.5127/pr.034313
D’Zurilla, T. J., Chang, E. C., and Sanna, L. J. (2004). “Social problem solving: current status and future directions,” in Social Problem Solving: Theory, Research, and Training, eds E. C. Chang, T. J. D’Zurilla, and L. J. Sanna (Washington, DC: American Psychological Association), 241–253. doi: 10.1037/10805-014
D’Zurilla, T. J., Maydeu-Olivares, A., and Kant, G. L. (1998). Age and gender differences in social problem-solving ability. Pers. Individ. Dif. 25, 241–252. doi: 10.1016/S0191-8869(98)00029-4
Dugas, M. J., Marchand, A., and Ladouceur, R. (2005). Further validation of a cognitive-behavioral model of generalized anxiety disorder: diagnostic and symptom specificity. J. Anxiety Disord. 19, 329–343. doi: 10.1016/j.janxdis.2004.02.002
Dugas, M. J., and Robichaud, M. (2007). Cognitive-Behavioral Treatment For Generalized Anxiety Disorder: From Science To Practice. New York, NY: Routledge, doi: 10.1023/A:1026514712357
Dupuy, J. B., and Ladouceur, R. (2008). Cognitive processes of generalized anxiety disorder in comorbid generalized anxiety disorder and major depressive disorder. J. Anxiety Disord. 22, 505–514. doi: 10.1016/j.janxdis.2007.05.010
Erdley-Kass, S. D., Kass, D. S., Gellis, Z. D., Bogner, H. A., Berger, A., and Perkins, R. M. (2018). Using problem-solving therapy to improve problem-solving orientation, problem-solving skills and quality of life in older hemodialysis patients. Clin. Gerontol. 41, 424–437. doi: 10.1080/07317115.2017.1371819
Fergus, T. A., Valentiner, D. P., Wu, K. D., and McGrath, P. B. (2015). Examining the symptom-level specificity of negative problem orientation in a clinical sample. Cognit. Behav. Ther. 44, 153–161. doi: 10.1080/16506073.2014.987314
Fergus, T. A., and Wu, K. D. (2010). Do symptoms of generalized anxiety and obsessive–compulsive disorder share cognitive processes? Cognit. Ther. Res. 34, 168–176. doi: 10.1007/s10608-009-9239-9
Gu, H., Wen, Z., and Fan, X. (2017). Examining and controlling for wording effect in a self-report measure: a Monte Carlo simulation study. Struct. Equ. Modeling 24, 545–555. doi: 10.1080/10705511.2017.1286228
Hawkins, D., Sofronoff, K., and Sheffield, J. (2009). Psychometric properties of the social problem solving inventory-revised short-form: is the short form a valid and reliable measure for young adults? Cognit. Ther. Res. 33, 462–470. doi: 10.1007/s10608-008-9209-7
Hearn, C. S., Donovan, C. L., Spence, S. H., and March, S. (2017). A worrying trend in social anxiety: to what degree are worry and its cognitive factors associated with youth social anxiety disorder? J. Affect. Disord. 208, 33–40. doi: 10.1016/j.jad.2016.09.052
Henry, J. D., and Crawford, J. R. (2005). The short-form version of the depression anxiety stress scales (DASS-21): construct validity and normative data in a large non-clinical sample. Br. J. Clin. Psychol. 44(Pt 2), 227–239. doi: 10.1348/014466505x29657
Hu, L., and Bentler, P. M. (1998). Fit indices in covariance structure modeling: sensitivity to underparameterized model misspecification. Psychol. Methods 3, 424–453. doi: 10.1037//1082-989X.3.4.424
Hu, L., and Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance struc-ture analysis: conventional criteria versus new alternatives. Struct. Equ. Modeling 6, 1–55. doi: 10.1080/10705519909540118
Humphrey, K. (2016). Investigating the efficacy of a targeted youth intervention on negative problem orientation. J. Public Ment. Health 15, 209–215. doi: 10.1108/JPMH-08-2016-0037
Kertz, S., and Woodruff-Borden, J. (2012). The role of metacognition, intolerance of uncertainty, and negative problem orientation in children’s worry. Behav. Cogn. Psychother. 41, 243–248. doi: 10.1017/S1352465812000641
Koerner, N., Tallon, K., and Kusec, A. (2015). Maladaptive core beliefs and their relation to generalized anxiety disorder. Cogn. Behav. Ther. 44, 441–455. doi: 10.1080/16506073.2015.1042989
Konstantellou, A., Campbell, M., Eisler, I., Simic, M., and Treasure, J. (2011). Testing a cognitive model of generalized anxiety disorder in the eating disorders. J. Anxiety Disord. 25, 864–869. doi: 10.1016/j.janxdis.2011.04.005
Ladouceur, R., Blais, F., Freeston, M. H., and Dugas, M. J. (1998). Problem solving and problem orientation in generalized anxiety disorder. J. Anxiety Disord. 12, 139–152. doi: 10.1016/S0887-6185(98)00002-4
Meade, A. W., Johnson, E. C., and Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. J. Appl. Psychol. 93, 568–592. doi: 10.1037/0021-9010.93.3.568
Meyer, T. J., Miller, M. L., Metzger, R. L., and Borkovec, T. D. (1990). Development and validation of the penn state worry questionnaire. Behav. Res. Ther. 28, 487–495. doi: 10.1016/0005-7967(90)90135-6
Nadort, M., Arntz, A., Smit, J. H., Giesen-Bloo, J., Eikelenboom, M., Spinhoven, P., et al. (2009). Implementation of outpatient schema therapy for borderline personality disorder with versus without crisis support by the therapist outside office hours: a randomized trial. Behav. Res. Ther. 47, 961–973. doi: 10.1016/j.brat.2009.07.013
Nezu, A. M., and D’Zurilla, T. J. (2006). Problem-Solving Therapy: A Positive Approach To Clinical Intervention. New York, NY: Springer Publishing Company.
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behav. Res. 47, 667–696. doi: 10.1080/00273171.2012.715555
Reise, S. P., Kim, D. S., Mansolf, M., and Widaman, K. F. (2016). Is the bifactor model a better model or is it just better at modeling implausible responses? Application of iteratively reweighted least squares to the Rosenberg Self-Esteem Scale. Multivariate Behav. Res. 51, 818–838. doi: 10.1080/00273171.2016.1243461
Robichaud, M., and Dugas, M. J. (2005a). Negative problem orientation (Part I): psychometric properties of a new measure. Behav. Res. Ther. 43, 391–401. doi: 10.1016/j.brat.2004.02.007
Robichaud, M., and Dugas, M. J. (2005b). Negative problem orientation (Part II): construct validity and specificity to worry. Behav. Res. Ther. 43, 403–412. doi: 10.1016/j.brat.2004.02.008
Rodriguez, A., Reise, S. P., and Haviland, M. G. (2016a). Applying bifactor statistical indices in the evaluation of psychological measures. J. Pers. Assess. 98, 223–237. doi: 10.1080/00223891.2015.1089249
Rodriguez, A., Reise, S. P., and Haviland, M. G. (2016b). Evaluating bifactor models: calculating and interpreting statistical indices. Psychol. Methods 21, 137–150. doi: 10.1037/met0000045
Romano, M., Moscovitch, D. A., Ma, R., and Huppert, J. D. (2019). Social problem solving in social anxiety disorder. J. Anxiety Disord. 68, 102–152. doi: 10.1016/j.janxdis.2019.102152
Saggino, A., Balsamo, M., Carlucci, L., Cavalletti, V., Sergi, M. R., da Fermo, G., et al. (2018). Psychometric properties of the Italian version of the young schema questionnaire L-3: preliminary results. Front. Psychol. 9:312. doi: 10.3389/fpsyg.2018.00312
Schmitt, N., and Kuljanin, G. (2008). Measurement invariance: review of practice and implications. Hum. Resour. Manage. Rev. 18, 210–222. doi: 10.1016/j.hrmr.2008.03.003
Vandenberg, R. J., and Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ. Res. Methods 3, 1–70. doi: 10.1177/109442810031002
Wen, Z. L., Hau, K.-T., and Herbert, W. M. (2004). Structural equation model testing: cutoff criteria for goodness of fit indices and chi-square test. Acta Psychol. Sinica 36, 186–194. doi: 10.1007/BF02911031
Zhong, J., Wang, C., Li, J., and Liu, J. (2009). Penn state worry questionnaire: structure and psychometric properties of the Chinese version. J. Zhejiang Univ. Sci. B 10, 211–218. doi: 10.1631/jzus.B0820189
Keywords: Negative Problem Orientation Questionnaire, negative problem orientation, bifactor model, measurement invariance, adolescents, China
Citation: Xiao H, Lin R, Wu Q, Shen S and Yan Y (2020) Negative Problems Orientation Questionnaire for Chinese Adolescents: Bifactor Model and Measurement Invariance. Front. Psychol. 11:608676. doi: 10.3389/fpsyg.2020.608676
Received: 21 September 2020; Accepted: 20 November 2020;
Published: 10 December 2020.
Edited by:
Michela Balsamo, University of Studies G. d’Annunzio Chieti and Pescara, ItalyReviewed by:
Wei Chen, Guizhou Normal University, ChinaJavier Ortuño Sierra, University of La Rioja, Spain
Candido J. Ingles, Miguel Hernández University of Elche, Spain
Copyright © 2020 Xiao, Lin, Wu, Shen and Yan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Rongmao Lin, lrm990527@hotmail.com; Youwei Yan, yanyouwei123@sina.com
†These authors share first authorship