Investigating Psychometrics of Career Decision Ambiguity Tolerance Scale

Park, In-Jo; Hai, Shenyang; Lee, Seungmi; Sohn, Youngwoo

doi:10.3389/fpsyg.2019.02067

ORIGINAL RESEARCH article

Front. Psychol., 10 September 2019

Sec. Educational Psychology

Volume 10 - 2019 | https://doi.org/10.3389/fpsyg.2019.02067

Investigating Psychometrics of Career Decision Ambiguity Tolerance Scale

$\nIn-Jo Park$ In-Jo Park¹

Shenyang Hai¹

Seungmi Lee²

Youngwoo Sohn²^*

¹Department of Psychology, Henan University, Kaifeng, China
²Department of Psychology, Yonsei University, Seoul, South Korea

This study aimed to validate career decision ambiguity tolerance scale-Korean form applicable to a Korean sample. In study 1, 17-items from the original 18-item career decision ambiguity tolerance scale were valid based on IRT. In study 2, using the confirmatory factor analysis, we showed that excluding item 4 from the original scale is better than including it in the three factors model. Given the results of study 1 and 2, the constructs in the 17-item career decision ambiguity tolerance scale-Korean form were valid. In study 3, career decision ambiguity tolerance positively predicted career decision-making self-efficacy, career indecision, and career adaptability, respectively, after controlling for calling and career search self-efficacy. Thus, the incremental validity of the career decision ambiguity tolerance scale-Korean form was ensured. In study 4, the reliability of the scale was retained as the test-retest (conducted over a 4-week period) demonstrated adequate results.

Introduction

In the twenty first century, it is a worldwide phenomenon for individuals to have difficulty in career decisions because they have difficulty collecting information to fit an appropriate career into a rapidly changing global environment and instability of economy. Korea is one such country, where companies are laying off senior employees and are stagnant in hiring new employees (Kim and Yoon, 2015). Individuals are confronted with career decision difficulty and should tolerate ambiguity in terms of determining their career (Xu and Tracey, 2014a). Tolerance for career decision ambiguity appears to be one of the most important career issues for both college students and employees considering turnover or seeking a second career after retiring. In this vein, it is necessary for psychologists and career counselors to detect and evaluate the degree to which individuals can tolerate ambiguity in their career decision process. Accordingly, Xu and Tracey (2015) conducted research in which they developed and validated a career decision ambiguity tolerance scale.

Ambiguity tolerance is a personality variable, reflecting an individual's emotional and perceptual aspect (Frenkel-Brunswik, 1949). Budner (1962) defined ambiguity tolerance as “the tendency to perceive ambiguous situations as desirable.” It influences reactions to a spectrum of situational features suitable in various life contexts (Furnham and Marks, 2013). Empirical research showed that ambiguity tolerance is related to better performance, willingness to violate ethical norms, and consumers' willingness to pay for remanufactured products (Teoh and Foo, 1997; Weisbrod, 2009; Hazen et al., 2012). On the other hand, intolerance of ambiguity is associated with challenge appraisal, life satisfaction, and positive affect (Bardi et al., 2009). Regarding career behaviors, ambiguity tolerance predicts career decision-making self-efficacy (Xu and Tracey, 2014a) and indirectly predicts career indecision via environmental and self-exploration (Xu and Tracey, 2014b).

Xu and Tracey (2015) recently adapted the ambiguity tolerance construct and introduced the idea of career decision ambiguity tolerance (CDAT) to career research. Based on Budner's (1962) tripartite model of ambiguity tolerance which consisted of three constructs, namely tolerance toward unfamiliarity, complexity, and inconsistent information, they defined CDAT as “people's evaluations of and responses to unfamiliar, complex, or inconsistent information in career decision making” (Xu and Tracey, 2015).

CDAT would be frequently utilized to explore individuals' career decision process. Accordingly, the CDAT scale, which was developed and validated by Xu and Tracey (2015), consists of three subscales containing 18 statements in total for three subscales. These three subscales are preference, tolerance, and aversion, in which each factor contains 6 statements. This original scale's validity and reliability were ensured through exploratory and confirmatory factor analysis, incremental validity, and test-retest reliability. However, though cross-cultural validation studies can greatly benefit career researchers and career counselors, no research has been conducted cross-culturally to extend the CDAT scale's validity in Korea. For instance, validating the Korean CDAT scale could allow researchers to investigate its role on career decision and to provide career counselors with a sound psychometric tool for evaluating their Korean clients.

We are thus interested in the Mokken and Rasch rating models in the Item Response Theory Analysis (IRT) for validating the CDAT scale-Korean form in this study. The IRT is useful for investigating scale validity because it provides psychometric characteristics with many indices (e.g., scalability, item difficulty, and point measure correlation). The Mokken model was originally used to analyze dichotomous data, and a model of polytomous data in a Likert scale was subsequently introduced (Molenaar, 1982). The Mokken model assesses whether each item evaluates the same underlying concept. The scalability of a scale is evaluated by Loevinger's coefficient H, which compares the actual Guttman errors to the items' expected errors (Crichton, 1999; Sijtsma et al., 2011). Also, existing scales are validated with psychometric evaluation using the Rasch rating model (Park et al., 2015). The Rasch rating model was developed for polytomous items. This model assesses whether the difference between any given threshold location for items and the mean of the threshold locations are coherent across items (Wright et al., 1994). The Rasch rating model reports several indices including item difficulty, point measure correlation, and difference item function, which is useful for confirming scales' construct validity.

The current study aimed to validate the CDAT scale-Korean form. To validate this scale, we first attempted to evaluate the psychometric properties for each CDAT scale item using the Rasch rating and Mokken models. Second, confirmatory factor analysis was performed to compare model fit between the different two models. Third, we explored incremental validation through hierarchical regression analysis. Lastly, a test-retest was conducted with 4-weeks' delay in order to confirm the CDAT scale-Korean form's reliability.

Study 1: Item Response Theory Analysis

Study 1 aimed to validate the CDAT scale-Korean form using the Mokken and the Rasch rating models. The Mokken model reports indices confirming the scalability of each item. The Rasch model provides indices estimating item difficulty and data fit relative to the model. These two models are useful for ensuring the scale's construct validity.

Methods

Participants and Procedures

In total, 323 participants were part of this study. All participants were attending psychology courses from 7 universities, including Calvin, Chonnam, Choonsing Cyber, Dankuk, Gwandong, and Gyeonsang Universities in South Korea. Furthermore, 87 (26.9%) were male and 236 (73.1%) were female. The mean age was 31.64 (SD = 11.91), though ages ranged from 19 to 64 years.

Surveys were administered after the authors explained the current study's goals. We received consent forms from all participants in this study. The participants accessed a website (www.surveymonkey.com) using their smartphone or laptop computer to respond to the survey questions. If a participant did not respond to any question, he or she could not finish the survey. The duration including the study's explanation and the questionnaire response time was about 7 min. Participants received course credits if they completed the entire survey questionnaire.

This study was conducted in accordance with the Declaration of Helsinki and was approved by Yonsei University's review board. All participants gave written informed consent in accordance with the Declaration of Helsinki.

Instrument

The career decision ambiguity tolerance scale

The CDAT scale (Xu and Tracey, 2015) is an 18-item measure designed to assess an individual's evaluation and response to the career decision process, based on the Budner's tripartite model (Budner, 1962). The CDAT scale consists of three domains, namely preference (e.g., “It is interesting to discover new strengths and weakness”), tolerance (e.g., “I am tolerant of the potential difference between my perception and the reality of a career”), and aversion (e.g., “I find difficult to make career decisions as things cannot be predicted clearly”); each domain, respectively contained 6-items. Following the translation guidelines of Beaton et al. (2000), the authors used forward-backward and counter-translation to translate the CDAT scale into Korean for the current study. Participants were asked to rate a 7-point Likert scale, which ranged from 1 (strongly disagree) to 7 (strongly agree). Xu and Tracey (2015) found a Cronbach's alpha of 0.79, 0.69, and 0.83, respectively for the preference, tolerance, and aversion factors. This study also showed a Cronbach's alpha of 0.79, 0.76, and 0.83, respectively for the preference, tolerance, and aversion factors.

Analyses

In the current study, we performed the Mokken model analyses using MSPWIN5 (Molenaar et al., 2000a). The Mokken model assumes there is an underlying continuum applied to the scale's items (Schmitz et al., 2000). In this study, the preference, tolerance, and aversion subscales were considered as underlying, latent constructs. This model reports a scale coefficient, Loevinger's H, to ensure the scalability for both the individual scale items and the scale itself. An H value should be ≥0.3 for it to be an acceptable value for determining the scalability for each item and the overall scale (Carter et al., 2015).

Rasch model analyses were conducted using WINSTEPS 3.81.0 for Windows (Linacre, 2014). Data fit, item difficulty, and response category relevance were calculated using the Rating Scale Model (RSM; Wright and Masters, 1982). In the RSM, the ability of the individual indicates the measured psychological trait's strength, and item difficulty indicates the trait strength required for individuals to typically agree with an item. The RSM is designed for polytomous items, which are effective in validating short versions of scales.

Results

Loevinger's H for all items is provided in Table 1. In the current study, item 4 (i.e., “I am not interested in knowing new information about myself”) had weak scalability (H = 0.27). The H values of all other items were ≥0.3, which indicated that the other 17-items have acceptable scalability. The overall scale H for preference, tolerance, and aversion, respectively, were 0.41, 0.37, and 0.47, which was acceptable as subscale. Rho was used as scale reliability; an estimated value above 0.70 is considered to indicate a reliable scale (Molenaar et al., 2000b). The subscales of preference (Rho = 0.74), tolerance (Rho = 0.78), and aversion (Rho = 0.83) all had good reliability.

TABLE 1

Table 1. H-coefficients for CDAT-Korean form items (N = 323).

Unidimensionality was examined through principal component analysis of residuals (PCA). Principal component analysis is used to verify the Rasch model. Because the Rasch model requires unidimensionality, principal component analysis is used to test whether a substantial factor exists in the residuals after the main measurement dimension has been estimated (Smith, 2002). Linacre (2014) suggested that the eigenvalue of unexplained variance explained by the first contrast should be below 20%. In this study, we separately conducted the Rasch model with each factor (i.e., preference, tolerance, and aversion factors). The obtained eigenvalue of unexplained variance explained by the first contrast for the preference, tolerance, and aversion factors, respectively, was 14.2, 18.4, and 12.7%; hence, the data met Linacre's criterion.

The absence of redundant items and item homogeneity can be confirmed by fit statistics (Linacre and Wright, 1994). Fit indices are calculated by comparing obtained responses of an item with the expected score generated by the Rasch model. The expected score expresses the item's construct validity and possible responses (Bond and Fox, 2007). Table 2 shows Infit and Outfit statistics for all 18 CDAT scale items. If the Infit and Outfit values of an item equal 1.0, the item perfectly fits within the expected score. If the fit value is below 0.5 or above 1.5, the item is not validly useable. Item 4 violated this criterion as it showed 1.67 and 1.66, respectively, for Infit and Outfit. The other 17-items for fit indices varied from 0.66 to 1.48; hence, 17-items met the Infit and Outfit criterion [i.e., 0.5 ≤ MNSQ (mean square residual) ≤ 1.5].

TABLE 2

Table 2. Statistics of Rasch model for 323 participants.

Item difficulty estimates and item characteristic curves (ICCs) were obtained for all 18-items. Item difficulty estimates for all items ranged from −0.50 to 0.51. Seven items returned item difficulty estimates of >0 (more difficult items) and the other 11 items returned difficulty estimates of <0 (easier items) (see Table 1). Item 10, with an item difficulty estimate of −0.50, was the easiest item; item 12, with an item difficulty estimate of 0.51, was the most difficult.

We also verified the CDAT scale's content validity. Content validity expresses whether the items evaluate all aspects of the construct they were intended to assess (Bryant, 2000). Point measure correlations are used to assess the items' content validity (Guiberson and Rodriguez, 2014). The point measure correlation value, ranging from −1 to 1, should be ≥0.3 to ensure adequate content validity (Bond and Fox, 2007; Linacre, 2014). In our study, the point measure correlation values of all items were above 0.30 (see Table 2).

Study 2: Confirmatory Factor Analysis

In study 1, item 4's scalability and fit indices failed to ensure its validity. The authors wondered whether the structural factor by excluding item 4 from the CDAT scale would exacerbate the validity of this scale. The purpose of study 2 was to compare one model (i.e., inclusion of item 4) with the other (i.e., exclusion of item 4) using confirmatory factor analysis.