Establishing circularity: development and validation of the circular work value scale (CWVS)

Schneider, Jannick; Striebing, Clemens; Hochfeld, Katharina; Lorenz, Timo

doi:10.3389/fpsyg.2024.1296282

ORIGINAL RESEARCH article

Front. Psychol., 05 April 2024

Sec. Quantitative Psychology and Measurement

Volume 15 - 2024 | https://doi.org/10.3389/fpsyg.2024.1296282

Establishing circularity: development and validation of the circular work value scale (CWVS)

Jannick Schneider^1,2^*

Clemens Striebing¹

Katharina Hochfeld¹

Timo Lorenz²

¹Center for Responsible Research and Innovation, Fraunhofer Institute for Industrial Engineering, Berlin, Germany
²Department of Psychology, MSB Medical School Berlin, Berlin, Germany

Objectives: Addressing the lack of German-language instruments, this study aims to develop a questionnaire that enables the measurement of work values. According to the theory of basic human values (Schwartz, 1992), a culturally fitting questionnaire is validated by covering constructs in the four broader dimensions of Social, Prestige, Intrinsic, and Extrinsic work values. Convergent, discriminant and incremental congruent validity are assessed.

Method: Data were collected in a cross-sectional online-based panel survey. Individuals working more than 20 h per week were included (N = 1,049). Using a genetic algorithm, an economical and valid questionnaire was designed to assess work values.

Results: The 11 work values are measurable with three items each. They provide a good fit to the data with support for strict measurement invariance. The empirical associations to estimate construct validity overall reflect expected relations to social and individualistic work motives, neuroticism, environmental awareness, and basic values. Furthermore, congruent incremental validity is supported with relations to value congruence of the person–organization fit, and multidimensional scaling supports the assumed theoretical circularity of the work values.

Implications: This study developed a questionnaire that enables a theory-based valid measurement of work values. The questionnaire allows practitioners to economically collect information about the value structure of employees or applicants. Future research should consider the development of work values over time and investigate whether more distinctive constructs provide a better fit in the nomological network.

1 Introduction

Values play a significant role in the realm of organizations and work, attracting considerable interest from researchers and practitioners alike. For one, practitioners can be interested in capturing the value structure in personnel selection to identify individuals with the “right” values (Anglim et al., 2022). Values can provide insight into cultural fit, likelihood of quitting, and possibly even job performance. Furthermore, congruence between values and behavior is associated with well-being and reduced stress (Bojanowska et al., 2022). Research supports correlations to relevant outcome variables based on values (Glazer et al., 2004; Fischer and Smith, 2006; Anglim et al., 2022) such as attitudes toward diversity (Anglim et al., 2019), organizational commitment, and organizational citizenship behaviors (Arieli et al., 2020). Consequently, personal values have broad application possibilities for researchers and practitioners in personnel selection or organizational development.

To measure the value structure of employees or applicants, researchers and practitioners often rely on questionnaires that capture the general value structure (e.g., Schwartz, 2021). However, these questionnaires are intended to measure context-free universal value dimensions (Schwartz, 2016). The goal of these questionnaires is to capture values that apply in all situations across all domains of life. Thus, no specific value expressions are represented in the work context (Consiglio et al., 2017), and their use may disfavor applicant responses in selection settings due to low job references (Uggerslev et al., 2012). Hence, contextual item formulations are preferable in the work context due to their higher predictive and content validity (Shaffer and Postlethwaite, 2012; Moldzio et al., 2021; Potočnik et al., 2021; Sackett et al., 2022). Examples are contextualized traits like conscientiousness and emotional stability, yielding higher incremental validity above the non-contextualized counterparts. Against this background, this article seeks to develop and validate a contextualized questionnaire using a genetic algorithm to measure personal values represented in work contexts.

2 Literature review: from basic to contextualized work values

Values are cognitive representations of motives and secondary drivers of behavior (Schwartz, 1992, 2021; Kooij et al., 2011). These values play a significant role in influencing human action and decision-making as cross-situational goals that vary in importance (Arieli et al., 2020). The value structure of individuals and other social units function as guiding principles through which behavioral outcomes are evaluated as more or less desirable (Schwartz, 1994).

Schwartz’s (1992) theory of basic human values established a widely used and empirically supported theoretical framework that is applied across multiple areas of psychology (e.g., political attitudes and organizational contexts; Davidov et al., 2008; Sagiv and Schwartz, 2022). A total of 10 universal, cross-context values (see Table 1) have been supported in their discriminant, predictive, and factorial validity in samples from more than 80 countries (Schwartz, 1992, 1994; Schwartz and Boehnke, 2004; Schwartz et al., 2012). The 10 values can be expanded to a refined theory of basic values with 19 more narrowly defined constructs that represent the motivational continuum in a more distinctive way and specify the broader dimensions (Schwartz et al., 2012). The theoretical values represent two pairs of higher value dimensions with contrasting motivational bases and desirable goals (Schwartz, 2021): Openness to Change (importance of independent thinking, acting, feeling, challenging, and changing) contrasted with Conservation (importance of self-retention, securing the past, order, and resistance to change), Self-Enhancement (importance of one’s own interests, relative success and dominance over others) in contrast to Self-Transcendence (concern for the well-being and interests of others).

Table 1

Table 1. Definitions of work values and corresponding basic values.

According to Schwartz (2021), values form a coherent structure in a circular arrangement. The values line up according to the compatibility or conflict potential of the underlying goals and higher value dimensions. Thus, values with conflicting goals are further apart in the circular arrangement than values representing more compatible goal states (Schwartz, 1992, 2021). This assumes that compatible values guide similar perceptions, preferences, and behaviors (underlying goals are more likely to be pursued in similar actions). In contrast, when values conflict, the pursuit of one goal opposes the pursuit of another goal (Schwartz, 1992, 2021; Maio et al., 2009).

De Clercq et al. (2008) argued that the theory of basic human values should be applied to the work context for consideration of employees’ values. The theory provides a universal, cross-cultural context that relates individual values at work to each other and can cluster individual and organizational values. Ros et al. (1999) described work values as the specific expression of basic values in the work context. Like basic values, they are guiding principles which are hierarchically ordered according to their importance. Furthermore, they reflect the subjective importance of work-related goals and describe what a person expects from his or her work.

The four dimensions of basic values (Self-Transcendence, Self-Enhancement, Openness to Change, and Conservation) reflect Social, Prestige, Intrinsic, and Extrinsic work values in the circular arrangement in work contexts (Ros et al., 1999; Johnson, 2001; Krumm et al., 2013; Borg et al., 2019; Sulistiobudi and Hutabarat, 2022). Social work values reflect the meaningfulness of positive social relationships and the possibility to contribute to society. Prestige-related work values represent goals regarding power, authority, influence, and success at work. Autonomy, enjoyment, and creativity are expressions of Intrinsic work values. In contrast, Extrinsic work values relate to job security and maintaining order in an employee’s life. The definitions of the basic values applied to the work context are shown in Table 1.

Albrecht et al. (2020) supported the extension of the work values by an 11th construct. Based on Exploratory Factor Analysis, they identified the value of Environmental Sustainability (subsumed under Universalism in the theory of basic human values; Schmidt et al., 2007) as an independent dimension. Here, the increasing subjective significance of aspects such as corporate social responsibility and environmental protection in work contexts is evident (Albrecht et al., 2020). Additionally, Albrecht et al. (2020) understood Safety as the importance of safety climate (interpersonal), which subordinates the value under the dimension of Self-Transcendence/Social. Nonetheless, Schwartz et al. (2012) emphasize that Safety can be considered both self-related and social. Simultaneously, the definition, according to Consiglio et al. (2017), contains further aspects beyond the physical safety climate. This will be considered in the following section.

2.1 Measurement of contextualized work values

As mentioned above, to measure values in a work-related manner, appropriate questionnaires with contextual relevance and high predictive validity are needed. These questionnaires should reflect the theory of basic human values in the work context. To our knowledge, there are no questionnaires available in the German work context that replicate and validate the factor structure of Schwartz’s basic values (2021).

2.1.1 German-language questionnaires

The German adaptation of the Super Work Value Measure (Seifert and Bergmann, 1983) is not aligned with the theory of basic human values and is outdated. With their Munster Work Value Measure, Krumm et al. (2013) captured single items per value and not the 10 broader value constructs. They extend the basic values by 11 additional ones, including values that are especially relevant to older workers. Stiglbauer et al. (2022) developed a questionnaire to assess work orientations partially based on individual’s values in the context of employer branding. They integrate various aspects of the meaning of work under generational differences. The relevant circularity of the theory of basic human values is not studied in the work context, and various theoretical assumptions are integrated in their approach. To some extent, the questionnaire focusses more generally on the expectations of an employer and not on the work itself. Moreover, there are variations in values across cultures and diverse intercultural perspectives on work (Shi et al., 2023). Conducting an initial assessment to ascertain the applicability and validity of Schwartz’s theory of basic human values within the context of German workplaces is imperative.

2.1.2 Questionnaires in other cultural contexts

Outside the German context, various questionnaires do not adequately represent the theory of basic human values or fail to replicate its proposed factor structure in work contexts (Arciniega and González, 2000; Avallone et al., 2010; Busque-Carrier et al., 2022). This is critical because Schwartz’s theory serves as a broad theoretical framework which specifies the interrelations of universal values. The broad value conceptualizations are essential to differentiate between distinct motivational bases and can be used to adequately aggregate work value items from a wide variety of work value lists. Therefore, the theory provides a valid foundation for researching work values under the premise of replicating the motivational continuum in work contexts (De Clercq et al., 2008). Consiglio et al. (2017) were able to support the application of universal values in the work context for the first time using ranking scales. More recently, Albrecht et al. (2020) extended the use of the questionnaire by Consiglio et al. (2017) with rating scales. This has the advantage of being able to compare longer lists of values, and participants do not have to discriminate between the importance of individual values (Alwin and Krosnick, 1985; Schwartz, 1994; Sagiv and Schwartz, 2022).

2.2 Aim of this study

In the present study, a German work value questionnaire will be developed and validated based on the Values at Work Scale from Albrecht et al. (2020). We pursue the following objectives.

2.2.1 Translation, adaptation, and advancement of the Values at Work Scale

We want to adapt the questionnaire from Albrecht et al. (2020) and develop a culturally fitting questionnaire with accurate validation procedures (MacKenzie et al., 2011; International Test Commission, 2017; Boateng et al., 2018). In particular, the consideration of content validity and the cultural fit of the item formulation plays a significant role. Moreover, the value of Environmental Sustainability needs to be operationalized more comprehensively for higher measurement accuracy. In the initial version, the value of Safety should include items for both intrapersonal and interpersonal Safety to meet theoretical propositions (Schwartz et al., 2012; Consiglio et al., 2017).

2.2.2 Construct validity

To evaluate factorial validity, the questionnaire will be generated as economically as possible and with a good fit to the training data based on a genetic algorithm (Schultze, 2017). As a result, we expect a good fit of the measurement model to the test data (H1) in Confirmatory Factor Analysis (CFA). Building on the theoretical circular arrangement of Schwartz’ values, we expect a continuous circular pattern (H2) comparable to Schwartz et al. (2001, 2012), Albrecht et al. (2020) and Borg et al. (2019) using Multidimensional Scaling (MDS).

Albrecht et al. (2020) and Consiglio et al. (2017) have advocated the expansion of construct diversity to capture the nomological validity of work values. Therefore, to assess convergent validity, correlations to specific motives will be considered within the circular array of values. We expect social work motives to correlate more positively with Social work values than with Prestige work values (H3a). We further hypothesize that individualistic work motives correlate more positively with Intrinsic work values than with Extrinsic work values (H3b). To test the convergent validity of the additionally posited value of Environmental Sustainability, the correlation to environmental awareness will be assessed. Here, we expect a positive correlation (H4).

We address discriminant validity and whether the corresponding questionnaires of the construct are empirically distinct (Rönkkö and Cho, 2022). It will be assessed in relation to neuroticism as an affective-oriented trait with personal tendencies to negative mood states (McCrae and Costa, 2003). Data from a comprehensive meta-analysis support the notion that this affective trait has no or only very small correlations with cognitive-based basic values (Parks-Leduc et al., 2015). Since values tend not to have direct implications for stress (Roccas et al., 2002; Sagiv et al., 2004), we also expect a sufficient empirical distinction of work values to neuroticism in our study (H5).

2.2.3 Criterion validity

To establish correlations with relevant outcome variables, work values will be considered in relation to value congruence in the person–organization fit (PO-Fit; Kristof, 1996; Cable and DeRue, 2002). The perceived complementary fit of one’s own values to organizational values emerges as an important variable in organizational settings (for relevant correlations, see Kristof-Brown et al., 2005, 2023; Arthur et al., 2006; Uggerslev et al., 2012; Etzel and Nagy, 2020; Straatmann et al., 2020; Ghielen et al., 2021). Specifically, we capture the congruent incremental validity of work values versus Schwartz’s (1994) basic values. Because work values represent a contextualized conceptualization, we expect a significant increase in explained variance compared to the four value dimensions of basic values (H6).

3 Method

We report how our sample size was determined, the basis on which cases are excluded, and all manipulations and measurement tools (Simmons et al., 2011; Kline, 2019, p. 64). We used a checklist to ensure transparency in our research (Aczel et al., 2020; see Appendix 1) and preregistered the study.¹ To develop a valid and short questionnaire for a useful application in organizational and research settings, we conducted a quantitative representative cross-sectional online panel survey in June 2023.

3.1 Sample

The addressed population is composed of working-age individuals (18–69) in Germany with at least 20 working hours/week to establish a sufficiently large reference to work activity. Based on different recommendations for CFA sample sizes (e.g., N:q-ratio, N_minimum = 200; Gnambs, 2013; Kline, 2016; Muthén and Muthén, 2017; Wang and Rhemtulla, 2021; Nye, 2022; R-package: semTools) and the need for a randomized split of the data set, as well as a buffer for potential case exclusion, a minimum of N = 990 was set (n_training = 450, n_test = 350, n_buffer = 190). The online panel sample was based on representative distributions concerning age and gender for Germany (study incentive: 2,50€). The final sample was acquired via e-mail invitations of panelists [N = 1,048; 46.66% women, 7.44% with migration background,² 29.7% with management responsibilities, M_{tenure(years)} = 12.43, SD_{tenure(years)} = 10.78, M_age = 44.14, SD_age = 12.43, M_{workinghours/week} = 37.64, SD_{workinghours/week} = 6.75]. The distribution of industrial sectors and educational levels is presented in Tables 2 and 3. To ensure the quality of the panel sample, we referred our survey design to established best practices (Porter et al., 2019; Aguinis et al., 2021; Ward and Meade, 2023).³

Table 2

Table 2. Distribution of industrial sectors (N = 1,048).

Table 3

Table 3. Distribution of educational level (N = 1,048).

3.2 Materials

3.2.1 Circular Work Value Scale

We translated the Values at Work Scale (Albrecht et al., 2020) into German following the guidelines of the International Test Commission (2017). The questionnaire underwent back-translation by two native English speakers with relevant cultural backgrounds. This process ensured an accurate reflection of the questionnaire’s meaning. Additional items were included for each work value dimension to facilitate algorithm-based item selection (ABIS) and to address all theoretical facets. Environmental Sustainability and Safety were specifically addressed with items more aligned with theoretical propositions. Construct definitions (Table 1) and existing German-language questionnaires for assessing work values (Seifert and Bergmann, 1983; Krumm et al., 2013; ISSP Research Group, 2017) guided the formulation of items. Selection was based on content validity. The questionnaire initially consisted of 77 items (7 per work value dimension) using a 7-point Likert scale (1 = “completely unimportant” to 7 = “very important”). To ensure content validity, five expert interviews were conducted with researchers from work and organizational psychology. These were followed by eight cognitive interviews with individuals from the target population⁴ (MacKenzie et al., 2011; Boateng et al., 2018). The received feedback led to iterative adjustments in wording and cultural appropriateness,⁵ resulting in the final set of items presented in Appendix A2.

3.2.2 Measures to test convergent and discriminant validity

To evaluate convergent validity, the Inventory for the Assessment of Work Motivation—Short Form (IEA-K; Kanning, 2016) was selected for relations with work motives. The extended version shows satisfying psychometric quality (Lang et al., 2018). For the correlations in the circular arrangement of the work values, the scale on individualistic motives (12 items, e.g., “take responsibility myself”; α = 0.89; ω = 0.92) and on social motives (six items, e.g., “exchange information with colleagues also about private matters,” α = 0.79; ω = 0.88) was selected. Answer scales ranged from 1 = “unimportant for me” to 5 = “extremely important for me.”

For correlations with the newly set up work value of Environmental Sustainability, five items of the German version of the New Ecological Paradigm questionnaire on environmental awareness (Schleyer-Lindenmann et al., 2018; α = 0.85; ω = 0.87) were included (e.g., “If things continue on their present course, we will soon experience a major ecological catastrophe.”; 1 = “do not agree at all” to 6 = “completely agree”). Discriminant validity is to be tested by correlations with the Big Five-dimension neuroticism. The German version of the Big Five Inventory Short (three items, BFI-S; Schupp and Gerlitz, 2014, e.g., “I am someone who often worries.”; α = 0.82; ω = 0.83; 1 = “not applying at all” to 7 = “fully applies”) was used.

3.2.3 Measures to test incremental congruent validity

The PO-Fit criterion in the incremental congruent validity test is assessed via three items from Cable and DeRue (2002) on value congruence (e.g., “The things that I value in life are very similar to the things that my organization values”; α = 0.93; ω = 0.93; 1 = “do not agree at all” to 7 = “completely agree”). Incremental validity is to be assessed against each of the four basic value dimensions of the Higher-Order Value Scale-17 (Lechner et al., 2022). Here, the four dimensions (Openness to Change, e.g., “It is important to her/him to develop her/his own opinions,” α = 0.83; ω = 0.85; Conservation, e.g., “It is important to her/him to maintain traditional values and ways of thinking,” α = 0.66; ω = 67; Self-Enhancement, e.g., “It is important to her/him to show that her/his performance is better compared to the performance of other people,” α = 0.74; ω = 0.76; Self-Transcendence, e.g., “It is important to her/him to help the people dear to her/him,” α = 0.82; ω = 0.85; 1 = “is not at all similar to me” to 6 = “is very similar to me”) are assessed using 17 items.

Furthermore, based on empirical evidence, different control variables were integrated. The age, gender, and tenure of respondents have been shown to impact basic and work values (Consiglio et al., 2017). We conducted a pre-test of the final online questionnaire with 13 participants to check the comprehensibility and questionnaire design. All items were presented in a forced choice format to ensure data completeness. The survey was administered in German.

3.3 Statistical analysis

Careless or inattentive response patterns and outliers were analyzed by multiple mechanisms (Schroeders et al., 2022; Ward and Meade, 2023). We used autocorrelation screening (R package: responsePatterns; Gottfried et al., 2022), long string analysis, intra-individual response variability and mahalanobis distance (R-package: careless; Curran, 2016). Conspicuous responses were further investigated. We excluded 195 data entries based on the applied techniques and short response times on individual pages. The final sample size included 853 participants.

Due to current debates on validity concerns of measures in psychological science (Flake et al., 2017; Hussey and Hughes, 2020; Shaw et al., 2020) and the need for short questionnaires (Fuchs and Diamantopoulos, 2009), the data will be analyzed with ABIS using a genetic algorithm. In the field of psychological assessment, algorithms are increasingly used in item selection and questionnaire development (Algner and Lorenz, 2022; Kerber et al., 2022; Pundt et al., 2022). Compared to classical approaches, algorithms have the advantage of being more objective and efficient with respect to defined criteria to find a (nearly) finite solution (Leite et al., 2008; Olaru et al., 2015). Empirical studies suggest that the use of algorithms leads to similar or better results in scale construction than traditional approaches (Sandy et al., 2014; Schroeders et al., 2016; Olaru and Danner, 2021). However, the need for a rigorous theory-driven item development covering all construct-specific properties must be considered (Dörendahl and Greiff, 2020).

The goal of ABIS is to select those items from an initial item pool that fulfill defined criteria (e.g., the best representation of the construct or best fit to the data). In this context, the selection of items and the development of an economic questionnaire can be defined as a combinatorial problem (Schroeders et al., 2016; Kerber et al., 2022). Based on a given set of items (here, 77 items), a questionnaire with 33 items should be developed with satisfying quality. Thus, the computation of the single best solution would be disproportionately time-consuming with average computational power (possible combinations: 96.549.157.373.046.880). Meta-heuristic like genetic algorithms are utilized to handle the complexities of such combinatorial optimization problems. Genetic algorithms are based on principles of natural selection (Holland, 1992; Schroeders et al., 2016). Since this is a meta-heuristic and estimation-based approach (Blum and Roli, 2003), it is not a procedure to find the single best solution (Yarkoni, 2010). However, the benefit is to increase the psychometric quality of the whole questionnaire under high efficiency (low demand of time and computational power; Dörendahl and Greiff, 2020) and the consideration of diverse item combinations (Olaru and Danner, 2021). The advantage of genetic algorithms is that items are not considered in isolation. Item quality is always evaluated considering specified criteria against the background of the entire questionnaire in CFAs (Schultze, 2017).

With a genetic algorithm (R-package: stuart), the initial collection of 77 items is to be reduced based on evolutionary selection processes with the goal of an optimal or near-optimal solution. The basis for the survival of an item in the item pool is its quality (called “fitness”; Galán et al., 2013). The algorithm is based on two processes: Variation (recombination and mutation) and selection. Variation promotes diversity and novelty of items, whereas selection rewards quality. The heuristic uses genes (items) that represent a certain variable and links them to a chromosome (scale of items). To allow variability, a predefined number of chromosomes are then randomly generated from the original item pool, which represents the 1st generation of items (usually 100–200 individuals; Yarkoni, 2010). The algorithm now pursues the goal of maximizing the psychometric quality of the questionnaire by evaluating the chromosomes against a “fitness” function. Based on the defined fitness function, each generation’s fittest chromosomes (item sets) are extracted and used as a basis for the next generation (enabling the selection process of the fittest items). To enable the process of variation and establish genetic diversity and mutation, the spontaneous exchange of items within a scale or between two scales is permitted. With a predefined number of iterations, this procedure identifies the fittest chromosome (item combination) with the highest quality (Schroeders et al., 2016).

Thus, we reduced the initial questionnaire to an economic version (11 factors à, three items).⁶ The collected dataset was randomly divided into a training and a test dataset using holdout-validation (n_training = 450; n_test = 403). In the training dataset, the item combinations were examined against a fitness/quality function based on the Chi-Square test statistic, Root-Mean-Square Error of Approximation (RMSEA), Standardized Root-Mean-Square Residual (SRMR), Comparative Fit Index (CFI), and the reliability of the subscales. Additionally, the function includes the assumed latent variable correlations according to the circular structure of the theory of basic human values. The final selected items will be analyzed for their factorial validity in cross-validation (R-package: stuart, function: cross validate) to the test data set. To check the fit of the factorial measurement model to training and test data, a CFA will be conducted in R (package: lavaan; estimator: MLR). Furthermore, measurement invariance will be evaluated between gender and age groups (Meredith, 1993). Using the cross validate function of the stuart-package, the tests for measurement invariance are not conducted sequentially as often applied (Cheung and Rensvold, 2002). In the approach given by stuart (Schultze, 2017), a measurement model with strict measurement invariance is assumed a priori. Therefore, any discrepancy between the scale and the assumed measurement model would manifest in the overall model fit. Due to non-normal data distributions in various scales, a robust estimator was used (skew and kurtosis in Mardia’s Test of multivariate normality, e.g., Authority: skew = −0.15, kurtosis = −0.57; Social Justice: skew = −0.82, kurtosis = 0.99). The fit indices are reported according to Kline (2016). Cutoffs are derived from Gäde et al. (2020, p. 649) for good and acceptable model fit with heterogeneous items.⁷ The circular theoretical ordering of work values (Schwartz et al., 2012; Albrecht et al., 2020) is to be identified per non-metric MDS (R-package: MASS). The aim is to display the correlation-based distances between the work values from a higher dimensional ordering on a two-dimensional space using an iterative estimation to reduce the stress value (Kruskal, 1964; Hout et al., 2013).

To analyze convergent and discriminant validity, latent and manifest correlations between work values and motives, neuroticism, and environmental awareness are considered. As values tend to be somewhat important in general (Sagiv and Schwartz, 2022) and individuals may differ in their response styles (Rudnev, 2021), a common variance factor is important to consider when analyzing personal values. Manifest correlations among work values and basic values will be assessed using ipsative, intraindividual mean-centered scores. For the theoretical assumptions of the theory of basic human values and its circularity, the application of this approach to control for common factor variance can be beneficial. Ipsatization converges ratings to preferences which is more aligned with the theoretical definition of values. Scores based on ipsatized data may be more resistant to common factor bias of response styles and social desirability (Rudnev, 2021). Problems with ipsatization can occur when estimating internal consistency, test–retest reliability or using multivariate techniques due to perfect collinearity. However, we only use ipsatization to assess convergent validity in bivariate correlations to test theoretical assumptions in the nomological net of work values and basic values. Regarding latent variables, bifactor models are potential procedures to control for individual response styles and common variance (Rudnev, 2021). However, only limited empirical evidence for their application in the context of human values is present (e.g., Lilleoja et al., 2016), and their usage must still be thoroughly evaluated (Mansolf and Reise, 2017). Nevertheless, for transparency issues, we report a bifactor model for our final work value scale based on the whole data set to estimate the variance of a potential common variance factor. Therefore, we included an extra method, g-factor, which has equal loadings for each item and is unrelated to the other factors (Schwartz et al., 2012). In this model, we fixed the factor loadings of the first indicator to 1, based on previous bifactor models in value research (Lilleoja et al., 2016).

For the evaluation of our hypothesis, differences in the magnitude of Pearson correlations will be evaluated. The confidence interval (CI) of the difference between Pearson correlations (based on Fisher’s r-to-z transformation) will be calculated to compare the associations (Zou, 2007; R-package: cocor). An upper level of the CI below zero indicates that the two correlations are not equal (e.g., Authority is lower correlated with social work motives than Social Justice). Due to power considerations, we assess discriminant validity with the confidence interval of the latent correlations in CFAs [CI_CFA(sys)]. In the analysis, the variances of the latent variables will be fixed to 1. We will inspect the upper/lower limits of the 95% CI of the estimated factor correlations according to Rönkkö and Cho (2022).

The incremental validity of the individual scales is assessed via hierarchical regressions and the additional explained variance (ΔR² _adjusted ). The R² _adjusted of the baseline model (consisting of one basic value dimension) is considered in comparison to the regression model with the corresponding work values (cf. Table 1). The prerequisites are examined according to Bühner and Ziegler (2009). Due to violations of normality assumptions assessed with Shapiro–Wilk test and heteroscedasticity assessed via Breusch–Pagan test, the regressions were performed via Bootstrapping (5,000 iterations). Based on the criticism of using conventional significance levels (Kline, 2019) for evaluating p-values, we calculate an Alpha that minimizes Type I and II errors (Mudge et al., 2012), considering the smallest effect size of interest (r = 0.30; ƒ² = 0.10; α_correlation = 0.001; α_regression = 0.001). This makes interpretation less arbitrary and more adjusted to context and data (R-package: JustifyAlpha; Maier and Lakens, 2022).

4 Results

Descriptive statistics of assessed scales are displayed in Tables 4 and 5. We conducted CFAs (estimator: MLR) for the work motive questionnaire (χ² = 1333.86, df = 134, p < 0.001, χ²/df = 9.95, CFI = 0.771, RMSEA = 0.102 [0.098, 0.107], and SRMR = 0.073), and the basic value questionnaire (χ² = 459.86, df = 113, p < 0.001, χ²/df = 4.07, CFI = 0.908, RMSEA = 0.06 [0.055, 0.065], and SRMR = 0.07). Neuroticism, environmental awareness, and value congruence were collectively analyzed with constrained covariances (χ² = 176.62, df = 44, p < 0.001, χ²/df = 4.01, CFI = 0.964, RMSEA = 0.059 [0.051, 0.068], and SRMR = 0.083). The work motive questionnaire especially shows weak factorial validity.

Table 4

Table 4. Descriptive statistics and ipsatized Pearson correlations of work values (n = 853).

Table 5

Table 5. Pearson correlations of work values and variables from the nomological network (n = 853).

4.1 Model fit and latent structure of the CWVS

The final set of selected items (see Table 6) shows a good fit for the training data. Cross-validation of the test data supports the assumption of strict measurement invariance between the two samples (see Table 7). Additionally, the analysis of measurement invariance between age and gender groups is in line with the conjecture of strict measurement invariance. Hence, the data support our first hypothesis. The bifactor model shows a 10% variance in the common variance factor with a good fit of the model to the data (see Table 7).

Table 6

Table 6. Final set of selected items in the CWVS in German and English wording, including standardized factor loadings, standard errors, and reliability estimates.

Table 7

Table 7. Results of model comparisons in cross-validation and bifactor model.

The stress index of the final MDS solution (stress = 0.13; see Figure 1) is less than the recommended criterion of 0.15 (Dugard et al., 2010), suggesting goodness of fit. Each quadrant of the ellipses includes one work value dimension supporting the circularity and the relationships of opposing value dimensions. Thus, Hypothesis 2 is supported.

Figure 1

Figure 1. MDS to assess the dimensionality and theoretical circularity of the work values.

4.2 Analysis of convergent and discriminant validity

Latent and Pearson correlation statistics are displayed in Tables 4, 5 and 8. Convergent validity is assessed with correlations from Table 5. Compared to the two Prestige work values Ambition and Authority, Social Justice (r_{Diff-Authority} = −0.07 [−0.14, −0.01], r_{Diff-Ambition} = −0.06 [−0.11, −0.002]) and Helping and Supporting (r_{Diff-Authority} = −0.14 [−0.21, −0.08], r_{Diff-Ambition} = −0.13 [−0.18, −0.07]) exhibit higher Pearson correlations with social work motives, while Environmental Sustainability (r_{Diff-Authority} = 0.02 [−0.05, 0.09], r_{Diff-Ambition} = 0.04 [−0.03, 0.10]) shows similar correlations. Correspondingly, Autonomy (r_{Diff-Rule Respecting} = −0.11 [−0.17, −0.05], r_{Diff-Traditional Values} = −0.09 [−0.15, −0.03], r_Diff-Safety = −0.06 [−0.11, −0.002]), and Variety (r_{Diff-Rule Respecting} = −0.18 [−0.23, −0.13], r_{Diff-Traditional Values} = −0.16 [−0.22, −0.11], r_Diff-Safety = −0.13 [−0.18, −0.08]) demonstrate higher Pearson correlations with individualistic work motives, whereas Enjoyment (r_{Diff-Rule Respecting} = −0.04 [−0.10, 0.01], r_{Diff-Traditional Values} = −0.02 [−0.08, 0.03], r_Diff-Safety = 0.01 [−0.04, 0.06]) as part of the Intrinsic work values and Rule Respecting, Traditional Values, and Safety exhibit lower correlations. The p-value of the latent and Pearson associations is lower than the computed optimal Alpha of 0.001. Conclusively, Hypotheses 3a and 3b are partially supported by our data as the work values Enjoyment and Environmental Sustainability are not strongly related to the corresponding work motives.

Table 8

Table 8. Latent correlations of work values and variables for assessing convergent and discriminant validity.

For Hypothesis 4, we inspected the correlation of the newly set up work value Environmental Sustainability with the construct of environmental awareness. Latent and Pearson correlations (r_latent = 0.61, r_Pearson = 0.52) support our assumption.

Discriminant validity is evaluated by analyzing the relations of the work values measured with the CWVS and the Big Five-dimension neuroticism (H5). The CI_CFA(sys) of the latent correlations are displayed in Table 8. Work values of Authority (r_latent = −0.23 [−0.32, −0.14]), Variety (r_latent = −0.18 [−0.26, −0.10]) and Autonomy (r_latent = −0.19 [−0.27, −0.10]) demonstrate significant negative latent correlations to neuroticism (p < 0.001). Nonetheless, the limits of the intervals are below the recommended Cutoff of 0.80. Therefore, empirical distinction according to Hypothesis 5 is supported.

4.3 Congruent incremental validity

For the sixth hypothesis, we conducted hierarchical linear regressions. The aim is to assess the additional explained variance of the work values compared to one basic value dimension in the criterion of value congruence. The results are displayed in Table 9. The inclusions of Social (ΔR²_adjusted = 0.10), Prestige (ΔR²_adjusted = 0.13), Intrinsic (ΔR²_adjusted = 0.10), and Extrinsic (ΔR²_adjusted = 0.18) work values show significant increases in the explained variance, supporting Hypothesis 6.

Table 9

Table 9. Results of hierarchical linear regression analysis for the prediction of the value congruence and change in the adjusted R².

4.4 Exploratory analysis

Due to the weak factorial validity of the IEA-K (Kanning, 2016), we decided to provide additional evidence for convergent validity. We analyzed latent and ipsative Pearson correlations of the CWVS with the basic value dimensions from the Higher-Order Value Scale-17 (Lechner et al., 2022).

As Table 5 illustrates, the latent and ipsative Pearson correlations in the circumplex of basic and work value dimensions support the theoretical assumptions. Prestige work values show positive Pearson/latent correlations with the basic value dimension of Self-Enhancement and negative Pearson and lower latent correlations with Self-Transcendence. The opposite accounts for Social work values. Intrinsic work values also correlate more positively with Openness to Change and negatively, respectively, lower with Conservation. Correspondingly, the opposite accounts for Extrinsic work values. As a result, convergent validity is supported by the additional analysis.

5 Discussion

The goal of this study was to develop a German-language work value questionnaire with accurate content, factorial and nomological validity that can be utilized in organizational and work-related settings. Based on the theory of basic human values (Schwartz, 1992), a questionnaire developed by Albrecht et al. (2020) was used as a starting point to capture the contextualized 11 work values representing the theoretical circularity of value constructs. The questionnaire development process was based on guidelines for cultural adaption and item generation (MacKenzie et al., 2011; International Test Commission, 2017; Boateng et al., 2018). A cross-sectional panel survey led to the inclusion of n = 853 cases. The questionnaire was developed using a genetic algorithm. Convergent, discriminant and congruent incremental validity were assessed, as well as the theoretical representation using MDS.

To our knowledge, this study is the first to support the full application of the theory of basic human values to the German work context. The developed questionnaire provides good psychometric quality with satisfying fit to test and training data and supports the assumption of strict measurement invariance between age groups and gender. The theoretical foundation is supported by MDS, as the work values provide a clear circular structure of conflicting and congruent underlying work-related goals and expectations. The dimension of Safety, which originally was understood as safety climate (Albrecht et al., 2020), is measured with three items, including interpersonal and intrapersonal Safety aspects. This potentially explains why the value can be found in the original Extrinsic domain. The work value of Enjoyment is more distinct from the other Intrinsic values. Implications of this conjuncture are discussed next.

Analysis of relations to work motives and basic values support convergent validity except for Enjoyment and Environmental Sustainability. Enjoyment does not correlate more strongly with individualistic work motives than Extrinsic work values. This may be due to its lower internal consistency as items capture different facets of the underlying work value (having fun at work and balancing work and recreational time). Additionally, as Enjoyment is more distant from Autonomy and Variety in the MDS solution, its theoretical and, therefore, empirical associations in the circumplex might be closer to Social work values. Environmental Sustainability is perhaps not associated with Social work motives beyond Authority and Ambition due to its strict focus on environmental aspects. The value does not capture interpersonal facets and is, therefore, potentially less related to social work motives. However, the MDS solution displays high distances, and ipsative Pearson correlations are significantly negative between Environmental Sustainability and Prestige work values. Nevertheless, it is essential to exercise caution when interpreting the correlations of the work values with work motives, as the short form of the IEA used possesses weak factorial validity in our study. Therefore, the exploratory analysis was conducted, and the basic value dimensions show a clear circular pattern of correlations with the work values. The newly established and more comprehensively assessed work value of Environmental Sustainability correlates moderately to highly positively with the construct of environmental awareness. Hence, convergent validity is backed by our data.

The latent correlations of the work values in the CWVS with the personality construct of neuroticism endorsed our assumption. Based on the 95% CI of latent correlations, the two questionnaires are empirically distinctive. The results are in line with a comprehensive meta-analysis addressing the association between personality traits and personal values (Parks-Leduc et al., 2015).

Finally, we analyzed congruent incremental validity. The additional explained variance of the work values compared to the basic value dimensions in the person–organization fit of value congruence supports our theoretical assumptions. Due to their contextualization, work values tend to provide more content-valid insights, potentially resulting in enhanced criterion validity. This is particularly relevant for work value research as questionnaire development approaches in the past did not study the criterion-related validity of work values compared to basic values (Consiglio et al., 2017; Albrecht et al., 2020; Stiglbauer et al., 2022).

5.1 Practical implications

Our study delivers the first evidence that the CWVS is more adequate when predicting work-related outcomes. As the questionnaire’s validity and theoretical assumptions are predominantly supported, we encourage the application of this questionnaire in practical and research settings. The short questionnaire with contextualized items can be used for a thorough assessment of employees’ or applicants’ value structure. The knowledge about what people value at work can provide powerful insights (Arieli et al., 2020; Anglim et al., 2022) and be beneficial for employers and employees (Bojanowska et al., 2022). The face validity of value assessment in work contexts may increase due to the contextualized work values. Hence, the assessment of what employees in an organization expect from their work or to deliver additional material for interventions can be useful applications of work values. For instance, team expectations can be clarified based on the CWVS by enabling a general, work-related framing. Furthermore, work design measures to improve value congruence between tasks and personal values can be adapted more purposefully due to the higher content validity of work values than basic values. The application of work values in career management interventions can provide insights into the personal development of individuals. As some values indirectly tend to affect psychological health, the work-related contextualization of values may be more useful for supporting individuals in their career choices.

5.2 Limitations and future research

Although we developed a valid new questionnaire, some limitations need to be acknowledged. The definitive version of the CWVS captures all dimensions quite broadly. Yet, algorithms for item selection only display a heuristic, which does not automatically result in the optimal solution. As we repeated the computation, we found that various scales (Enjoyment, Social Justice, Helping and Supporting, and Safety) were partially unstable. Here, various solutions provided good scale quality. Accordingly, it may be appropriate to discriminate between more narrowly defined work values in correspondence to Schwartz’s refined theory with 19 values. Differentiations in Enjoyment were illustrated previously; Social Justice and Helping and Supporting potentially address colleagues, customers, and the broader society. Additionally, Safety could be further differentiated into interpersonal and intrapersonal safety (complementary to Albrecht et al., 2020), and Traditional Values could be understood as compliance with organizational culture or broader societal culture and norms. Therefore, the use of this questionnaire and the theory of basic human values in the work contexts must be evaluated constantly as values are changing and new things may become more important (Kristof-Brown et al., 2023). Items of Enjoyment and Traditional Values show lower internal consistencies than other constructs. In particular, evaluations of the bifactor model showed variations in standardized factor loadings and standard errors compared to other constructs and the classic CFA approach. The measurement of the underlying construct with the selected items may need further investigation as the dimensionality of work values can be more complex and heterogeneous than the 11 broader dimensions. This specifically accounts for Enjoyment with lower factor loadings, lower internal consistency, more distance in the MDS solution and a lack of higher correlations to individualistic work motives.

Furthermore, our goal was to provide evidence for convergent validity beyond basic human values (Consiglio et al., 2017; Albrecht et al., 2020). The correlations of the work values, to a great extent, implicate convergent validity. Nevertheless, the factorial validity of the used scales from the IEA-Short Form (Kanning, 2016) is low in this study. As a result, we appeal to be cautious when interpreting these relations. However, the correlations to the basic values provide unambiguous evidence for convergent validity, as expected by theoretical assumptions. Future research should also consider different variables in the nomological net, potentially assessing only specific variables per work value (comparable to our approach with Environmental Sustainability and environmental awareness).

We reported a bifactor model to control for a common variance factor with a good model fit to the data. Approximately 10% of the variance is attributable to this common variance factor. This is comparable to other common variance factors in value research (Lilleoja et al., 2016). However, the application of bifactor models is currently under debate concerning their accuracy in estimation and model specification (Mansolf and Reise, 2017; Eid et al., 2018; Rudnev, 2021). Hence, future research should consider a more accurate and comparable estimation of latent value factors by assessing proper approaches to control for common variance factors in value research.

As we collected all data from a single source at one time, one might argue that a common method bias is inherent to our data (Podsakoff et al., 2003). We acknowledge the discussion around this topic (Spector and Brannick, 2010). For potential improvements, we advocate that researchers survey relations of work values to other variables through multi-source and longitudinal data and across cultures. Here, the influence of previous organizational cultures on work value development should be considered. Variables like the length of employment, company size, and other socialization influences of previous organizations could be worth including as control variables above gender, age, and tenure in the current organization. The development of work values over time and the implications of work values in daily decision-making are promising research strings (as for value congruence and PO-Fit; Kristof-Brown et al., 2023). In the case of criterion validity, it would be beneficial to assess incremental validity over a period. We provided initial evidence for additional explained variance in the value congruence to the organization, but more comprehensive criteria must be assessed against the basic values (e.g., job performance, organizational citizenship behavior, or more narrowly defined attitudes toward diversity and organizational justice). Moreover, cross-cultural validations of the CWVS would be appropriate for evaluating measurement invariance.

Additional questionnaires are needed to assess the extent to which work value-based behavior can be relevant for organizational practice and personnel development. Correspondingly, a questionnaire to assess the perceived organizational culture based on the work value dimensions can deliver relevant insights into discrepancies between the importance and the lived reality of one’s personal work values. This would enable teams or employees to unfold aspects that oppose personal values and, therefore, cause strains.

6 Conclusion

Overall, our data promote the application of the theory of basic human values to the German work context. We used a genetic algorithm to derive a valid questionnaire, which supports strict measurement invariance between gender and age groups. The analysis of convergent, discriminant, and congruent incremental validity, in addition to the MDS results, strongly corroborates our hypothesized relations. The study results in a promising questionnaire for broader work value assessments with practical relevance.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation by request.

Ethics statement

The studies involving humans were approved by MSB Medical School Berlin Ethical Board. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

JS: Conceptualization, Data curation, Formal analysis, Methodology, Writing – original draft. CS: Data curation, Resources, Supervision, Writing – review & editing. KH: Resources, Supervision, Writing – review & editing. TL: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Supervision, Validation, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2024.1296282/full#supplementary-material

Footnotes

1. ^https://aspredicted.org/blind.php?x=ZMJ_9TJ

2. ^Migration background is defined as having at least one parent who is a first-generation immigrant.

3. ^Three attention check items; additional data collection to compensate for attention failure; “Contact Researcher”-option; an accurate estimate of needed time; pilot test with 10–30 participants; analysis for carelessness in answers; asking participants if they were attentive and offered data deletion option; an explanation for careful data entries, which participants had to check off.

4. ^When selecting the interviewees, care was taken to ensure the greatest possible diversity in terms of age, professional position, educational background, and gender.

5. ^Work value “Safety”: corporate social benefits were mentioned several times as important; work value “Traditional Values”: implementation of religious beliefs in working life was perceived as irritating by some respondents. As the aspect of religiosity is losing importance in everyday life in Germany (Hodapp and Zwingmann, 2019), the item was removed.

6. ^We tested three- and four-item versions per dimension. As both had similar relations in the nomological network, we decided to use a three-item version. The three-item version seems rather stable except for the scales of Safety, Enjoyment, and Helping and Supporting. Here, multiple solutions were distinguishable. This will be addressed in the discussion.

7. ^Deviation of the overall model from the perfect fit (χ²-test), Quotient of the test statistic (good: χ²/df ≤ 2; acceptable: χ²/df ≤ 3), CFI (good: ≥0.95; acceptable: ≥0.90), SRMR (good: ≤0.05; acceptable: ≤0.10) (Hu and Bentler, 1999), and RMSEA (good: ≤0.05; acceptable: ≤0.08) (Steiger, 1990).

References

Aczel, B., Szaszi, B., Sarafoglou, A., Kekecs, Z., Kucharský, Š., Benjamin, D., et al. (2020). A consensus-based transparency checklist. Nat. Hum. Behav. 4, 4–6. doi: 10.1038/s41562-019-0772-6

PubMed Abstract | Crossref Full Text | Google Scholar

Aguinis, H., Villamor, I., and Ramani, R. S. (2021). MTurk research: review and recommendations. J. Manag. 47, 823–837. doi: 10.1177/0149206320969787