Psychometric Properties of the Independent and Interdependent Self-Construal Questionnaire: Evidence From the Czech Republic

Lacko, David; Čeněk, Jiří; Urbánek, Tomáš

doi:10.3389/fpsyg.2021.564011

ORIGINAL RESEARCH article

Front. Psychol., 03 June 2021

Sec. Quantitative Psychology and Measurement

Volume 12 - 2021 | https://doi.org/10.3389/fpsyg.2021.564011

This article is part of the Research Topic New Challenges in Globalized Societies: Cross-cultural Studies and Test Adaptation View all 13 articles

Psychometric Properties of the Independent and Interdependent Self-Construal Questionnaire: Evidence From the Czech Republic

$\r\nDavid Lacko,*$ David Lacko^1,2*

Jiří Čeněk^3,4

Tomáš Urbánek²

¹Department of Psychology, Faculty of Arts, Masaryk University, Brno, Czechia
²Institute of Psychology, Czech Academy of Sciences, Brno, Czechia
³Department of Social Studies, Faculty of Regional Development and International Studies, Mendel University in Brno, Brno, Czechia
⁴Department of Information and Library Studies, Faculty of Arts, Masaryk University, Brno, Czechia

This article introduces a validation study of the Czech version of an independent and interdependent self-construal questionnaire (SCS, Vignoles et al., 2016) conducted on 330 Czech subjects. In this study, the reliability, convergent validity and factor validity were verified. However, the confirmatory factor analysis revealed unsatisfactory factor structure (RMSEA = 0.053 [0.048, 0.057], SRMR = 0.080, CFI = 0.775, TLI = 0.755). These results are discussed with respect to other adaptations of individualism/collectivism scales in countries beyond typical West-East dichotomy. Hence, the article not only critically discusses the shortcoming of the Czech and original versions of the questionnaires, but also the general issues of the individualism-collectivism construct in the cross-cultural context as a whole.

Introduction

Formulated in the 1970s by Hofstede, the cultural dimension of individualism/collectivism (I/C) has become a popular theoretical concept in cross-cultural psychology and a useful tool to structure and measure the psychological characteristics of members of various cultures (Bond, 2002). Consequently, I/C is used as a predictor for many other psychological and behavioral variables (Oyserman et al., 2002). The I/C dimension was originally defined at a national level as a single bipolar dimension. Hofstede (1983) defined individualism as the quality of a relationship between an individual and his or her immediate social environment (family, friends, community, etc.). The theory of independent (de facto individualism) and interdependent (de facto collectivism) self-construal (Markus and Kitayama, 1991) later became the dominant approach in individual I/C research. It is based on the “social orientation hypothesis” (Varnum et al., 2010), which states that cultures differ in social orientations and their development. While some (individualistic) cultures adopt an independent social orientation and tend to emphasize self-direction, self-expression and autonomy, other (collectivistic) cultures endorse the development of interdependent social orientation and emphasize harmony, relatedness and connection with others. Even though individualism is currently rising in most societies, the mentioned cross-cultural differences remain detectable (Santos et al., 2017).

Despite the popularity of the construct, there is unfortunately no widely accepted method of measuring the individual level of I/C. Oyserman et al. (2002) identified 27 I/C scales and performed a content analysis of I/C domains. They found seven individualism and eight collectivism components accounting for 88% of the items across the scales. None of these 27 scales can be considered a single standard of measurement. The overall agreement on I/C operationalization differs in the selected scales from component to component, which has drawn attention to the fragmentation of the concept of I/C and its operationalization.

The debate on the number of factors and their structure is still ongoing and the existing research has suggested that the concept of an independent and interdependent self might be one-dimensional (Hofstede, 1983), two-dimensional (Markus and Kitayama, 1991; Lu and Gilmour, 2007), three-dimensional (Kashima and Hardie, 2000; Noguchi, 2007), four-dimensional (Singelis et al., 1995), five-dimensional (Bartoš, 2010) or possibly even seven-dimensional (Vignoles et al., 2016).

Such ambiguity raises a question about the true underlying factor structure and therefore calls for further investigation on independent samples (Bollen, 1989b). The importance of this step is even more crucial in cross-cultural research, where securing the equivalence of constructs as well as the measurement invariance is necessary to be able to compare country or cultural group means (van de Vijver and Tanzer, 1997; van de Vijver and Leung, 2001; Čeněk and Urbánek, 2019). In order to acquire some evidence of structural equivalence, statistical methods such as Confirmatory Factor Analysis (CFA), Multi-Group Confirmatory Factor Analysis (MG-CFA), Measurement Invariance (MI), etc., need to be applied (Fischer and Karl, 2019). Performing CFA is vital in cross-cultural research as a first step in verifying the possibility of comparing results across different questionnaire translations. This article is focused mainly on this step. The next steps after setting the configural model should lie in constraining the factor structure, factor loadings and intercepts and thus verifying the configural, metric and scalar invariance measurement across various cultural groups.

Individualism and Collectivism in the Czech Republic

Several previous studies have already been conducted on Czech participants with mixed results. Hofstede’s (Hofstede et al., 2010) approach assigned an index of 58 to the Czech Republic, suggesting a slightly above-average level of individualism. In cross-cultural comparisons, Czechs have been shown to be more individualistic than the rest of countries in Central Europe (Kolman et al., 2003). Dumetz and Gáboriková (2017) study confirmed that the Czech Republic is more individualistic than Slovakia, but others found the exact opposite result (Bašnáková et al., 2016). Furthermore, the Czech Republic seems to be less individualistic than the Netherlands (Bašnáková et al., 2016) and less collectivistic and, similarly, individualistic as East Asians (Lacko et al., 2020). Another study found that Czechs are not only more individualistic, but simultaneously also more collectivistic than Czech Vietnamese (Čeněk, 2015).

Unfortunately, these mixed results might be caused by the lack of valid and reliable tools to measure I/C, because adaptation attempts are relatively sparse for the Czech population. The first exception is an adaptation of INDCOL (Singelis et al., 1995) done by Bartoš (2010). However, his validation has a factor structure that does not fully correspond to the original scale, and its results are therefore not fully comparable in cross-cultural research. The psychometric properties of INDCOL are described in the next chapter.

The second exception, an attempt to adapt an I/C questionnaire into Czech, is a translation and cross-cultural verification of the Independent and Interdependent Self Scale (IISS; Lu and Gilmour, 2007) performed by Lacko and Čeněk (2020), who compared Czech and East Asian university students. Although the scale exhibited satisfactory reliability for both independent self-construal (Czech α = 0.815, Chinese α = 0.929) and interdependent self-construal (Czech α = 0.795, Chinese α = 0.906), the IISS showed configural non-invariance (RMSEA = 0.043 [0.025, 0.057], SRMR = 0.144, CFI = 0.636, TLI = 0.617). In their second study, performed on a larger sample consisting of only Czechs, they found similar unsatisfactory fit indices of IISS (RMSEA = 0.064 [0.061, 0.066], SRMR = 0.104, CFI = 0.460, TLI = 0.432).

The third exception is found in several adaptations of traditional methods measuring the cultural values, where individualism represents one of the cultural values. These Czech adaptations include VSM-94 (Value Survey Module 1994; Hofstede, 1994; adapted by Kolman et al., 2003), SVS (Schwartz Value Survey; Schwartz, 1992; adapted by Hnilica et al., 2006) and PVQ (Portraits Value Questionnaire; Schwartz et al., 2001; adapted in two studies by Řeháková, 2006; Anýžová, 2014). However, neither the original questionnaire manuals, nor the Czech adaptations of VSM-94 and SVS reported any relevant psychometric properties. The first study of PVQ reports only unsatisfactory internal reliability coefficients (α = 0.35–0.70; Řeháková, 2006). Even though the second study revealed acceptable MG-CFA fit indices (CFI = 0.924, RMSEA = 0.017), it suggested insufficient MI results across countries (metric MI ΔCFI = 0.009, scalar MI ΔCFI = 0.155) and these results were applied only to 10 out of 23 countries in total (Anýžová, 2014), which therefore raises doubts about the validity and reliability of those instruments.

As mentioned above, the two-dimensional model of the IISS failed in the factor structure validation on the Czech sample, and Hofstede’s one-dimensional model is claimed to be outdated and considered obsolete and invalid by many scholars (e.g., Singelis et al., 1995; McSweeney, 2002; Blodgett et al., 2008). Hence, the goal of this paper is to conduct an adaptation and psychometric analysis of a relatively new tool for individual level I/C measurement, the Self-Construal Scale (SCS; Vignoles et al., 2016; for a description see the Method section). Furthermore, we tried to verify its convergent validity with the current Czech adaptation of the Individualism-Collectivism Scale (INDCOL; Singelis et al., 1995; adapted by Bartoš, 2010).

Materials and Methods

Participants

Data were collected from 330 Czech participants. This number of participants should be satisfactory for several reasons: (1) our proposed models are simple and they are composed only of several first-order factors and indicators (Kline, 1998); (2) even though the rules of thumb are not fully reliable while planning research, they usually indicate an amount of 100–200 participants as an absolute minimum (Little, 2013; Brown, 2015) and moreover, as Kline (1998) pointed out, the number of about 200 participants is not only very often used in the SEM framework, but it might also be reliable in certain circumstances; (3) no missing values were observed in our dataset (Brown, 2015; Kline, 1998); (4) I/C scales usually yield high internal-consistency reliability which decreases demands on sample size (Brown, 2015; Kline, 1998); (5) in single group models standard errors are significantly reduced with more than 150 responses (Little, 2013); and (6) our models yielded a huge number of degrees of freedom (Hoyle, 2012; Kline, 1998).

The research sample was 77% (n = 254) female. The participants were 18–65 years old (M = 24.29, SD = 6.536). Regarding the field of study, or vocation, 34.5% (n = 114) of the participants were psychologists/students of psychology, followed by students of/employees in the field of languages and history and of international studies (both 8.8%). Concerning the level of education, 58.8% (n = 194) had completed high school and 39.7% (n = 131) had achieved a university degree. Regarding the participants’ religion, political preference and family status, most of them identified themselves as an atheist (43.6%), had no political preference (40.6%; or were liberals 31.2%) and were single (58.5%). The comprehensive descriptive statistics are shown in Table 1:

TABLE 1

Table 1. Demographic characteristic of sample.

Procedure

Data collection was conducted between December 2018 - February 2019. The participants were mostly gathered through university social groups and social websites (i.e., the non-probability convenience sampling method) which resulted in a research sample with an over-represented student part of the population compared to other groups. The participants were informed about the ethical aspects of the research, especially data anonymization, their voluntary participation and the option to end the questionnaire at any time without giving a reason. In order to proceed further with the administration, they had to consent with their participation in the research. All items were administered randomly to avoid possible response biases caused by context influences and preceding questions (for review, see Uskul and Oyserman, 2006). The whole testing procedure took approximately 20 min.

In order to minimize any potential method bias caused by an imperfect translation procedure (van de Vijver and Hambleton, 1996) a parallel translation method was applied. The English original Self-Construal Scale (SCS) was independently translated by two bilinguals with backgrounds in social sciences. Both translations were subsequently compared and, in the case of any inconsistencies, discussed by the authors of the study until an agreement on the formulation was reached. We put a special emphasis on minimizing any potential shifts of meaning between the English and Czech versions of the scale. The Individualism-Collectivism Scale (INDCOL) was used in the original Czech version (Bartoš, 2010). Both scales were administered online. In addition to SCS and INDCOL, all relevant socio-demographic variables were collected (see Table 1).

Measures

The Self-Construal Scale (SCS)

The SCS was developed by Vignoles et al. (2016) and validated on 9573 (Study 1, n = 2294; Study 2, n = 7279) participants across 55 cultural groups in 33 nations. The SCS is primarily based on the concept of independent and interdependent self (Markus and Kitayama, 1991). The authors built on other traditional I/C scales during the formulation of its items (e.g., Singelis et al., 1995).

The SCS consists of thirty-eight, nine-point, Likert-type numerical items scaled from 1 (not at all) to 9 (exactly), with three intermediate anchor-points (3 – a little, 5 – moderately, 7 – very well). The SCS contains half reversed items, which should enhance the validity of the factor structure and minimize the acquiescence bias (Smith et al., 2013).

The authors used exploratory and confirmatory factor analytic techniques, MG-CFA, multilevel analysis and other statistical procedures (such as modeling acquiescence as a common method factor in CFA or ipsatization for reliability estimation) in their validation study. Even though the authors did not perform an analysis of MI, they discussed it in relation to the items’ factor loadings, which in their opinion suggested a satisfactory invariance. However, no reliability estimation was performed in the validation study. The SCS was also tested for response biases in their follow-up study (cf. Smith et al., 2016) as well as concurrent validity with the I/C dimensions measured by individualism values and in-group collectivism practices, where the r coefficients were between 0.425 and 0.752.

Using Principal Components Analysis (PCA) and Principal Axis Factoring (PAF) in the first study, the authors identified seven dimensions of the SCS, namely “Self-reliance vs. Dependence on others,” “Self-containment vs. Connection to others,” “Difference vs. Similarity,” “Commitment to others vs. Self-interest,” “Consistency vs. Variability,” “Self-direction vs. Receptiveness to influence” and “Self-expression vs. Harmony.” The CFA partially confirmed the factor structure in the second study. The authors presented two respective models: model 1, which was comprised of 38 items, and model 2, with 26 items. The first model yielded good fit indices despite an insufficient CFI (SRMR = 0.050; RMSEA = 0.046; CFI = 0.790). However, the authors claimed that a 0.90 threshold for CFI is often unreachable and unrealistic in multidimensional questionnaires used in various cultural samples and they justified a CFI value near 0.80 as acceptable for cross-cultural multidimensional questionnaires. The second model showed better fit indices (SRMR = 0.033, RMSEA = 0.033, CFI = 0.922) and can be considered valid from the point of view of factor structure. Nevertheless, this model only has 26 items and one subscale is consequently comprised of only two items, whereas an unabbreviated model contains four to six items per subscale. Our view is that 26 items are insufficient for a seven-dimensional questionnaire, and we therefore focused on the longer version of the SCS. At the same time, we concur with the authors that the SCS is currently one of the most comprehensive tests available for I/C dimension measurement.

The Individualism-Collectivism Scale (INDCOL)

The INDCOL was introduced by Singelis et al. (1995) and later improved by Triandis and Gelfand (1998). The original scale contains 32 items, and the improved and shortened version contains 27 items. All items are nine-point, Likert-type questions. Both questionnaires measure four dimensions: horizontal collectivism (HC – empathy, cooperation, sociability), horizontal individualism (HI – independence, uniqueness, self-sufficiency), vertical collectivism (VC – submissiveness) and vertical individualism (VI – competitiveness). The validation study was conducted on 267 participants by Singelis et al. (1995). The reliability of scales was not ideal (HI α = 0.67, VI α = 0.74, HC α = 0.74, VC α = 0.68) nor were the CFA fit indices [χ2 (458) = 898.88, GFI = 0.79, AGFI = 0.75, RMSR = 0.089]. Based on the CFA results, the item pool was reduced from 94 to 32 items. The questionnaire was improved by Triandis and Gelfand (1998) on 543 participants in total (Study 1, n = 326; Study 2, n = 127; Study 3, n = 90). They selected 27 items with the highest factor loadings and also reported higher reliability coefficients (HI α = 0.81, VI α = 0.82, HC α = 0.80, VC α = 0.73). The 27 INDCOL items also showed good convergent and divergent validity through correlations with I/C scenarios. However, they did not repeat the CFA for the 27-item questionnaire, nor did they perform MI.

The Czech validation study was conducted by Bartoš (2010) on 1081 participants. He modified the nine-point, Likert type items to seven-points and reduced the number of items to 24. He applied Exploratory Factor Analysis (EFA) to examine the factor structure of the Czech version of the INDCOL and found five factors. He separated HI into HI1 (uniqueness; 3 items) and HI2 (independence; 2 items). VI (7 items), HC (7 items) and VC (5 items) remained the same as in the original study. He also conducted reliability estimates, but two scales did not meet the minimum criteria (VI α = 0.79, HI1 α = 0.71, HI2 α = 0.60, VC α = 0.63, HC α = 0.76). Although the Czech version of the scale seems to have limited psychometric properties, we decided to use it in this study for two reasons: (1) to verify its factor structure as reported by Bartoš (2010) using CFA on an independent Czech sample, and (2) to test its convergent validity with SCS, because it is, despite its limitations, the only available criteria for Czech samples.

Analytical Procedure

In order to examine the factor structure of both questionnaires, we performed a CFA with a robust, weighted, least square mean and variance (WLSMV) estimator, which is suitable for ordinal and non-Gaussian distributed data from Likert-type scales (Finney and DiStefano, 2013), because according to multivariate Henze-Zirkler tests, data were non-normally distributed at the subscale level (univariate Shapiro-Wilk tests confirmed these findings at the item level) for both questionnaires, and which is also less biased than robust maximum likelihood (MLR; Li, 2016). As the criteria for evaluating a good model fit, many more or less strict cut-offs are used. We used the Tucker-Lewis Index (TLI) ≥ 0.95, Comparative Fit Index (CFI) ≥ 0.95, Root Mean Square Error of Approximation (RMSEA) ≤ 0.60, Standardized Root Mean Square Residual (SRMR) ≤ 0.80 (Hu and Bentler, 1999) and Adjusted Goodness-of-Fit Index (AGFI) ≥ 0.90 (Hooper et al., 2008) fit indices for the evaluation of a good model fit in this study.

Internal consistency of subscales was assessed with Cronbach’s α and McDonald’s ω. We used the 0.70 threshold of internal consistency as a satisfactory indicator of reliability. We also performed a reliability analysis with ipsatization in order to reduce culture-specific response and acquiescence biases (Fischer, 2004; Fischer and Milfont, 2010). Standardized within-subject ipsative scores were calculated for each item of each individual according to the following formula:

i p s a t i v e s c o r e = \frac{r e s p o n s e - M o f s c a l e f o r e a c h i n d i v i d u a l}{S D o f s c a l e o r e a c h i n d i v i d u a l} .

Convergent validity between and within measures was verified with nonparametric Spearman’s correlation analyses, while each subscale score was entered into analysis as arithmetic mean. We interpreted correlation coefficients higher than 0.50 as indicators of minimally acceptable convergent validity and coefficients higher than 0.70 as sufficient evidence for convergent validity (Carlson and Herdman, 2010). The statistical analysis was conducted in R (v 3.6.1; R Core Team, 2020), specifically the packages lavaan (Rosseel, 2012), semTools (Jorgensen et al., 2018), psych (Revelle, 2020), ShinyItemAnalysis (Martinkova and Drabinova, 2018), and MVN (Korkmaz et al., 2014).

Results

The descriptive statistics (means, standard deviations, skewness and kurtosis) of all scales are shown in Table 2. The item analysis within classical test theory approach (i.e., descriptive statistics of items, several types of discrimination, etc.) is reported in Supplementary Appendix II.

TABLE 2

Table 2. The descriptive statistics of subscale scores.

Factor Structure

The Czech version of the SCS showed satisfactory RMSEA and SRMR. The relative chi-square (χ²/df) was 1.913, which suggested a good global fit of the model (Kline, 1998). However, the model showed unsatisfactory CFI and TLI values. Nevertheless, we would like to emphasized that CFI of the Czech version of the SCS was almost the same as the CFI of the Vignoles et al. (2016) original version (see Table 3). A common factor with acquiescence as a common method factor was used on the reversed items following the procedure used by Vignoles et al. (2016) in order to reduce acquiescence bias (see Welkenhuysen-Gybels et al., 2003). This model also did not yield satisfactory fit indices.

TABLE 3

Table 3. The SCS and SCS modified model fit indices compared to the original version by Vignoles et al. (2016).

Almost all of the items’ factor loadings besides three instances were above the recommended 0.40 threshold (Fornell and Larcker, 1981). Even if we take into consideration the stricter thresholds, for instance 0.50 (Hair et al., 2018), we would obtain only four more such instances. Furthermore, the current factor loadings often being higher than the originals obtained by Vignoles et al. (2016; see Supplementary Appendix I). All item parameters, covariances (with two exceptions) and variances were statistically significant). Therefore, no items had to be removed from the model in order to improve its fit.

An analysis of the potential cross-loadings with a modification index (mi) and expected parameter change (epc) could bring deeper insight into model misfit. We found that item 32 (for items wording see Supplementary Appendix I) from “Self-expression vs. Harmony” had potential cross-loadings on subscales “Self-containment vs. Connection to others” (mi = 155.075, epc = 0.869), “Self-interest vs. Commitment to others” (mi = 148.989, epc = 1.048) and “Consistency vs. Variability” (mi = 87.303, epc = −0.627). Analogously, item 15 from Self-direction vs. Receptiveness to influence had potential cross-loadings on subscales “Self-containment vs. Connection to others” (mi = 73.090, epc = 1.079), “Consistency vs. Variability” (mi = 68.539, epc = −0.746) and “Self-interest vs. Commitment to others” (mi = 57.856, epc = 1.239). Item 35 from “Self-interest vs. Commitment to others” had potential cross-loadings on “Self-direction vs. Receptiveness to influence” (mi = 61.102, epc = 1.124), “Self-expression vs. Harmony” (mi = 43.283, epc = 0.789) and “Self-containment vs. Connection to others” (mi = 39.786, epc = −1.486). Finally, item 25 from “Consistency vs. Variability” had potential cross-loadings on subscales “Self-direction vs. Receptiveness to influence” (mi = 55.498, epc = 0.563), “Self-interest vs. Commitment to others” (mi = 49.360, epc = 0.453) and “Self-containment vs. Connection to others” (mi = 48.192, epc = 0.485). These findings are discussed in detail in the Discussion section.

The analysis of modification indices showed that the SCS contained multiple cross-loaded items. If all of the above-mentioned cross-loadings are included in the model (CZ SCS Mod.; see Table 2 above), the majority of fit indices would be better than the original study by Vignoles et al. (2016). However, even these improved fit indices could still be considered unsatisfactory. Furthermore, using an exploratory approach (i.e., not confirmatory, cf. Bollen, 1989a; Byrne, 2010) we proposed four alternative models with individualism dimension as a one second-order factor of all subscales (i.e., CZ SCS 1 second-order factor), with individualism as a general factor in the bifactor model (i.e., CZ SCS bifactor), with individualism (non-reversed items) and collectivism subscales (reversed items in two-dimensional model (i.e., CZ SCS 2 factors) and with two common factors with acquiescence as a common method factor separately on reversed and positive items (i.e., CZ SCS 3, see Table 2). None of these models fit the data well.

The same CFA procedure was applied for the INDCOL scale with similar results as the SCS. The relative chi-square (χ²/df) was 2.118, which suggested a good global fit. The RMSEA and SRMR were satisfactory, however, the CFI and TLI were not. Hence, the current CFA results did support neither the 5-factor configural model provided by Bartoš (2010) nor the original 4-factor model provided by Singelis et al. (1995; see Table 4).

TABLE 4

Table 4. The INDCOL fit indices compared to the original version by Singelis et al. (1995).

Reliability

Concerning the reliability of the SCS, Cronbach’s α varied between 0.667 and 0.855, while McDonald’s ω fell between 0.651 and 0.854 (see Table 5). The values of both coefficients showed satisfactory internal consistency in most of the subscales. Three subscales were slightly below the minimum threshold of 0.70. In case of ipsatization, α varied between 0.265 and 0.786. The results with ipsative scores were less satisfactory than the raw score results. This suggested that the questionnaire’s items might have been potentially influenced by a response bias (especially the subscales “Self-direction vs. Receptiveness to influence” and “Self-expression vs. Harmony”).

TABLE 5

Table 5. Reliability estimations of the Czech version of the SCS subscales.

Concerning the reliability of the INDCOL, α varied between 0.502 and 0.812 (for ipsative α between 0.432 and 0.642), while ω fell between 0.483 and 0.814 (see Table 6). The values of both coefficients demonstrated satisfactory internal consistency only for the VI and HC subscales. VC and HI demonstrated unsatisfactory internal consistency, and the ipsative scores suggested that they might have been influenced by a response bias.

TABLE 6

Table 6. Reliability estimations of the Czech version of the INDCOL subscales.

Convergent Validity Between Measures

In the following section, the results of the correlation analyses between subscales of the original and adapted version of SCS, between subscales of the original and adapted version of INDCOL, and between scales of SCS and INDOL are reported. A comparison of Spearman’s ρ to the original correlation coefficients (by Vignoles et al., 2016) in the subscales is shown in Table 7. All differences between obtained and original coefficients were smaller than 0.250. Besides a few exceptions, the Czech version showed relatively similar patterns of correlations to the original version. All of these associations were statistically significant and ranged from small to medium effect sizes (with exceptions of two insignificant associations and one association with high effect size).

TABLE 7

Table 7. Comparison of the estimated correlations in the Czech version (below diagonal) and the original version (above diagonal) of the SCS by Vignoles et al. (2016).

A comparison of the Spearman’s ρ correlations and original correlations of the Czech INDCOL version among the subscales is shown in Table 8. Even though current correlations were generally higher than correlations reported by Bartoš (2010), the corelations coefficients were still rather small. We also observed three differences between original coefficients and coefficients obtained in this study which were higher than 0.250. However, our results appear to be more in line with the I/C theory, because negative correlations between the HC (collectivism) and individualistic subscales (VI and HI2) were observed (instead of positive as reported by Bartoš, 2010).

TABLE 8

Table 8. Comparison of the estimated correlations in the current (below diagonal) and original study (above diagonal) of the Czech version of INDCOL by Bartoš (2010).

Relationships were also expected between the SCS subscales and the INDCOL subscales as a demonstration of convergent validity. We assumed that HC and VC (i.e., collectivism) should be negatively correlated with all SCS subscales, whereas HI1, HI2 and VI (i.e., individualism) should correlate positively. As shown in Table 9, our expectations about directions were confirmed. However, these r_s coefficients were relatively small, and some of them non-significant. Only four associations were higher than 0.50 threshold (“difference vs. similarity” and “horizontal individualism: uniqueness”; “self-direction vs. receptiveness to influence” and “vertical collectivism”; “self-containment vs. connection to others” and “horizontal collectivism”; and “self-interest vs. commitment to others” and “horizontal collectivism”); none of them were above the recommended 0.70 value. Hence, these results do not support the assumption of the convergent validity of the SCS and INDCOL. It appears that both scales measure slightly different constructs.

TABLE 9

Table 9. Coefficients of Spearman’s rho in the SCS and INDCOL.

Discussion

The Psychometric Properties of the SCS

A validation study of the SCS was conducted and the psychometric properties of the SCS were examined. As Bollen (1989b) pointed out, the replication on independent samples is the only way to check whether original associations are a sampling fluke or not. He also emphasized the necessity of such research, because despite the fact that replications are often considered very valuable, such studies appear far too seldom. This type of research therefore serves as a contribution to the cross-cultural examination of the I/C concept and its results are necessary for a deeper understanding of I/C across various cultures.

In summary, both questionnaires demonstrated limitations in their reliability and validity. These shortcomings could have stemmed from the lack of reliability and validity of the original versions (Singelis et al., 1995; Triandis and Gelfand, 1998; Bartoš, 2010; Vignoles et al., 2016) rather than our cross-cultural adaptation.

In more detail, despite that Czech version of the SCS showed satisfactory reliability in four subscales (the rest of the subscales were only slightly below the 0.70), similar correlations between subscales were observed and similar fit indices were obtained in comparison with the original study (Vignoles et al., 2016). Additionally, our study revealed several crucial psychometric shortcomings of the scale which suggest its insufficient validity and reliability. This might be to some extent a consequence of the psychometric properties of the original instrument.

Four main issues were identified. First, the reliability estimation with ipsative scores showed poor internal consistency suggesting that the SCS may be influenced by response biases. Second, the CFA results were unsatisfactory, and therefore cross-cultural comparisons using this questionnaire might be biased, non-invariant and invalid (Fischer and Karl, 2019). Both the original and Czech versions of the SCS have serious shortcomings in their factor structure, as suggested by, for example, the CFI.

Consequently, the third issue stemmed from an analysis of the modification indices, which revealed some cross-loadings. For example, the item “You follow your personal goals even if they are very different from the goals of your family” might saturate not only “Self-interest vs. Commitment to others,” but also “Self-containment vs. Connection to others,” “Self-direction vs. Receptiveness to influence” and “Self-expression vs. Harmony.” This finding seems logical, because a person who answers negatively to the mentioned item is probably not only more committed to others, but also leans toward harmony and receptiveness to an influence and is more connected to others. The presented analysis suggests that many items have similar cross-loadings. We believe that this is probably more likely caused by the poor theoretical background in the latent variables than vague and ambiguous item wording. Consequently, SCS factors are vaguely defined and lack divergent validity because they are based primarily on psychometric results. Therefore, even simply worded items (e.g., the item “You always ask your family for advice before making a decision”) have potential cross-loadings on other subscales. Furthermore, the semantic qualities of some factors seem to be quite similar (e.g., “Self-direction vs. Receptiveness to influence” and “Self-interest vs. Commitment to others”), and perhaps may be adequate to reduce the number of I/C factors. Although the process of validation of a multidimensional cross-cultural questionnaire like SCS is very tedious, it should not be limited just to the psychometric evaluation of factor structure, model fit, reliability, etc., but it should also be theoretically well grounded.

And four, despite that all directions of relationships between SCS and INDCOL were as expected, i.e., the dimensions of horizontal and vertical collectivism correlated with interdependent self, whereas the dimensions of horizontal and vertical individualism correlated with independent self, the correlation coefficients were mostly small or moderate. This suggests, that both scales probably measure slightly different and insufficiently related constructs The above-mentioned issues with the SCS lead us to questions about the I/C concept itself, because similar issues were observed in multiple previous studies (see below).

General Issues of the I/C Concept

Research of I/C has been criticized by many scholars. Generally, there is no questionnaire in the literature measuring I/C that repeatedly meets the demanding requirements of cross-cultural research (i.e., CFA, MG-CFA, MI across different cultures, controlling for response bias, etc.). Many studies do not sufficiently conduct or report these important psychometric properties, and do not conduct any adequate multi-level analysis (Oyserman et al., 2002; Levine et al., 2003a,b; Chen and West, 2008; Cozma, 2011). Another relevant critique argues that the I/C research ignores or even lacks concurrent and discriminant validity of the scales (Oyserman et al., 2002; Levine et al., 2003b; Bresnahan et al., 2005; Schimmack et al., 2005). Furthermore, the conceptual unclarity of the I/C research is also often criticized (e.g., Oyserman et al., 2002; Voronov and Singer, 2002; Brewer and Chen, 2007; Oyserman and Lee, 2008).

As mentioned above, the original validation studies of INDCOL (Singelis et al., 1995; Triandis and Gelfand, 1998), SCS (Vignoles et al., 2016), VSM (Hofstede, 1994), SVS (Schwartz, 1992), and IISS (Lu and Gilmour, 2007) did not meet the minimum criteria of CFA or did not even perform such a procedure. Similar patterns can be found in other I/C questionnaires, for example in the W-M (Wagner and Moch, 1986), COS (Communal Orientation Scale; Clark et al., 1987), RISC (Relational-Interdependent Self-Construal; Cross et al., 2000), AICS (Auckland Individualism and Collectivism Scale; Shulruf et al., 2007) and ICIA scales (Collectivism Interpersonal Assessment Inventory; Matsumoto et al., 1997).

On the other hand, there are also several exceptions. For example, the Human Relations Questionnaire (HRQ; Noguchi, 2007) identifies three factors, namely the focus on others, helping others and self-focus, with satisfactory CFA indices: χ²(24) = 49.93, GFI = 0.973, CFI = 0.964, RMSEA = 0.052. Nevertheless, the author did not perform MG-CFA and MI analysis across the United States and Japanese versions, and its internal consistency was also not sufficient (α = 0.44-0.76). Three factors were also identified in the RIC scale (Relational, Individual and Collective Self-Aspects; Kashima and Hardie, 2000), namely relational, individual and collective self-aspects. The RIC scale showed satisfactory CFA fit indices [χ²(24) = 79.26, CFI = 0.94, NFI = 0.91]; however, the CFA was supported by a nine-item model (only 3 items per subscale), not the original 30-item model (CFI = 0.72).

Other examples can be the PVQ (Schwartz et al., 2001), or the Collective Self-Esteem Scale (CSES, Luhtanen and Crocker, 1992). The PVQ provided an acceptable MG-CFA but insufficient MI results (Anýžová, 2014), while the four-dimensional CSES reached close to satisfactory CFA results (NFI = 0.74–0.91, TLI = 0.84–0.92, CFI = 0.87–0.93). However, the CSES factor structure was not confirmed in the African American ethnic group, and therefore we can doubt its usability in a cross-cultural equivalence (Utsey and Constantine, 2006). Additionally, both questionnaires also focus primarily on different constructs that only possess a partial overlap with the I/C concept, i.e., cultural values in the case of PVQ and collective self-esteem in the case of CSES.

In this paper we conducted an attempt to validate an adaptation of a relatively new I/C scale on a Czech sample which is a sample not fitting into the group of West and East countries (such as the United States, England, Japan, China) that are studied the most often in the field. In this section we want to provide information about other similar research going beyond this dichotomy, both successful and unsuccessful ones. We omit adaptations without a CFA procedure or its adequate equivalent, which unfortunately represents the vast majority of studies (Chen and West, 2008). I/C scales were already successfully adapted for instance in Turkey (e.g., Li and Aksoy, 2006; Akın et al., 2010), Jordan, Lebanon, and Syria (e.g., Harb and Smith, 2008), Hong Kong and Ghana (e.g., Affum-Osei et al., 2019) or Switzerland and South Africa (results were satisfactory only for one of two used I/C scales; see Györkös et al., 2012).

Nevertheless, these successful attempts are relatively rare compared to the amount of studies that failed to do so. In spite of the fact that the authors themselves often interpreted the quality of the adaptations as satisfactory and part of studies is indeed methodologically and statistically sound, a deeper inspection reveals issues in the factor structure of the adapted scales. The adaptations usually did not yield satisfactory fit indices. In some cases, the number of factors and items were substantially changed compared to the original scales. Despite the fact that these model modifications and changes lead in some cases to the satisfactory fit indices of the “new” scales, this rather data-driven approach needs to be considered exploratory (see e.g., Bollen, 1989a; Byrne, 2010).

Similar psychometric problems with the adaptation of the I/C scale as in the current study were observed for instance in Poland (cf. Pilarska, 2011), Spain (cf. Gouveia et al., 2003), India (cf. Sivadas et al., 2008), Malaysia (cf. Miramontes, 2011; Ramley et al., 2020), Mexico (cf. Miramontes, 2011), Singapore (cf. Soh and Leong, 2002), Italy (cf. Bobbio and Sarrica, 2009; D’Amico and Scrima, 2015; Germani et al., 2020), France (cf. Gibas et al., 2016), Philippines (cf. Miramontes, 2011; Bernardo et al., 2012; Datu, 2014), Australia (cf. Freeman and Bordia, 2001; Miramontes, 2011), Portugal (cf. Gonçalves et al., 2017), Thailand (cf. Christopher et al., 2011) or Argentina (cf. Chiou, 2001).

The current, rather unsatisfactory results might have deeper causes than just the psychometric quality of the original SCS scale. Even though past studies assumed I/C being a stable cross-cultural construct with an ambition to categorize nations along the collectivistic and individualistic spectrum, these assumptions were not entirely confirmed. It seems that the relatively simplistic East-West dichotomy doesn’t truly exist (Matsumoto, 1999; Takano and Osaka, 1999, 2018; Heine et al., 2002; Oyserman et al., 2002; Levine et al., 2003a,b), and the I/C construct is far less stable than assumed (Yamagishi, 1988; Gardner et al., 1999; Oyserman and Lee, 2008). Consequently, some authors with respect to the previously mentioned shortcomings and critiques of I/C research came to the conclusion that the concept of I/C itself does not exist and suggest not using it in research (e.g., Levine et al., 2003a,b). Therefore, doubts about the validity of using I/C as a predictor of other constructs in current cross-cultural studies should be raised.

Limitations and Future Directions

The results of our study are based on the unrepresentative sample gained through the non-probability convenience sampling method which resulted in various imbalances of demographic characteristics, especially in the overrepresentation of women, young participants and participants with a university education. An analysis performed on different populations might result in a different factor structure. However, I/C research usually validates the scales on samples of university students; for example, INDCOL (Singelis et al., 1995; Triandis and Gelfand, 1998) was created on the basis of such samples and SCS (Vignoles et al., 2016) used this population in the first phase of their validation study. Furthermore, despite the fact that sample size was considered satisfactory, SEM usually needs large samples and therefore is an a priori power analysis based on model simulations highly recommended (Hoyle, 2012; Little, 2013; Brown, 2015; Kline, 1998), which was not performed in the current study.

The future research should, in the first step, focus on a redefinition and reconceptualization of the I/C construct (e.g., Oyserman et al., 2002; Voronov and Singer, 2002; Brewer and Chen, 2007; Oyserman and Lee, 2008), while it is quite possible that such redefinition would not be universal for different cultural groups. Consequently, after the theoretical clarification of the I/C concept, the main aim of the future research should lie in a sounder methodological and statistical approach such as routinely using SEM techniques.

One of the possible ways to achieve this is the development of a new self-report instrument with satisfactory psychometric properties with the potential to be adapted in multiple cultures (Schimmack et al., 2005; Chen and West, 2008; Cozma, 2011). An important characteristic of this instrument would be its resistance to a reference-group effect (see Heine et al., 2002). Additionally, such an instrument would need to yield satisfactory results in repeated replications on independent samples from both the same and other cultures (Bollen, 1989b). Furthermore, adequate statistical approaches need to be used while comparing means across various cultural groups, such as MG-CFA with scalar measurement invariance (see Fischer and Karl, 2019). The research needs to be robust enough to cover the whole spectrum of variables that can potentially affect the level of I/C in order to reduce the cultural attribution fallacy (i.e., unpacking studies; see Matsumoto and Juang, 2013). Since the validation procedure usually does not end with one (un)successful validation study, but it represents an iterative process of bringing new evidence of validity and reliability the research in the field is far from concluded. However, we believe that without such an approach it is not possible to bring valid information about the real nature of I/C in culturally diverse populations via self-report scales.

The second possible approach to solve the current unsatisfactory situation in I/C research could lie in a shift from quantitative self-report questionnaires based on verbal responses to the usage of entirely different group of methods (e.g., Matsumoto, 1999; Bond, 2002; Fiske, 2002; Heine et al., 2002). For example, Talhelm et al. (2018) observed the differences in I/C with an observational design of the real-life behavior of participants; Partikova (2019) used interpretative phenomenological analysis of semi-structured interviews, and Hsu and Barker (2013) identified differences in I/C using the content analysis of TV advertisements. Furthermore, meta-analysis of other “cultural products” by Morling and Lamoreaux (2008) revealed higher effect sizes than meta-analyses performed on self-report scales. Another example can be found in the work of Klein et al. (2018) who created a new “WEIRDness score” (WEIRD: Western, educated, industrialized, rich and democratic; see Henrich et al., 2010) which might be included into multi-group statistical analysis as a predictor in a similar fashion as Hofstede’s dimensions. We believe that it might be possible to create a similar country-level index specifically related to the I/C. Maybe not on the basis of self-report questionnaire data, but rather from an in-depth qualitative analysis of several indicators and consequent inter-rater agreement of experts from various cultures.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by The Research Ethics Committee of Masaryk University (Ref. No.: EKV-2018-011, Proposal No.: 0257/2018). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements. Written informed consent was implied via completion of the questionnaires.

Author Contributions

DL contributed to data collection, article drafting, and data analysis. JČ contributed to questionnaire adaptation, data collection, and article drafting. TU contributed to article revisions and data analysis. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the Czech Science Foundation under Grant GC19-09265J (The influence of socio-cultural factors and writing system on perception and cognition of complex visual stimuli).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2021.564011/full#supplementary-material

References

Affum-Osei, E., Aboagye, M. O., Antwi, C. O., and Asante, E. A. (2019). Validating the Auckland individualism–collectivism scale (AICS): testing factor structure and measurement invariance in Hong Kong and Ghanaian Samples. Psychol. Stud. 64, 187–199. doi: 10.1007/s12646-019-00494-2