Does higher endorsement with collaboration lead to better performance on collaborative problem solving? An explanatory item response approach to cross-cultural comparisons

Introduction: Collaborative problem solving (CPS) is an essential competency in the 21st century. However, the understanding of how cultural background shapes individuals’ collaboration awareness and its relationship with collaborative problem-solving skills are underexplored by psychometrics.

Methods: This study employs Explanatory Item Response Modeling to examine the impact of cultural orientations on students’ endorsement with collaboration in PISA 2015.

Results: Results show that students endorsed valuing teamwork more than valuing relationships. Western countries with individualist cultures generally demonstrated higher endorsement with collaboration than Eastern countries with collectivist cultures. China has the highest collaboration endorsement, followed by the US, Canada, Korea, and Japan. In contrast, Japan has the highest CPS assessment scores, followed by Korea, Canada, the US, and China.

Discussion: Findings revealed that students from collectivist cultures do not have higher endorsements of collaboration compared with students from individualist cultures. Also, higher endorsement of collaboration does not necessarily lead to better success in the CPS assessment. Further research is needed to understand the gap between students’ attitudes towards collaboration and their achievement based on cultural values and schools’ CPS training.

Introduction

Collaborative problem solving (CPS) is regarded as one of the core competencies in the 21st century (von Davier and Halpin, 2013; Griffin and Care, 2014). CPS is the process of “approaching a problem responsively by working together and exchanging ideas” to achieve mutual goals by two or more parties (Hesse et al., 2015). It is often conceptualized as a highly loaded skill that systematically combines problem-solving competencies and communicative skills in collaboration and teamwork situations. This process requires an understanding of the context, establishing, and making use of knowledge, as well as endeavoring individual efforts and contributing to the group to achieve a specific goal (Griffin et al., 2013; Hilse et al., 2014).

During the last decade, CPS skills have been gaining attention in education, and an increasing number of curricula and assessment programs have included CPS as part of their agenda (OECD, 2017). In 2015, CPS tasks were first integrated into the twenty-first-century skills framework as a key competency and were introduced as a principal assessment section by the Assessment and Teaching of 21st-Century Skills (ATC21S) and the Organization for Economic Co-operation and Development (OECD) boards (Griffin and Care, 2014). Since then, many studies have been conducted on exploring potential factors that influence CPS and searching for effective tools to teach and assess CPS skills (Kirschner et al., 2012; Rosen and Rimor, 2016; Stefik et al., 1987).

Concurrently, discussions are also ongoing to understand CPS from different perspectives, especially on factors that are attributed to the success of collaboration (Griffin et al., 2013; Sycara et al., 2013). Previous studies suggest that confounding factors such as gender, cultural background, and language proficiency can influence the process as well as the outcomes of collaboration (Chang et al., 2017; Hao et al., 2017). However, cross-national empirical studies focusing on the relationship between students’ academic achievement and their attitudes towards CPS attitudes lagging.

The present study aims to examine the impact of cultural orientations on individuals’ awareness of collaboration from five representative countries (i.e., China, Japan, Korea, the United States, and Canada). The collaboration scale from Program for International Student Assessment (PISA) 2015 (OECD, 2017) was used as an instrument to measure levels of endorsement and awareness of collaboration. Explanatory item response modeling (EIRM) was used to analyze students’ responses to the items on the collaboration scale based on item-level and person-level covariates and to reveal students’ potential attitudinal differences towards collaboration on two dimensions (i.e., valuing relationships and valuing teamwork). This study was guided by the following research questions:

1) To what extent do students endorse with valuing relationships versus valuing teamwork?

2) To what extent do students from the different cultural background vary on their endorsement with valuing relationships and valuing teamwork?

3) Does higher endorsement with collaboration associate with better performance on collaborative problem solving?

The significance of the present study is threefold. Theoretically, we examined the impact of cultural orientations on students’ attitudes towards collaboration through a validated, large-scale international survey. Results can be used for planning future research on fostering attitudes towards collaboration and CPS competency. Methodologically, the present study is the first study that employs EIRM to analyze the polytomous responses in a large-scale survey. The explanatory item response models provide rich information on how students’ responses in the collaboration scale differed based on both item properties and student characteristics. Pedagogically, findings facilitate educators and policymakers make informed decisions on teaching and learning of collaborative problem-solving.

Literature review

Importance of endorsement with collaborative problem solving

Collaboration is defined as a “coordinated, synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem” (Roschelle and Teasley, 1995, p. 70). CPS is often dichotomized into the social domain of collaboration skills and the cognitive domain of problem-solving skills. The problem-solving process involves cognitive abilities including task regulation, perspective-taking, and reflecting. The collaboration process is defined as the activity of working together as a team instead of individuals to achieve a common goal (Care et al., 2015). The process of collaboration can be further divided into three main aspects: participation, perspective-taking, and social regulation. Specifically, participation is defined as an individual’s willingness to offer and share information and thoughts, and the extent of their involvement as part of a team. Perspective-taking skills, as another dimension of exchange information, refers to an individual’s ability to internalize others’ opinions and make adaptations or adjustments for their purposes (Dehler et al., 2011).

In the Framework for Teachable Collaborative Problem Solving Skills, one of the key factors that ensures a successful collaboration is shared mental representation, that is, the participant’s mental representation should be aligned to other group members’ mental spaces within the same group (Hesse et al., 2015). Many studies have shown that participants usually demonstrate a higher performance of collaboration or collaborative problem solving when higher levels of shared mental representations are achieved (Klimoski and Mohammed, 1994). This type of alignment could take effect in each step of collaboration including goal setting, plan execution, communication, and progress evaluation. In this sequence of joint actions, participant’s awareness, or willingness to collaborate, plays a crucial role in positively fostering the successful fulfillment of the CPS task.

Endorsement with CPS shaped by cultural orientation

Given the importance of endorsement with collaboration to CPS skills, many theoretical, as well as empirical studies, were dedicated to exploring factors that influence individual’s attitudes towards collaboration in different domains (e.g., Bollen et al., 2019; Cradock-Henry et al., 2017; Mattessich and Monsey, 1992). Among many causes, it is found that culture plays an important role in shaping individuals’ awareness of collaborations and cognitive development through social interactions (Patel et al., 2012). Culture can be referred to as national or regional culture, which serves as a vital factor that shapes the collective programming of human minds and behavior patterns, and often is represented by a set of values or beliefs about the willingness to communicate, openness to knowledge sharing, and trustfulness of interpersonal interaction (Hofstede, 2011).

To date, several studies were conducted to define culture and to investigate how culture steers individuals’ social and cognitive behaviors at large. Hofstede’s multi-dimensional model (Hofstede, 2011), a seminal framework in cross-cultural psychology, assesses national cultures across six dimensions: Power Distance, Uncertainty Avoidance, Individualism versus Collectivism, Masculinity versus Femininity, Long Term Orientation, and Indulgence versus Restraint. Originating from an extensive survey spanning over 50 countries and 100,000 participants, the model interprets cultural differences and their impact on societal behaviors and values. It postulates that these dimensions shape not just ideologies but also the collective behaviors within a society, making human actions predictable to an extent.

The impact of these cultural dimensions, particularly Individualism versus Collectivism, has been profound on collaborative practices and problem-solving, as subsequent studies indicate the profound influence of a society’s orientation on individuals’ approach to group tasks (Hofstede, 2011). These studies reveal that individualists prioritize personal goals and task achievement, often above group harmony, while collectivists tend to value group cohesion and interpersonal relationships, potentially at the expense of task outcomes. This cultural orientation influences not only personal relationships and work dynamics but also educational approaches, particularly in collaborative learning settings (e.g., Arieli and Sagiv, 2018; Arpaci et al., 2018; Zhang and Yin, 2019).

Collaborative climates in China, Japan, Korea, US, and Canada

Hofstede’s studies utilized the World Values Survey to measure cultural values across 76 countries, introducing the Individualism vs. Collectivism (IDV) index. They found a link between a country’s wealth, geographical location, and its level of individualism, with richer Western countries tending towards individualism and Eastern countries towards collectivism. The U.S., Australia, the UK, and Canada scored high on individualism, while East Asian countries like Korea and China scored low. Subsequent research has continued to explore these cultural dimensions in representative nations. Chen (2000) conducted a comparative study and claimed that Chinese children grow up in a collectivist culture, and thus their psychological and cognitive development is different from children from individualist countries such as the United States. In terms of behavioral representations, Asian societies often display shyness, self-restraint, and high levels of socialization, whereas North American societies encourage self-assertion and personal independence. Oyserman et al. (2002) meta-analysis found that Canada and the U.S. are among the world’s most individualistic societies, valuing individual rights over social ties. However, compared to Japan and Korea, these differences in individualism versus collectivism are not as pronounced in certain behaviors. Only China distinctly favored collectivism, which has implications for understanding social behaviors in collaboration and conflict resolution across cultures.

Explanatory item response modeling

Item response theory (IRT) models are commonly adopted tools to analyze survey responses. Traditional IRT models are widely used for item parameter estimation but often criticized for failing to incorporate additional variables that are likely to influence responses to the items. Unlike traditional IRT models, explanatory IRT models allow researchers to use both item-level (i.e., item properties) and person-level (i.e., student characteristics) when estimating item parameters and latent trait levels (De Boeck and Wilson, 2004). EIRM has originated from the generalized linear mixed modeling (GLMM) framework in which items (i.e., repeated measures) are assumed to be nested within students (i.e., random effects) within a multilevel structure. This approach enables the use of explanatory variables (i.e., covariates) at the item and person levels to predict student responses in the same model. To date, many studies have applied EIRM to a wide range of areas, including differential item functioning (French and Finch, 2010), contextual effect detection (Randall et al., 2011), and proficiency scaling (Hartig et al., 2012). However, these studies employed EIRM to model the effects of item-level and person-level variables using dichotomous item responses (i.e., correct or incorrect). To date, only several studies have utilized the EIRM approach to model polytomous responses, survey items based on a Likert scale.

Tuerlinckx and Wang (2004) first attempted to analyze a publicly-available verbal aggression dataset using two polytomous IRT models including the Rating Scale Model (RSM; Andrich, 1978) and Partial Credit Model (PCM; Masters, 1982) with item-level and person-level covariates. In their study, the explanatory form of PCM (EPCM) can be formulated as the log-odds of choosing response j over (j − 1) on item i for person n:

$\begin{array}{ll}\log \left(\frac{P_{nij}}{P_{ni\left(j-1\right)}}\right)={\mathrm{Z}}_{\mathrm{n}\mathrm{i}\mathrm{j}}{\theta }_{\mathrm{n}}-{\mathrm{X}}_{\mathrm{n}\mathrm{i}\mathrm{j}}^{\prime }{\delta }_{\mathrm{i}}+{\tau }_{\mathrm{i}\mathrm{j}}& (1)\end{array}$

where ${\theta }_{\mathrm{n}}$ represents the latent trait of person n, ${\mathrm{Z}}_{\mathrm{n}\mathrm{i}\mathrm{j}}$ is a matrix of fixed and random effects related to the latent trait ${\theta }_{\mathrm{n}}$, ${\delta }_{\mathrm{i}}$ is the threshold parameter between the first (j = 0) and second (j = 1) response categories for item i, ${\mathrm{X}}_{\mathrm{n}\mathrm{i}\mathrm{j}}^{\prime }$ is the matrix of fixed and random effects related to individual items, and ${\tau }_{\mathrm{i}\mathrm{j}}$ is the matrix of the distances between the first threshold parameter and the other threshold parameters for item i. Tuerlinckx and Wang (2004) model allowed the estimation of not only item threshold parameters but also the effects of covariates related to items and persons. However, the model could only estimate the effects of item-related covariates on the first threshold parameter. That is, the model contains no explanatory variables to describe the common variability beyond the initial threshold parameter.

Stanke and Bulut (2019) used the same verbal aggression dataset and estimated four polytomous IRT models including RSM, PCM, EPCM, and cross-classified PCM with a new parameterization that enables the estimation of the effects of explanatory variables on both the first threshold parameter and the subsequent threshold parameters. That is, the model can account for the variability in all of the threshold parameters based on item-level covariates. In Equation 2, the log-odds of choosing response j over (j − 1) for item i for person n can be calculated as:

$\begin{array}{ll}\log \left(\frac{P_{nij}}{P_{ni\left(j-1\right)}}\right)={\mathrm{Z}}_{\mathrm{n}\mathrm{i}\mathrm{j}}{\theta }_{\mathrm{n}}-\left({\mathrm{X}}_{\mathrm{n}\mathrm{i}\mathrm{j}}^{\prime }{\delta }_{\mathrm{i}}+{\mathrm{W}}_{\mathrm{n}\mathrm{i}\mathrm{j}}^{\prime }{\tau }_{\mathrm{i}\mathrm{j}}\right)& (2)\end{array}$

where ${\mathrm{W}}_{\mathrm{n}\mathrm{i}\mathrm{j}}^{\prime }$ is a matrix of fixed and random effects related to the threshold parameters and the remaining elements are the same as those defined for Equation 1. Compared to Tuerlinckx and Wang (2004) approach, this method is more flexible in terms of explaining how all threshold parameters differ based on item-level covariates. In this study, Stanke and Bulut (2019) approach was employed to examine the impact of cultural orientations on attitudes towards collaboration. Students’ responses to the survey items in the Collaboration Scale of PISA 2015 were analyzed based on item-level (item type: valuing relationships vs. teamwork) and person-level (country type: individualism vs. collectivism) covariates.

The present study

To sum, the impact of culture on individuals’ attitudes towards collaboration revealed by previous studies suggests that the cultural factor provides the fundamental contexts for arousing awareness of collaboration and promoting individuals’ psychological engagement in collaborative tasks (Hofstede, 2011). Therefore, in the present study, we make two hypotheses: (1) students from collectivist cultures will have a higher endorsement of collaboration compared with students from individualistic cultures and (2) higher endorsement of collaboration will lead to better success in the CPS assessment.

Methods

Sample

The sample of the current study consists of students from five countries that participated in PISA 2015: China (n = 9,841), Japan (n = 6,647), Korea (n = 5,581), Canada (n = 20,058), and the United States (n = 5,712).

In total, 47,839 students took both the CPS assessment and the Collaboration Scale in the selected countries. The demographic information of the students and their CPS assessment scores from the five countries reported by OECD (2017) are shown in Table 1. To prepare the dataset, we first detected the missing values and found that the variables Country ID and Gender contained no missing data, whereas students’ responses to Item 1—Item 8 contained missing responses. We calculated that the proportion of missing data was lower than 5% (about 2%). Therefore, we deleted the samples with missing data and conducted further analyses.

Table 1

Table 1. Students’ demographic characteristics by countries.

Overview of collaboration scales in PISA 2015

PISA is a triennial assessment of competencies and attitudes that aims to measure “how well 15-year-old students approaching the end of compulsory schooling are prepared to meet the challenges of today’s knowledge societies” (OECD, 2017). Covering nearly half a million students in 2015, PISA uses a two-hour test on core subjects and a questionnaire suite. It has probed problem-solving since 2003, with a shift to computer-based assessment in 2012, defining it as cognitive processing to resolve non-obvious problems. In 2015, PISA expanded to include collaborative problem-solving (CPS), assessing individuals’ capacity to pool knowledge and efforts in a group problem-solving context. It also introduced the Collaboration Scale, assessing students’ attitudes towards valuing relationships and teamwork.

The Collaboration Scale measures students’ self-report levels of “establishing and maintaining shared understanding,” “taking appropriate action to solve the problem,” and “establishing and maintaining team organization” (OECD, 2017, p. 44). There were eight items from two domains of collaboration which measures students’ attitudes towards valuing relationships and measuring students’ attitudes towards teamwork, which are presented in Table 2. In the present study, we calculated the reliability indices (Cronbach’s Alpha) of the two scales among the selected countries and revealed that the reliability for scale of attitudes towards valuing relationships (Item1—Item4) is 0.7, and the reliability for scale of attitudes towards valuing teamwork (Item5—Item8) is 0.8, indicating satisfactory internal reliabilities of the two scales. Moreover, we also examined the construct validity of the two scales using Confirmatory Factor Analysis. The model yielded satisfactory model fit indices, with CFI = 0.94, TLI = 0.92, AGFI = 0.96, RMSEA = 0.08, and SRMR = 0.08. The standardized factor loadings of the eight items in the two scales are presented in Table 2.

Table 2

Table 2. Items in the disposition of collaboration and teamwork blocks.

Item-level and person-level covariates

To address the research questions, the effects of two explanatory variables were analyzed using EPCM. The item-level covariate item type is a two-way factor that explains whether an item measures attitudes towards valuing relationships or teamwork. Two sets of person-level covariates were used to code the countries based on the cultural orientations represented in Hofstede’s culture dimensions and their country identities. More specifically, country type refers to whether the selected country has an Eastern collectivist culture or a Western individualist culture; and country ID represents the identity of the selected country.

Item type

To explore the effect of item-level differences, we categorized items 1 to 4 as item type = 0 (valuing relationships), and items 5 to 8 as item type =1 (valuing teamwork). The effect of item type will be explored in EPCM as an item-level covariate.

Country type

Two categories of country type were created on the dimension of collectivism vs. individualism. The US and Canada were coded as individualist Western countries (country type = 1); and China, Korea, and Japan were coded as collectivist Eastern countries (country type = 0).

Country ID

Country ID was introduced in the EPCM to account for the variability in attitudes towards collaboration attitudes across the selected countries. We coded China as Country 1, Korea as Country 2, Japan as Country 3, Canada as Country 4, and the US as Country 5.

Data analysis

Partial credit model

The baseline model PCM was used to understand the psychometric properties. PCM yields the estimate of the first threshold parameter and the distances between the first threshold parameter and the subsequent threshold parameters. The goal of estimating PCM prior to explanatory IRT models was to review the threshold parameters of the items in the Collaboration Scale without considering the effects of any item-level and person-level covariates.

Explanatory partial credit model

To answer the four research questions of this study, a weighted version of EPCM was used to account for the total variance based on an item-level variable item type, and two person-level variables country type and country ID. PISA 2015 adopted a two-stage stratified sampling design with country as the first layer and schools as the second layer. To prevent misleading results caused from the unbalanced sample sizes, the Country Weight variable in the dataset was introduced in the EPCM to weight students’ responses of collaboration scales as instructed by the technical report of PISA 2015 for country-wise comparisons (OECD, 2013). The log-odds of choosing response j over (j − 1) on item i for person n in the weighted EPCM can be calculated by:

$\begin{array}{ll}{{\displaystyle \sum }}_{m=1}^n{d}_m\log \left(\frac{P_{nij}}{P_{ni\left(j-1\right)}}\right)={{\displaystyle \sum }}_{m=1}^n{d}_m\left({\mathrm{Z}}_{\mathrm{n}\mathrm{i}\mathrm{j}}{\theta }_{\mathrm{n}}+{\mathrm{X}}_{\mathrm{n}\mathrm{i}\mathrm{j}}^{\prime }{\delta }_{\mathrm{i}}+{\mathrm{W}}_{\mathrm{n}\mathrm{i}\mathrm{j}}^{\prime }{\tau }_{\mathrm{i}\mathrm{j}}\right)& (3)\end{array}$

where ${\theta }_{\mathrm{n}}$ represents students’ attitudes of collaboration where higher values indicate higher levels of positive attitudes towards collaboration, ${\mathrm{Z}}_{\mathrm{n}\mathrm{i}\mathrm{j}}$ is the matrix of the person-level predictors (i.e., country type and country ID), ${\delta }_{\mathrm{i}}$ is the estimate of the first threshold parameter between the response categories of “Strongly Disagree” and “Disagree” for item i, ${\mathrm{X}}_{\mathrm{n}\mathrm{i}\mathrm{j}}^{\prime }$ is the matrix of the item-level predictor (i.e., item type) for the first threshold, ${\tau }_{\mathrm{i}\mathrm{j}}$ represents the distances between the first threshold parameter and the thresholds for the subsequent response categories (i.e., “Disagree” to “Agree” and “Agree” to “Strongly Agree”), and ${\mathrm{W}}_{\mathrm{n}\mathrm{i}\mathrm{j}}^{\prime }$ is the matrix of the item-level predictor for the thresholds of the subsequent response categories. Equation 3 also includes a vector of survey weights, $d_m$, for each country $\left(m=1,2,3,4,5\right)$ to correct for the impact of sampling error as instructed in the technical report of PISA 2015 (OECD, 2017). The use of survey weights makes the sample of students in the PISA 2015 database representative of the population of middle school students from the selected countries.

Model summary

In the present study, five models were fitted to the weighted EPCM to estimate the effect of item-level covariates and person-level covariates. Equation 4 summarizes the first model that include item-level covariate only. More specifically, ${\theta }_n$ is calculated as the student n’s general attitudes towards collaboration, ${\delta }_1$ and ${\delta }_2$ are the estimates of the first threshold locations (from “Strongly Disagree” to “Agree”) for Item Type 0: valuing relationship and Item Type 1: valuing teamwork, respectively, ${\tau }_{11}$ and ${\tau }_{12}$ are the estimates of distances between step thresholds (from “Disagree” to “Agree” and from “Agree” to “Strongly Agree”) for Item Type 0: valuing relationship, ${\tau }_{21}$ and ${\tau }_{22}$ are the estimates of distances between step thresholds for Item Type 1: valuing teamwork, and ${\epsilon }_j$ is the error term. In addition, $w_m$ is the weight index assigned with respect to student n’s country identity.

$\begin{array}{l}{w}_m\left[\begin{array}{l}\log \left(\frac{P_{n\left(\mathit{\text{Strongly}}\mathit{\text{Disagree}}\right)}}{P_{n\left(\mathit{\text{Disagree}}\right)}}\right)\mathit{\text{or}}\log \left(\frac{P_{n\left(\mathit{\text{Disagree}}\right)}}{P_{n\left(\mathit{\text{Agree}}\right)}}\right)\mathit{\text{or}}\\ \log \left(\frac{P_{n\left(\mathit{\text{Agree}}\right)}}{P_{n\left(\mathit{\text{Strongly}}\mathit{\text{Agree}}\right)}}\right)\end{array}\right]\\ =w_m({\theta }_n+{\delta }_1{\left(\mathit{\text{Type}}0\right)}_i+{\delta }_2{\left(\mathit{\text{Type}}1\right)}_i+{\tau }_{11}{\left(\mathit{\text{Type}}0\right)}_i+{\tau }_{21}{\left(\mathit{\text{Type}}1\right)}_i\\ +{\tau }_{12}{\left(\mathit{\text{Type}}0\right)}_i+{\tau }_{22}{\left(\mathit{\text{Type}}1\right)}_i+{\epsilon }_j)\end{array}\text{ (4)}$

Equation 5 can be explained in the same manner as Equation 4, only with an addition of a person-level covariate Country Type. The parameter ${\delta }_3$ estimates the location of first threshold for Country Type: 1, which is the effect of Western/individualist cultural orientations on students’ attitudes towards collaboration compared with the Eastern/collectivist culture.

$\begin{array}{l}{w}_m\left[\log \left(\frac{P_{n\left(\mathit{\text{Strongly Disagree}}\right)}}{P_{n\left(\mathit{\text{Disagree}}\right)}}\right)or\mathrm{\hspace{0.33em}}\log \left(\frac{P_{n\left(\mathit{\text{Disagree}}\right)}}{P_{n\left(\mathit{\text{Agree}}\right)}}\right)\mathrm{\hspace{0.33em}}\mathit{\text{or}}\mathrm{\hspace{0.33em}}\log \left(\frac{P_{n\left(\mathit{\text{Agree}}\right)}}{P_{n\left(\mathit{\text{Strongly Agree}}\right)}}\right)\right]\\ =w_m({\theta }_n+{\delta }_1{\left(\mathit{\text{Type}}0\right)}_i+{\delta }_2{\left(\mathit{\text{Type}}1\right)}_i+{\delta }_3{\left(\mathit{\text{CountryType}}1\right)}_n\\ +{\tau }_{11}{\left(\mathit{\text{Type}}0\right)}_i+{\tau }_{21}{\left(\mathit{\text{Type}}1\right)}_i+{\tau }_{12}{\left(\mathit{\text{Type}}0\right)}_i+{\tau }_{22}\left(\mathit{\text{Type}}1{)}_i+{\epsilon }_j\right)\end{array}\text{ (5)}$

Equation 6 includes an interaction term Item Type: valuing teamwork * Country Type: Western to further examine whether there is interaction effect between students’ cultural orientations and their attitudes towards two sub types of collaboration.

$\begin{array}{l}{w}_m\left[\mathrm{\hspace{0.33em}}\log \left(\frac{P_{n\left(\mathit{\text{Strongly Disagree}}\right)}}{P_{n\left(\mathit{\text{Disagree}}\right)}}\right)\mathit{\text{or}}\mathrm{\hspace{0.33em}}\log \left(\frac{P_{n\left(\mathit{\text{Disagree}}\right)}}{P_{n\left(\mathit{\text{Agree}}\right)}}\right)\mathrm{\hspace{0.33em}}\mathit{\text{or}}\mathrm{\hspace{0.33em}}\log \left(\frac{P_{n\left(\mathit{\text{Agree}}\right)}}{P_{n\left(\mathit{\text{Strongly Agree}}\right)}}\right)\right]\\ =w_m({\theta }_n+{\delta }_1{\left(\mathit{\text{Type}}0\right)}_i+{\delta }_2{\left(\mathit{\text{Type}}1\right)}_i+{\delta }_3{\left(\mathit{\text{CountryType}}1\right)}_n\\ +{\delta }_4{\left(\mathit{\text{Type}}1\times \mathit{\text{CountryType}}1\right)}_n+{\tau }_{11}{\left(\mathit{\text{Type}}0\right)}_i+{\tau }_{21}{\left(\mathit{\text{Type}}1\right)}_i\\ +{\tau }_{12}{\left(\mathit{\text{Type}}0\right)}_i+{\tau }_{22}\left(\mathit{\text{Type}}1{)}_i+{\epsilon }_j\right)\end{array}\text{ (6)}$

Equations 7, 8 examine the effect of Item Type and Country ID, with and without interactions, on students’ attitudes of collaboration. In both equations, China (Country ID: 1) was set as the baseline for comparisons.

$\begin{array}{l}{w}_m\left[\log \left(\frac{P_{n\left(\mathit{\text{Strongly Disagree}}\right)}}{P_{n\left(\mathit{\text{Disagree}}\right)}}\right)\mathit{\text{or}}\mathrm{\hspace{0.33em}}\log \left(\frac{P_{n\left(\mathit{\text{Disagree}}\right)}}{P_{n\left(\mathit{\text{Agree}}\right)}}\right)\mathrm{\hspace{0.33em}}\mathit{\text{or}}\mathrm{\hspace{0.33em}}\log \left(\frac{P_{n\left(\mathit{\text{Agree}}\right)}}{P_{n\left(\mathit{\text{Strongly Agree}}\right)}}\right)\right]\\ =w_m({\theta }_n+{\delta }_1{\left(\mathit{\text{Type}}0\right)}_i+{\delta }_2{\left(\mathit{\text{Type}}1\right)}_i+{\delta }_3{\left(\mathit{\text{Country}}2\right)}_n\\ +{\delta }_4{\left(\mathit{\text{Country}}3\right)}_n+{\delta }_5{\left(\mathit{\text{Country}}4\right)}_n+{\delta }_6{\left(\mathit{\text{Country}}5\right)}_n\\ +{\tau }_{11}{\left(\mathit{\text{Type}}0\right)}_i+{\tau }_{21}{\left(\mathit{\text{Type}}1\right)}_i+{\tau }_{12}{\left(\mathit{\text{Type}}0\right)}_i+{\tau }_{22}\left(\mathit{\text{Type}}1{)}_i+{\epsilon }_j\right)\end{array}\text{ (7)}$

$\begin{array}{l}{w}_m\left[\log \left(\frac{P_{n\left(\mathit{\text{Strongly Disagree}}\right)}}{P_{n\left(\mathit{\text{Disagree}}\right)}}\right)\mathit{\text{or}}\log \left(\frac{P_{n\left(\mathit{\text{Disagree}}\right)}}{P_{n\left(\mathit{\text{Agree}}\right)}}\right)\mathit{\text{or}}\log \left(\frac{P_{n\left(\mathit{\text{Agree}}\right)}}{P_{n\left(\mathit{\text{Strongly Agree}}\right)}}\right)\right]\\ =w_m({\theta }_n+{\delta }_1{\left(\mathit{\text{Type}}0\right)}_i+{\delta }_2{\left(\mathit{\text{Type}}1\right)}_i+{\delta }_3{\left(\mathit{\text{Country}}2\right)}_n\\ +{\delta }_4{\left(\mathit{\text{Country}}3\right)}_n+{\delta }_5{\left(\mathit{\text{Country}}4\right)}_n+{\delta }_6{\left(\mathit{\text{Country}}5\right)}_n\\ +{\delta }_8{\left(\mathit{\text{Type}}1\times \mathit{\text{Country}}2\right)}_{ni}+{\delta }_9{\left(\mathit{\text{Type}}1\times \mathit{\text{Country}}3\right)}_{ni}\\ +{\delta }_{10}{\left(\mathit{\text{Type}}1\times \mathit{\text{Country}}4\right)}_{ni}+{\delta }_{11}{\left(\mathit{\text{Type}}1\times \mathit{\text{Country}}5\right)}_{ni}\\ +{\tau }_{11}{\left(\mathit{\text{Type}}0\right)}_i+{\tau }_{21}{\left(\mathit{\text{Type}}1\right)}_i+{\tau }_{12}{\left(\mathit{\text{Type}}0\right)}_i+{\tau }_{22}\left({(\mathit{\text{Type}}1)}_i+{\epsilon }_j\right)\end{array}\text{ (8)}$

Data analysis procedures

The present study utilized the eirm package (Bulut, 2021), which is an extension of the lme4 package (Bates et al., 2015) in R. Before estimating the models, polytomous item responses were transformed into a pseudo-polytomous form using the polyreformat function in the eirm package given the link function underlying the explanatory IRT models is the logit function with a binomial distribution. Table 3 demonstrates the transformation of the original polytomous to pseudo-polytomous responses.

Table 3

Table 3. Transformed survey responses for EPCM.

The five EIRMs were not completely nested within each other. Therefore, we compared the models based on the relative fit indices of Akaike Information Criterion (AIC; Akaike, 1974) and Bayesian Information Criterion (BIC; Schwarz, 1978). Both AIC and BIC are relative fit indices penalized by the number of parameters. The smaller AIC and BIC values are, the better the model fits the data.

Results

Psychometric properties of items in the collaboration scale

In PCM, the locations of the first threshold parameters “Strongly Disagree/Disagree” can be interpreted as item easiness. That is, the higher the estimate is, the easier it is to endorse the item for students. Generally, the easiness of all thresholds of the eight items are within [−3, 3], suggesting the items in the two scales are set appropriately with no invalid items. Table 4 presents the estimates of the first threshold parameters ($\mathrm{\delta }$) and the distances between the first threshold and the subsequent parameters ($\mathrm{\tau }$) for all items in the Collaboration Scale. Results show that students from the selected countries are least likely to choose to Disagree over Strongly Disagree with the statement “I prefer working as part of a team to working alone” (Item1:1.94 logits), followed by “I find that teamwork raises my own efficiency” (Item7:2.25 logits) and “I enjoy cooperating with peers” (Item8:2.34 logits) among all items based on the Strongly Disagree/Disagree threshold. In the contrast, students are most likely to choose to Disagree over Strongly Disagree with the statement “I am a good listener” (Item2:2.90 logits), followed by “I take into account what others are interested in” (Item4:2.88 logits) and “I enjoy considering different perspectives” (Item6:2.85 logits).

Table 4

Table 4. Locations of “Strongly Disagree/Disagree,” “Disagree/Agree” and to “Agree/Strongly Agree” thresholds for the partial credit model.

The distances to Disagree/Agree categories for the eight items vary from 1.06 to 2.25 logits, which shows the eight items within collaboration scales have different distances from Disagree to Agree. Item 1 (1.06 logits) and Item 7 (1.12 logits) have the smallest distances from Disagree to Agree, indicating it is more difficult for students to change their attitudes from Disagree to Agree on the statements “I prefer working as part of a team to working alone” and “I find that teamwork raises my own efficiency.” However, Item 3 and Item 8 have the larger distances between Disagree and Agree categories, indicating that it is easier to raise students’ levels of agreement on the statement of “I enjoy seeing my classmates be successful” and “I enjoy cooperating with peers”.

It is noted that the distances between Agree and Strongly Agree categories are all negative values ranging from −1.63 to −0.81. PCM does not require strict ordering of response categories and thus it allows researchers to estimate threshold parameters that may not follow the same order as the response categories. The negative threshold parameters suggest that the thresholds are not increasingly ordered as the response categories, suggesting it is harder for students to choose Strongly Agree category to Agree category compared with other thresholds. Additionally, the results from PCM demonstrate that students generally hold positive attitudes towards collaboration, it is easier for them to choose the response category Strongly Disagree over Disagree compared with the other thresholds.

Explanatory partial credit model

Table 5 summarizes the results estimated by the five explanatory models using EPCMs. As explained earlier, the explanatory models included different item-level covariates, person-level covariates, and their interactions. Relative fit indices are presented at the bottom of Table 5 to facilitate the comparison of non-nested models.

Table 5

Table 5. Summary of the five EPCM results.

Effect of item type

The results of Model 1 show that the items measuring valuing teamwork (${\delta }_{\text{Teamwork}}=2.49$) are easier to endorse than the items measuring valuing relationships (${\delta }_{\text{Relationship}}=2.39$) by students from all selected countries on choosing Disagree over Strongly Disagree. However, students are more likely to choose to Agree over Disagree on the items related to valuing relationships (${\tau }_{\text{Relationship}\_1}=-.51$) compared with valuing teamwork (${\tau }_{\text{Teamwork}\_1}=-.77$), which is in the reversed order compared with the first threshold location. For the final thresholds, the items related to valuing teamwork (${\tau }_{\text{Teamwork}\_2}=-3.56$) are easier to endorse compared with the items related to valuing relationships (${\tau }_{\text{Relationship}\_2}=-3.80$). The results show that a higher level of endorsement on collaboration is necessary to strongly agree on valuing relationships than valuing teamwork, regardless of the country effects.

Effect of item type and cultural orientations

The second model examines the effects of both item type and country type (Western/Individualist vs. Eastern/Collectivist) on the levels of endorsing collaboration items. The results concerning item type are like those of Model 1. Regarding person-level covariate, the main effect of Country Type: Western shows that there is a minor yet significant positive effect on Western countries (${\delta }_{\mathit{\text{Western}}}=0.06$, SE = 0.01, p < 0.001) compared with Eastern countries. That is, students from Western countries are $e^{0.06}$= 1.06 times more likely to endorse with collaboration than students from Eastern cultures.

Model 3 introduces an interaction term between item type and country type in addition to the main effects of these covariates. In Model 3, it is easier to agree on the items on valuing teamwork than valuing relationships, and the effect on Country Type: Western increases to ${\delta }_{\mathit{\text{Western}}}=0.15$, which suggests that students from Western countries are $e^{0.15}$= 1.16 times more likely to endorse with collaboration than students from Eastern countries.

Besides main effects, there is a negative effect on the interaction term (${\delta }_{\mathit{\text{Teamwork}}\ast \mathit{\text{Western}}}=-0.16$, SE = 0.02, p < 0.001). The results indicate that compared with Eastern cultures, students from Western cultures are less likely to endorse with valuing teamwork than valuing relationships. In both models that incorporated item-level and person-level covariates, the interactions between item type and the distances from the first threshold parameter show a similar pattern in Model 1 that it requires a higher level of endorsement on collaboration to choose to Agree over Disagree on the items related to valuing teamwork and Strongly Agree over Agree on the items related to valuing relationships.

Effect of item type and country identities

Models 4 and 5 examine item-level and person-level covariates including item type and country ID. The main effects of item type in the two models are similar to the previous models. Model 4 suggest that when setting China as the baseline of comparison, there are negative effects on all the selected countries, including Korea (${\delta }_{\mathit{\text{Korea}}}=-0.20$, SE = 0.03, p < 0.001), Japan (${\delta }_{\mathit{\text{Japan}}}=-0.43$, SE = 0.03, p < 0.001), Canada (${\delta }_{\mathit{\text{Canada}}}=-0.24$, SE = 0.02, p < 0.001), and the US (${\delta }_{US}=-0.19$, SE = 0.03, p < 0.001).

Furthermore, among the countries, students from Japan hold the most negative attitudes towards collaboration, followed by Canada, Korea, and the United States. Therefore, we concluded that students from China are most likely to value collaboration among all the selected countries.

Model 5 provides more information on how students from different countries differ between valuing relationships and valuing teamwork. Results from Model 5 show that compared with China, there are negative effects on the United States (${\delta }_{US}=-0.03$, SE = 0.03, p < 0.001), Canada (${\delta }_{\mathit{\text{Canada}}}=-0.05,$ SE = 0.02, p < 0.001), Korea (${\delta }_{\mathit{\text{Korea}}}=-0.12$, SE = 0.03, p < 0.001), and Japan (${\delta }_{\mathit{\text{Japan}}}=-0.30$, SE = 0.03, p < 0.001) with increasing order. Also, results suggest that compared with China, Korea (${\delta }_{\mathit{\text{Teamwork}}\ast \mathit{\text{Korea}}}=-0.16$, SE = 0.04, p < 0.001), Japan (${\delta }_{\mathit{\text{Teamwork}}\ast \mathit{\text{Japan}}}=-0.09$, SE = 0.04, p < 0.001), Canada (${\delta }_{\mathit{\text{Teamwork}}\ast \mathit{\text{Canada}}}=-0.29$, SE = 0.03, p < 0.001) as well as the United States (${\delta }_{\mathit{\text{Teamwork}}\ast US}=-0.25$, SE = 0.04, p < 0.001) are less likely to value teamwork than relationships.

Discussion and conclusion

The present study used the EIRM approach to examine the impact of cultural orientations on students’ attitudes towards collaboration. There are several trends in the findings. First, students from all countries generally endorsed valuing teamwork more than valuing relationships. Second, students from Western countries with individualist cultures are more likely to value collaboration than Eastern countries with collectivist cultures. Third, China has the highest endorsement of collaboration, followed by the United States, Canada, Korea, and Japan. However, based on the CPS assessment scores, Japan has the highest scores, followed by Korea, Canada, the United States, and China, which is in the exactly reversed order of their endorsement with collaboration. Therefore, we concluded that the two hypotheses of the present study are rejected that (1) students from collectivist cultures are not ensured to have a higher endorsement of collaboration compared with students from individualist cultures and (2) higher endorsement of collaboration does not necessarily lead to higher achievement in the CPS assessment.

Findings from the present study were compared with previous studies on attitudinal differences in collaboration among various cultural orientations. In Hofstede’s longitudinal survey on 76 countries’ inclinations to collaboration, they proposed a novel paradigm to model their positions on the collaboration dimension of the societies (Hofstede, 2011). The results reveal that the pattern differences between Western and Eastern countries are polarized. Generally, individualism tends to prevail in developed and Western countries, with the United States having the highest IDV score. On the other hand, collectivism prevails in less developed and Eastern countries, whereas Japan takes a middle position on this dimension. Hofstede et al. attributed this distribution of countries’ values mainly to their wealth.

PISA 2015 provides a new window to observe middle school students’ attitudes towards collaborative problem-solving in two folds—valuing teamwork and valuing relationships, which reveals two dimensions of collaboration instead of regarding it as a single factor. In addition to attitudes, PISA 2015 also included a collaborative problem-solving assessment to measure students’ performance on CPS tasks based on some real-life scenarios. Findings from the current study revealed that students from Eastern countries did not necessarily have a higher endorsement of collaboration as indicated in the previous cross-national surveys. Instead, only China demonstrated a strong representation of collectivist culture and showed the highest levels of endorsement with collaboration. In contrast, Japan and Korea showed the least agreement on items related to collaboration among the five selected countries.

An unexpected finding of the present study revealed that valuing collaboration did not lead to better success on CPS performance in PISA 2015. Countries with lower endorsement of collaboration demonstrated better performance on the CPS assessment. Previous studies on the relationship between attitude and achievement held different positions. Some studies show that attitude is a positive predictor of achievement (e.g., Lupo et al., 2017; Graham et al., 2007), while others claim attitude does not ensure greater achievement (e.g., Martin and Rainey, 1993). Findings from the present study indicate the gap between students’ attitudes shaped by cultural background and achievement based on schools’ CPS training for students.

Practical implications

Collaborative problem solving requires cognitive skills and non-cognitive social skills, where attitudes towards collaboration shaped by cultural orientations play a significant role. The present study shed light on understanding the relationship between psychological attributes (i.e., endorsement with collaboration) and performance on collaborative problem solving. Findings revealed that attitudes towards collaboration is not determined by national culture. Further, CPS is not simply determined by psychological characteristics like attitudes. Future research is necessary to understand and address the pedagogical and instructional gaps in CPS training among different countries, after accounting for the effects of confounding factors such as cultural orientations.

Limitation

Some limitations were identified. First, the present study selected limited countries to represent the Western and Eastern cultures. Future research could be conducted among more countries and regions for global comparisons. Second, the sample selection and data analyses in the present study used country as the minimum unit, whereas overlooking the internal cultural variation within large countries like China, the US, and Canada. Further inspections could be conducted to investigate the differences of attitudes towards collaborations within countries.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

CL: Writing – review & editing, Writing – original draft, Visualization, Validation, Supervision, Software, Resources, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Data curation, Conceptualization. OB: Writing – review & editing, Validation, Supervision, Software, Resources, Methodology, Investigation, Formal analysis.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work was supported by the MOE (Ministry of Education in China) Liberal Arts and Social Sciences Foundation (Grant Number: 24YJC880093), Shanghai Pujiang Program (Grant Number: 22PJC058), Shanghai Jiao Tong University Social Science Youth Talent Cultivation Project (Grant Number: 2023QN010), and Shanghai Jiao Tong University New Faculty Startup Project (Grant Number: 22X010503470).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Akaike, H. (1974). Stochastic theory of minimal realization. IEEE Trans. Autom. Control 19, 667–674. doi: 10.1109/TAC.1974.1100707

Crossref Full Text | Google Scholar

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika 43, 561–573. doi: 10.1007/BF02293814

Crossref Full Text | Google Scholar

Arieli, S., and Sagiv, L. (2018). Culture and problem-solving: congruency between the cultural mindset of individualism versus collectivism and problem type. J. Exp. Psychol. Gen. 147, 789–814. doi: 10.1037/xge0000444

PubMed Abstract | Crossref Full Text | Google Scholar

Arpaci, I., Baloğlu, M., and Kesici, Ş. (2018). The relationship among individual differences in individualism-collectivism, extraversion, and self-presentation. Personal. Individ. Differ. 121, 89–92. doi: 10.1016/j.paid.2017.09.034

Crossref Full Text | Google Scholar

Bates, D., Maechler, M., Bolker, B., and Walker, S. (2015). Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48. doi: 10.18637/jss.v067.i01

Crossref Full Text | Google Scholar

Bollen, A., Harrison, R., Aslani, P., and van Haastregt, J. C. (2019). Factors influencing interprofessional collaboration between community pharmacists and general practitioners—a systematic review. Health Soc. Care Community 27, e189–e212. doi: 10.1111/hsc.12705

PubMed Abstract | Crossref Full Text | Google Scholar

Bulut, O. (2021). eirm: Explanatory item response modeling for dichotomous and polytomous item responses, R package version 0.3.0. doi: 10.5281/zenodo.455628

Crossref Full Text | Google Scholar

Care, E., Griffin, P., Scoular, C., Awwal, N., and Zoanetti, N. (2015). “Collaborative problem solving tasks” in Assessment and teaching of 21st century skills: methods and approach (Dordrecht: Springer Netherlands), 85–104.

Google Scholar

Chang, C. J., Chang, M. H., Chiu, B. C., Liu, C. C., Chiang, S. H. F., Wen, C. T., et al. (2017). An analysis of student collaborative problem solving activities mediated by collaborative simulations. Comput. Educ. 114, 222–235. doi: 10.1016/j.compedu.2017.07.008

Crossref Full Text | Google Scholar

Chen, X. (2000). “Growing up in a collectivist culture: socialization and socioemotional development in Chinese children” in International perspectives on human development. eds. A. L. Comunian and U. P. Gielen (Lengerich, Germany: Pabst Science Publishers), 331–353.

Google Scholar

Cradock-Henry, N. A., Greenhalgh, S., Brown, P., and Sinner, J. (2017). Factors influencing successful collaboration for freshwater management in Aotearoa, New Zealand. Ecol. Soc. 22:14. doi: 10.5751/ES-09126-220214

Crossref Full Text | Google Scholar

De Boeck, P., and Wilson, M. (eds.) (2004). “A framework for item response models” in Explanatory item response models: a generalized linear and nonlinear approach (New York, NY: Springer), 3–41.

Google Scholar

Dehler, J., Bodemer, D., Buder, J., and Hesse, F. W. (2011). Guiding knowledge communication in CSCL via group knowledge awareness. Comput. Hum. Behav. 27, 1068–1078. doi: 10.1016/j.chb.2010.05.018

Crossref Full Text | Google Scholar

French, B. F., and Finch, W. H. (2010). Hierarchical logistic regression: accounting for multilevel data in DIF detection. J. Educ. Meas. 47, 299–317. doi: 10.1111/j.1745-3984.2010.00115.x

Crossref Full Text | Google Scholar

Graham, S., Berninger, V., and Fan, W. (2007). The structural relationship between writing attitude and writing achievement in first and third grade students. Contemp. Educ. Psychol. 32, 516–536. doi: 10.1016/j.cedpsych.2007.01.002

Crossref Full Text | Google Scholar

Griffin, P., and Care, E. (2014). “The ATC21S method” in Assessment and teaching of 21st century skills: Methods and approach (Dordrecht: Springer Netherlands), 3–33.

Google Scholar

Griffin, P., Care, E., Bui, M., and Zoanetti, N. (2013). “Development of the assessment design and delivery of collaborative problem solving in the assessment and teaching of 21st century skills project” in Epedagogy in online learning: New developments in web mediated human computer interaction (IGI Global), 55–73.

Google Scholar

Hao, J., Liu, L., von Davier, A. A., and Kyllonen, P. C. (2017). “Initial steps towards a standardized assessment for collaborative problem solving (CPS): practical challenges and strategies” in Innovative assessment of collaboration. eds. A. von Davier, M. Zhu, and P. Kyllonen (Dordrecht: Springer Netherlands),Cham: Springer), 135–156.

Google Scholar

Hartig, J., Frey, A., Nold, G., and Klieme, E. (2012). An application of explanatory item response modeling for model-based proficiency scaling. Educ. Psychol. Meas. 72, 665–686. doi: 10.1177/0013164411430707

Crossref Full Text | Google Scholar

Hesse, F., Care, E., Buder, J., Sassenberg, K., and Griffin, P. (2015). “A framework for teachable collaborative problem solving skills” in Assessment and teaching of 21st century skills: methods and approach. eds. P. Griffin and E. Care (Dordrecht: Springer Netherlands), 37–56.

Google Scholar

Hilse, J., Holt, D., Fischer, A., and Funke, J. (2014). Assessing collaborative problem solving. In Proceedings of the Annual Meeting of the Cognitive Science Society. (Vol. 36).

Google Scholar

Hofstede, G. (2011). Dimensionalizing cultures: the Hofstede model in context. Online Read. Psychol. Cult. 2:8. doi: 10.9707/2307-0919.1014

Crossref Full Text | Google Scholar

Kirschner, P. A., Buckingham-Shum, S. J., and Carr, C. S. (Eds.) (2012). Visualizing argumentation: Software tools for collaborative and educational sense-making. Heidelberg: Springer Science & Business Media.

Google Scholar

Klimoski, R., and Mohammed, S. (1994). Team mental model: construct or metaphor? J. Manag. 20, 403–437. doi: 10.1177/014920639402000206

Crossref Full Text | Google Scholar

Lupo, S., Jang, B. G., and McKenna, M. (2017). The relationship between Reading achievement and attitudes toward print and digital texts in adolescent readers. Lit. Res. 66, 264–278. doi: 10.1177/2381336917719254

Crossref Full Text | Google Scholar

Martin, E. D., and Rainey, L. (1993). Student achievement and attitude in a satellite-delivered high school science course. Am. J. Dist. Educ. 7, 54–61. doi: 10.1080/08923649309526810

Crossref Full Text | Google Scholar

Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika 47, 149–174. doi: 10.1007/BF02296272

Crossref Full Text | Google Scholar

Mattessich, P. W., and Monsey, B. R. (1992). Collaboration: what makes it work. A review of research literature on factors influencing successful collaboration. 919 Lafond, St. Paul, MN 55104: Amherst H. Wilder Foundation.

Google Scholar

OECD (2013). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. Paris: OECD Publishing Available at: http://dx.doi.org/10.1787/9789264190511-en.

Google Scholar

OECD (2017). PISA 2015 assessment and analytical framework: Science, reading, mathematic, financial literacy and collaborative problem solving. Rev. Edn. Paris: OECD Publishing Available at: http://dx.doi.org/10.1787/9789264281820-en.

Google Scholar

Oyserman, D., Coon, H. M., and Kemmelmeier, M. (2002). Rethinking individualism and collectivism: evaluation of theoretical assumptions and meta-analyses. Psychol. Bull. 128, 3–72. doi: 10.1037/0033-2909.128.1.3

PubMed Abstract | Crossref Full Text | Google Scholar

Patel, H., Pettitt, M., and Wilson, J. R. (2012). Factors of collaborative working: a framework for a collaboration model. Appl. Ergon. 43, 1–26. doi: 10.1016/j.apergo.2011.04.009

Crossref Full Text | Google Scholar

Randall, J., Cheong, Y. F., and Engelhard, G. Jr. (2011). Using explanatory item response theory modeling to investigate context effects of differential item functioning for students with disabilities. Educ. Psychol. Meas. 71, 129–147. doi: 10.1177/0013164410391577

Crossref Full Text | Google Scholar

Roschelle, J., and Teasley, S. D. (1995). “The construction of shared knowledge in collaborative problem solving” in Computer supported collaborative learning. ed. C. O’Malley. (Berlin, Heidelberg: Springer), 69–97.

Google Scholar

Rosen, Y., and Rimor, R. (2016). “Teaching and assessing problem solving in online collaborative environment” in Professional development and workplace learning: Concepts, methodologies, tools, and applications (Hershey, Pennsylvania: IGI Global), 254–269.

Google Scholar

Schwarz, G. (1978). Estimating the dimension of a model. Ann. Stat. 6, 461–464. doi: 10.1214/aos/1176344136

Crossref Full Text | Google Scholar

Stanke, L., and Bulut, O. (2019). Explanatory item response models for polytomous item responses. Int. J. Assess. Tools E. 6, 259–278. doi: 10.21449/ijate.515085

Crossref Full Text | Google Scholar

Stefik, M., Foster, G., Bobrow, D. G., Kahn, K., Lanning, S., and Suchman, L. (1987). Beyond the chalkboard: computer support for collaboration and problem solving in meetings. Commun. ACM 30, 32–47. doi: 10.1145/7885.7887

Crossref Full Text | Google Scholar

Sycara, K., Gelfand, M., and Abbe, A. (Eds.) (2013). Models for intercultural collaboration and negotiation. New York, NY, USA: Springer.

Google Scholar

Tuerlinckx, F., and Wang, W. C. (2004). “Models for polytomous data” in Explanatory item response models: A generalized linear and nonlinear approach. eds. P. De Boeck and M. Wilson (New York, NY: Springer New York), 75–109.

Google Scholar

von Davier, A. A., and Halpin, P. F. (2013). Collaborative problem solving and the assessment of cognitive skills: psychometric considerations. ETS Res. Rep. Ser. 2013:i-36. doi: 10.1002/j.2333-8504.2013.tb02348.x

Crossref Full Text | Google Scholar

Zhang, Y., and Yin, H. (2019). Collaborative cheating among Chinese college students: the effects of peer influence and individualism-collectivism orientations. Assess. Eval. High. Educ. 45, 54–69. doi: 10.1080/02602938.2019.1608504

Crossref Full Text | Google Scholar