Validating the Inventory of School Motivation with Mainland Chinese Students

Li, Feifei; Kauffman, Yashu; Thomas, Michael; Dempsey, Michael S.; Wang, Shiyuan; Kauffman, Douglas F.

doi:10.3389/feduc.2017.00025

ORIGINAL RESEARCH article

Front. Educ., 20 June 2017

Sec. Educational Psychology

Volume 2 - 2017 | https://doi.org/10.3389/feduc.2017.00025

This article is part of the Research Topic Self-Regulation in Education and Health View all 23 articles

Validating the Inventory of School Motivation with Mainland Chinese Students

$\r\nFeifei Li*$ Feifei Li¹*

Yashu Kauffman²

Michael Thomas³

Michael S. Dempsey⁴

Shiyuan Wang⁵ $Douglas F. Kauffman\r\n$ Douglas F. Kauffman⁴

¹Educational Testing Service, Princeton, NJ, United States
²Massachusetts Institute of Technology, Cambridge, MA, United States
³University of Illinois-Chicago, Chicago, IL, United States
⁴Boston University School of Medicine, Boston Medical Center, Boston, MA, United States
⁵Rowan University School of Osteopathic Medicine, Stratford, NJ, United States

The purpose of this study was to validate the Inventory of School Motivation (ISM) (Mainland China), which was developed specifically for use with Mainland Chinese students. Development of the instrument was grounded in personal investment theory and built on the ISM instrument, which has been shown to accurately tap the achievement goal constructs hypothesized by the theory. Our data analysis indicated that the subscales in our instrument do represent dimensions associated with achievement goals: task, effort, competition, social power, affiliation, social concern, praise, and token. Participants were 458 undergraduates from five universities in eastern China. A series of nested confirmatory factor analyses supported a multidimensional school motivation structure. The results indicated both convergent and concurrent validity for the instrument with Mainland students. Relationships were found between ISM goal constructs and global motivation goals, family-oriented goals, self-concepts, and learning approaches. Our findings support that Mainland Chinese students’ achievement goal orientation is consistent with that found in other cultures, suggesting that the instrument and the theory that informs it may further cross-cultural research in this area. At the same time, though, our findings show that Mainland students endorse some goal orientations differently from other groups, suggesting an inventory instrument specific to Mainland China is both important and necessary.

Introduction

There is growing recognition that culture influences achievement motivation. Guay (2016) argues that learners’ motivation “is not based solely on such intrapersonal factors as innate characteristics, but also on contexts (including cultural ones) in which students are supposed to develop their competencies. Thus, the cultural context is expected to shape motivation (i.e., cultural specificity)” (p. 157). Western concepts of achievement motivation, however, have been applied to diverse groups without reference to differences in culture, an approach that can mischaracterize the true nature of achievement motivation in different groups (Maehr and McInerney, 2004). Empirical evidence is building to show that findings drawn from culturally neutral studies may not accurately describe the true nature of achievement motivation in diverse groups (e.g., Maehr and Nicholls, 1980; McInerney et al., 1997; Bempechat and Drago-Severson, 1999; Salili and Hoosain, 2002; Guay, 2016; Lam et al., 2016). Researchers have shown specific interest studying motivation in Asian cultures. Studies in this area report that some Asian groups differ from their Western counterparts in the way they construe self, others, and the interdependence of the two (Markus and Kitayama, 1991; Li, 2002; Chang et al., 2003; Ho and Hau, 2008; Cheng and Lam, 2013). Previous research has characterized these differences as being collectivist and individualistic respectively, though such labels can overgeneralize (e.g., Markus and Kitayama, 1991). A second reason for this interest is the continued concern that students in some Western societies do not perform at the same high levels as those in Asian cultures (Li, 2002; Chan and Lam, 2008; Ho and Hau, 2008; Qu and Pomerantz, 2015; Kember, 2016).

Several achievement studies have been conducted with non-Mainland Chinese students (e.g., Chang and Wong, 2008; Ho and Hau, 2008; King et al., 2012; Cheng and Lam, 2013). Evidence suggests that Mainland and non-Mainland Chinese differ culturally. Li and Bray (2007) pointed out two cultural poles in their findings, fully domestic and fully international, with Mainland Chinese students falling into the former category and Hong Kong and Macau students occupying a position somewhere in between. Hong Kong is characterized by a blended social system of the East and the West due to its history as a British colony. In the meantime, Mainland China has gone through the establishment of a communist government, the Culture Revolution, and “Open Door” policy, which also have great impact in the culture of Mainland China. Zeng (2016) recognizes the cultural history shared by Mainland China and Hong Kong, which includes Confucian philosophy, a collectivist social orientation, and endorsement of authority, but Zeng finds evidence of significant differences as well, for example, Mainland China is a more traditional Confucian society than Hong Kong in terms of the degree of equality between people in a society and in terms of the extent to which a society reinforces the traditional masculine work. Using Hofstede’s cultural dimensions framework, Zeng concludes that China and Hong Kong may be better described as “sibling” cultures, which means cultures originated from a same culture, but developed under different systems, and thus shares similarities but has differences. We argue for assessing achievement motivation in Mainland China separately from Chinese groups acculturated outside the Mainland.

The present study, which is grounded in personal investment (PI) theory, represents a step toward better understanding the way Mainland Chinese students construe achievement motivation. Social-cognitive in nature, PI emphasizes the role of context in meaning construction across cultures and reflects the multidimensional nature of achievement goals (McInerney et al., 2005; King and McInerney, 2014; Da Silva, 2016). Assessment tools have been developed specifically to represent constructs associated with PI, notably the Inventory of School Motivation (ISM). ISM has been tested across widely diverse groups, yielding evidence for its reliability and validity in Western and non-Western contexts (McInerney and Liem, 2009; King and McInerney, 2014; McInerney, 2016). It has been validated primarily outside Mainland China, however [see, for example, Watkins et al. (2002), McInerney and Ali (2006), and King and Watkins (2013)]. Yeung et al. (2016) represent an exception. In their study, students from southern China were used in their investigation of reciprocal relationships between effort and achievement. To assess effort, they used four items from the ISM, but without first validating the instrument in a Mainland context. We argue that construct validity of the ISM needs to be evaluated with Mainland Chinese participants before accepting its appropriateness for that group.

Three research questions guided the present study:

1. What is the latent structure underlying Chinese students’ academic motivations?

2. Is there convergent validity for the ISM related to the Chinese context?

3. Is there concurrent validity for the ISM related to the Chinese context?

The Asian Context

Asian students consistently outperform their Western peers on academic measures. The Programme for International Student Assessment (2016), for example, reported that Asian countries held the five top rankings in math literacy, four of the five top rankings in science literacy, and the five top rankings in reading literacy. Research on this achievement gap has converged on cultural differences related to sense-of-self. Religious and philosophical traditions in Asian societies, it is argued, have produced a collectivist culture that strongly influences academic motivation (Chen et al., 1995). In these societies, self is meaningful to the degree that it is construed in appropriate social relationships with significant others (Markus and Kitayama, 1991; Lee, 1996; Hau and Ho, 2008; Ho and Hau, 2008; Zeng, 2016). Academic achievement in collectivist societies is perceived as a social endeavor that is a path to social approval and status (Lin and Fau, 1990; Chen et al., 1995). As a result, individuals feel a strong sense of obligation for academic proficiency (Salili and Lau, 2003; Ho and Hau, 2008; Tao and Hong, 2014), an obligation that has been implicated in the distinctly competitive nature of Asian school systems (Watkins and Biggs, 1996; Ho and Hau, 2008). A unique characteristic of Asian learners is that they adopt both mastery and performance goals simultaneously, a finding that has been termed the “paradox of the Chinese learner.” Empirical evidence supports the view that cultural context (i.e., collectivism) accounts for achievement motivation observed in Asian societies (King and McInerney, 2012; King et al., 2012, 2013; King and Watkins, 2013).

PI Theory

Personal investment is a social-cognitive theory concerned with how individuals choose to allocate their effort, ability, and time for different activities (Maehr and McInerney, 2004; Braskamp, 2009; King and McInerney, 2014; McInerney, 2016). The theory is an important foundation of ISM and has been proven valuable in cross-cultural settings because it is sensitive to both universal and relative goal behaviors. Even though research based on PI theory indicates that diverse groups of learners endorse similar educational goals and values, motivational patterns that represent culturally specific motivational dimensions still vary both within and across groups. PI emphasizes the subjective meaning that persons attach to achievement situations, which reflect their culturally determined belief systems. Maehr and McInerney (2004) identify three components of meaning that make up culturally bound belief systems: (1) perceived goals, (2) sense-of-self, and (3) facilitating conditions. These components are themselves shaped by situation specific tasks at hand, relevant personal experience and knowledge, and the sociocultural context that frames them all.

Perceived goals, also referred to as personal incentives, represent the motivational focus of activity. Personal goals are influenced by the value individuals invest in activities as well as cultural definitions of failure and success relevant to activities (Maehr and McInerney, 2004; McInerney et al., 2005; McInerney, 2016). Four broad goal categories have been proposed: task (intrinsic or mastery), ego (social competence or performance), social solidarity (interpersonal relationships), and extrinsic incentives (Maehr and McInerney, 2004; McInerney et al., 2005; King et al., 2013; McInerney, 2016). The relationship between goals varies across cultures (Maehr and McInerney, 2004; Braskamp, 2009; Li, 2016; Liem, 2016). In a society where learning and employment opportunities are highly competitive, for example, goals are not necessarily mutually exclusive, as evidenced by how individuals in Asian societies value and internalize both ego and task goals (King and Watkins, 2013; King et al., 2013; Li, 2016; McInerney, 2016).

Sense-of-self refers to individuals’ perceptions, beliefs, and feelings related to personal identity, which are influenced by one’s culture (McInerney et al., 1997; Maehr and McInerney, 2004; King et al., 2012, 2013). Sense-of-self is presumed to comprise multiple factors, including sense of purpose related to a task that will be undertaken in the future (e.g., future school-related tasks), self-reliance (e.g., self-regulation in school settings), and self-esteem (e.g., affective judgments about general academic ability). These factors interact with the goals detailed above to shape motivational orientation.

Environmental factors can inhibit or facilitate individuals’ internal motivation to act. In an academic setting, these facilitating conditions include affect for school, perceived value of school, and the support and caring that significant others provide for academic success (McInerney et al., 2005; King et al., 2012, 2013; McInerney, 2016). Students’ like or dislike for school may be subject-specific or it may manifest more globally. Affect, along with the perception of school’s pertinence to success in life, represent powerful facilitating conditions. Students’ perceptions of support and caring from significant others—parents, peers, and teachers—play an influential role in determining their achievement motivation.

Inventory of School Motivation

McInerney et al. (1997) developed the ISM to explore if students from diverse cultural backgrounds endorse goals differentially and to examine the relationship of goals to academic motivation and achievement. Development of the instrument was grounded in PI theory and was designed explicitly to assess the four goal constructs associated with the theory: mastery, ego, social, and extrinsic. Findings from these studies have established achievement motivation as multidimensional, forming eight sub-categories: mastery (task, effort), performance (competition, social power), social (affiliation, social concern), and extrinsic (praise, token rewards). Task and effort relate to task interest and degree of willingness to work hard in school. Competition and social power refer to individual goals to outperform others and assume leadership roles. Affiliation and social concern are associated with individuals’ preference for cooperation with other students and seeking group success with support and caring. Praise and token rewards relate to individuals’ goals to seek social recognition and tangible rewards (McInerney, 2008, 2016; McInerney and Liem, 2009; King et al., 2012; King and McInerney, 2014).

Validating ISM

McInerney et al. (2003) developed the General Achievement Goal Orientation Scale (GAGOS) to test the convergent validity of ISM. Results revealed a hierarchical structure underlying ISM and GAGOS. Task and effort sub-goals loaded on mastery orientation, while the remaining sub-goals loaded on performance orientation. Ali and McInerney (2005) replicated these findings with Anglo-Australian, immigrant, Aboriginal Australian, Native Americans, and Anglo-Americans. Results revealed slight positive correlations among each set of sub-goals, providing evidence for the multidimensional structure of ISM. King et al. (2012) compared Hong Kong Chinese with Filipino students using ISM and found positive correlations between the sub-goals and a higher-order structure they formed using mastery, ego, social, and extrinsic goals. The researchers also investigated the relationship between perceived goals and sense-of-self using items related to academic study. They found that positive sense-of-self was positively related to the higher-order structure of goals and negative sense-of-self was negatively associated with this structure. Performance and extrinsic goals, however, did not correlate with negative sense-of-self for the Hong Kong students. The latter finding suggests that the Hong Kong Chinese students did not perceive goal orientations as being mutually exclusive.

Exploratory factor analyses have yielded structures that are congruent with the framework hypothesized for ISM (McInerney and Sinclair, 1991). The multidimensional (McInerney et al., 1997) and hierarchical (McInerney and Ali, 2006) structure of the instrument was further tested through confirmatory factor analyses (CFAs), and the results demonstrated the applicability of the model in cross-cultural settings. The derived ISM goal factors, moreover, were shown to predict important academic outcomes across the different cultural groups, such as school confidence, perceived value of school, feelings toward school, desired occupation, school completion, school achievement, and absenteeism (McInerney and Sinclair, 1991; McInerney, 1994a,b, 1995; McInerney and Swisher, 1995). McInerney et al. (1997) and Ali and McInerney (2005) concluded their findings indicating that the multidimensional first-order (FO) structure was invariant across cultural groups, but the fit indices for the two studies did not represent strong support for their position. We conjecture that that item loadings and factor correlations differed as a result of the cultural uniqueness of the groups studied (Anglo-Australian, immigrant, Aboriginal Australian, Native Americans, and Anglo-Americans).

Based on the previous literatures, we hypothesize that, like other students in Asian cultures, and also as a result of education and employment resources that are highly competitive, Mainland Chinese students do not perceive goal orientations as mutually exclusive; both performance and mastery goals are internalized. In addition, student’s perceptions of support and caring from significant others, including family, peers, and teachers, play a crucial role in determining their achievement motivation patterns.

Materials and Methods

Participants

Participants were 458 undergraduates from five universities in eastern China. The sample consisted of 196 males (42.7%) and 262 females (51.7%) ranging in age from 18 to 29 years old (Mean = 20.3, SD = 1.3). Across the full sample, 76.6% of the students were freshmen, and 23.4% were in their sophomore year or later. Students were recruited from a national top university (36.2%), from a medical school (9.6%), and from three state-funded universities (54.2%). Participants were enrolled in several programs: arts and sciences (40.8%), engineering (20%), business and management (15.1%), medicine (9.6%), education (7.9%), and law (5.9%). Most of the students were originally from East China (77.9%). A majority of students had a father with high school or higher education (63.6%), and approximately half had a mother with high school or higher education (54.2%). Over half of the students were the only children in their family (57.7%).

Measures

Five instruments were used in the current validation study: ISM, the Family-Oriented Motivation Scale, GAGOS, sense-of-self scale, and the learning process questionnaire (LPQ) (Biggs, 1987). Participants responded to all items using a 5-point Likert-type scale (1 = Strong Disagree, 2 = Disagree, 3 = Neither Agree nor Disagree, 4 = Agree, 5 = Strongly Agree).

Inventory of School Motivation

The most recent version of the ISM comprises 43 items (King et al., 2012), from which we selected 36 items that most closely match the motivational goals found in the original ISM (McInerney et al., 1997). These 36 items were validated using cross-cultural samples found in other CFA models (McInerney, unpublished)¹. Two of the 36 items were discarded to improve reliability; “I like being given the chance to do something again to make it better.” and “It makes me unhappy if my friends aren’t doing well at school.” Cronbach’s alpha was 0.88, overall, demonstrating reliability in general, and alphas for the subscales ranged from 0.58 to 0.79 (Table 1).

TABLE 1

Table 1. First-order factor solution for Inventory of School Motivation factor loadings (M2) and Cronbach’s Alpha for each subscale (N = 458).

Family-Oriented Motivation Scale

Watkins et al. (2002) suggested that future research using ISM with Chinese participants should include a subscale that is more family-oriented. The scale was proposed as a way of assessing motivation patterns related to effort and social status, which are particularly valued in Chinese culture (Hau and Ho, 2008; Ho and Hau, 2008; Zeng, 2016). We included a three-item subscale that assesses one’s motivation to please family from the “Attitude Toward Mathematics Survey” (Miller et al., 1996). Cronbach’s alpha coefficient was 0.75 in the current sample, indicating sufficient reliability.

General Achievement Goal Orientation Scale

General Achievement Goal Orientation Scale was developed to measure global orientations to motivation. This scale comprises 15 items related to global motivation (i.e., a perceived motivation for school) and three general motivational constructs (i.e., mastery, performance, and social motivation). McInerney (2003) conducted a confirmatory factor analysis that offered strong evidence for its construct validity. In this study, GAGOS was also used to assess the relationship of global goal orientations to the eight motivation factors of ISM. With respect to the current Chinese student sample, one item in performance general subscale was taken out due to its attrition to the reliability. Cronbach’s alphas ranged from 0.66 to 0.86, and an overall alpha of 0.85 suggested good reliability of GAGOS. CFA on the current Chinese students sample resulted in fit statistics that indicated good model fit [normed fit index (NFI) = 0.96, non-normed fit index (NNFI) = 1.01, CFI = 1.00, RMSEA = 0.00], as was shown in Table 2. Refer to McInerney’s study (McInerney, 2003) for the CFA model.

TABLE 2

Table 2. Fit indices of General Achievement Goal Orientation Scale (GAGOS), Sense-of-self Scales, Learning Process Questionnaire (LPQ), and Family-Oriented Motivation Scale (N = 458).

Sense-of-Self Scale

This scale assesses individuals’ self-concepts, including sense of purpose (i.e., valuing schools for the future), self-reliance (i.e., self-regulation within academic settings), and self-esteem (i.e., negative and positive feelings about general academic ability at school). The scale included 24 items, which have been validated (McInerney and Ali, 2006). One item was deleted to improve reliability. Cronbach’s alphas ranging from 0.60 to 0.73 suggested sufficient reliability in the current sample. A CFA model with three factors (i.e., sense of purpose, self-reliance, and self-concept) was fitted to the data. The fit indices supported its construct validity (NFI = 0.89, NNFI = 0.97, CFI = 0.97, RMSEA = 0.026) (Table 2).

Learning Process Questionnaire

Learning Process Questionnaire measures students’ approach to learning and was developed based on the tertiary framework of deep, surface, and achieving strategies (Biggs, 1987). LPQ consists of 18 items with 6 items in each factor. LPQ has been validated across diverse cultural groups (Biggs et al., 2001). Learners engaging in a deep motive strategy are intrinsically interested and want to maximize meaning to earn high academic scores. Surface learners, in contrast, use a motive strategy that exhibit lower effort such as rote learning to prevent failure. Learners who use achieving motives focus their efforts on achievement and also make effective use of space and time. Reliability analysis suggested that the surface strategy items were not reliable, so items related to surface strategy were removed from the analysis. One item of the deep motive items was also removed. Subscales of deep learning with five items and achievement learning approaches with six items were reliable, with Cronbach’s alphas at 0.73 and 0.70, respectively, and its latent structure with two factors (deep and achieving strategies) was also supported (NFI = 0.90, NNFI = 0.94, CFI = 0.95, RMSEA = 0.041) (Table 2).

Procedure

Scale Development

Versions of ISM developed for Hong Kong Chinese (Watkins et al., 2002) were used to develop the current scale using simplified Mandarin Chinese that would be appropriate for Mainland China. We invited five Chinese doctoral students studying in the United States and three faculty members teaching English in China to do translation and back translation between English and simplified Mandarin Chinese. The authors compared the original and the back-translated English versions. Discrepancies were discussed between the authors and the translators until agreement was reached on the final Chinese version.

Instrument Administration

A package that contains the demographic survey, ISM, LPQ, GAGOS, sense-of-self scale, and Family-Oriented Motivation Scale was administered to participants in their classrooms by the first author, with the assistance of teaching staff at each school. To standardize the delivery, assisting teachers received a copy of the instrument, along with written instructions. The assisting teachers were also briefed by the authors on the structure, purpose, and administration of the survey prior to its administration with students. Each administration took approximately 25 min.

Analyses

CFAs of the Motivational Goals

Confirmatory Factor Analyses assess the extent to which the observed indicators (items) reflect the structure of the underlying constructs. As was hypothesized and validated by McInerney and his colleagues, underlying the ISM were eight FO-specific factors, and this structure was invariant across different cultural groups (McInerney et al., 1997; Dowson and McInerney, 2004; McInerney and Ali, 2006). Results from some validation studies also indicate that the eight FO factors can define the general second-order (SO) factors (Dowson and McInerney, 2004; McInerney and Ali, 2006).

To determine the construct validity of ISM in the Mainland Chinese sample, the CFA models were tested and compared. The CFAs were performed with EQS (Bentler, 2004), using maximum likelihood estimation for the categorical data with the raw data matrix as input. First, we evaluated whether the 34 items could be explained by the eight FO factor model (i.e., task, effort, competition, social power, affiliation, social concern, praise, and token) and by the a priori model of the ISM design (M2), which is highly restricted and in which the measurement error terms associated with indicators are uncorrelated (Table 1). Second, we examined whether these eight FO factors could be explained by four SO factors (Table 3) (M3). Third, to improve the model, we fixed the values of the parameters whose estimates appeared to be non-significant in the previous model and made some respecifications suggested by modification indices or Lagrange multiplier statistics provided in EQS (M4), given these specifications are theoretically sound and do not contradict the results of previous research in the particular substantive domain (Biggs, 1992; Raykov and Marcoulides, 2006). To assess the relative and absolute goodness-of fit of the nested models, we used the unidimensional model as the null model, in which all 34 items reflected one global motivation factor (M1). Fit values for all four models were estimated and are presented in Table 4.

TABLE 3

Table 3. First-order and second-order factors of motivation goals in Inventory of School Motivation (M3).

TABLE 4

Table 4. Summary of models and goodness-of-fit statistics (N = 458).

In this study, we used confirmatory factor analysis for categorical items that were implemented based on the tetrachoric or polychoric correlations for categorical data. This approach takes advantage of the distinct item response patterns and frequencies. Many studies had factor analyses performed assuming data from the Likert-type scales were continuous rather than categorical, which may easily result in solutions that misrepresent the actual structure underlying the population and in turn might lead to incorrect conclusion (Mooijaart, 1983; Mislevy, 1986; Bernstein and Teng, 1989; Bentler, 1990).

The following model fit indices are presented in Table 4: Akaike’s (Akaike, 1987) information Criterion (AIC) and Bozdogan’s (Bozdogan, 1987) consistent version of the AIC (CAIC), Bentler and Bonett’s (Bentler and Bonett, 1980) NFI and NNFI, the comparative fit index (CFI; Bentler, 1990), and the root mean square error of approximation (RMSEA; Browne and Cudeck, 1993). Both AIC and CAIC consider statistical goodness-of-fit as well as model parsimony; CAIC also takes the sample size into account (Bandalos, 1993). The RMSEA is the most widely endorsed criterion of fit as it is least affected by the sample size (Bollen, 1989; Marsh et al., 1996). CFI, which takes the sample size into account, was suggested as an index preferred over NFI which tends to underestimate fit in small samples (Bentler, 1990). NNFI is a variant of the NFI that takes model complexity into account. As a rough guideline, it has been suggested that CFI and NFI in the mid-0.90s or above may represent a reasonably good model fit (Hu and Bentler, 1999). An RMSEA value of less than 0.05 is indicative of the well-fitting model (Browne and Cudeck, 1993).

Convergent and Concurrent Validities

Convergent validity was tested by correlating the eight specific motivational goals found in ISM with the four general motivational goals of GAGOS and the family-oriented motivational goals. To determine the concurrent validity, we examined the relationship between the eight specific motivational goals and the four self-concept factors from the sense-of-self scale and the learning approach factors from LPQ. To ensure that derived scores were meaningful, reliability for each subscale was tested and items reducing reliability were removed. Cronbach’s alpha, means and SDs for the subscales are presented in Table 1 with the number of items retained in parentheses. CFA was performed on each scale to ensure good model fit with the Mainland Chinese student data (Table 2). The factor scores on each subscale were calculated by summing item scores weighted by the factor loading of each item so that measurement error of each item could be partialed out from the item score. Factor scores were standardized by area conversion before they were correlated (Thorndike, 1982).

Results

As presented in Table 1, the mean scores of the ISM scales are higher for task (4.21), social concern (3.84), token (3.66), and effort (3.65); but lower for social power (2.7), affiliation (2.99), competition (3.32), and praise (3.57).

The CFAs on the nested models (Table 4) revealed that the unidimensional model (M1) did not fit as well as the FO multidimensional model (M2). CFA on the SO hierarchical model (M3) failed to converge and did not result in a proper solution, which often happens with CFA that use small sample size datasets or poorly defined latent variables (Anderson and Gerbing, 1984; Boomsma, 1985; Gerbing and Anderson, 1987). For M2, RMSEA, which is least related to the sample size, was 0.05 with a 90% confidence interval ranging between 0.049 and 0.054, which satisfied the good fit criterion of RMSEA < 0.05. The other fit statistics were NFI = 0.75, NNFI = 0.83, and CFI = 0.85. Although they did not reach the cutoff criterion of 0.95, considering the small sample size (N = 458), these values still suggest an acceptable model fit (Marsh et al., 1988). With small sample sizes, the RMSEA is the most salient criterion of goodness-of-fit and often the one preferred in judging the model fit. In addition, the solution provided by M2 accounts for all of the item-factor loadings that were positive and significant, which is consistent with a priori expectations. These results supported a multidimensional structure underlying the ISM responses from the current Chinese student sample.

To improve the model, restrictions were applied to M2. According to the recommendations provided by the modification indices in EQS, respecifications were made on factor correlations and measurement error correlations to the degree that they were theoretically reasonable as shown in M4 (Figure 1). As for the factor correlations, correlations among the eight FO factors were largely positive, with just a few insignificant ones. Pairs with insignificant correlations were represented by different higher-order factors posited in Maehr’s or McInerney’s goal theories. The insignificant correlations in M2 were fixed at 0 in the modified model (M4). Table 5 shows the correlation results for task vs. social power, task vs. affiliation, effort vs. affiliation, competition vs. affiliation, competition vs. social concern, and token vs. affiliation. In addition, some of the measurement errors (uniqueness) in M4 were considered to be correlated if the variances on the two items could be accounted for by some common factor other than any of the eight specific motivation factors (Figure 1). For example, the incremental χ² statistic from the Lagrange multiplier test suggested that two items, “I try harder with interesting schoolwork” and “I don’t mind working a long time at schoolwork that I find interesting,” had significantly correlated measurement errors. By looking at the content of the two items, it is likely that “interesting schoolwork” may account for the correlation between the two measurements. The fit statistics of the modified model (M4) were RMSEA = 0.040 with 90% CI at (0.036, 0.045), NFI = 0.79, NNFI = 0.89, and CFI = 0.90 (Table 4) improved significantly over the FO multidimensional model (M2).

FIGURE 1

Figure 1. The modified first-order factor structure for the Inventory of School Motivation (M4).

TABLE 5

Table 5. Correlations between the factors of the Confirmatory Factor Analysis solution (M4) (N = 458).

Looking at the factor correlations (Table 5), some of the hypothesized factors that related to the same higher-order factor were highly correlated. The correlation between task and effort, for example, was 0.88; between competition and social power it was 0.57; and the correlation between praise and token was 0.95. These correlations suggested that the higher-order motivation factors should exist. We also noticed a few correlations that were very different from the previous multi-group validation study (McInerney and Ali, 2006), and these may indicate Mainland Chinese learners differ from those in other societies. We found, for example, that effort and competition were correlated at 0.73 (0.41 in McInerney and Ali, 2006); token and effort were correlated at 0.78 (0.34 in McInerney and Ali, 2006); token and task were correlated at 0.66 (0.26 in McInerney and Ali, 2006); and social power and token were correlated at 0.31 (0.64 in McInerney and Ali, 2006).

Item loadings are shown in Table 1. There were also some item-factor loadings that were apparently lower than those from the previous multi-group validation study (McInerney and Ali, 2006). One item, for example, “The harder the problem, the harder I try,” loaded at 0.26 with Effort (0.65 in McInerney and Ali, 2006). The item, “I work hard to try to understand new things at school,” loaded 0.44 with effort (0.76 in McInerney and Ali, 2006). “I do my best work at school when I am working with others” loaded 0.44 with affiliation (0.76 in McInerney and Ali, 2006).

Convergent Validity

Table 6 presents the correlations of standardized scores between the eight motivation factors of ISM and the other motivation scales used in this study, including Family-Oriented Motivation Scale and GAGOS. The factor scores of ISM correlated significantly with the factor scores in GAGOS in the pattern hypothesized. Task (0.54) and effort (0.61) were highly correlated with mastery goals. Competition had high correlation (0.57) and social power had moderate correlation (0.30) with performance goals. Affiliation was highly correlated (0.59) and social concern was moderately correlated (0.19) with social goals. All factors were significantly correlated with global motive in general, which was highly correlated with effort (0.63). There were also some high correlations that may indicate a motivation pattern for Mainland Chinese students that differ from other groups. Praise and token were highly correlated with both general mastery and performance goals, while effort was highly correlated with performance goals. Competition, for example, was highly correlated with all general goals except the general social goals. We also noticed that family-oriented goal was significantly correlated with competition, praise, and token, but had no correlation with task, effort, or social concern. This may suggest that families are an important motivator for Mainland Chinese students. These significant correlations demonstrate convergent validity.

TABLE 6

Table 6. Correlations of standardized scores between the eight motivation factors and factors from GAGOS, Sense-of-self Scales, Learning Process Questionnaire and Family-Oriented Motivation Scale, and their Cronbach’s Alpha indicators (N = 458).

Concurrent Validity

The eight factors in ISM were found significantly correlated with self-concept factors in the Sense-of-Self Scale and learning approaches in LPQ. As was shown in Table 6, sense of purpose was significantly and positively correlated with task, effort, competition, and social power, while the correlations of sense of purpose with praise and token ranged in magnitude from high to moderate. Sense of reliance was more highly correlated with task and effort than with any other motivation factors. Negative self-esteem was negatively correlated with task, effort, and social concern, but it was insignificantly correlated with the other factors. Positive self-esteem showed a positive significant correlation with all eight motivation factors. As for the relation between motivation and learning approach, achievement learning approaches seemed to be more highly correlated with motivation factors (except social concern) than did deep learning approaches.

Discussion

Confirmatory Analysis findings from this study support the validity of an ISM instrument modified for Mainland Chinese students. The consistency of the subscales was tested and after removing a few items the subscales showed good reliability. High correlations between the motivational factors of the scales showed convergent and concurrent validity (ISM with Family Orientation Scale and GAGOS scales, and ISM with Sense-of-Self and LPQ scales, respectively). These findings suggest the ISM (MC) represents a motivation assessment tool that is attuned to Mainland Chinese cultural factors and robust at the same time. In this study, Mainland Chinese students endorsed the same educational values and goals as do other groups, which has generally been the case in other validation studies with ISM (Maehr and McInerney, 2004).

The mean scores of subscales indicate that, like students in other cultural groups, the mastery-related goals (task and effort) and social concern have highest endorsement from the Mainland Chinese students; and social power goal has the lowest endorsement. Compared with students in other social groups (McInerney and Ali, 2006), our sample of Mainland Chinese students seem to endorse competition and token at a higher level than students from other social groups, but endorse affiliation goals less than other student groups.

It is notable that some correlations in the present study were much higher than were found by McInerney and Ali (2006). These differences may indicate that motivational goals in Mainland China are perceived differently from the way they are perceived by other cultural groups. Convergent analysis findings revealed that effort and competition were highly correlated; token was also highly correlated with effort and task. These constructs represent both mastery and performance goal orientations, suggesting that for Mainland Chinese students these goal orientations are not mutually exclusive when the academic achievements are concerned. This supposition is further supported by findings from the concurrent analysis, which yielded high correlations for praise and token with both mastery and performance goals. The latter findings may suggest that Mainland Chinese students perceive themselves to be intrinsically motivated toward academic success as it is framed by social and extrinsic goals. Additional findings from the convergent analysis may provide insight into this phenomenon. We found a high correlation for family goal orientation with competition, praise, and token—though not effort. These correlational findings suggest that Mainland Chinese students’ performance and extrinsic motivation orientations are framed by family goal orientations. These correlational findings, taken together, suggest that Mainland Chinese students adopt both mastery and performance orientations, which are also intertwined with social and extrinsic goals in academic settings (cf., Yeung et al., 2016). This study is important because it offers strong evidence for the validity of the ISM (MC) and its use in Mainland education systems. We found Mainland Chinese students’ mastery and performance goal orientations generally match those found in other cultural groups, confirming prior studies that suggest these goal orientations are universal in nature. We also found, as expected and in line with previous research, that Mainland Chinese students differed in important ways from groups in how they perceive achievement goals. These findings can be used to better predict students’ academic motivation on the Mainland, and this information may be used to identify and provide assistance to students at risk of failure. Teaching methods, furthermore, may benefit from this study. Teachers must be fully aware of the facts that students learning motivations are associated with cultures and values, and that students bring diverse values and goals to the classroom. Teachers should adapt their teaching strategies to the motivation patterns of their students. After the “open-door” policy was implemented in the 1980s, in response to the rapid process in modernization, openness, and globalization as a result of economic reforms, there has been a trend to adopt the individualist Western-oriented values by Mainland Chinese. Correspondingly, Chinese educational system began to emphasize values oriented to individuality, student-centered classroom activities, independent thinking, and creativity and move away from authority-centered classrooms that rewards rote learning and competition (Liem and Nie, 2008). Our findings suggest that the students’ motivation patterns should be examined and the ideas of education reform should be revisited. The ISM (MC) may have great value in assessing students’ achievement orientations and developing and implementing teaching strategies that align with students’ achievement orientations.

Future Research

This study needs to be replicated to ensure the ISM (MC) accurately assesses and predicts Mainland students’ achievement goals. Attention should be paid to verifying differences in academic goal orientation between Mainland students and other groups. Additionally, comparative studies with Mainland students and other groups should also be conducted (e.g., Western and other Asian groups). Comparative studies may, for example, help researchers to better understand the nature of and the relationship between goal orientations in different cultures, which may lead to a better understanding of the teaching and learning strategies that support these goals. It is possible, for example, that some teaching and learning strategies in one culture may prove beneficial for specific application in other groups. PI theory, and instruments designed to tap its constructs, continue to prove useful in understanding how achievement goals in academic settings are similar and different across cultures. We will continue to explore the phenomena described revealed in this study, and to identify its applicability to teaching and learning.

Ethics Statement

IRB approval was received and additional permission was granted by the schools where data was collected.

Author Contributions

FL administered the instrument, collected data, implemented analyses, and wrote the manuscript. DK was the mentor of this study. YK and MT contributed to the design of the study. MD provided edits to the article and helped with the discussion section. SW helped with the literature synthesis.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer, LL, and handling editor declared their shared affiliation, and the handling editor states that the process nevertheless met the standards of a fair and objective review.

Footnote

^McInerney, D. (2005). Inventory of School Motivation: Guidelines for Use and Validation Checks. University of Western Sydney. Unpublished.

References

Akaike, H. (1987). Factor analysis and AIC. Psychometrika 52, 317–332. doi: 10.1007/978-1-4612-1694-0_29