The Ongoing Quest for Culturally-Responsive Assessment for Indigenous Students in the U.S.

Trumbull, Elise; Nelson-Barber, Sharon

doi:10.3389/feduc.2019.00040

POLICY AND PRACTICE REVIEWS article

Front. Educ., 07 June 2019

Sec. Assessment, Testing and Applied Measurement

Volume 4 - 2019 | https://doi.org/10.3389/feduc.2019.00040

This article is part of the Research Topic Cultural Contexts and Priorities in Assessment View all 6 articles

The Ongoing Quest for Culturally-Responsive Assessment for Indigenous Students in the U.S.

$\nElise Trumbull$ Elise Trumbull¹^*

Sharon Nelson-Barber²

¹Independent Researcher, San Rafael, CA, United States
²Senior Program Director Culture & Language in Education, WestEd, Redwood City, CA, United States

Efforts in the U.S. to design curriculum, instruction, and assessment based on Indigenous systems of knowledge and ways of teaching and assessing learning have been mounted wherever Indigenous peoples live. Yet, Western-style education in those places often continues to dominate, to the detriment of Indigenous students' engagement and school completion. Assessment, in particular, has long aroused great concern because many common assessments are not only ineffective but also destructive for Indigenous students—especially when they are used to make high-stakes decisions that affect students' life outcomes. Among such decisions are eligibility for passage from one grade to the next, high school graduation, and college admission. Much is known about how to make assessment culturally-responsive for Indigenous students, but it is often the case that successful programs and practices are jettisoned when new country-wide or state-wide policies are instituted. In the U.S., the most egregious recent case of public policy's interfering with highly successful education of American Indian and Alaska Native students was the No Child Left Behind Act of 2000. Driven by demands to attain high performance on standardized tests, teachers truncated or abandoned strong culture-based instruction in favor of instruction thought to prepare students to do well on the tests. This is just one example of how decision makers under external pressures tend to revert to “best practices” or “the One Best Way,” evoking historical movements to extinguish Indigenous languages and cultures. This article discusses obstacles to culturally-responsive assessment for Indigenous students, describes examples of efforts in the U.S. and elsewhere to improve assessment for Indigenous students, explores the concept of “culturally-valid assessment,” and interleaves recommendations for going forward constructively within various sections of the paper.

Teaching and learning are deeply cultural processes (Rogoff, 2003). Educating children and youth, whether within the home and community or in school, has cultural underpinnings that shape the educational goals of participants and how they go about achieving them. But the culturally diverse schools of the U.S. operate largely on the basis of European-American values and norms that are rarely even identified as cultural (Spindler and Spindler, 2013). These Western values implicitly guide approaches to every aspect of schooling, from academic content to the organization of classrooms, expected styles of speaking and interacting, instructional practices, and means of assessing student learning. For Indigenous students, the gap between home and school culture is arguably greater than that of any other group, and the nature and impact of this gap are magnified by the history of genocide perpetrated on Indigenous peoples (Dunbar-Ortiz, 2014).

The cultural knowledge and perspectives of Indigenous¹ students (American Indian, Alaska Native, and Native Hawaiian), which are important to their learning and development, are most often ignored or given minor acknowledgment in U.S. schools, even when states institutionalize policies intended to promote integration of heritage knowledge and culture into the curriculum (Lee, 2015)². Such legislation is often underfunded, and teachers expected to act on it may not even be aware of it years after it has been passed (Jojola et al., 2011). Thus, school-based education misses the mark for a great many Native students. As McCarty (2018) says, “… Native children…bring [particular]… habits of mind, body, and spirit to school (p. 274) that develop through their active participation in meaningful activities of their cultural communities. Much of their education is “typically a seamless part of everyday life” (Levinson, 2000, p. 5) embedded in context (Nelson-Barber et al., in press) and connected to a historical and cultural sense of place (Deloria and Wildcat, 2001). For Native children, including many residing in urban settings apart from intact Native communities, Western schooling—disconnected as it often is from daily experience— is not harmonious with their cultural world views and ways of interacting with adults and peers to learn and demonstrate learning (Nelson-Barber and Trumbull, 2007; Faircloth and Tippeconnic, 2010; Stumblingbear-Riddle and Romans, 2012; Cajete, 2015). Responding to a wide-ranging review of First Nations education issues in Canada (Battiste, 2002). Barnhardt and Kawagley (2005) say:

[It is no wonder that Indigenous students the world over] have, for the most part, demonstrated a distinct lack of enthusiasm for the experience of schooling in its conventional form—an aversion that is most often attributable to an alien institutional culture rather than any lack of innate intelligence, ingenuity, or problem-solving skills on the part of the students (p. 10).

Nowhere is the disconnection between Native ways of knowing and Western ways of teaching more evident than in the arena of student assessment, most egregiously in the realm of large-scale tests (Nelson-Barber and Trumbull, 2007), particularly when those tests are used to make high-stakes decisions, such as who gains entrance to college (Atkinson and Geiser, 2009). The history of the assessment (academic as well as psychological) of Indigenous students in the U.S. is fettered with accounts of the failure of “usual” practices to elicit accurate information about their abilities or learning (e.g., Dumont, 1972; Chavers and Locke, 1989). Because they perceive high-stakes tests as biased, many Indigenous students face them with anger and dread, not with the “intellect and imagination” they might bring to an authentic task (Brayboy and Maaka, 2015, p. 77, citing a fictitious but aptly biting story by Sherman Alexie). Similarly, classroom assessment can also fail Native students, when teachers do not have a sophisticated understanding of these students' culture-based approaches to learning or when they feel pressured by state policies to standardize instruction and assessment (McCarty, 2009; Nelson-Barber and Johnson, 2016).

In the U.S., progress toward culturally responsive curriculum and instruction and culturally valid assessment has taken place, in many cases, in fits and starts—with instances of favorable legislation and policies (Lee, 2015) and effective practices alternating with lack of funding, new laws that undercut the effective practices, and resurgence of dominant culture calls for “high standards” defined and measured in Western terms (Beaulieu, 2008; McCarty, 2009; Nelson-Barber and Trumbull, 2015). It was well-documented how the No Child Left Behind law of 2000 resulted in the gutting of many highly-effective programs that promoted bilingualism and biliteracy in Native languages and served to engage students who otherwise might have dropped out of school (and the drop-out rate increased, as those programs were eliminated) (Balter and Grossman, 2009; McCarty, 2009).

Unfortunately, the tendency to seek the “One Best Way” (Rogoff, 2003, p. 347)—historically a European-style civilization with all its values, institutions and practices—has meant imposition of the dominant culture's ways on the schooling enterprise. Education's search for “best practices,” no matter the nods to diversity, is simply a form of the “One Best Way” approach on a smaller scale. As suggested, one of the most serious blows dealt by such an approach is often to the survival of the very language of a non-dominant group—perhaps the most important cultural creation and vehicle of culture (Chavers and Locke, 1989; McCarty, 2009; Nelson-Barber and Johnson, 2016). In fact, the backdrop to every issue in the schooling of Indigenous students the world round is a history of the colonial imposition of institutions and practices that not only do not fit the needs of Indigenous students but actively devalue the cultural resources (language primary among them) that are so integral to their learning and development. It is not hyperbole to state that the ancestors of today's Indigenous students were often the targets of genocide; failing that, their ways of life were targets for extinction. This cultural and historical trauma (Brave Heart and DeBruyn, 1998) is within memory of elders in many Indigenous communities, and it continues to affect how Indigenous students and their families approach schools throughout not only the U.S. but also many other regions in the world.

In this article, we discuss ongoing issues in the assessment of Indigenous students, offer examples of promising efforts and steps toward better policies and practices, and synthesize a few recommendations as we go along. We must stress that what we present and the data we use to bolster our arguments are drawn largely from our own research and other professional experiences of the past two plus decades—primarily in the U.S. (including Alaska and Hawaii), but also in the U.S. entities in Micronesia. Assessment should be connected to what and how students have learned in the classroom, hence we address culturally-responsive instruction to some degree.

Recognizing Indigenous Ways of Knowing in Schools

Indigenous knowledge systems may be underestimated or misunderstood by teachers who are steeped in a Western epistemological perspective; yet among every Indigenous group, complex systems for understanding and responding to nature have supported their survival for countless generations (Cajete, 2000; Wilson and Kamana, 2001; Kawagley, 2006). Throughout Canada and the U.S., Indigenous educators have long worked to educate their dominant culture peers about the value of Indigenous knowledge systems (particularly in science and mathematics) and to show the way toward teaching both Indigenous and Western ways of knowing (Aikenhead, 1997, 2006; Barnhardt and Kawagley, 2005; Lipka et al., 2007; Snively and Williams, 2008).

In the intertwined educational triad that is curriculum, instruction, and assessment, efforts at culturally-responsive pedagogy have often addressed only the first element—curriculum (Banks, 2010). It is easier to identify cultural content related to the history or cultural practices of students' communities than instructional and assessment approaches that are likely very different from what teachers have been taught in their teacher preparation programs. However, Indigenous ways of teaching, learning, knowing, and expressing knowledge are key elements that teachers need to learn about, if they want to engage their Native students successfully (Trumbull et al., 2002; Kawagley, 2006; Lomawaima and McCarty, 2006; Aikenhead and Elliott, 2010; Banks, 2010; Nicholas, 2014). And teaching Indigenous knowledge systems in parallel with Western knowledge systems—with opportunities for comparison and for examining which system is more useful under what circumstances—can make for powerful education for both Native and non-Native students (Barnhardt and Kawagley, 2005). As an example of how both knowledge systems contribute to an understanding of a natural phenomenon, Barnhardt and Kawagley (2005) offer the following.

For example, when choosing an eddy along the river for placing a fishing net, it can be explained initially in the Indigenous way of understanding by pointing out the currents, movement of debris and sediment in the water, the likely path of the fish, the condition of the river bank, upstream conditions affecting water levels, the impact of passing boats, and so on. Once students understand the significance of the knowledge being presented, it can then be explained in western terms, such as flow, velocity, resistance, turbidity, sonar readings, and tide tables, to illustrate how the modern explanation adds to the traditional understanding (and vice versa). All learning can begin with what the student and community already know and have experienced in everyday life (p. 12).

Cultural Validity in the Assessment of Native Students

We and several colleagues, representing the disciplines of sociolinguistics, applied linguistics, psychometrics, and education, have collaborated, for more than two decades in some cases, on efforts to define and effectuate “cultural validity” in assessment (e.g., Estrin and Nelson-Barber, 1995; Solano-Flores and Nelson-Barber, 2001; Solano-Flores and Trumbull, 2003; Nelson-Barber and Trumbull, 2007; Huang et al., 2010; Trumbull et al., 2015). Achieving cultural validity in assessment means, first, recognizing that tests and assessments are cultural artifacts and that the ways in which students respond to them are affected by their cultural knowledge and experiences. It means accounting for students' cultural and linguistic backgrounds, epistemologies, educational experiences, communication styles, and socioeconomic situations in the processes of assessment development and implementation (Solano-Flores, 2011, p. 3). Cultural validity is challenged at each phase of the assessment process. Table 1 presents a summary of the phases of an assessment process, along with sample questions that might accompany each phase in the pursuit of cultural validity.

TABLE 1

Table 1. Some elements of a process for promoting cultural validity in assessment.

Of course, additional questions need to be asked at every step along the way. They will reflect the particular needs and goals of the local context, the teachers, and the students concerned. Given the longstanding, well-identified negative consequences of the assessment of Indigenous students, any assessment developer—teacher or other—should be alert to the many possible sources of bias at every stage of the assessment process.

Language as a Key to Cultural Validity

Language is one of the greatest keys to cultural validity in assessment (Trumbull and Solano-Flores, 2011). Nearly all assessments depend upon language in some way, and language differences among students are associated with differences in the ways they construe the language of an assessment question or task (Solano-Flores and Trumbull, 2003). Native students in the U.S. are an extremely linguistically diverse group, speaking or having some familiarity with hundreds of heritage languages and numerous varieties of English. They may be instructed in English, or they may be among those who have access to programs that use their heritage languages. Many of the most important and successful efforts to teach Native students in a culturally-responsive manner have used students' heritage languages for extended periods of time—either as the primary language of instruction for several years, or for a significant portion of time each day (e.g., Wilson and Kamana, 2001; McCarty and Roessel, 2015). In some cases, when students have not learned the heritage language fully at home, it has been taught to them in school (Holm and Holm, 1995; McCarty, 2018).

One tragic casualty of recent assessment policies is the use and revitalization of heritage languages in instruction and assessment. In the name of high standards, there has been a frenzy to standardize instruction and assessment more than ever—even as cognitive research shows that students construct new knowledge on the basis of what they already know (Cobb, 1994) and through the means available to them (including all languages to which they have been exposed) (Cummins, 2005; Solano-Flores and Trumbull, 2008). Depriving Native students who have been using their heritage language in the classroom of such a fundamental resource that is linked to life experience and identity, is both cruel and educationally indefensible.

Language and Assessment in Native Communities in the U.S.

In schools where Native languages are used and taught, assessments should be designed to parallel that use. But decisions about the language to be used on an assessment are often not straightforward. If students have learned a heritage language in school or home, but English has been used as the primary medium of instruction, they should be assessed in English. However, assessment of students who know more than one language is complex because their knowledge of the world is distributed across their languages. Research has shown that for such students, assessment in the language of schooling alone is less reliable than for monolingual students and that more assessment data are required to make accurate judgments of students' learning (Solano-Flores and Trumbull, 2008). Language is a fraught topic within Native communities in the US. Language policies of the U.S. government have led to extensive, intentional, unnecessary, language loss in nearly every Native group, with resulting threats to the maintenance of dynamic cultures and cultural transmission to following generations (Chavers and Locke, 1989; McCarty, 2002, 2018; Lee, 2015). So, any discussion of language and aspects of schooling among Native educators, parents, and communities may not only entail complex educational decision-making but also evoke deep pain (Adams, 1995; cf., Coles-Ritchie and Charles, 2011; Bombay et al., 2014).

A Brief Summary of Recommendations for Approaching Language Issues in Assessment

Some recommendations for increasing the potential for cultural validity with Native students, related to language are (Solano-Flores and Trumbull, 2008; Trumbull and Solano-Flores, 2011): (1) Assess students in the language in which they have been (primarily) taught; (2) If assessments are to be developed in two languages, do so in a parallel vs. sequential process, i.e., develop them in tandem, going back and forth between language versions to ensure they are assessing the same constructs; (3) Do not assume that a translation from English to the Indigenous language will be valid. Translating is full of pitfalls, and the two language versions are not likely to be equivalent; (4) Recognize that students speak different varieties of English or other languages, and assessment in some “standard” variety may not be understood as well as their own language variety (jeopardizing validity); (5) Understand that students exposed to more than one language have their knowledge organized differently—that they know some things in one language and other things in the other language(s); (6) Strive to use clear language, particularly if assessing in English, without eliminating important content vocabulary or simplifying concepts; (7) Periodically interview students about how they have construed a question or instructions on a written assessment for feedback about how to improve how assessments are written; and (8) If at all possible, allow students to choose the language in which they wish to be assessed or to use their home language as well as English, if they so desire.

Large-Scale Testing Woes

The use of large-scale testing—a longtime anathema to Native students (Chavers and Locke, 1989)—has not abated in recent decades. In fact it has ramped up in the U.S. and Canada to the point where some Native families complain that their children are “tested to death” (Claypool and Preston, 2011, p. 89). McCarty (2018) refers despairingly to “the current policy moment in which students' accomplishments are measured against normalized English monolingualism and high-stakes standardized tests” (p. 276). Teachers in Indigenous settings are in a bind: they want to resist policies they see as compromising an authentic experience for their students; yet they may feel pressure to comply with regulations (McCarty, 2009; Coles-Ritchie and Charles, 2011). Native teachers may even fear losing their jobs, if they teach in ways that they know will reach their Native students (Lipka et al., 2005). Lomawaima and McCarty (2014) have called for “zones of safety” and “zones of sovereignty,” in which Native educators can freely explore culturally-responsive pedagogy without fear of reprisal.

Research on large-scale assessment for English language learners (ELLs), which has implications for Native students, has increasingly pointed to the need to tailor assessments to sub-populations of ELLs to obtain valid scores on academic tests (Ercikan et al., 2014; Kopriva et al., 2016). One way of addressing this need is through computerized systems (Kopriva, 2014), but such systems are costly to develop and likely a long way off. In the meantime, educators concerned about fairness in the assessment of Native students will continue to raise objections to standardized systems that, in general, serve only to penalize them (Penfield and Lee, 2009; Claypool and Preston, 2011).

The use of “universal design principles” (Thompson et al., 2002) by test developers represents an effort to make all test questions or items accessible to all students. Universal design addresses psychometric concerns (does the item test what it purports to?) as well as the language of tests (seeking clarity, simplicity, readability), visual format (e.g., clear fonts), and potential bias against subgroups of students. This last element, potential bias, is a hard nut to crack. Traditional methods of determining bias, such as DIF (differential item functioning) analyses, may not identify bias against Indigenous students because of limitations in population sampling, for instance. Usually, Indigenous students represent a small sample and come from many different cultural-linguistic communities—making identification of potential bias against particular subgroups difficult (Huang et al., 2010).

Some Examples of Efforts in the U.S. and Beyond to Indigenize Assessment

As acknowledged above, our research has been limited in geographic scope to the U.S. and U.S.-affiliated entities, but we want to acknowledge some of the efforts of educators beyond the U.S. to bring assessment of Indigenous students into line with the cultural values and practices of their own communities. In the U.S., some promising programs and strategies have been mounted at the state level (e.g., Alaska and Hawai'i), others within Indigenous nations (e.g., Navajo), and some at individual schools (e.g., the Kamehameha School in Hawai'i). Outside the U.S. some efforts have been developed at a national level (e.g., New Zealand). The following examples are not intended to be representative but to illustrate a range of efforts and associated issues. Each context is, of course, different; but common challenges and goals tend to be identifiable across contexts where Indigenous values come into conflict with colonial or Western values. These few examples show both commonalities and particularities across Indigenous settings around the world.

Within the U.S.

The state of Alaska has a long record of researcher-community collaboration to establish educational standards and promote curriculum and instruction reflecting Alaska Native knowledge systems and local approaches to learning (Lipka et al., 1998; Barnhardt and Kawagley, 2005; Barnhardt, 2014). Research has evaluated the impact of Alaska Native mathematics curricula in English on students of all backgrounds' learning and shown that all students with this experience performed better on standardized tests than peers not exposed to the curriculum (Lipka and Adams, 2004). Teachers' informal, culturally-responsive assessment practices integrated with instruction have been documented through observation and video analysis (e.g., Lipka et al., 2005).

A few recent studies document approaches to assessment practices that have been successful with Native students, both children and adults. Repeated throughout many accounts are references to the importance of “relational pedagogy” (McCarty, 2018, p. 277, regarding a Navajo Nation perspective), which emphasizes the importance of family and community, mutual respect and cooperation, and concern for social justice linked to place and environment. In one study, a group of six Yup'ik Eskimo teachers and one non-Native teacher worked together during an intensive in-service summer course to conceive of ways to indigenize assessment, that is, to align assessment practices with Yup'ik values (Coles-Ritchie and Charles, 2011). They used a “funds of knowledge” approach (González et al., 2005) that led to focusing on community, context, and the classroom as a collective that works to support all students. Their goal was to replace the usual individualistic, decontextualized, competitive approach to assessment with authentic assessments that were student-centered, based on an Indigenous curriculum, and that highlighted students' strengths (Coles-Ritchie and Charles, 2011, p. 36). The language of instruction was English, though participants may have used Yup'ik words to enhance communication.

For some time Indigenous educators in the state of Hawai'i have capitalized on the revitalization of local Indigenous knowledge systems and practices by infusing context-adaptive approaches in the formal education they provide to Kanaka Maoli (Indigenous Hawaiian) students. These culture-based methods, which align with Hawaiian worldviews and cultural values and consider learners' prior experiences, home language, and culture, yield improved educational outcomes and socio-emotional development (Takayama and Ledward, 2009; Kana'iaupuni et al., 2010; Kana'iaupuni and Ledward, 2013) and contribute to student resilience (Tibbetts et al., 2007, 2009; Johnson, 2013). Of course, full use of students' heritage languages would be a most important step in Indigenizing schooling—something that should be possible in communities where all or many members speak a heritage language, and some schools in Hawai'i provide instruction in the Hawaiian language, with associated assessments. Among these are Pūnana Leo Hawaiian language immersion private preschools and Ka Papahana Kaiapuni in the public schools through grade 5. Hawaiian language immersion assessments are being formalized by University of Hawai'i faculty, Hawai'i Department of Education staff, and cultural and technical experts.

Effective assessment methods grounded in Kanaka Maoli perspectives, such as Makawalu (“having eight eyes,” meaning that one is able to approach a problem in multiple ways) (Kaiwi, 2006; Kaiwi and Kahumoku, 2006) and cultural frameworks such as Nā Hopena A'o (Hawai'i Department of Education, 2015) are now in use in Hawai'i public and Hawaiian-focused schools. Program-specific, culturally aligned assessment processes have been devised by Native \Hawaiian Education Act grantees (Johnson et al., 2014). Assessment tools such as these are under continuous development and intended for broad dissemination among Kanaka Maoli contexts (Ng-Osorio and Ledward, 2011; Johnson, 2013).

In schools in the U.S.-affiliated Pacific island region (former Trust Territories), teachers have explored classroom assessment processes intended to be responsive to their multicultural classrooms. They have acted as facilitators, using a system of modeling, observing, and giving feedback to students. Rather than asking direct questions, they listen to students in small groups and help develop conversations around topics and concepts they want students to learn more about. Reflection and journal writing are promoted. These are formative assessment strategies designed to evaluate both what students know and how they know it (Pacific Islands Climate Education Partnership., 2015) (Also see discussion of formative assessment beginning on p. 9).

Outside the U.S.

Māori educators in New Zealand and Sámi³ educators in Norway have faced the common challenge of educational policies that elevate the dominant language in standards and assessment over their own languages (Özerk and Whitehead, 2012). Educators feared that national standards and tests in the dominant language (English in New Zealand, Norwegian in Norway) would drive instructional practices similar to those documented in the U.S. after the implementation of the No Child Left Behind law (McCarty, 2009). But both groups have succeeded in maintaining a strong stand in favor of the representation of their own languages and cultures in schooling. Through a long and fitful process, the Māori attained the right to national standards in Māori for Māori immersion schools, and leaders have negotiated for a culturally responsive assessment program that is aligned with instruction.

The Sámi have maintained the right to teach in their own languages, but they needed assessments in those languages, linked to their culture-based curriculum. There are three linguistically distinct Sámi languages—not mutually intelligible—so three assessments were needed. At first, to save money, the Norwegian government proposed to translate the standard Norwegian tests into the Sámi languages, but Sámi educational leaders prevailed against this questionable practice and were able to begin assessment development in their languages, led by their own experts. Translations are often notoriously flawed, and as mentioned earlier it is highly preferable in terms of validity to develop comparable tests in two languages simultaneously rather than translate from a dominant language test to a minority language test—a process that is usually given short shrift (Solano-Flores et al., 2002, 2009).

In Saskatchewan, researchers investigated First Nations educators' and community members' concepts of an ideal education for First Nations students. They held focus groups with two elders (grandparents), representatives of First Nations organizations, and educators (teachers and administrators), with the plan of “juxtapose[ing] these ideas with the predominant learning and assessment tactics used in a Saskatchewan school” (Claypool and Preston, p. 85). Elders characterized First Nations learning as a personal and reflective process of self-exploration, starting with the individual and progressing to entail relationships with family, community, and beyond (Claypool and Preston, 2011). Reciprocity and cooperation are the hallmarks of these relationships. Such self-discovery is in the context of a worldview that integrates the physical, emotional, spiritual, and cognitive. Evidently, English is the primary language of instruction.

Representatives of First Nations organizations spoke of the schools as sites of “incessant testing” (Claypool and Preston, 2011, p. 89) of First Nations students in their second language (English) on tests that did not reflect their cultural experiences and of the negative impact of testing on these students. They said that before assessment could succeed, the students needed to have a sense of belonging in the school and pride in their cultural heritage; such a sense of belonging depended on a trusting relationship between teachers and the parents of students. When it came to the educators, they recited a litany of standardized academic tests and other formal tests and checklists, including assessments of students' health and language skills they were required to administer. Teachers' classroom assessment strategies were more student-centered and closer to First Nations values, allowing for student choice, goal-setting, and reflection as well as peer cooperation and peer assessment. Formats included oral presentations, written tests, demonstrations, and portfolios. But teachers necessarily focused on cognitive development and in a manner linked to a predictable school schedule, wherein certain curriculum was taught at specific times, and assessment fed into report card grading periods. This lock-step, linear organization was also cited as not culturally-responsive to First Nations students. Claypool and Preston conclude that teachers of First Nations students need to address the conflicts caused by existing assessment practices—including adding emotional and spiritual dimensions to assessments.

The researchers urge that teachers move beyond a focus on a “zone of cognitive competence” toward a “zone of trustful intuition,” which acknowledges the spiritual and emotional domains of learning (p. 92). Students should be encouraged to tune into their emotional reactions and instincts as they learn and make decisions in the classroom. And teachers need to “facilitate a classroom environment infused with respect, relationships, and reciprocity, all of which stimulate positive learning and assessment experiences for students” (p. 92). Many others have written about the distance between this sort of worldview and a Western approach, which separates the cognitive from other aspects of knowing and approaches learning in bits and pieces in a linear fashion and suggested how schooling in the U.S. might incorporate Native knowledge and ways of knowing (e.g., Barnhardt and Kawagley, 2005; Lipka et al., 2007; Marin and Bang, 2015).

Formative Assessment: Potential for Cultural Validity

We have long advocated for assessment tools and processes that are chosen for their flexibility in adapting to students' contexts and experiences (Estrin and Nelson-Barber, 1995⁴; Koelsch and Estrin, 1996; Nelson-Barber and Trumbull, 2007). It is worth stopping for a moment to consider the dual meaning of the term “assessment” in this context. It can refer to a particular instrument (a short quiz, a formal test, a group activity) or to the general process of evaluation of learning. Likewise, the term “formative assessment” can refer to a specific formative assessment tool or the process of assessing students formatively in general. “Formative” refers to the property of shaping students' learning going forward on the basis of what is learned from the assessment process.

Because it can be tightly linked to the curriculum taught and conducted in ways that are culturally appropriate, classroom assessment is more likely to approach cultural validity than assessment designed for large groups of students from multiple cultural and linguistic backgrounds. Formative assessment, in particular, has the potential to be tailored to students' and teachers' needs. It entails ongoing evaluation of students' learning through any method suitable to gaining the information needed to judge a student's progress and determine what additional instruction may be needed to meet learning goals. In the ideal, it provides feedback that helps learners evaluate their own learning and engages them in self-reflection and setting learning goals (Stobart, 2006).

Formative assessment can be formal or informal. Formal formative assessment is planned in advance and may take the form of actual tests or performance tasks at various points during a grading period. But formative assessment is often informal and so integrated with instruction that it does not look like “assessment.” It tends to be improvised and is not necessarily recorded by the teacher (Ruiz-Primo and Furtak, 2007, p. 59). For instance, a teacher may move about the classroom and observe students' working on various tasks or sit in on a small group discussion; she may or may not take anecdotal notes as students try to solve a problem together (Trumbull and Lash, 2013). She may pose questions to students working in small groups, pairs, or as a whole class—often “on the fly,” as she sees the need to gauge students' understanding (Heritage, 2007). Such observations, notes, and answers to questions will contribute to judgments about students' learning. Informal assessment is continuous and likely the greatest source of teachers' judgments about student learning (see, e.g., Ruiz-Primo and Furtak, 2007). It is also an arena in which culture-based assumptions about how participants (teacher and students) should interact may promote or interfere with the accurate exchange of information.

Student-teacher interactions are based on implicit cultural norms related to how people in different role groups (adults/children; teachers/students; parents/teachers) should communicate with each other and exchange information (Greenfield et al., 2000). It is at the level of these informal social interactions—rife with invisible cultural shaping—that accurate information about student learning may or may not be exchanged between teacher and student. In successful interactions, students are able to show their learning, and teachers are able to give feedback that helps students judge their own learning and set goals for future learning.

Formative assessments can range from teacher observations during instruction to guided small-group discussion, cognitive interviews with students, peer group self-assessment, demonstrations or non-verbal presentations, and pencil-and-paper quizzes (Basterra, 2011). But the most common form of formative assessment is a teacher's posing of oral questions during class discussions. This strategy has been touted as perhaps the most important formative tool (Heritage and Heritage, 2011). Yet, it has been noted by numerous researchers that direct questioning is ineffective in engaging Native students (Dumont, 1972; Philips, 1983). In fact, in a recent study in which we participated, teachers' use of students' oral responses during discussion to evaluate student learning and plan instruction was the one factor that was negatively associated with the performance of American Indian and Alaska Native students on NAEP mathematics assessment (Huang et al., 2011).

Selecting Formative Strategies

Wholesale adoption of particular formative assessment strategies with Native students is clearly not advisable. But because formative assessment is so flexible, constrained only by the kind of evidence of learning needed, it can be shaped to be responsive to the cultural context. If students do not respond adequately, the strategy at hand can be tossed out or modified. Because formative assessment is an ongoing process, without the high stakes of a standardized test, mistakes in evidence-gathering can be corrected without undue harm to students.

An alternative to oral questioning of individual students is for the teacher to address the whole group and allow for choral response—students answering at once. Ms. Nancy Sharp, a Yup'ik Eskimo teacher in Manokotak, Alaska, was documented using this choral response strategy during her teaching of a unit on Parkas and Patterns (Lipka et al., 2005). Over the course of < 10 min, she and her students have 35 exchanges—sometimes a teacher question followed by one or more student responses, sometimes a teacher short explanation followed by a student comment or short question. Ms. Sharp is showing students pieces of a parka and coaching them on how to describe what they see and how those pieces will be used in what appears to be a highly relaxed, cooperative session. She is engaging in ongoing, informal formative assessment of students' grasp of the task at hand.

In a recent study, 40 (12 Native, 28 Non-Native) teachers of Native students in 10 schools in Alaska, New Mexico, and Arizona were interviewed about the assessment practices they used in their classrooms (Trumbull et al., 2015). Six of the schools had between 69 and 100% Native students; three had between 18 and 32% Native students; one had only 4% Native students. Among the formative assessment strategies teachers reported using were various forms of observation, several techniques that relied very little on verbal interaction, peer and self-evaluation strategies that entailed goal-setting, and opportunities for student reflection and students' offering of feedback to the teacher about instruction. Teachers did report using direct questioning, but some lowered the spotlight on individual students by having them use white boards on which they could write a response and hold it up. Two teachers from the Southwest, one Native and one non-Native opined that it was likely students' lack of confidence or their self-consciousness that prevented them from engaging in classroom questioning. Oddly, neither seemed to consider that there could be a cultural preference at play.

Conclusion

It is a blatant understatement to say that approaches to the assessment of Indigenous students in the U.S. have fallen far short of an ideal of culturally-responsive, culturally-valid practice. The failure to establish fair and effective assessment policies and practices for Indigenous students is particularly frustrating for two reasons: first, because of the potentially damaging consequences of assessments that are ill-matched to students' needs and second, because of the fact that we already know a great deal about what it would take to work toward that ideal.

There are a few encouraging signs in the realm of large-scale assessment in a related arena, the assessment of bilingual students and English learners. Finally, the testing community has acknowledged that tests designed for native English speakers may not be appropriate for students who are still learning English. Whereas, assessment of students through the use of more than one language is clearly the most desirable approach, for a host of reasons (including time, cost, and specialized human resources) this approach is not likely to be widely implemented. Instead, test developers are actively looking for ways to improve the validity of those tests for English learners by modifying the language and structure of tests or offering a range of accommodations, based on a considerable body of research conducted during the past two decades. Attention is being given to the language of tests in order to eliminate non-construct-related sources of language difficulty, with educators, in particular, moving toward using “plain English” in test directions and questions (Pennock-Roman and Rivera, 2011).

Accommodations such as glossaries and bilingual dictionaries that help students identify the meaning of everyday or academic words that may not be common in their environments (and that have nothing to do with the subject matter per se) have been shown to be useful in reducing test bias (Kieffer et al., 2009). For students more proficient in a language other than English, providing assessments in their home languages has, not surprisingly, been shown to result in better performance than use of assessments in English (Kieffer et al., 2009; Robinson, 2010). But, for both political and financial reasons, this practice, which would increase the validity of assessment for many students, is an infrequent practice.

In the realm of classroom assessment, formative assessment has gained currency—boding well for Indigenous students, because formative assessment by its very definition is closely aligned with classroom instructional content and processes. Although psychometricians debate what constitutes validity in formative assessment, the argument can be made that this form of assessment has the potential for greater validity than any standardized educational assessment because it can take any form that engages a student. Thus, it is more likely to elicit information about his or her learning. Moreover, the same constructs can be examined from different perspectives, using different tools; and the teacher and student can work together to determine when and how to assess a construct.

Teachers in Indigenous settings who have used “alternative” forms of assessment, often embedded in instruction, are more likely to get support from districts and administrators for their assessment methods, since they can be explained via formative assessment principles (cf. Trumbull et al., 2015). But most teachers of Indigenous students need extended professional development to learn about successful strategies already identified and to explore and experiment with methods appropriate to their own settings. Such professional development cannot be generic but must have the strong participation of Indigenous educators, who understand issues of language, culture, and the history of Indigenous schooling. Research on professional development in general (Darling-Hammond et al., 2009) and on professional development related to classroom assessment (Wylie et al., 2009) show that states and districts need to expend considerable resources on long-term and intensive teacher support to achieve meaningful assessment reform.

Ultimately, achieving cultural validity in the assessment of Indigenous students in the U.S. and across the globe is a matter of social justice (Brayboy and Maaka, 2015). Assessment legislation or educational policies that do not take into account the specific needs of Indigenous populations are not destined to promote such cultural validity (we have only to look at the impact of No Child Left Behind). Indigenous students deserve assessment systems–linked to culturally-responsive curriculum and instruction– that are fair and equitable. Much is known about how to carry out the task. What remains is the will to act on that knowledge.

Author Contributions

The article stems from work on which the two authors have collaborated for more than 25 years and expresses their joint thinking on issues in the assessment of Indigenous students. ET did much of the writing, but SN-B's input was equally important.

Funding

Some of the material in this document is based upon learnings from work supported by the National Science Foundation under grant awards No. 1020392, 1348622, and 1626939. Any opinions, findings, and conclusions or recommendations expressed in this material are those of author(s) and do not necessarily reflect the views of the National Science Foundation.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

1. ^We use “Native” and “Indigenous” interchangeably, often choosing the term used by the author(s) we cite. “Aboriginal” and “First Nations” are terms used in Canada.

2. ^See also Alaska DEED (Department of Education and Early Development) (2012). Guide to implementing the Alaska cultural standards for educators. Juneau, AK: Alaska Department of Education and Early Development; Montana State (1999). Indian Education for All (IEFA) Act. Recognition of American Indian cultural heritage (MCA 20-1-501). Helena, MT: Montana State.

3. ^The Sámi are an Indigenous group living in three northern regions of Norway.

4. ^Trumbull previously published under the name “Elise Trumbull Estrin.”

References

Abedi, J. (2003). Impact of Students' Language Background on Content-Based Performance: Analyses of Extant Data. Executive Summary (CSE Technical Report No. 603). Los Angeles, CA: University of California, Center for the Study of Evaluation/National Center for Research on Evaluation, Standards, and Student Testing.

Adams, D. W. (1995). Education for Extinction: American Indians and the Boarding School Experience, 1875–1928. Lawrence, KS: University Press of Kansas.