Leading an Assessment Reform: Ensuring a Whole-School Approach for Decision-Making

The ability of teachers to use assessment data to inform decisions related to learning and teaching defines teaching effectiveness. However, to maximise the benefits of teacher decision-making, there is a need to ensure that all teachers across the school are supported to engage in a whole-school approach to ensuring that all students across different stages are supported. This paper reports on a case study of a school in building an assessment culture with a strong focus on using a range of data for teacher decision-making. We used an auto-ethnography to reflect on our experiences in leading this assessment reform. Using the lens of activity theory, we have identified structural, organisational, social and behavioral factors that contribute to the success of the program.

Introduction

Teacher assessment knowledge and skills are critical for improving student learning (Black and Wiliam, 1999; Hattie, 2008). A range of theoretical and empirical evidence support the effectiveness of using assessment to increase student outcomes. However, the effectiveness of using assessment relies on the ability of teachers to constantly adapt their teaching in response to student learning needs and learning development (Mandinach and Gummer, 2016). Teachers do this by using and making sense of different data sources to inform the design of their learning and teaching activities to support individual students, a process that has been proven to increase student learning and engagement (van Gee et al., 2016). The ability of teachers to use assessment data to inform their decisions related to learning and teaching, commonly known as teacher assessment data literacy, defines teaching effectiveness.

Studies on teacher assessment data literacy highlights several issues including a low level of proficiency and self-efficacy (Mandinach and Gummer, 2016), misconception of the process (Kippers et al., 2018) and competing workload demands (Kippers et al., 2018). Despite the importance of data literacy, reforms in schools are often fragmented with teachers feeling that they are not fully supported. To maximise the benefits of teacher decision-making, there is a need to ensure that all teachers within the school are assisted to develop a whole-school approach to ensure that all students across different stages of learning receive the support they need. There is also a problem with of varied understanding of the assessment process, when a common understanding of assessment language and processes is needed for successful implementation of any assessment reform (Davison, 2013).

To address the issues of teacher support and competing understanding of teacher data literacy, this case study will describe a whole-school approach undertaken by one school in Australia to implement an assessment reform focused on building teacher capacity to develop and implement a school data tracking system to help teachers make informed decisions.

Implementing an Assessment Reform

The roles of principals and other school leaders in implementing educational reforms are undoubtedly one of the biggest factors that enable teachers to effectively implement changes in their practice. The people, processes and tools available to support teachers and students are critical for the successful implementation of the program. This has been highlighted by (Davison, 2013) in the context of AfL program. She emphasises that for the successful implementation of AfL reforms, program implementers should use the principles of AfL to develop, implement, monitor and evaluate the assessment literacy program. Consistent with a view of effective student learning in AfL, any reform should consider where individual teachers are in terms of their AfL competence, where they need to go, and how best to get there. Hence, any assessment literacy programs aimed at supporting teachers to enhance their assessment competence should begin with evaluating their current level of performance and identifying their training and support needs using a tool that clearly defines the criteria and standards for teacher AfL literacy.

AfL should be embedded in curriculum and assessment institutionally and pedagogically. Teacher AfL literacy programs should start by setting and sharing appropriate learning outcomes, success criteria, and performance standards. These learning outcomes are the teachers’ assessment knowledge and skills, the success criteria are the indicators of these knowledge and skills, and the performance standards are the quality of assessment practices that will be used to monitor teacher AfL development.

Feedback should be used extensively to provide useful information to individual teachers. As proposed by Davison (2013), one characteristic of a successful assessment reform is “constructive qualitative feedback which helps stakeholders (these include teachers) to recognize the next steps needed for reform and how to take them” (p. 265). The effectiveness of feedback on teachers’ performance is supported by studies such as Jensen (2011) in Australia, who found that if teachers receive feedback related to their performance, their effectiveness could rise by up to 30%.

Teacher AfL literacy programs should also develop the self and peer assessment capability of teachers (Davison, 2013). Teachers should be encouraged to regularly reflect on their practices to assess how effective they are and how well they are progressing in using AfL to improve their professional learning.

AfL literacy should provide teachers with continuing opportunities to engage in further education. Contrary to the common practices of most formal training, professional development, AfL programs are most effective if embedded in teachers’ everyday classroom activities. Black et al. (2003) emphasise that professional development for teachers to adopt and adapt AfL should be framed in such a way that teachers will be fully engaged in a range of activities where they are treated as learners themselves rather than simply telling them how to use assessment and assessment information.

The importance of continuous sharing and reflecting on their practices by teachers and their peers goes far beyond acquiring explicit knowledge. The community of learners they create gives them opportunities to share and acquire tacit knowledge, which cannot be transferred so easily through formal training and conferences. Superficially, it may seem easy to create such a learning environment, but there are a number of critical factors that influence its effectiveness. Amongst these are trust, early involvement, due diligence (Foos et al., 2006), personal interest and shared values (Dhanaraj et al., 2004), intrinsic motivation (Osterloh and Frey, 2000) and fit to the organization (Ambrosini and Billsberry, 2007). It is, therefore, imperative that systems identify and adopt the philosophical changes required for effective assessment AfL literacy. Systemic changes should foster trust, develop and communicate shared values, support intrinsic motivation, and find ways for individuals to fit into the school system. The latter requires not only helping teachers to change their assessment practices, but developing personal attributes, which are necessary pre-requisites for AfL literacy.

Assessment literacy is not only necessary for teachers but for all other stakeholders, including administrators, students and parents (Davison, 2013). The linkage of assessment literacy to key responsibilities (Popham, 2009) defines its true nature. People with different stakes in education have different needs and so require levels of assessment literacy. Davison (2013), as the lead consultant in assessment reform in Hong Kong, Singapore, and Brunei, found that the most important factor contributing to failure of assessment reform is misconceptions amongst policy makers about what AfL really is. Due to lack of understanding of what it takes to implement assessment reform, policy makers may think that simply changing the assessment practices of teachers will make the assessment reform successful. This, however, is not the case because teachers are not autonomous, nor working in isolation. What is required is the establishment of strong AfL culture at all levels of stakeholders and across the system (Davison and Leung, 2009).

An effective AfL literacy program should recognise the diversity of teachers, who, just like students, have individual needs, diverse learning characteristics and different classroom contexts in which they operate. Hence, AfL literacy programs should use the concept of differentiated instruction and adopt various strategies that suit teachers’ needs. Above all, program implementers should have a strong belief that all stakeholders can improve their assessment literacy (Davison, 2013).

In summary, the effectiveness of a teacher AfL literacy program starts with a clear understanding of the basic principles of AfL by stakeholders, re-engineering the educational culture and re-aligning educational practices to AfL principles to provide teachers with an environment that models AfL culture, providing the necessary support services to teachers, thus enabling teachers to actualize their learning.

Building Teacher Decision-Making

Research on assessment highlights the need to build teacher assessment decisions. For every learning and teaching episode, teachers need to constantly engage in ongoing decisions to provide the necessary support for individual students. The importance of teacher decision-making was highlighted in the seminal paper by Black and Wiliam (1999) on formative assessment.

Practice in a classroom is formative to the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited (p. 109).

In that definition, it is evident that gathering and analysing data and using the results to inform the next steps of learning are critical for the effectiveness of assessment to improve student outcomes. Building on the work of Black and Wiliam, the Assessment Reform Group proposed a definition of assessment for learning (AfL) as “the process of seeking and interpreting evidence for use by learners and their teachers to decide where the learners are in their learning, where they need to go and how best to get there” (ARG, 2002, p. 2) This definition highlights the need for teachers to constantly gather and use student data to inform all learning processes. This has been clearly articulated in the more recent conceptualisation of teacher assessment for learning literacy that “accounts for knowledge and skills in making highly contextualised, fair, consistent and trustworthy assessment decisions to inform learning and teaching to effectively support both students and teachers’ professional learning” (Alonzo, 2016).

Even with the emphasis given in the literature on teacher decision-making, research evidence shows that teachers still have relatively low proficiency in this area (Mandinach and Gummer, 2016). Also, there is more to understanding data for decision-making than simply knowing how to interpret grades, marks and high-stakes tests (Bowers, 2009; Kippers et al., 2018). The low proficiency of teachers and the reliance on standardised student data for decision-making compromises teachers’ self-efficacy in decision-making. What is needed in the field is a more practical approach to using a range of data, from “in-class contingent formative assessment to formal summative assessments used for formative purposes” (Davison, 2007), with teacher professional development in the area of AfL and decision-making recently given renewed emphasis (Kippers et al., 2018).

From the first conceptualisation of the role of teacher decision-making, there has been a great demand for teacher professional development in assessment decision-making. In a comprehensive list of content for building teacher AfL literacy, (Popham, 2009) explicitly listed a number of skills relating to teacher decision-making. Several education bureaucracies have been implementing assessment literacy development programs to train teachers using various modalities including the development of resources and advice. However, despite all these initiatives, the quality of teachers’ decision-making remains relatively low.

Building teachers’ assessment decision-making is part of a bigger assessment reform that needs to happen in schools. Teacher assessment knowledge and skills must be improved for teachers to make informed decisions. Teachers’ AfL literacy (Popham, 2011), which include teachers’ data literacy (Mandinach and Gummer, 2013) need to be at a certain level of competence. Davison (2013) emphasises that for the successful implementation of assessment reform, program implementers should use the principles of AfL to develop, implement, monitor and evaluate the assessment literacy program. From this perspective, it is necessary that teachers should be considered as learners, that is, just like the students they teach they need support to become independent and self-regulated teacher learners. To achieve this, teacher AfL literacy programs should be organised around a number of the key principles of AfL. Davison (2013) has expanded on seven core AfL principles and contextualised them to a teacher AfL literacy program, as follows:

1. Consistent with a view of effective student learning in AfL, a teacher AfL literacy program should consider where individual teachers are in terms of their AfL competence, where they need to go, and how best to get there. PD should begin with evaluating their current level of performance and identifying their training and support needs.

2. AfL should be embedded in curriculum and assessment institutionally and pedagogically. Teacher AfL literacy programs should start by setting and sharing appropriate learning outcomes, success criteria, and performance standards. These learning outcomes are the teachers’ assessment knowledge and skills, the success criteria are the indicators of these knowledge and skills, and the performance standards are the quality of assessment practices that will be used to monitor teacher AfL development.

3. Feedback should be used extensively to provide useful information to individual teachers. As proposed by Davison (2013), one characteristic of a successful assessment reform is “constructive qualitative feedback which helps stakeholders (these include teachers) to recognize the next steps needed for reform and how to take them” (p. 265). The effectiveness of feedback on teachers’ performance is supported by studies such as Jensen (2011) in Australia, who found that if teachers receive feedback related to their performance, their effectiveness could rise by up to 30%. Just as in the classroom, teacher AfL literacy learning needs to utilize feedback in order for teachers to have discussions about their performance in relation to the learning outcomes, criteria, and standards.

4. Teacher AfL literacy programs should also develop the self and peer assessment capability of teachers (Davison, 2013). Teachers should be encouraged to regularly reflect on their practices to assess how effective they are and how well they are progressing in using AfL to improve their professional learning. Similar to student self and peer assessment, teachers need to have criteria and standards readily available to guide them in their reflective practice. Self and peer assessment is only possible if there is an assessment tool that can be used to guide them to assess their performance.

5. AfL literacy should provide teachers with continuing opportunities to engage in further education. Contrary to the common practices of most formal training, professional development, AfL programs are most effective if embedded in teachers’ everyday classroom activities. Black et al. (2003) emphasise that professional development for teachers to adopt and adapt AfL should be framed in such a way that teachers will be fully engaged in a range of activities where they are treated as learners themselves rather than simply telling them how to use assessment and assessment information. In other words, teachers should undergo authentic learning that fosters inquiry, experimentation, collaboration, and reflection (James et al., 2007).

6. The importance of continuous sharing and reflecting on their practices by teachers and their peers goes far beyond acquiring explicit knowledge. The community of learners they create gives them opportunities to share and acquire tacit knowledge, which cannot be transferred so easily through formal training and conferences. Superficially, it may seem easy to create such a learning environment, but there are a number of critical factors that influence its effectiveness. Amongst these are trust, early involvement, due diligence (Foos et al., 2006), personal interest and shared values (Dhanaraj et al., 2004), intrinsic motivation (Osterloh and Frey, 2000) and fit to the organization (Ambrosini and Billsberry, 2007). It is, therefore, imperative that systems identify and adopt the philosophical changes required for effective assessment AfL literacy. Systemic changes should foster trust, develop and communicate shared values, support intrinsic motivation, and find ways for individuals to fit into the school system.

7. Assessment literacy is not only necessary for teachers but for all other stakeholders, including administrators, students and parents (Davison, 2013). The linkage of assessment literacy to key responsibilities (Popham, 2009) defines its true nature. People with different stakes in education have different needs and so require levels of assessment literacy. Furthermore, Davison argues that issues in AfL implementation should be used to develop an assessment literacy program for policy makers. At the highest level, the nature of AfL should be clearly understood so as to facilitate the legislation of some pre-requisites needed to institutionalise AfL implementation.

8. An effective AfL literacy program should recognise the diversity of teachers, who, just like students, have individual needs, diverse learning characteristics and different classroom contexts in which they operate. Hence, AfL literacy programs should use the concept of differentiated instruction and adopt various strategies that suit teachers’ needs. Above all, program implementers should have a strong belief that all stakeholders can improve their assessment literacy (Davison, 2013).

The Context of the Study

This paper reports on an assessment literacy program in one public primary school in Australia which focused on developing and implementing a school data tracking system to help teachers make informed decisions. The school is part of a wider Learning Community with four other schools within the area, all together comprising 283 teachers and 4,521 students.

The assessment literacy program has been the focus of professional development for the last three years with the goal of building a stronger assessment culture within the school. It aims to build a common understanding of the principles of assessment for learning amongst school leaders, teachers, students and parents/carers. Five assessment leaders were designated to work collaboratively with a university partner. Every term, they engage in professional development and then develop an action plan on how to support other teachers with their assessment literacy. Resources and advice are provided.

In the past, teachers at the school participated in a number of professional learning programs on assessment but there was no common understanding nor collective effort to implement effective assessment practices, as there was no clarity or agreement on what assessment practices were effective in supporting student learning. There were various interpretations of what effective assessment looked like and teachers’ practices were diverse, but the use of summative assessment was very common. Furthermore, the collection and use of assessment data was done individually by teachers and data sharing across different stages was not evident. Whole school collaboration was missing and whole school student assessment data was not tracked annually to accurately monitor student development throughout their schooling. This meant that individual student assessment data was not accessible to all staff within the school. Student data were kept by individual teachers and were not accessible to teachers taking the same students the following year.

To address these issues, the school became involved in an educational partnership with a university to build teacher capacity in assessment for learning (AfL) practices and implementation, with the goal to build a strong assessment culture across school. The overall approach of this program was the use of situated learning to connect the new knowledge and skills gained by assessment leaders and teachers to their current responsibilities (Lave and Wenger, 1991; Flores, 2005). This approach is based on a socio-cultural theory of learning that highlights the critical role of peer conversation, negotiation and consensus-making to co-construct knowledge (Lunenberg et al., 2017). To meet the objectives of the program and establish contextual sustainability, five teachers were trained to take leadership roles in assessment. They serve as mentors for their colleagues, facilitate discussions in professional learning groups to identify, discuss and resolve issues and misconceptions related to assessment, and engage collaboratively with their colleagues to develop, implement and evaluate assessment strategies. A teacher AfL literacy tool and process are used to help teachers identify their current level of assessment knowledge and skills and key areas of concern. This process helps develop the critical and evaluative skills required to monitor their assessment literacy (Timperley et al., 2008). Professional learning activities are designed in a way that teachers and school leaders can analyse and evaluate their current level of performance for specific skills against the set of criteria indicated in the tool. From their analysis, individual teachers develop their learning goals for continued engagement in PD activities. To support the suite of professional development activities, the program co-develop resources and templates with evidence-based advice to make the processes accessible for school leaders, teachers, parents and students. These resources have been validated, piloted and evaluated for their effectiveness in supporting the objectives of the program.

Methods

We use autoethnography to reflect on our experiences as the University Partner (UP), the Instructional Leader (IL) and one of the Assessment Leaders (AL). Autoethnography was chosen as a method because it allows us to draw from our own experience the data needed, analyse it and understand the cultural shift (Campbell, 2016) in assessment practices in one school. It allows a researcher-practitioner to tell their accounts using critical inquiry embedded in theory and practice (McIlveen, 2008).

We adopted the approach of Ellis et al. (2010) by using autoethnography both as a process and a product. As a process, we reviewed program documents, teacher and student data and classroom observation records to guide us in our reflection (Goodall, 2001). These documents facilitated our analysis of our lived experiences about the program. As a product, we reflected on these documents and started to write our own personal account of the program using rich narratives. We focus on what processes, products, engagement and commitment have established the current assessment culture in our school where teachers are strategically designing a range of assessment tasks to elicit student learning and using all these pieces of evidence to make informed decisions to support individual students. To achieve rigour of our reflection, we convened and discussed our own reflection using the activity theory as the theoretical lens to interpret our experiences. The use of a specific theory to interpret our reflection ensured coherence and consistency of the results. More importantly, it allowed us to achieve the aim of this paper. Any insights that we did not agree, we discussed it and deleted those that were problematic. All these processes contributed to the trustworthiness of the results.

Theoretical Framework

Engestrom’s (1987) activity theory was chosen as a framework for this study, based on Vygotsky’s (1978) conceptualisation of the primacy of culture rather than individual cognition in mediating action, learning and meaning making. In other words, learning is facilitated by social interactions of individuals within the community. This model is useful for understanding how different factors work together to influence various socially and culturally mediated activities to achieve the intended outcomes. It has been extensively applied on learning and development in work pracices.

Specifically, this theory describes the roles of the objects (experiences, knowledge and physical products), tools (documents, resources, etc), and community (people or stakeholders). The subjects, which are the people engaged in the activity, work as part of the community to achieve the object or the outcome of the activity. The quality of the interactions among objects, tools and the community determine the quality of the outcomes. Thus, this analytical framework is useful for reflecting on different elements of social learning systems to understand the patterns of social activities and development, which consequently brings the intended outcomes.

In this paper, the subjects are the teachers who are engaged in professional learning to build their assessment capability particularly on using assessment data to inform learning and teaching. These teachers work within the larger learning community of the other four schools supported by the university which leads the professional development (PD) activities. PD resources including a website are co-developed by the university and assessment leaders of the school.

Results

This section highlights our reflections on our experience and expertise in leading the assessment reform in one school. We focus our reflection on what factors have greatly influenced the development of school assessment culture particularly on using data to inform teachers’ decision.

Motivation and Aim of the Program

The need for common understanding of assessment principles, practices and processes of AfL in the school provided the impetus for the development of the program. This has been initiated with the commitment of the executive staff members to professional development that engaged all teachers in the network of five schools in the wider learning community. The professional development program focused on building a culture of assessment collaboration through implementing assessment schedules, which originally focused on building teachers’ capacity in using learning intentions and success criteria, questioning techniques, self and peer assessment, and teacher feedback. The need to focus on teacher decision-making was brought about by teachers’ concerns with what to do with the large amount of assessment data gathered throughout the teaching period. In addition, there were existing system-generated data including the results of national assessment tests teachers were expected to use to inform their assessment decision-making but they were uncertain how to reconcile the two sources of data.

The External Partnerships

The school’s need for expert support stimulated the development of the university partnership. Together with the university, the school executive team created a school assessment framework and an assessment schedule for three-year implementation. The professional discussions and collaboration were underpinned by the conceptual framework of AfL, building a school culture of clarity and high expectations for AfL implementation. One of the most critical features of the university-school partnership was the designation of the five teachers to be assessment leaders and undertake regular professional development program facilitated by the university partner. We form the team together and the university knowledge from extensive research is used as input for discussing the most appropriate approach and content to address the assessment literacy needs of teachers across the school. The development of the program is based on teacher self-assessment using the teacher AfL literacy tool (Alonzo, 2016). This tool was used as it is theoretically and empirically supported and adheres to the AfL concept of rubrics with clear criteria and descriptions of five level of performance standards.

After a PD program each term, we develop an action plan which outlines the approach and content for in-school professional development including monitoring individual teachers’ implementation of AfL. Another feature of the collaboration is working with the other four schools in the learning community. Each year, the five schools join the Learning Community Professional Development Day. Assessment experts from different universities are invited to deliver keynotes, with workshops tailored to the needs of teachers conducted. In addition, the collaboration with other network schools has contributed significantly to building the assessment culture of the school. Within these teams, individual staff were chosen to share their successful assessment strategy for the group. We identify best practices from other schools and incorporate them in our action plan.

To further our collaboration, we have built a community of trust where our classrooms are open for observation. As the University Partner, “I observe classes of teachers and give feedback at the end of the session. I adopted a dialogic-feedback approach where teachers lead the discussion of their performance and I clarify some of their misconceptions and give them actionable feedback to further improve their practice.” As an Instructional Leader, “I find it valuable that our university partner observes a number of teachers to ensure that our practices in our school are adhering to the principles of AfL. We have also identified teachers who have best practices and use them as exemplars for other teachers to observe while teaching. During Stage Meetings, we discuss our learnings from observing other teachers.”

To better develop a strategy for whole-school data collection for decision-making, benchmarking activities were conducted in two schools which are known for their data-driven decision-making initiatives. We looked at the approaches used by each school in terms of data collection, analysis and decision-making. From this experience, we developed a spreadsheet that could help teachers analyse the results of pre and post-tests to calculate the learning gains and effect size. The results of comparing pre and pots-tests are just the starting point for teachers to make decisions for individual students. They have to draw from their professional judgment based on several sources of assessment data including anecdotal records, observation, interviews, self and peer assessment, to validate the results of pre and post-tests.

While this analysis is helpful, there is a need for a more sophisticated assessment tool to link the results to individual students’ learning needs. As a result of this, the school has subscribed to an external online assessment provider, Essential Assessment, to reinforce the need of teachers for data. The content of the online assessment program is aligned to the Australian curriculum across stages of schooling. Teachers use it to determine the knowledge and skills of students against the Australian Curriculum achievement standards and use the results along with their classroom assessment data to develop differentiated learning, teaching and assessment activities. Teachers reflect on various data sources and use their professional judgment to identify learning needs and support of individual students.

The Internal Mechanisms

To provide a mechanism for a whole-school approach, a school goal of building an assessment literate school culture was embedded into all teachers’ Professional Development Program (PDP) goals. This is a mandatory document for all teachers to set their goals for the whole year. The whole-school assessment framework and assessment schedule was co-developed by all teachers to establish consistency between all staff and co-create school expectation on assessment practices. This process ensured that common understanding of assessment knowledge was shared across the school to establish a culture of trust and value. Assistant principals check that the schedules are implemented, and that assessment data are the focus of stage group meetings and are explicitly used to inform decisions. In addition to ensuring teachers assessments are outlined on a whole school assessment schedule, the school executives created stage-based PDP goals of collecting whole stage data that is accessible to the whole school. This provided transparency for student learning across the whole school. Every student's learning, growth and attainment became a stage-based collective priority. All student results became a critical part of collaborative stage-based planning days.

My role as Instructional Leader (IL) was funded to provide in class professional development to all staff while they are teaching. “I used my theoretical and practical knowledge of AfL and to flexibly model the various domains of teacher AfL literacy in the classroom. I follow up all lessons with debriefing to ensure collegial discussions focusing on ensuring teachers become the learners while I provide feedback for further improvement.” The basic principle I communicate with teachers is to use data to guide teaching to “close the gap” between prior knowledge and the intended learning outcomes. I modeled how data-driven decision-making fits well into the AfL initiative in our school. During this professional development I clearly discussed how the five most common AfL practices can me more effective if driven by student data. I clearly demonstrate how sharing learning outcomes can be effectively used if teachers have a clear record of individual students’ prior knowledge. I demonstrate how student assessment data informs the identification of the gap between prior knowledge and learning outcomes, which these gaps will guide teachers to support individual students to set their learning goals. Then, I show how the success criteria scaffold what the learning will look like. I demonstrate also how to differentiate the success criteria based on students’ prior knowledge. Further, I emphasise how to give feedback linked to the success criteria and learning goals of students. More importantly, I encourage teachers to build the capacity of students to engage in self and peer assessment with the aim for students to gather data related to their learning to monitor their progress. The results of self and peer assessment need to be moderated with reference to the teacher’s assessment record. The results of moderation are then used to set future goals of individual students.

I have observed that this method is useful for teachers as it gave them a clear direction on how to implement AfL practices with a strong focus on gathering a range of assessment data and using them to inform every aspect of learning and teaching. In the beginning of the project it was observed that there is a wide disconnect between theory and practice and teachers have the difficulty to implement a coherent learning and teaching activity where each assessment activity supports and builds on from each other. Providing teachers with explicit links on how these practices fit together enhanced their understanding of teaching and delivering AfL practices. They have also understood the range of data that needs are elicited and how to integrate all these different data and make sense of them and use their insights to inform learning and teaching activities to support individual students to learn more effectively.

Based on my experience as an Assessment Leader, there are structural and internal mechanisms that influence the implementation of assessment reform in our school. The clarity and the connection of the theory and the practical side of AfL practices in the classroom were imperative to its successful implementation. Teachers need to have a clear understanding of the importance of AfL and how this will benefit all students. Differentiated staff professional development is also an important factor ensuring that all staff are supported at their level of understanding and using those staff who were competent as Assessment Leaders in the school provided staff extra support in implementing AfL practices. I organise mentoring sessions with my team to ensure that I could differentiate professional development to each staff member. It was important to build, maintain and sustain momentum around AfL practices throughout the school year to ensure successful implementation.

As an AL it was important to ensure that my team had clarity around what was expected with AfL. I approached my team members with the idea of collaborative practice to best ensure AfL is used effectively in all classrooms I supervised. Using a simple questionnaire tool, I gained an understanding of what they did and did not know and used that information as a guiding tool to support my classroom teachers. I did this by organising teaching observations of myself using AfL tools in my classroom and having prior and conclusion conversations to discuss what they saw and heard. We set high expectations and it is expected that what we discuss after the lesson observations and theoretical conversations will be implemented. Another mechanism is the collaboration among assessment leaders. We have specific time for discussions around AfL in team meetings, what is working, who among the staff need further specific guidance with and students who may require further differentiation.

Our roles, processes and high expectations support individual teachers. Some teachers require more prompting and support. There are those who did not fully understand the benefits of using data to inform their learning and teaching activities. If we identify any teacher who has misconceptions of AfL, we discuss it in a supportive and respectful environment. Reflecting on the experience of those teachers, they were able to come on board and implement the practices consistently and effectively. I would say that building an assessment culture requires shared responsibility and accountability across all classrooms in the school. It was challenging in the beginning of implementation of the program were many teachers required reminders to consistently apply the AfL practices into their everyday practice particularly the gathering and recording of assessment data.

Adapting the Program

The on-going monitoring of the program allows for its flexibility and growth. As the university partner that delivers the professional development, I regularly seek feedback from IL and ALs about the perceived needs of teachers and use that to plan the future direction of the program. To allow for more objective feedback, another researcher was employed to explore the status of the assessment culture in the school. The focus was on students’ perception of their engagement in assessment. The data from this engagement were used to inform the future direction of the program.

One example of how we exemplify the use of data to inform the direction of the program is how we use the results of teacher self-reflection using the Teacher AfL tool. The initial results showed that the use of rubrics by teachers was s quite low. During the professional development day, this was extensively discussed to try to identify the contributory factors to this. What appeared to be the problem was the absence of a school-wide common understanding and expectations of quality writing. As the university partner, I discussed the practices of one school we benchmarked, where they have common rubrics for different writing types and how it is differentiated across different year levels to reflect different levels of proficiency. As the Instructional Leader, I facilitated the creation of common rubrics for each type of writing. Together with the teachers, we identified the criteria and established the levels of performance across different year levels. My role as an Assessment Leader became easier to communicate with other teachers what rubrics to use and how to use it. Every stage meeting, we reflect on our experience in using these rubrics and provide critical feedback to the IL on how to further improve the rubrics. This is an on-going process and we have found across the time we are using the rubrics that it has improved significantly both its contents and the way we used it across different year levels. It facilitated the consistency of teacher judgment, and we felt that it contributed greatly to the reliability and validity of the data we for our decision-making.

We have demonstrated above how data gathered from teachers’ feedback was used to improve rubrics collaboratively. This allowed teachers to co-create differentiated success criteria for each aspect of writing. It deepened their knowledge of the content and provided better support for students to aim and achieve for higher outcomes. As the Instructional Leader, I have observed that teachers can now provide specific feedback using the rubrics and can point out to students their specific areas for improvement. The consistency of use of the common rubrics ensure also that students are assessed based on the same expectations and outcomes. More broadly, the rubrics are used for moderation for consistent teacher judgment. Teachers are fair and develop consistent reporting of student learning and attainment to parents. This process became an integral part of stage meetings.

The process of using rubrics is now well understood by teachers, and the rubrics themselves have become a formative assessment tool for every writing lesson for both the teacher and the students. Teachers use the rubrics to mark the pre-test writing of students and discuss with them their goals based on the results. Teachers record students’ pre and post-test student data on a stage-based data tracking sheet. The student growth data became a strong focus of stage-based discussions. As an Assessment Leader. I have observed also that students’ engagement in self and peer assessment has improved significantly as because of their familiarity of the rubrics. They can clearly articulate their learning with reference to the criteria and standards.

The Resources Needed

Further to professional development, we recognised the need to build resources that will support teachers. We work collaboratively, to identify what resources will be accessible for teachers. Apart from the common rubrics discussed above, we have developed resources aligned to the domains of teachers’ AfL literacy. We have developed various resources including background information about the program, various AfL practices, forums for raising questions and issues, a blog for sharing teachers’ best practices and links to various empirical evidence. We put it in a secure website for accessibility and convenience. The frontpage of the website is shown in Figure 1.

FIGURE 1

FIGURE 1. List of resources developed to support teachers.

Observable Outcomes

As a result of the school processes, practices, policy and people leading the assessment reform in the school, teachers became more capable of understanding and using assessment data to guide learning.

This is evident in the discussion during Stage meetings on how teachers refer to student data as their bases for adapting their teaching. The conversation is focused around the clarity of the aims of eliciting and gathering different types of data, making sense of these data and identifying specific actions with the aim to support individual students. We have observed also that teachers now are more confident to use their agency to trial and use more formative assessment practices across the school. Whatever assessment strategy teachers use, it became a school expectation to record observations on formative assessments grids, scaffolds or the teacher's personal record book in all lessons. After two years of implementation, data about students learning has begun to shift the learning and teaching within the classroom. My observation as an Instructional Leader is that, the more the teachers learn about individual students in terms of their background, learning development and needs, then they are able to provide specific feedback that further scaffolds student learning. The students themselves become teachers of their own learning. They became better at self-assessment and self-directed learners.

This is where the teacher decision making process has started to impact student learning. When the teacher uses classroom-based assessment data including anecdotal records, they gain a deeper understanding of student needs and are better able to effectively differentiate the school’s stage-based teaching programs. Teachers have the knowledge of individual students and are able to flexibly adjust various aspects of learning, teaching and assessment activities to account for individual differences whilst meeting high expectations.

Student goal setting across the whole school has become a new school target. Teachers work closely with students to set their learning goals. Through this process, students understand what they need to learn and how success looks at each lesson. This develops students’ independence, accountability and responsibility into their learning success. It gives them a clear guide to know when they are meeting the learning outcomes. We have observed that those students who have clear focus on their goals, engage more on challenging tasks. The conversation with their teachers around their progress using data as evidence enable them to establish their next learning goals. This on-going conversation with individual students enables teachers to see the gaps in student knowledge rather than at the end of the term when post-test is administered. This in-class data gathering gives more data for teachers to use to support students.

Teachers link the goals of the students to the National Literacy and Numeracy Learning Progressions. Setting the goals on the Progressions ensured teachers were able to accurately track student attainment of these goals. It also supported whole school consistency in embedding school-based student data in a central and easily accessible location that stays with the student throughout their schooling years.

The assessment data became the focus for engaging parents as well. Parent feedback forms containing learning progress and learning goals are sent home to seek inputs on the learning goals of their children. This is a process that is valued across the whole school community.

Discussion

Our reflection in leading a whole school approach to building an assessment culture with a strong focus on using a range of data for teacher decision-making highlights the different aspects of the program that contribute how it is gaining a significant traction. Based on our reflection, developing teacher decision-making knowledge and skills are influenced by system, organizational structure and interpersonal factors (Marsh and Farrell, 2015; Schildkamp, 2019). These factors are enabled by people leading the reform, processes institutionalised, tools developed and used, and principles adhered to.

Using activity theory, the first requisite for building teachers’ decision-making skills is ensuring that everyone in the school has common understanding of the principles of effective assessment practices. For teachers, various contributary factors build their AfL literacy, which consequently enable them to engage in data-driven decision making processes starting with eliciting and gathering individual students’ learning to making sense of the different types of data and use any insights to inform critical decisions related to improvising student learning and outcomes (Schildkamp, 2019).

The clarity of the aims of the program and consistency of implementation across the school contribute to the development of common understanding among teachers, students and parents. This is a critical phase of the program implementation where all teachers have high level of understanding of not only the aims of the program but also the principles of effective assessment practices (Davison, 2013). The Instructional Leader, Assessment Leaders and teachers are the subjects of this initiative. The IL and ALs play critical roles in leading and implementing the assessment plan and they work collaboratively to achieve the aims of the program. They provide the social structure needed for the successful implementation of the program (Poulton, 2020). They serve as mentors to model assessment practices and also monitor the consistency of practice and implementation across school as research shows a lack of leadership in assessment reform contributes to its failure (Marsh and Farrell, 2015).

The tools co-developed by these subjects and by the university partner facilitated the development of the common understanding and language of assessment across the school. The availability of the tools is important for supporting teachers to successfully implement their PD assessment goals. For example, the resources provide teachers with materials to deepen their theoretical and empirical knowledge in assessment and decision-making. The common rubrics develop consistency of judgment among teachers, which ensures the trustworthiness of assessment decisions. The co-design process provides the initial link between theory and practice where research evidence and teacher experience are used as inputs for the development of different tools. Closing the gap between theory and practice is an important consideration in assessment reform (Oo, 2020). The effectiveness of the tools depends on their accessibility. For example, the Essential Assessment becomes a handy tool for all teachers that they use anytime they want to check the progress of their students. The design of this assessment tool contributes to its adoption because it lessens teachers time to mark, analyses results and identifies micro skills that individual students have achieved and suggests learning goals. Another key to the effectiveness of these tools lies on creating a culture of continuous improvement of these tools. The opportunity during stage meetings to reflect on what aspects of the tools need to be revised values individual teachers’ voice and hence, creating a culture of trust.

The internal mechanisms constituted by the rules like the use of common rubrics, stage meetings, embedding of assessment goals to PDP goals, moderation and classroom observations emerged through collective agreement. These are not imposed rules but rather developed through respectful and dialogic conversation with teachers, thus more likely to be acted on by all teachers. They are agreed with a common understanding that these rules will help everyone in achieving the aims of the program. These rules have created clarity of expectations and built positive relationships amongst IL, ALs and teachers which has contributed greatly to the success of the program (Poulton, 2020). Through these rules, everyone becomes responsible and accountable of student learning. Any rules that are counter-intuitive and are not supporting the aim of the program are discussed and modified.

The university partnership and participation in a wider school network’s activity created the wider community for collaboration, which facilitates translation of theory into practice and sharing of best assessment practices. The collaboration between university partner and the school allow for the critique of research evidence and how it can be applied in the school context. IL, ALs and teachers try some strategies and then later evaluate their effectiveness (a separate paper on university-school partnership is in-preparation to highlight this critical aspect of assessment reform). Drawing from their experience they can verify the theoretical knowledge generated from research in the university. Through this process, research outputs are translated into practical skills for teachers, which in turn enhances their AfL practices and decision-making. The community and the relationship created provide the environmental factors that facilitate the growth of the program (Marsh and Farrell, 2015).

The participation of different key people supporting the program with the responsibilities clearly articulated is a division of labour which provides the critical personnel to implement and evaluate every aspect of the program. The specific roles played by the leadership team including the principal and school executives, the allocation of resources and the funding and appointment of assessment leaders are the school-level factors underpinning its success (Cosner, 2011). The trust of the principal and the clarity of roles of the IL and ALs have contributed to their capacity to take full responsibility of the program. Whilst strong support is provided to teachers, IL and ALs encourage teachers to use their agency to try implement any assessment strategies that they think could help their students based on the data. The degree of flexibility and tapping into teacher agency facilitate changes in teacher practices (Priestley et al., 2012). This flexibility allows for teachers to develop their adaptability in assessment, an important consideration to ensure effective implementation of assessment (Loughland and Alonzo, 2019). In this process, teachers are constantly reflecting on how assessment can be best implemented in different context. This adheres to the context-drive nature of assessment.

All these factors are illustrated in Figure 2.

FIGURE 2

FIGURE 2. The interrelationships of the factors that contribute to the success of the program.

Conclusion

Although this paper is based on our collective reflection only, it provides an extensive overview on how to lead a school-wide assessment reform to build a strong assessment culture that can grow teachers’ capacity in decision-making. We have demonstrated the function of various tools, the school commitment and internal processes, the partnership created to support the school and the roles of the key players. To further substantiate our claims, however, empirical data is needed to be gathered to provide evidence of the impact of the program on the practices of teachers and how their practices increases student outcomes, which is the focus of our current work.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author Contributions

DA: Conceptualisation, methodology, analysis and interpretation, writing-reviewing and editing. JL: Conceptualisation, methodology, analysis and interpretation, writing-reviewing and editing. EO: Conceptualisation, analysis and interpretation, writing-reviewing and editing.

Conflict of Interest

DA is the editor of this journal. He inhibited in the process of editorial review.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Alonzo, D. (2016). Development and application of a teacher assessment for learning (AfL) literacy tool. Available at: http://unsworks.unsw.edu.au/fapi/datastream/unsworks:38345/SOURCE02?view=true (Accessed July 20, 2020).

Google Scholar

Ambrosini, V., and Billsberry, J. (2007). “Person-organisation fit as anamplifier of tacit knowledge,” in Paper presented at the 1st global e-conference on fit.

Google Scholar

ARG (2002). Assessment for learning: 10 principles. London, United Kingdom: Nuffield Foundation.

Black, P., Harrison, C., Lee, C., Marshall, B., and Wiliam, D. (2003). Assessment for learning: putting it into practice. Oxford, United Kingdom: Oxford University Press.

Black, P., and Wiliam, D. (1999). Assessment for learning: beyond the black box. Cambridge, United Kingdom: University of Cambridge School of Education.

Bowers, A. J. (2009). Reconsidering grades as data for decision making: more than just academic knowledge. J. Educ. Adm. 47, 609–629. doi:10.1108/09578230910981080

CrossRef Full Text | Google Scholar

Campbell, E. (2016). Exploring autoethnography as a method and methodology in legal education research. Asian J. Leg. Educ. 3 (1), 95–105. doi:10.1177/2322005815607141

CrossRef Full Text | Google Scholar

Cosner, S. (2011). Teacher learning, instructional considerations and principal communication: lessons from a longitudinal study of collaborative data use by teachers. Educ. Manag. Adm. Leadersh. 39 (5), 568–589. doi:10.1177/1741143211408453

CrossRef Full Text | Google Scholar

Davison, C. (2013). “Innovation in assessment: common misconceptions and problems,” in Innovation and change in English language education. Editors K. Hyland, and L. Wong (Oxon, United Kingdom: Routledge), 263–275.

Google Scholar

Davison, C., and Leung, C. (2009). Current issues in English language teacher‐based assessment. TESOL Q. 43 (3), 393–415. doi:10.1002/j.1545-7249.2009.tb00242.x

CrossRef Full Text | Google Scholar

Davison, C. (2007). Views from the chalkface: English language school based assessment in Hong Kong. Lang. Assess. Q. 4 (1), 37–68. doi:10.1080/15434300701348359

CrossRef Full Text | Google Scholar

Dhanaraj, C., Lyles, M. A., Steensma, H. K., and Tihanyi, L. (2004). Managing tacit knowledge and explicit knowledge transfer in IJVs: the role of relational embeddedness and the impact on performance. J. Int. Bus. Stud. 35 (5), 428–442. doi:10.1057/palgrave.jibs.8400098

Google Scholar

Ellis, C., Adams, T., and Bochner, A. (2010). Re: forum qualitative sozialforschung/forum: qualitative social research. Forum Qual. Soc. Res. 12, 1–8. doi:10.17169/fqs-12.1.1589

CrossRef Full Text | Google Scholar

Engeström, Y. (1987). Learning by expanding: An activity-theoretical approach. Helsinki: Orienta- Konsultit.

Flores, M. A. (2005). How do teachers learn in the workplace? findings from an empircal study carried out in Portugal. J. In-service Educ. 31, 485–508. doi:10.1080/13674580500200491

CrossRef Full Text | Google Scholar

Foos, T., Schum, G., and Rothenberg, S. (2006). Tacit knowledge transfer and the knowledge disconnect. J. Knowl. Manag. 10 (1), 6–18. doi:10.1108/13673270610650067

CrossRef Full Text | Google Scholar

Goodall, B. H. L. (2001). Writign the new ethnography. Walnut Creek, CA: AltaMira.

Hattie, J. (2008). Visible learning: a synthesis of over 800 meta-analyses relating to achievement. Hoboken, NJ: Routledge.

CrossRef Full Text

James, M., Black, P., Carmichael, P., Drummond, M. J., Fox, A., MacBeath, J., et al. (2007). Improving learning how to learn: classrooms, schools and networks. London, United Kingdom: Routledge.

CrossRef Full Text

Jensen, B. (2011). Better teacher appraisal and feedback: Improving performance. Retrieved from Grattan Institute, Melbourne: Available at https://grattan.edu.au/wp-content/uploads/2014/04/081_report_teacher_appraisal.pdf

Kippers, W. B., Wolterinck, C. H. D., Schildkamp, K., Poortman, C. L., and Visscher, A. J. (2018). Teachers’ views on the use of assessment for learning and data-based decision making in classroom practice. Teach. Teacher Educ. 75, 199–213. doi:10.1016/j.tate.2018.06.015

CrossRef Full Text | Google Scholar

Lave, J., and Wenger, E. (1991). Situated learning: legitimate peripheral participation. Cambridge, United Kingdom: Cambridge University Press.

CrossRef Full Text

Loughland, T., and Alonzo, D. (2019). Teacher adaptive practices: a key factor in teachers’ implementation of assessment for learning. Aust. J. Teach. Educ. 44 (7), 18–30. doi:10.14221/ajte.2019v44n7.2

CrossRef Full Text | Google Scholar

Lunenberg, M., Murray, J., Smith, K., and Vanderlinde, R. (2017). Collaborative teacher educator professional development in Europe: different voices, one goal. Prof. Dev. Educ. 43 (4), 556–572. doi:10.1080/19415257.2016.1206032

CrossRef Full Text | Google Scholar

Mandinach, E. B., and Gummer, E. S. (2013). A systemic view of implementing data literacy in educator preparation. Educ. Res. 42 (1), 30–37. doi:10.3102/0013189x12459803

CrossRef Full Text | Google Scholar

Mandinach, E. B., and Gummer, E. S. (2016). What does it mean for teachers to be data literate: laying out the skills, knowledge, and dispositions. Teach. Teacher Educ. 60, 366–376. doi:10.1016/j.tate.2016.07.011

CrossRef Full Text | Google Scholar

Marsh, J. A., and Farrell, C. C. (2015). How leaders can support teachers with data-driven decision making. Educ. Manag. Adm. Leadersh. 43 (2), 269–289. doi:10.1177/1741143214537229

CrossRef Full Text | Google Scholar

McIlveen, P. (2008). Autoethnography as a method for reflexive research and practice in vocational psychology. Aust. J. Career Dev. 17 (2), 13–20. doi:10.1177/103841620801700204

CrossRef Full Text | Google Scholar

Oo, C. Z. (2020). “Assessment for learning literacy and pre-service teacher education: Perspectives from Myanmar. PhD thesis. Sydney, Australia:School of Education, University of New South Wales.

Google Scholar

Osterloh, M., and Frey, B. S. (2000). Motivation, knowledge transfer, and organizational forms. Organ. Sci. 11, 538–550. doi:10.1287/orsc.11.5.538.15204

CrossRef Full Text | Google Scholar

Popham, W. J. (2009). Assessment literacy for teachers: faddish or fundamental? Theory Pract. 48 (1), 4–11. doi:10.1080/00405840802577536

CrossRef Full Text | Google Scholar

Popham, W. J. (2011). Assessment literacy overlooked: a teacher educator’s confession. Teacher Educ. 46 (4), 265–273. doi:10.1080/08878730.2011.605048

CrossRef Full Text | Google Scholar

Poulton, P. (2020). Teacher agency in curriculum reform: the role of assessment in enabling and constraining primary teachers’ agency. Curric. Perspect. 40 (1), 35–48. doi:10.1007/s41297-020-00100-w

CrossRef Full Text | Google Scholar

Priestley, M., Edwards, R., Priestley, A., and Miller, K. (2012). Teacher agency in curriculum making: agents of change and spaces for manoeuvre. Curric. Inq. 42 (2), 191–214. doi:10.1111/j.1467-873X.2012.00588.x

CrossRef Full Text | Google Scholar

Schildkamp, K. (2019). Data-based decision-making for school improvement: research insights and gaps. Educ. Res. 61 (3), 257–273. doi:10.1080/00131881.2019.1625716

CrossRef Full Text | Google Scholar

Timperley, H., Wilson, A., Barrar, H., and Fung, I. (2008). Teacher professional learning and development: best evidence synthesis on professional learning and development. Available at: https://researchspace.auckland.ac.nz/bitstream/handle/2292/12537/TPLandDBESentire.pdf?sequence=4 (Accessed June 16, 2020).

Google Scholar

van Geel, M., Keuning, T., Visscher, A. J., and Fox, J.-P. (2016). Assessing the effects of a school-wide data-based decision-making intervention on student achievement growth in primary schools. Am. Educ. Res. J. 53 (2), 360–394. doi:10.3102/0002831216637346

CrossRef Full Text | Google Scholar

Vygotsky, L. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.