Supporting a state in developing a working theory of improvement for promoting equity in science education

Introduction: This study was undertaken to explore the potential of developing a working theory of improvement for creating a more equitable system of science education at the level of a US state. We ask: How can tools from a long-term research-practice partnership support a state team in initiating improvement research toward promoting a more equitable system of science education?

Methods: This design study took place in winter 2024 in a single state. External partners supported leaders of a single state in the US Northeast to support a process of articulating aims, specifying primary and secondary drivers, and identifying change strategies to promote a more equitable system of science education in the state, grounded in the vision of A Framework for K-12 Science Education (National Research Council, 2012). In this paper, we rely on descriptive analyses of joint meetings and a focus group with state leaders describe the tools supporting the process of development, the team’s use of the tools to generate an early draft of the Driver Diagram, and issues surfaced while developing it with a team of interest holders in the state.

Results: Two meanings of equity emerged as significant within the series of meetings: that of the importance of universal access to professional learning and the importance of students having opportunities to experience culturally relevant instruction. The issues surfaced highlighted the need for infrastructures for professional learning to reach a diverse group of interest holders in science, including teachers, school leaders, and district leaders across the state. They also saw curriculum materials that connect to students’ everyday lives and community priorities as key drivers for equitable change in the system, around which professional learning activities should be organized. The team also surfaced several policy changes needed to implement change strategies, only some of which team members felt they had some authority.

Discussion: Where past researchers have observed that equity can disappear as a focus during implementation of reforms, this study found that developing an aim statement and driver diagram helped energize and refocus a team’s implementation efforts geared toward a vision for science teaching and learning that is focused on ensuring all students can engage in meaningful science learning that is culturally and locally relevant to them.

1 Introduction

“There is nothing so practical as good theory” (Lewin, 1943, p. 118) might be an apt characterization of the aim of theory in improvement research in education. The focus and use of theory in improvement research is different from other forms of basic and applied social research, in that it is intended to directly inform the design of improvement strategies and support the agency of participants in design (Eddy-Spicer and Penuel, 2022). Further, theory is intended to help a network to develop a common language to describe its shared work, a common language to which members can identify with and commit to (Russell et al., 2017; Bennett and Provost, 2015). As work unfolds, representations of theories serve as places to record conjectures, questions, successful practice ideas, as well as agreements of a network (Thompson et al., 2019; Sandoval and van Es, 2021).

For theory to guide improvement that seeks to transform educational systems so that they are more just for students of all races, ethnicities, genders, classes, and abilities, more explicit ideas about educational equity and how to promote it are needed. These include ideas related to how to expand opportunities to learn valuable disciplinary knowledge and skills, as well as how to transform teaching so that it connects more explicitly with the cultural and linguistic funds of knowledge of students (National Academies of Sciences Engineering and Medicine, 2022). More broadly, improvement research must ask questions related to “how,” “for whom,” and “with whom” in concert with questions about who defines improvement and who decides the purposes for change (Philip et al., 2018).

A focus on educational equity is relatively new within improvement research in education (Jabbar and Childs, 2022) and presents distinct challenges to developing practical theories to guide improvement. For one, it pushes us to think beyond the “small changes” called for in many guides to improvement and toward transformation (Engeström, 2017). It also requires teams that engage in coordinated changes of system-level infrastructures with multiple kinds of interest holders. And it requires a particular kind of “working theory” that is not only adaptive to what is being learned through change efforts, but also one that anticipates and adapts to the turbulence, turnover, and policy feedback loops inherent in equity-oriented change efforts (Green, 2014; Oakes and Rogers, 2007; Daly et al., 2016).

This design research study addresses the question: How can tools from a long-term research-practice partnership support a state in developing an aim statement and driver diagram to inform the choice of change strategies to promote a more equitable system of science education? Here, tools refer to cultural artifacts that mediate improvement processes. The paper describes the work and dilemmas of a research-practice team working closely with leaders of a single state in the US Northeast to support a process of articulating aims, specifying primary and secondary drivers, and beginning to identifying change strategies to promote a more equitable system of science education in the state, grounded in the vision of A Framework for K-12 Science Education (National Research Council, 2012). We present both academic theories informing the work, as well as the emergent “working theory” of the group, situating within the broader policy and political context for this work. Finally, we present a collective analysis of perceptions of the value, tensions, and limits of the work for promoting equitable change at the level of a state system of science education.

2 Conceptual framework

Our conceptual framework encompasses a vision for equitable teaching and learning in the classroom, as well as a framework for organizing for change across levels of a system in a state to achieve that vision. Developing a vision for equity, that is, for ensuring that adults in systems bear a collective responsibility for the education of each student, regardless of background, is a key leadership practice (Galloway and Ishimaru, 2015).

2.1 Vision for equitable teaching and learning in the classroom

The broader reform vision guiding this state’s improvement effort comes from principles described in A Framework for K-12 Science Education (National Research Council, 2012), the National Academies consensus volume that became the basis for the development of the Next Generation Science Standards (NGSS; NGSS Lead States, 2013). Those principles are that (a) children are born investigators, (b) science education should focus both on ideas and practices, (c) student understanding develops incrementally over many years, (d) science and engineering require both knowledge and practice, (e) instruction should connect to students’ interests and experiences, and (f) science education should promote equity.

The Framework emphasizes the need for teaching to connect with students’ interests, identities, and experiences to promote equity. The overarching commitment is expressed in the idea that:

[A] major goal for science education should be to provide all students with the background to systematically investigate issues related to their personal and community priorities. They should be able to frame scientific questions pertinent to their interests, conduct investigations and seek out relevant scientific arguments and data, review and apply those arguments to the situation at hand, and communicate their scientific understanding and arguments to others. (p. 278)

Accomplishing this aim requires addressing different sources of inequity at both the classroom and system levels. In the classroom, there is a need to redesign instruction to better grow out of students’ lived experiences and to leverage students’ cultural funds of knowledge and diverse ways of communicating (National Research Council, 2012, p. 284; Bang et al., 2017). At the school and district level, leaders must address differences in opportunities to learn. As other scholars have pointed out (Bullock, 2017; Tate, 2001), in STEM education such differences are highly racialized, and so addressing them demands attention to racial and ethnic differences in opportunities to learn.

In practice, policy makers, practitioners, and researchers rely on different discourses or frames for equity (National Academies of Sciences Engineering and Medicine, 2022; National Academies of Sciences Engineering and Medicine, 2024). Sometimes, the pursuit of equity aims for reduction in achievement gaps, while for others, it aims for expanded opportunity within existing systems and models of teaching and learning (e.g., through advanced coursework). Still others advocate for expanding science education to support heterogeneous ways of knowing that reflect students’ family and cultural ways of knowing, being, and valuing. And some discourses focus on justice, emphasizing how science education should help students see how science is implicated in movements for social and ecological justice, or how science can bring about more sustainable futures. When considering aims and strategies for improvement, these equity lenses can be powerful tools for both guiding design and reflecting on the comprehensiveness and coherence of improvement theories (National Academies of Sciences Engineering and Medicine, 2024).

2.2 Organizing for equitable change across levels of a system

Some scholars have compared the development of and advocacy for the NGSS to a social movement (Haverly et al., 2022). In fact, science education leaders in the country relied on strategic alliances with colleagues within professional networks, as well as intentional political distancing from the contested Common Core standards, to help build broad support for the adoption of the NGSS (Hardy and Campbell, 2020; Hardy and Campbell, 2022). These networks were a source of both practical guidance and research findings that leaders used politically to gain support for the vision of the Framework (Hopkins et al., 2018).

For such efforts to succeed, leaders—whether inside government or outside it—must perceive themselves to have agency. As agents, they need to act to identify and frame problems in ways that gain the attention of decision makers, while also articulating how solutions can address those problems, taking advantage of time-limited windows of opportunity to do so (Majone, 1989; Kingdon, 1984/2010). They also need to be able to articulate solutions in a way that energizes and organizes others to support broad implementation of policies, once enacted (Mintrom, 1997; Smith, 1991; Mintrom and Vergari, 1996). They need an understanding of the ideas, motives, and concerns of others in their local policy contexts and skill in responding to them (Mintrom, 1997). If government leaders working inside state level systems can move beyond seeing themselves as simply implementers of others’ decisions (cf., Weiss, 1980), to agents who can influence distributed, coordinated decision making processes to highlight and disrupt inequitable policies (Galloway and Ishimaru, 2015), the vison set forth in the Framework is more likely to be fully realized.

For leaders to promote equity-oriented change, leaders need additional capabilities for developing and holding visions that require imagination of possibilities for systems that do not yet exist. In practice, it is common for the horizons of change to be limited to what can be implemented in systems as they currently are, and for change visions to constrict to focus on what is possible now, sacrificing vision for more radical change (Chang and Philip, n.d.; Nikolakis, 2020). Conversely, holding a sense of possibility for an education system in which all students experience meaningful science learning that connects to their interests, identities, and experience can matter for moving systems toward equity. Stable leadership is also beneficial in promoting equity-oriented change. A comparative case study of six state systems found states with relatively stable leadership (meaning limited turnover) and constancy in vision were able to implement more equity-focused projects in their states than states with high turnover and little consistency with respect to a vision for equitable teaching and learning in science (Wingert et al., 2020).

More broadly, improvement research in education, as we seek to illustrate with this study, is a collaborative approach that can facilitate the development of both agency and a bold vision for equitable change. It is an approach that needs to be guided by both theory and the practical and political constraints of state leaders and their partners, as well as by a sense of possibility and responsibility for change. Here, we illustrate how the development of aim statements and Driver Diagrams can facilitate such activity.

3 Study context and methods

This study is an example of improvement research that took place within a larger, long-term research-practice partnership in education. It is a descriptive case study aimed principally at describing the process of engagement between researchers and state leaders and their reflections on the process, obtained through a focus group. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

3.1 Local policy context and practice team

The current study took place more than 10 years after the publication of the NGSS and 8 years after the adoption of the standards by this US Northeastern state. The Framework for K-12 Science Education (National Research Council, 2012) was the basis for these standards, and the Next Generation Science Standards (NGSS) or standards based on the Framework have been adopted by 49 states and the District of Columbia. The policy context is one of broad importance to science education, but also one in which there is reflection and renewed attention to what is needed to support implementation of the vision of the Framework equitably (National Academies of Sciences Engineering and Medicine, 2022).

While not one of the 26 lead states that helped to develop the NGSS, the focal state was an early adopter of the standards. Advocacy for the adoption of the NGSS was widespread in the state from different interest holders (e.g., state office of education representatives, higher education scholars, district leaders, teachers, and parents). Beyond the early adoption, the State Department of Education was able to leverage funding from a Math Science Partnership (MSP) Grant to engage higher education, district leaders, and teachers across the state in learning more about the NGSS and creating three-dimensional curriculum materials that could be implemented in classrooms across the state.

Building on the MSP grant project, the science specialist at the state education agency partnered with the state science center to offer NGSS-aligned professional learning opportunities for science teacher leaders and teachers across the state. In all, 1,765 individuals participated in these professional learning opportunities prior to the outbreak of the COVID-19 pandemic in early 2020.

The pandemic and its aftermath proved to be an inflection point in implementation. As was true for implementation efforts across the US, the COVID-19 pandemic shifted attention away from concerted efforts to support implementation of the NGSS across the state. After the pandemic, concern over “learning loss” in core subject areas of reading and mathematics dominated policy attention and funding decisions at the state and local level. This period also ushered in turnover at the state department of education and in district science leadership across the state. As a result, it was not until August 2023 that attention was given to rebooting efforts to support the implementation of the NGSS in the state. This set the stage for and provided the context in which this work of a state articulating aims, specifying primary and secondary drivers, and beginning to identify change strategies to promote a more equitable system of science education, is situated.

A State Leadership Team (SLT) carried out the work described in this community case study. It included 18 different interest holders that represented some of the key constituencies in science education in the state, including three state leaders from the department of education, and one representative from another state agency. Typically, state leaders in science education have responsibilities for standards development and for collaborating to develop or contract for accountability testing in science (Hopkins, 2016). The SLT also included one representative from a professional development provider funded as part of a regional service center; nine curriculum leaders at the district level from both large urban districts and medium-sized districts; and two university-based faculty active in both preservice and in service teacher education in science, one of whom is an author of this paper; one principal; and one science teacher. Among the local education leaders were also leaders of the state teachers’ association in science and the state’s science supervisors’ association.

All the SLT members were active in the state’s NGSS implementation efforts prior to the COVID-19 pandemic. As efforts in August 2023 turned to rebooting the state-level support for the implementation of the NGSS, the group reassembled, even though their earlier engagement in state-level NGSS implementation support was not identical to the roles they took on as part of the SLT. With the SLT reassembled, state leaders reached out to leaders of a research-practice partnership to initiate the work that is reported in this case study and described in more detail next.

3.2 External partner: ACESSE research-practice partnership

The first two of the authors of this paper provided support to the state in the case study to reboot efforts to implement the NGSS through an established research-practice partnership among two universities, a professional organization, and state education agency leaders operating in a Network Improvement Community focused on improving equity and coherence in state systems of education (Penuel et al., 2018). The aim of this NSF-funded partnership, known as Advancing Coherence and Equitable Systems of Science Education (ACESSE), focused on supporting states with advancing both vertical and horizontal coherence within their state systems of science education. Vertical coherence refers to the ideal that people across different levels of a state education system share a common vision for teaching and learning; horizontal coherence refers to alignment among key components of infrastructure that guides teaching, such as standards, curriculum, professional development, and assessments (National Research Council, 2006). For purposes of the grant, ACESSE employed and supported states in developing a multifaceted conception of equity as involving many possible “projects” related to such issues as racial justice, challenging ableism, disrupting ableism, and supporting culture-based pedagogies (Bell, 2019).

As part of the ACESSE improvement activities, a set of resources were jointly developed and tested with state leaders to aid in their ability to craft a common vision for equitable science teaching and learning, aligned to the vision of the Framework, among diverse interest holders within their states. One of these was a protocol and workshop for developing Aim Statements. This protocol and workshop were intended to support state-level teams, such as the one that is the focus of the current case study, in articulating a measurable improvement aim for a problem to be solved and key indicators to which the team would hold itself accountable. We emphasized with states the need for aim statements to be “a declaration of a lofty purpose” intended to “inspire individuals to see themselves as part of a larger narrative—as members of a community engaged in a highly valued pursuit (Bryk et al., 2015, p. 150). After developing aim statements with state teams, we recommended and devised a protocol for constructing Driver Diagrams with them. Driver Diagrams include (1) the aim statement; (2) primary drivers that most influence the attainment of the aim; (3) secondary drivers that influence primary drivers; and (4) change strategies that are intended to influence secondary drivers. For these, we emphasized building each layer of a diagram carefully from the aim statement, starting with primary drivers reflecting classroom level practices that few state team leaders likely have the capacity to influence directly. We emphasized that they should specify secondary drivers as ones they had some authority to influence, through change strategies that they identified. As aim statements need to specify measurable improvement targets, another tool we introduced to leaders was a set of candidate Practical Measures. Practical measures provide for “direct measurement of intermediary targets” to guide evaluation of improvement ideas and inform revisions to them (Yeager et al., 2013, p. 12). As few states have systems for measuring anything other than distal outcomes like achievement (National Academies of Sciences Engineering and Medicine, 2024), the partnership engaged in an iterative process of building initial aim statements without specific targets, developing Driver Diagrams, and then choosing and gathering baseline data from Practical Measures to form the basis for refined practical measures. These different tools are all available on the project’s public website at https://sites.google.com/view/acesseproject/leadership-capacity-development-for-equity.

3.3 Data sources

A primary data source for the study were the iterations of the initial Aim Statement and the Driver Diagram developed by the SLT. The process of developing practical measures had not yet been initiated prior to the conclusion of the current study. To generate the Aim Statement and inform the development of the Driver Diagram, we also relied on an anonymous survey administered in Fall 2023 to members of the SLT.

In addition, at the conclusion of the engagement with the external partners, the second author led a focus group to elicit reflections on the process using a semi-structured interview protocol. The focus group consisted of state agency leaders who had responsibility for facilitating the process. The protocol asked leaders to reflect on what they saw as key outcomes of the process, as well as how, if at all, the improvement research tools helped them. Their reflections were shared with the entire SLT, who were invited to add additional reflections or challenge the state’s or researchers’ interpretations. One SLT member offered additional comments and reflections incorporated into the account below.

3.4 Approach to analysis of data

We used the lens of equity frames to characterize the artifacts produced by the team. Frames refer to ways that people characterize aspects of problems to be solved, ethical reasons to act, and solutions to educational problems (Coburn, 2006). Five frames identified in a recent NASEM report on equity in STEM (National Academies of Sciences Engineering and Medicine, 2024) were used as lenses here: (1) reducing gaps between groups; (2) expanding opportunity and access; (3) embracing heterogeneity in STEM classrooms; (4) learning and using STEM to promote justice; and (5) Envisioning sustainable futures through STEM.

To develop descriptions of the case we used notes from meetings and reflections from the focus groups to reconstruct significant moments in the early stages of the improvement process reported here. We identified significant moments that focused on the reported benefits and challenges of the process from the SLT or partners’ point of view. We did so, because in a research-practice partnership, a key criterion for effectiveness is supporting the practice partner’s organization in achieving its goals (Henrick et al., 2023).

4 Findings

In this section, we describe our process (Figure 1) and the key findings are integrated into the description.

Figure 1

Figure 1. Improvement initiation process.

A first step in our process was to work with the SLT to develop an initial version of an aim statement. As noted above, this draft would not include specific, measurable targets that represented an improvement from a baseline, but would be used to inform the development of a Driver Diagram. This is a deviation in some respects from standard practice in improvement research, which begins with articulating a measurable improvement goal as part of the process of articulating an aim (Bryk et al., 2015). It was necessary to pursue an iterative approach, because the state did not have a system of measures for capturing baseline data on its goals or its mediating processes.

A first challenge faced by the joint research-practice team was how to design the process of generating an aim statement to allow for maximum input from members of the SLT, and to facilitate a process that would not be unduly influenced by the fact that the state education agency was leading the process. The SLT had limited time to meet, and in a short afternoon meeting, there was the risk that a few voices might unwittingly dominate discussion. To accomplish this goal, the ACESSE team agreed to field a sample survey. The following are the anonymous three-question survey for the SLT to respond to in Fall 2023, to elicit both their ideas for an aim statement and ideas regarding the resources and policy changes that would be needed to accomplish those aims:

1. What things would you like to see happening in ALL [state] classrooms 5 years from now? Be sure to say what students would be doing and also say how the teacher would be supporting the work.

2. What resources or activities would teachers need to make that vision a reality?

3. What changes to school, district, and state policies would be needed, so that ALL teachers could access those resources and activities? FOCUS ON CHANGES THAT YOU THINK ARE POSSIBLE, EVEN IF DIFFICULT TO ACHIEVE.

We chose to focus on SLT’s members visions, because past research shows that these are important guides for change (Munter and Correnti, 2017), and answers to questions about vision can help focus efforts to promote educators’ growth (Penuel et al., 2020). Further, we focused on what people felt they had agency and capacity to achieve, since these are important constraints on the scope of improvement work (Bryk et al., 2015).

Twelve of the 15 SLT members responded to the survey, and from the survey, the first two authors of the paper drafted a possible “starter” aim statement to reflect common ideas across the responses. These represented all but the team members from the state education agency, who purposefully chose not to participate. We also drafted some initial ideas about primary and secondary drivers that the SLT might incorporate into their Driver Diagram, based on our own previous syntheses of research related to the broad aims, which focused on both the roles of curriculum and professional learning (Lynch et al., 2019; Gonzalez et al., 2022). We note this is a different approach from what has been used in prior efforts to construct causal analyses of systems using tools like a Fishbone Diagram (Ishikawa, 1968). However, it reflects the need for infusing evidence into decision making related to improvement, a key demand of contemporary educational policies (Haskins et al., 2009).

We next met with the leaders of the SLT to help them plan a meeting in which they could work on revising the initial language for the aim statement, to be followed by another meeting of the SLT to develop the Driver Diagram. We suggested presenting the initial aim statement as “sacrificial offering” to be worked on and revised by the group.

Again, time was a consideration in adopting this approach: presenting a preliminary Driver Diagram was taken primarily because time was limited and coming up with a statement “from scratch” was not feasible. A more desirable approach was to begin with a draft aim that reflected the individual thinking of SLT members that emerged from the survey. At the same time, the meeting with the SLT to consider and revise the aim statement unfolded in a way that was largely as intended and, to the state leaders at least, successful. On the one hand, state leaders observed that the team spent a lot of time wrestling with specific aim statement language, more so than they had expected initially. But they also concluded that while it was a little “tedious,” the time spent revising the aim statement was well-worth it, because it built the kind of commitment and sense of purpose that they hoped for in bringing the SLT back together. Their aim statement was in two parts, one focused on students and the other focused on teachers:

All students engage in discourse with peers, ask questions, explore related ideas, and use science and engineering practices to make sense of phenomena and solve problems relevant to their lives.

All teachers have capacity and are actively supporting student sensemaking with culturally- and locally-relevant three-dimensional learning opportunities supported by high-quality instructional materials.

The pair of aim statements reflects two equity frames. The first is that of expanding opportunity and access, as reflected in the language, “all students” and “all teachers.” Notably, the first aim statement reflects goals beyond access, calling for students to engage in meaningful participation in learning activities to make sense of phenomena, one of the “equity projects” emphasized in the ACESSE project. Another conception of equity reflected in the second aim statement relates to embracing heterogeneity, with the SLT articulating that teachers know how to engage in culturally relevant teaching.

To the leaders of the state team, settling on the two-part aim statement was “really key” for the SLT, both in focusing their attention and in energizing the group. As one state leader put it,

I was so impressed with the work that the synergy, the energy, the collaboration in that room was palpable. It really was something special, and I hadn’t felt that in a long, long time. It validated the whole process for me when we walked away with our aim statements. As this was the right place, the right time, and the right focus for what we need to do.

Once the team settled on an aim statement, the first authors presented a partially developed Driver Diagram for state leaders to use in their next SLT meeting and reviewed a plan for how to build out the Diagram. In the next meeting, the SLT elicited several primary and secondary drivers, but after we reviewed their work, we observed there were a mix of ideas for change strategies, with little distinction between primary and secondary drivers. Reflecting on the meeting, the state leaders had lots of questions: “What do we mean by primary driver? What do we mean by secondary driver? How do they interact together? How do we leverage them?” This uncovered a different kind of challenge that is common in improvement research, namely helping improvement teams learn the language and tools of improvement research (Russell et al., 2017).

In a meeting after generating the draft with the external partners, the authors helped the team sort through the initial draft and provided some additional guidance regarding Driver Diagrams. We clarified the concept of primary and secondary drivers, emphasizing the need to focus on primary drivers most proximal to impacting students and teachers and secondary drivers as (Lewin, 1943) processes, tools, and resources that can influence the primary drivers and (Eddy-Spicer and Penuel, 2022) that the SLT members could influence through enactment of change strategies. We also took their initial Driver Diagram and re-arranged some of the elements and combined others, so that they could see what we meant by the distinction between primary and secondary drivers, using the SLT members’ ideas as the foundation for the Diagram.

State leaders indicated that the subsequent SLT meeting was much more successful, with the team taking time to revise the Diagram to include links among aims, primary drivers, and secondary drivers. The team organized the drivers by actors, with primary driver actors being building level administrators, instructional leaders, and teachers. Secondary driver actors included roles represented in the SLT itself (i.e., state agencies, support agencies—including regional professional learning providers, and districts) (Figure 2).

Figure 2

Figure 2. Draft Driver Diagram.

Notably, different equity frames show up only in some of the actors’ responsibilities. These include state agency representatives who are to “identify and cultivate external partnerships supportive of Principal, Coaches/Dept Head and Teacher PL, knowledge of cultural and local knowledge/assets” (heterogeneity frame) and district leaders who are to “develop or identify professional learning offerings accessible to all teachers” (opportunity and access frame). Additionally, equity-based responsibilities show up in the roles of local instructional leaders, such as science coaches, who are to “provide or arrange structured practice-based professional learning for/with science teachers anchored around culturally relevant and place-based high-quality instructional materials” and teachers who are to participate in professional learning that helps them support special education students and emergent multilingual learners (heterogeneity frame). At the same time, the fact that each actor does not have a specified role in promoting equity implies more elaboration of the Driver Diagram and Aim Statements are needed, particularly since a key assumption needed to guide equity-oriented work in STEM is that everyone has a role to play in the system to promote equity (National Academies of Sciences Engineering and Medicine, 2024).

The process of developing a Driver Diagram led to important, but also challenging questions for the state leaders to consider. As one leader put it,

What should the scope be, how far do we go? From sort of the classroom out to boards of education, State board, legislature? You know those types of bigger policy issues? You know, you're getting farther away from the classroom, but those things can still influence it. You know. What about external partners, those types of things, other organizations or private industry?

At the same time, it provoked several questions about the limitation of the SLT’s authority and capacity. One state leader asked, “What do we need to do to be able to bring people in that can impact the places that we cannot right now as a team?” and “How do we identify what is doable and possible?” These, they said, were “the biggest challenges for us” that the process of developing a Driver Diagram provoked. Partly, these questions were provoked by the necessity to specify actors with authority and influence over key drivers, and the recognition that many of them were not at the table. But it also provoked questions about resources that would be needed to ensure, for example, that all teachers in the entire state had opportunities to learn how to engage in culturally responsive teaching.

Thus, a significant finding from the work underscores the need to take into account both agency and capacity in defining the scope of an equity-oriented improvement effort at the state level. Past studies have highlighted the increasing responsibilities given to state-level actors for improvement of teaching and learning outcomes (Smarick and Squire, 2014). However, power and authority in a state for implementing equitable change is distributed (National Academies of Sciences Engineering and Medicine, 2024), and so a team approach is required, as is greater specification of responsibilities for action within the effort we studied.

Reflecting on the whole process, SLT leaders emphasized the key role that external partners from ACESSE played in supporting the improvement work. State leaders commented that they were familiar with the tools and resources we had developed but did not feel confident enough to facilitate the process with the team. They had brought in one of the SLT team university-based faculty members, someone familiar with the state’s reform efforts and improvement methods as a co-facilitator, to add to the capacity of the team in the room.

Another SLT member said that at the conclusion of the process, “The SLT came away with a shared sense of purpose, direction, pride in their accomplishments, and excitement for the next steps; which include expanding capacity and developing Change Ideas.” This reflection, along with reflections made by team members along the way about elements of the process, underscore the power of collective participation in the process of improvement in generating commitment to improvement.

5 Conclusion

Improvement efforts do not always focus on equity, and while equity is often part of visions for reform, during implementation, it often disappears as a central focus of organizing for systemic change efforts (Bell, 2019). In this improvement engagement between a research-practice partnership and members of a single state leadership team in science education, the process of developing an aim statement and Driver Diagram helped energize and refocus a team’s implementation efforts geared toward a vision for science teaching and learning that is focused on ensuring all students can engage in meaningful science learning that is culturally and locally relevant to them. It helped them identify key actors and resources needed to influence the infrastructures that inhibit all students from having opportunities to engage in meaningful science learning. This engagement shows a case of improvement research processes helping rejuvenate implementation efforts in a way that keeps equity at the center, within a state with a strong commitment to do so.

Data availability statement

The datasets presented in this article are not readily available because these data are identifiable and would put political actors at risk. Requests to access the datasets should be directed to d2lsbGlhbS5wZW51ZWxAY29sb3JhZG8uZWR1.

Ethics statement

The studies involving humans were approved by University of Colorado Boulder Human Research and IRB. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

WP: Conceptualization, Funding acquisition, Writing – original draft, Writing – review & editing. TN: Funding acquisition, Investigation, Writing – review & editing, Writing – original draft. TC: Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This material is based in part upon work supported by the National Science Foundation under Grant Number DRL-1920249. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Acknowledgments

We wish to thank all members of the ACESSE team from the University of Washington, University of Colorado, and Council of State Science Supervisors for their intellectual leadership of the overall project, particularly Philip Bell, Principal Investigator.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Bang, M., Brown, B. A., Calabrese Barton, A., Rosebery, A., and Warren, B. (2017). “Toward more equitable learning in science: expanding relationships among students, teachers, and science practices” in Helping students make sense of the world using next generation science and engineering practices. eds. C. Schwarz, C. Passmore, and B. J. Reiser (Washington, DC: NSTA), 33–58.

Google Scholar

Bell, P. (2019). Infrastructuring teacher learning about equitable science instruction. J. Sci. Teach. Educ. 30, 681–690. doi: 10.1080/1046560X.2019.1668218

Crossref Full Text | Google Scholar

Bennett, B., and Provost, L. (2015). What’s YOUR theory? Qual. Prog. 48, 36–43.

Google Scholar

Bryk, A. S., Gomez, L. M., Grunow, A., and LeMahieu, P. G. (2015). Learning to improve: How America's schools can get better at getting better. Cambridge, MA: Harvard University Press.

Google Scholar

Bullock, E. C. (2017). Only STEM can save us? Examining race, place, and STEM education as property. Educ. Stud. 53, 628–641. doi: 10.1080/00131946.2017.1369082

Crossref Full Text | Google Scholar

Chang, M. A., and Philip, T. M. Caught between expansive world-building and the status quo: using figured worlds to understand world-building in co-design contexts. J. Learn. Sci.

Google Scholar

Coburn, C. E. (2006). Framing the problem of reading instruction: using frame analysis to uncover the microprocesses of policy implementation. Am. Educ. Res. J. 43, 343–349. doi: 10.3102/00028312043003343

Crossref Full Text | Google Scholar

Daly, A. J., Finnigan, K. S., and Liou, Y.-H. (2016). “How leadership churn undermines learning and improvement in low-performing school districts” in Thinking systemically: Improving districts under pressure. eds. A. J. Daly and K. S. Finnigan (Washington, DC: AERA), 183–208. doi: 10.3102/978-0-935302-46-2

Crossref Full Text | Google Scholar

Eddy-Spicer, D., and Penuel, W. R. (2022). “Section I introduction” in The foundational handbook on improvement research in education. eds. D. J. Peurach, J. L. Russell, L. Cohen-Vogel, and W. R. Penuel (Lanham, MD: Rowman & Littlefield), 23–28.

Google Scholar

Engeström, Y. (2017). Improvement versus transformation. Éduc. Didactique 11, 31–34. doi: 10.4000/educationdidactique.2718

Crossref Full Text | Google Scholar

Galloway, M. K., and Ishimaru, A. M. (2015). Radical recentering: equity in educational leadership standards. Educ. Adm. Q. 51, 372–408. doi: 10.1177/0013161X15590658

Crossref Full Text | Google Scholar

Gonzalez, K., Lynch, K., and Hill, H. C. (2022). A meta-analysis of the experimental evidence linking STEM classroom interventions to teacher knowledge, classroom instruction, and student achievement. Providence, RI: Annenberg Institute at Brown University, 22–515.

Google Scholar

Green, M. G. (2014). Caring leadership in turbulent times: Tackling neoliberal education reform. Charlotte, NC: Information Age Publishing.

Google Scholar

Hardy, I., and Campbell, T. (2020). Developing and supporting the next generation science standards: the role of policy entrepreneurs. Sci. Educ. 104, 479–499. doi: 10.1002/sce.21566

Crossref Full Text | Google Scholar

Hardy, I., and Campbell, T. (2022). Professional capital as political capital: science standards reform in the United States. Am. J. Educ. 128, 361–388. doi: 10.1086/719157

Crossref Full Text | Google Scholar

Haskins, R., Paxson, C., and Brooks-Gunn, J. (2009). Social science rising: A tale of evidence shaping public policy. Princeton, NJ: The Future of Children.

Google Scholar

Haverly, C., Lyle, A., Spillane, J. P., Davis, E. A., and Peurach, D. J. (2022). Leading instructional improvement in elementary science: state science coordinators' sense-making about the next generation science standards. J. Res. Sci. Teach. 59, 1575–1606. doi: 10.1002/tea.21767

Crossref Full Text | Google Scholar

Henrick, E., Farrell, C. C., Singleton, C., Resnick, A. F., Penuel, W. R., Arce-Trigatti, P., et al. (2023). Indicators of research-practice partnership health and effectiveness: Updating the five dimensions framework. Boulder, CO and Houston, TX: NCRPP and NNERPP.

Google Scholar

Hopkins, M. (2016). Findings from a survey of state science leaders. Boulder, CO: National Center for Research in Policy and Practice.

Google Scholar

Hopkins, M., Wiley, K., Penuel, W. R., and Farrell, C. C. (2018). Brokering research in science education policy implementation: the case of a professional association. Evidence Policy 14, 459–476. doi: 10.1332/174426418X15299595170910

Crossref Full Text | Google Scholar

Ishikawa, K. (1968). Guide to quality control. Tokyo, Japan: JUSE Press, Ltd.

Google Scholar

Jabbar, H., and Childs, J. (2022). “Critical perspectives on the context of improvement research in education,” in The foundational handbook of improvement research in education. eds. D. J. Peurach, J. L. Russell, L. Cohen-Vogel, and W. R. Penuel. (pp. 241–261). Rowman & Littlefield.

Google Scholar

Kingdon, J. W. (1984/2010). Agendas, alternatives, and public policies. 2nd Edn. Harlow, UK: Longman.

Google Scholar

Lewin, K. (1943). Psychology and the process of group living. J. Soc. Psychol. 17, 113–131. doi: 10.1080/00224545.1943.9712269

Crossref Full Text | Google Scholar

Lynch, K., Hill, H. C., Gonzalez, K. E., and Pollard, C. (2019). Strengthening the research base that informs STEM instructional improvement efforts: a meta-analysis. Educ. Eval. Policy Anal. 41, 260–293. doi: 10.3102/0162373719849044

Crossref Full Text | Google Scholar

Majone, G. (1989). Evidence, argument, and persuasion in the policy process. New Haven, CT: Yale University Press.

Google Scholar

Mintrom, M. (1997). Policy entrepreneurs and the diffusion of innovation. Am. J. Polit. Sci. 41, 738–770. doi: 10.2307/2111674

Crossref Full Text | Google Scholar

Mintrom, M., and Vergari, S. (1996). Advocacy coalitions, policy entrepreneurs, and policy change. Policy Stud. J. 24, 420–434. doi: 10.1111/j.1541-0072.1996.tb01638.x

Crossref Full Text | Google Scholar

Munter, C., and Correnti, R. J. (2017). Examining relations between mathematics teachers' instructional vision and knowledge and change in practice. Am. J. Educ. 123, 171–202. doi: 10.1086/689928

PubMed Abstract | Crossref Full Text | Google Scholar

National Academies of Sciences Engineering and Medicine (2022). Science and engineering in preschool through elementary grades: The brilliance of children and the strengths of educators. Washington, DC: The National Academies Press.

Google Scholar

National Academies of Sciences Engineering and Medicine (2022). Taking stock of science standards implementation: Proceedings of a virtual summit. Washington, DC: The National Academies Press.

Google Scholar

National Academies of Sciences Engineering and Medicine (2024). Equity in preK-12 STEM education: Framing decisions for the future. Washington, DC: National Academies Press.

Google Scholar

National Research Council (2006). Systems for state science assessment. Washington, DC: National Academies Press.

Google Scholar

National Research Council (2012). A framework for K-12 science education: practices, crosscutting concepts, and core ideas. Washington, DC: National Research Council.

Google Scholar

NGSS Lead States (2013). Next generation science standards: For states, by states. Washington, DC: National Academies Press.

Google Scholar

Nikolakis, W. (2020). Participatory backcasting: building pathways towards reconciliation? Futures 122, 102603–102611. doi: 10.1016/j.futures.2020.102603

Crossref Full Text | Google Scholar

Oakes, J., and Rogers, J. (2007). Radical change through radical means: learning power. J. Educ. Chang. 8, 193–206. doi: 10.1007/s10833-007-9031-0

Crossref Full Text | Google Scholar

Penuel, W. R., Bell, P., and Neill, T. (2020). Creating a system of professional learning that meets teachers’ needs. Phi Delta Kappan. 101, 37–41. doi: 10.1177/0031721720923520

Crossref Full Text | Google Scholar

Penuel, W. R., Bell, P., Neill, T., Shaw, S., Hopkins, M., and Farrell, C. C. (2018). Building a networked improvement community to promote equitable, coherent systems of science education. AASA J. Scholarship Pract. 15, 30–38.

Google Scholar

Philip, T. M., Bang, M., and Jackson, K. (2018). Articulating the “how,” the “for what,” the “for whom,” and the “with whom” in concert: a call to broaden the benchmarks of our scholarship. Cogn. Instr. 36, 83–88. doi: 10.1080/07370008.2018.1413530

Crossref Full Text | Google Scholar

Russell, J. L., Bryk, A. S., Dolle, J. R., Gomez, L. M., LeMahieu, P. G., and Grunow, A. (2017). A framework for the initiation of networked improvement communities. Teach. Coll. Rec. 119, 1–36. doi: 10.1177/016146811711900501

Crossref Full Text | Google Scholar

Sandoval, C., and van Es, E. A. (2021). Examining the practices of generating an aim statement in a teacher preparation networked improvement community. Teach. Coll. Rec. 123, 1–32. doi: 10.1177/016146812112300606

Crossref Full Text | Google Scholar

Smarick, A., and Squire, J. (2014). The state education agency: At the helm, not the oar. Washington, DC: Thomas B. Fordham Institute.

Google Scholar

Smith, J. A. (1991). The idea brokers. New York, NY: The Free Press.

Google Scholar

Tate, W. F. (2001). Science education as a civil right: urban schools and opportunity-to-learn considerations. J. Res. Sci. Teach. 38, 1015–1028. doi: 10.1002/tea.1045

Crossref Full Text | Google Scholar

Thompson, J., Richards, J., Shim, S.-Y., Lohwasser, K., Von Esch, K., Chew, C., et al. (2019). Launching PLC networks: footholds into creating and improving knowledge of ambitious and equitable teaching practices in a RPP. AERA Open 5, 1–22. doi: 10.1177/2332858419875718

PubMed Abstract | Crossref Full Text | Google Scholar

Weiss, C. H. (1980). Knowledge creep and decision accretion. Knowledge 1, 381–404. doi: 10.1177/107554708000100303

PubMed Abstract | Crossref Full Text | Google Scholar

Wingert, K., Riedy, R., Campanella, M., and Penuel, W. R. (2020). “Equity across state systems: possibilities and tensions in understanding scale” in The interdisciplinarity of the learning sciences. eds. M. Gresalfi and I. S. Horn (Nashville: TN International Society of the Learning Sciences), 2453–2460.

Google Scholar

Yeager, D., Bryk, A. S., Muhich, J., Hausman, H., and Morales, L. (2013). Practical measurement. Stanford, CA: Carnegie Foundation for the Advancement of Teaching.

Google Scholar