Empowering rural students through computational thinking and real-world STEM applications: insights from an innovative high school curriculum

Rural students often face challenges receiving high-quality education in science, technology, engineering, and math (STEM). However, without meaningful STEM educational opportunities, rural students might not develop the knowledge and skills needed to compete in a technology-driven workforce. Learning by Making (LbyM), an innovative intervention funded by the U.S. Department of Education’s Investing in Innovation and Education Innovation and Research Funds, addresses gaps in STEM learning in rural settings at the early high school level by supporting teaching and learning around computational thinking and real-world STEM applications. A randomized controlled trial (RCT) in which students were randomly assigned to the treatment (received intervention) or control group (business as usual) explored the fidelity and impact of the implementation of LbyM with 9th-grade students in three rural and three high-needs high schools in California. While the quantitative analysis did not find a significant impact on student achievement, possibly due to the small sample size and the challenges of program implementation during the COVID-19 pandemic, qualitative findings highlighted several considerations for improved rural and high-needs STEM equity. For example, the LbyM’s focus on place-built students’ ability to make sense of local phenomena by applying computational thinking and coding skills and by collecting, analyzing, and interpreting data to develop solutions to problems related to their lives. Teachers reported that the focus on real-world applications increased student engagement and self-efficacy. At the same time, LbyM built teacher self-efficacy through professional learning and sharing; teachers developed computational thinking, modeling, experiment, research, and coding skills alongside their students and increased their confidence in delivering NGSS content.

1 Introduction

In the U.S., around one-third of students are educated in rural settings, and over half of all school districts are located in rural areas (Johnson et al., 2014). Studies indicate that rural students often face challenges in receiving high-quality education in science, technology, engineering, and math (STEM) due to various obstacles, including inadequate technology, limited teacher training, and remote locations that are far from educational resources (Marksbury, 2017). The poverty rate among rural youth was reported at 21.1% in 2019, which is higher than the 16.1% observed among their urban counterparts (Davis et al., 2022). Furthermore, attracting and retaining well-qualified STEM educators in rural schools is challenging (Monk, 2007), and advanced STEM courses are seldom available in these settings (Gibbs, 2005). Rural schools, especially those facing economic hardships, often have reduced access to technology, leading to a gap in computer literacy for their students (Bouck, 2004). Without meaningful STEM educational opportunities, rural students may not develop the knowledge and skills needed to compete in a technology-driven workforce and may face barriers to future success.

In addition to challenges in rural settings, the issue of educational achievement gaps across different demographic groups continues to demand attention at national and state levels (Bohrnstedt et al., 2015). A focused effort is underway to find effective strategies that promote equity and access to education for all students (Heaster-Ekholm, 2020; Lee and Barnes, 2015; Ryoo and Calabrese Barton, 2018; Vossoughi et al., 2016). The COVID-19 pandemic further compounded these issues, disrupting the education of K-12 students on an unprecedented scale and exacerbating educational inequity throughout the U.S. (Bacher-Hicks et al., 2020; Engzell et al., 2020; Maldonado and De Witte, 2022). Thus, it is especially important now to address educational disparities and ensure equitable learning opportunities for every student, particularly in the face of a rapidly evolving, technology-centric global economy.

Within this critical context, Learning by Making (LbyM), an innovative intervention funded by the U.S. Department of Education’s Investing in Innovation and Education Innovation and Research Funds, aims to address gaps in STEM learning in rural and high needs settings at the early high school level (Hellman et al., 2024). It does so by encouraging students to use both three-dimensional science understandings (Duschl, 2008) and computational thinking (Seiter and Foreman, 2013) to make sense of local phenomena and develop solutions to real-world problems (Avery, 2013).

The LbyM’s design is informed by the place-based learning theory—an educational approach that emphasizes using the local community and environment as a foundation for learning (Sobel, 2004). Place-based STEM learning connects academic content with real-world contexts by leveraging the unique features of the students’ surroundings—such as local ecosystems, community challenges, and regional industries (Semken and Freeman, 2008). Key elements of place-based learning theory include experiential learning, student-centered inquiry, and collaboration between schools and local organizations (Sobel, 2004; Gruenewald and Smith, 2014). While some research on place-based STEM learning suggests that it can improve learning and narrow equity gaps (e.g., Gallay et al., 2021; Johnson et al., 2020), more rigorous research is needed to understand how place-based learning can impact student outcomes and how curriculum development, teacher training, and school-community partnerships can best be supported.

Offered as a yearlong high school curriculum, LbyM includes three foundational units and three experiment units to guide students in applying scientific, programming, and engineering practices to natural phenomena in their immediate environment. The LbyM curriculum integrates disciplinary core ideas from physical science with crosscutting concepts, and it situates them within the framework of science and engineering design practices. Experiment activities link directly to local events that set the stage for each unit. The focus of the curriculum is not on memorizing content but on equipping students with the skills necessary to collaboratively construct their understanding and self-efficacy. Each lesson in the curriculum has an overarching sensemaking goal to guide students in understanding the purpose behind each activity. When students are able to explain why they are performing a task and are able to connect activities to broader science and engineering learning goals, they are more likely to develop a deeper understanding of concepts and build critical thinking skills that can be applied beyond the classroom. Through active engagement in making sense of science phenomena, the focus shifts from rote learning to engaging in building evidence-based explanations and solving problems and mysteries of the natural world (Hawkins, 2014).

Another of the centerpieces of the LbyM curriculum is its focus on student-initiated scientific investigations that require computational thinking and coding to solve interdisciplinary problems. Computational thinking in high school STEM is a problem-solving method involving “decomposition” or breaking down complex problems into smaller, manageable parts (Shute et al., 2017). This approach emphasizes the use of logical thinking and algorithmic processes to tackle challenges effectively. When computational thinking is centered on phenomena relevant to students’ lived experiences, it motivates them to make observations, build models and simulations, gather, analyze, and communicate information, and test and redefine their ideas (Hawkins, 2014). By incorporating computational thinking into high school STEM education, students learn essential problem-solving skills applicable across various disciplines. Computational thinking helps students develop a systematic approach to addressing challenges, foster creativity and innovation, and prepare for future careers in STEM fields where computational skills are highly valued. To mediate the inclusion of computational thinking into the science classroom, LbyM utilizes a customized open-source web application, LbyM (https://app.lbym.org/), for both software and hardware-based lessons.

2 Research questions

To examine the implementation and impacts of LbyM on student STEM learning, the current study addressed the following research questions:

RQ 1—To what extent were the key components of LbyM implemented with fidelity?

RQ 2—What is the impact of LbyM on the science achievement of 9th-grade students compared to that of the business-as-usual condition?

We presented the analytic approach to answer each question and the findings in section 5.

3 Materials and methods

3.1 Study design

We designed this study following the What Works Clearinghouse (WWC) guidelines (What Works Clearinghouse, 2022) for implementing a randomized controlled trial (RCT) to meet “standards without reservations,” the highest ranking from WWC. RCTs are considered the gold standard in research that studies the causality of the intervention’s impacts. During the 2022–2023 school year, 9th-grade students in three rural and three high-needs high schools in California with at least two 9th-grade science teachers were recruited and randomly assigned to LbyM or business-as-usual science classes. We ensured at least one treatment and control class within each school.

3.2 Procedures

3.2.1 The LbyM curriculum

The LbyM yearlong high school curriculum includes three foundational units and three experiment units. Unit One introduces students to basic coding using the programming language Logo (Papert, 1972; Papert, 1980) to mediate the inclusion of computer science into the science classroom. Logo is specifically designed for education and supports data transfer from sensors used during student-designed investigations. Students learning Logo do not encounter the difficulties that come with learning complex programming languages, as it is a significantly simpler language. Students learn basic coding to create simulations and explore models. In subsequent units, they expand their coding skills to collect, analyze, and interpret data while using mathematics and computational thinking. In Unit Two’s focus on electron flow, students use microcontrollers with computers and build simple circuits to carry out investigations. They also learn how to measure circuit properties using a digital multimeter and utilize additional programming constructs. Then, in Unit Three, students connect sensors, microcontroller boards, and computers to measure light intensity and temperature. Additionally, they write Logo code to read and calibrate the sensors, as well as obtain and graph data.

Through the three foundational units, students grapple with computer science concepts such as variables, conditionals, arrays, strings, control structures, algorithms, and packetized data structures. They also learn scientific and engineering approaches to using computers/sensors to make measurements, solve problems, and model physical and biological systems. Given that some students may have limited working knowledge of foundational computer operations, the curriculum also includes an optional Unit 0 to introduce the operating system and computer basics.

Students then apply the foundational skills they acquired in Units 1–3 to conduct experiments designed around scientific phenomena. Each experiment features a top-level theme—water and soil, light and energy, and microbial fuel cells—and explores a range of applied science content areas in agriculture, biology, chemistry, and physics related to that topic. For example, in the five Water and Soil lessons, a “starter experiment” introduces water evaporation as the anchoring phenomenon and investigates its effects on temperature. Lesson Two guides students in making sense of the phenomenon and modeling the experiment at the macro and micro levels. Related phenomena are presented in Lesson Three, in which students apply scientific concepts to local issues by developing models and posing questions around the origins and quantities of water in their area and the effects that water has on different types of ground cover. Finally, in Lessons Four and Five, learners explore erosion by designing their own experiments modeled on Lesson One’s starter experiment. This involves modifying Logo code, creating an experimental setup using soil moisture and temperature sensors to accomplish an investigatory aim, carrying out an investigation, synthesizing what was learned throughout the unit, and presenting scientific and technical findings to the class.

Across the curriculum, students use the evidence they collect from conducting experiments to model and explain scientific phenomena. As part of this process, each lesson presents a series of challenges with escalating difficulty, culminating in a series of “Going Further” exercises to accommodate students’ ranges of abilities in a given classroom. Creating a course structure that supports student-driven, personalized learning is a key consideration when designing an equitable curriculum for rural classrooms, which often contain students of mixed foundational knowledge and achievement levels.

A final key component of the curriculum is college and career readiness for STEM fields. Teachers attend a training on college and career awareness and provide students with exposure to STEM careers through guest speakers, field trips, videos, and/or movies. These opportunities focus on local STEM opportunities that students can pursue in their rural or high-needs context.

3.2.2 Teacher professional learning experiences

Teachers implementing the LbyM receive professional development prior to and during implementation to develop pedagogical content knowledge in computational thinking for modeling, simulations, and communicating graphical information. The LbyM professional development uses teacher-to-teacher trainings to transition participants from “classroom teacher” to “teacher leader” and to further their competencies in professional development facilitator and resource teacher roles. The LbyM adheres to effective professional learning guidelines: professional learning should be intensive, teacher-centered, collaborative, job-embedded, data-driven, and classroom-focused (Akiba and Liang, 2016; Darling-Hammond et al., 2017). Teacher training features a robust set of scheduled sessions that includes an initial week-long, in-person summer institute, as well as synchronous, virtual training and support sessions that regularly recur during the school year. During the summer institute, teachers participate in unit and experiment walkthroughs with guided support from facilitators. They also attend presentations and training sessions about Computing, Science, Technology, Engineering, Mathematics (CSTEM), Career and Technical Education (CTE) pathways, and students’ college awareness and work-based skills development.

The follow-up trainings that occur during the school year focus on unit-by-unit and experiment walkthroughs, in addition to teacher-led discussions of best practices and areas for improvement. These sessions provide expert-guided learning and maker space for teachers to strengthen their pedagogical content knowledge in computational thinking, increase their self-efficacy to minimize future needs for technical assistance, and draw on their personal experiences to share best practices for teaching. One of the priorities of the LbyM is to use cloud-based videoconferencing platforms such as Zoom to proactively bridge the geographical distance that often silos educators who teach in isolated rural communities (Monk, 2007). To reach these communities, the LbyM team has transitioned its follow-up trainings from in-person sessions to a low-bandwidth online system. The new web-based delivery model is designed to expand LbyM training accessibility and sustainability, creating a virtual Networked Improvement Community of rural teachers that centers around computational-based science learning. Over the implementation school year, implementing teachers received six full-day professional learning experiences via Zoom.

3.3 Participants, study sample, and post-hoc power analysis

The majority (59–74%) of students from participating rural schools qualified for Free or Reduced-Price Lunch, and at least 40% (between 41 and 71%) of the students identified as Latinx. Three teachers in total (one from each school) taught the LbyM course, whereas five teachers taught the typical science classes that the schools would typically offer. Table 1 indicates the number of students randomly assigned to LbyM or control classes during the study’s 1-year period. Following the study design and data collection timelines, we collected data from those students and their teachers.

Table 1

Table 1. Number of students who were randomly assigned to treatment or control classes.

Among those students, 130 in treatment and 101 in control had valid and non-missing outcome scores. The post-hoc power analysis was conducted using the PowerUp! tool (Dong and Maynard, 2013). With this sample, the study was powered to detect effect sizes as small as 0.23 standard deviations (the minimum detectable effect size).

3.3.1 Baseline equivalence

Due to data attrition (missing pretest scores, missing posttest scores, or both), a regression model similar to that used to examine the treatment impacts was employed to study the baseline equivalence between the treatment and control groups. Those students with non-missing pretest and posttest scores were used in this analysis. The baseline measure comprised science scores from the Iowa Test of Basic Skills (ITBS). Table 2 summarizes the findings for baseline equivalence.

Table 2

Table 2. Baseline equivalence between treatment and control groups.

The baseline difference was not greater than 0.25 standard deviations. This indicates that as long as the pretest of the outcome is included as a covariate in the impact model, it will meet the WWC baseline equivalence requirement (What Works Clearinghouse, 2022). To meet the requirement and improve the precision of the impact estimate, we included the pretest as a covariate in the impact model.

4 Measures

4.1 Implementation measures

4.1.1 Teacher logs and implementation trackers

All participating teachers were asked to complete an online implementation log for five consecutive teaching days twice during the academic year, once in the fall and once in the spring. In addition, treatment teachers were asked to complete an implementation tracker throughout the year. Both instruments were designed to catch the details of day-to-day science instruction. The logs focused on lessons taught during each day of the log period, modifications teachers made to the curriculum, the ways in which implementation activities engaged students with Next Generation Science Standards (NGSS) science and engineering practices, and teacher use of curriculum-provided resources. In the implementation trackers, treatment teachers recorded their day-by-day implementation of the LbyM curriculum. In addition, they were asked to note anything they added to the curriculum and which activities they skipped, if any.

4.1.2 Teacher interviews

To understand the implementation context, as well as the barriers to and facilitators of implementation, we conducted interviews with six teachers after the implementation was completed—three teachers from the treatment group and three from the control group. The interviews generally focused on (a) classroom characteristics and overall context, (b) teachers’ experience and implementation of the LbyM curriculum or business-as-usual curriculum, including curricular content and alignment with NGSS, (c) student engagement, and learning, particularly around college and career readiness, and (d) feedback on implementation successes and challenges.

To analyze the qualitative data from the teacher interviews, we used grounded theory (Charmaz, 2014), which constructs meaning by asking questions about data that has been systematically collected and analyzed. In grounded theory, qualitative data is collected and coded, and the emerging patterns and themes are examined within the context of the research questions. Using grounded theory allowed us to situate the findings within the lived experience of the participating teachers and students and to incorporate contextual factors that impacted implementation into our understanding of the results. We reviewed interview transcripts to conduct qualitative data analysis of the treatment and control teacher interviews and identified emergent themes and patterns across classrooms. We then developed codes based on the emergent themes and utilized NVivo software to assist with the analysis of interview data.

4.2 Outcome measure

4.2.1 Student science assessment

The primary outcome measure was the single-subject Iowa Assessment for Science (ITBS Science). Delivered online via the Riverside Insights platform, this test has 43 items and takes approximately 35 min to complete. The reliability coefficient for fall administration is 0.863, and for spring administration is 0.875 (Welch et al., 2018). Students at study schools completed the test (Level 14, grade 8, form E) at the beginning of the school year to measure students’ prior knowledge. The same test was administered as a post-assessment within the last 4 weeks of the school year. The tests were administered to both treatment and control groups at a school at the same time.

5 Results and discussion

5.1 Fidelity of implementation (RQ 1)

We identified three key components to measure the fidelity of implementation throughout the 2022–2023 school year: (1) Teacher professional development, (2) the LbyM curriculum implementation, and (3) STEM pathways (Tables 3–5). Each component was discussed in detail below. The overall findings are summarized in Table 6. During the year of implementation, fidelity thresholds for components 1 and 3 were met, but component 2 was not met. This might dilute the expected treatment impact on student learning outcomes.

Table 3

Table 3. Key fidelity component 1: Professional development.

Table 4

Table 4. Key fidelity component 2: Curriculum implementation.

Table 5

Table 5. Key fidelity component 3: STEM pathways.

Table 6

Table 6. Fidelity of implementation results.

5.1.1 Key component 1: Teacher professional development

The implementation fidelity of the teacher professional development component was measured by teacher attendance at training sessions, as indicated in attendance records. As shown in Table 3, all three treatment teachers attended all 5 days of the 2022 summer institute and at least three follow-up training days, meeting the professional development fidelity threshold; thus, the program met the 80% threshold for Key Component 1.

5.1.2 Key component 2: Curriculum implementation

Curriculum implementation fidelity was measured by the number of the LbyM units implemented by teachers, as indicated by teacher implementation logs. In total, three base units and two experiment units were available to teach during the 2022–2023 school year. Teachers were expected to implement all three base units plus at least one experiment unit. As shown in Table 4, one teacher implemented all three base units and two experiment units, meeting unit-level fidelity. The other two teachers implemented all three base units but did not implement any experiment units. This implementation level (33.3%) was insufficient to meet the 80% threshold for Key Component 2.

5.1.3 Key component 3: STEM pathways

The STEM pathways component fidelity was based on four STEM college and career exposure indicators during the school year. Treatment teachers were expected to attend a training on college and career awareness and to provide students with exposure to STEM careers through a guest speaker session, field trip, videos, and/or movies. As shown in Table 5, during the implementation school year, all three teachers attended the college and career readiness training. In addition, all three teachers facilitated at least one opportunity for their students to learn about STEM careers. This resulted in 100% of the teachers meeting the unit-level fidelity and the program meeting the threshold of 80% for Component 3.

5.1.4 Contextual challenges to the LbyM implementation

The results indicate that the implementation challenges were mainly associated with student absenteeism and learning and classroom behavior maturity delays from COVID-19.

5.1.4.1 Student absenteeism

Teachers across both treatment and control groups reported that student absenteeism presented significant teaching challenges during the academic year. Some teachers indicated that at any given time, 25–50% of the class experienced attendance issues. Common factors contributing to student absences included COVID-19 illness and exposure, busing issues due to staff shortages, and challenges faced in the home. One treatment teacher noted:

Attendance was really bad. Attendance especially … first period … because some of those students, I would see them during the latter part of the day, probably I can see three or four who are perennially absent for my first period, but I would see them later in the day.

Another control teacher shared:

I would say absenteeism is at an all-time high. I’ve never seen my attendance as bad as it was this year. I have probably about 24 students enrolled in first period and maybe 15 of them, 12 of them show up each day. Maybe. I noticed when I was filling out the report cards today that there are 59 absences, or out of 90: 30 absences, 29 absences, 58 tardies. They don’t want to come to class. They come to class really late. We don’t have a difference of marking them whether they’re 5 min late or 50 min late. So yeah, absenteeism is a huge issue this year…

Comments from teachers during the study year suggest that the COVID-19 context may have endured in its influence on ongoing student absenteeism challenges; students who became ill with COVID often had to miss several days of class, and those who had been exposed to COVID were also unable to attend class for several days at a time. As one teacher explained:

It has been like playing a game of tag with the whole COVID thing…we have this one student in [my] class that tested positive. Who are all the kids around that student? Okay. Now they’re all quarantined.

In addition to COVID being a contributing factor to student absenteeism, bus lines at schools were repeatedly suspended due to staffing shortages, preventing many students from getting to school. The lack of reliable transportation exacerbated the challenge of student absenteeism. One teacher reported:

The thing that comes to mind is… [that] we have had busing issues - the school and with all the schools and keeping staff. We have repeatedly had to cancel some of our bus lines and kids cannot get to school.

The challenges of student absenteeism were further exacerbated by the communication barriers resulting from the COVID-19 pandemic, which hindered effective dialogue among teachers, students, and their families. The unpredictable circumstances brought about by the health crisis obstructed the usual channels of communication between educators and families, leading to instances where teachers could not ascertain the causes of student absences.

In addition, teacher interviews revealed challenges in students’ home environments related to self-management, which may have been another contributor to chronic absenteeism. Many parents in the community worked multiple jobs and thus had less capacity to manage their child’s attendance amid the complex context introduced by COVID-19. For example, one teacher shared:

If a parent is working two jobs and can’t stay at home and doesn’t have control over their high school student well enough to force them to go to school…that’s the [reason] that I get, ‘I can’t control my kid. What do you want me to do?’

This teacher’s reflection highlights the limits both parents and teachers were feeling regarding their influence on student attendance. Another teacher shared that some students would “disappear for days at a time, [with some kids travelling] and missing 6 weeks of school…every winter,” suggesting that student attendance was a significant issue.

5.1.4.2 Learning and classroom behavior maturity delays from COVID-19

Disruptions to student learning and challenging classroom behavior resulting from the COVID-19 pandemic also presented obstacles for teachers in both treatment and control groups. Students experienced approximately 3 years of school disruption from March 2020 through the declared end of the public health emergency in May of 2023. During that time, they attended school in a mix of remote, hybrid, and in-person modes. These transitions between learning models presented significant challenges with notable variations in approaches among schools.

Teachers across both treatment and control groups shared that many students lacked prerequisite academic knowledge generally expected at the high school level. For example, during a lesson on insulators and conductors, some students were unfamiliar with the periodic table and atomic numbers, which resulted in the teacher having to re-teach and demonstrate material that the students were expected to have mastered in prior grade levels. One teacher mentioned, “My heart just breaks for these kids because they are behind.” Another teacher noted that her students were so lacking in prerequisite knowledge that some had not yet mastered basic skills for operating a computer:

It was so hard in the beginning because these kids [are so behind], and they’ve never even used a mouse before…like they didn’t know how to minimize, or they didn’t know what a desktop was…They didn’t even know to make sure [they’re] plugged in and the power strip is on. So, they’re just elementary items.

Time spent catching students up on prerequisite academic and technical knowledge took away time needed to successfully teach through each lesson, which assumes a certain mastery level of foundational knowledge. This impacted teachers’ ability to pace and fully implement the intended curriculum successfully. In addition to delayed prerequisite academic knowledge, teachers faced challenges handling classroom behavior issues. A teacher from the treatment group described the extent of the challenge:

I’m sure the whole pandemic and remote learning thing has… made a big impact on these kids and not in a positive way… For the most part, they are not interested in doing their work. They’re not interested in grades. They just want to create havoc. It has just been really difficult because I’m trying to balance all these things of, okay… Usually freshmen, they’re sort of immature anyway, and so you have to train them… but these kids, it’s like they’re still in the seventh grade…

This teacher further shared that some of the students in her high school class were still exhibiting “little-kid behaviors” by throwing things or touching and hitting other students, actions which proved difficult to manage. Another treatment teacher shared:

This year, [the students] were low-driven…. I practically had to stand behind them in order for them to do the work, and when I was not here, which I was not more than a handful of times this year, they didn’t do anything, even basic activities. Well, the word search and the crossword puzzle that has nothing to do with STEM…

Teachers from the control group shared similar sentiments regarding students’ classroom behavior after returning to in-person learning:

It’s not really the course [that was a challenge for me this year] – it’s not the material. It is the ability of the students to want to be students.

Overall, teachers’ experiences revealed that following the return to in-person learning, many teachers were required to dedicate extra resources to getting their students up-to-speed with general classroom behavior, diverting teachers’ energy and time from classroom instruction.

5.2 Program impact (RQ 2)

5.2.1 Program impact on student science learning

A single-level regression model was used to estimate the impact of the treatment on the student level. The scale scores from the ITBS Science assessment served as the primary outcome measure. The pretest of ITBS Science served as a covariate in the impact model. In addition, gender (male versus female), ethnicity (Latinx versus non- Latinx, White versus non-White), and receiving free or reduced-price lunch status (yes or no) were included as covariates. The regression model takes the following form:

${\mathit{\text{Outcome}}}_i={\beta }_0+{\beta }_1{\mathit{\text{PROGRAM}}}_i+\sum {\beta }_s{\mathit{\text{Student}}}_i+{\epsilon }_i$

Outcome represents the science post-assessment scores for the i_th student, PROGRAM is a dichotomous variable representing assignment to the treatment condition (1 for treatment versus 0 for control), and Student represents a vector of student-level covariates measured at the baseline as described above. β₀ represents the intercept, β₁ represents the treatment impact by LbyM, and ε_i is the student random effect.

The missing-indicator method (White and Thompson, 2005) was used to account for missing values on the covariates (not the outcome variables) in the impact models. The missing-indicator method retains all observations with missing values on covariates in the analysis. Indicator variables were created for missing values on each variable (0 = observed, 1 = missing), and missing values on the covariates were coded to a constant. Both the re-coded covariates and the missing value indicator variables were included in the regression model. In a randomized controlled trial, in which randomization helps ensure that the baseline covariates are balanced, the missing-indicator method appears to refine the precision of impact estimates and standard errors (White and Thompson, 2005).

Observations with missing values on outcome variables were excluded from the impact analyses. Deletion of observations with missing outcome variables has been shown to result in accurate impact estimates and standard errors when outcomes are missing at random, conditional on the covariates (Allison, 2002; von Hippel, 2007).

The results indicate no significant difference between the treatment and control groups on the science post-assessment (difference = 2.12, p = 0.508). Although the treatment students scored 2 points higher than the control students (235.33 versus 233.21), this difference was not significant at the 0.05 level (Table 7).

Table 7

Table 7. Program impacts on student achievement outcomes.

5.2.2 Program supports for students and teachers

While the quantitative data provides the results related to the LbyM’s impact on student science learning, qualitative analysis of teacher interviews provides information on how LbyM supports students’ college and career readiness skill development, teacher self-efficacy for incorporating NGSS in the classroom, and student engagement.

5.2.2.1 Development of students’ college and career readiness skills

5.2.2.1.1 The LbyM group

Most teachers in the treatment group reported that the LbyM curriculum supported the development of college and career readiness skills, such as problem-solving, analytic thinking, teamwork, self-confidence, and technical and content-specific knowledge. For example, one teacher described how LbyM provided students with opportunities to build problem-solving and teamwork skills:

If they’re into coding or they’re into computers, [the LbyM curriculum] gives them a little bit of a background, and college readiness is problem-solving. They learn how to splice, cut wires… They learn how to work with others… So yes, they learned those problem-solving skills… so soft skills, like learning to work with others.

Similarly, another teacher felt that the LbyM curriculum positively challenged students and increased their self-confidence regarding their ability to succeed in college. This teacher shared:

… definitely [helps] with career [skills], and it would’ve been amazing to get out in the community so they could see what they’re doing in class… But I saw a lot of kids gain kind of self-confidence and a little bit of a grit that they might not have had, or were sort of unsure of, that would apply to taking on tougher classes, tougher challenges and thinking about, ‘Oh, I could be a college student.’

The third teacher in the treatment group emphasized students’ growth in analytical thinking and problem-solving:

Yes, [I feel that the LbyM curriculum provided students with experiences to support college and career readiness]. Again, there’s a lot of analytical thinking and logic. They’re challenged to troubleshoot all the time – solve problems that they’re not sure how to at the beginning.

Overall, these teacher reflections highlight the general success of the LbyM curriculum: teachers felt the curriculum’s content area focus and its built-in opportunities for supporting students’ problem-solving, and teamwork improved students’ college and career readiness skills.

5.2.2.1.2 Business-as-usual group

Compared to the positive responses from treatment teachers, teachers in the control group expressed more ambivalence regarding whether the business-as-usual science curriculum improved students’ college and career readiness. Overall, the consensus among teachers in the control group was that the curricular content may not have directly supported students’ college and career readiness skills but that the curriculum indirectly helped to foster students’ foundational life and science skills. For example, teachers shared:

I have to go really back to basics and it really isn’t dependent on this particular curriculum, but what some of them are learning. [For example], how to organize stuff, … how to take notes…there’s a guide on how to set up notebooks…

So far as the curriculum actually helping them in college, I mean – basics of scientific procedures and processes, and those engineering or crosscutting concepts, [then] definitely – because they’re practicing those kinds of strategies and techniques. But not the specific content unless they’re going to go into one of these fields: oceanography, geology, meteorology, astronomy.

These reflections suggest that teachers did not feel that the curriculum content directly supported college readiness but that students did gain some foundational science and life skills by working through the curriculum. Teachers in the control group observed growth in their students’ foundational skills and executive function but did not note curriculum-specific college and career readiness impacts:

I do not think that this particular curriculum necessarily gives them a leg-up on anything in college. Really [it’s] more, are they willing to do their work, take notes and learn, and then do better in college? But yeah, I do not think I could feel really positive about that being the case.

The control teachers’ more equivocal responses contrast with the treatment teachers’ confidence that the LbyM curriculum did directly build career and college readiness skills.

5.2.2.2 Adequacy of curriculum for delivering NGSS content and development of teacher self-efficacy for incorporating NGSS in the classroom

5.2.2.2.1 The LbyM group

Overall, most treatment teachers reported that they felt comfortable and confident in their ability to teach NGSS content and the curriculum. Treatment teachers who expressed confidence in their abilities to teach NGSS content cited support from LbyM. Treatment teachers also generally felt that the LbyM curriculum adequately covered NGSS content and were satisfied with the way that LbyM incorporated NGSS Science and Engineering Practices (SEPs) and Cross-cutting Concepts (CCCs). For example, one treatment teacher articulated that he felt comfortable teaching the LbyM curriculum and felt that the curriculum adequately covered SEPs and CCCs:

Yeah, because they were really something that they could do hands-on and they could do planning and they could apply it, which is something that’s lacking in traditional science classes. So much emphasis on the concept, but not much emphasis on practical things.

Treatment teachers also shared concrete examples of NGSS practices they felt particularly competent to teach, such as modeling and data analysis, as well as satisfaction with how the NGSS content areas were covered in the curriculum. For example, one teacher shared:

I think [that] yeah, for the most part for what the class is, it does a good job of trying to hit on most of the standards. And yeah, I think it gives the kids a lot of opportunities to make connections. I didn’t hyper analyze…which concept is this unit hitting on. But my sense of the class is [that] it’s broad, and hits on a lot of what they need to be exposed to.

Another treatment teacher shared that he did not feel like there were “many other science classes at all on [their] campus that even incorporat[ed] the engineering practices that LbyM [did],” noting the LbyM’s role in spotlighting NGSS standards. The third teacher cited specific examples of students using NGSS practices such as data analysis while using the LbyM curriculum:

Yeah, there was data analysis, like they were using the measurements that they were reading to… determine if it was directly or indirectly proportional and then, use those to reason, like [in] the latter part of [unit] 3.5, whether the temperature was consistent… the room temperature was consistent with the temperature of the block.

Furthermore, teachers found the hands-on nature of the LbyM curriculum to be an especially important asset for developing the self-efficacy of both students and teachers. One teacher elaborated:

[Hands-on activities are] really the shiny spot of it all, because [students] are getting some of those NGSS-aligned standards by doing the work, doing the fun stuff like, ‘Hey, why don’t we put this in here?’

Teachers in the treatment group also noted that by teaching the LbyM curriculum, the teachers themselves gained skills in coding, electronics, circuits, and problem-solving with software and hardware, which helped to improve teacher confidence overall. When reflecting on his experiences teaching the LbyM curriculum, one teacher reported that he “learned a lot about coding” and “how electronic stuff works.” Another teacher reflected that he did not know much about coding before teaching the LbyM curriculum, but he has since learned some basic coding and circuitry. The third treatment teacher reflected on their growth in self-efficacy through using the curriculum:

I have become much more competent in problem solving. [For example,] I have a question for Saturday’s meeting, and I’m not intimidated to ask this question at all: ‘Actually, I don’t know the answer.’ So, I feel very comfortable with this curriculum that I feel like I can figure things out. I remember the first time teaching the curriculum, when things didn’t work out, I was so clueless. And it was a lot harder back then. It’s a lot user-friendly now, and plus I’ve been doing it for a while, but yeah, I feel like I’m competent in problem solving, both with software and hardware.

5.2.2.2.2 Business-as-usual group

While treatment teachers received support from LbyM for teaching NGSSS content, control teachers did not find that level of support in their curriculum. While most control teachers shared that they felt their students were generally engaged in NGSS practices, those who reported feeling comfortable with teaching NGSS in their classrooms reported that they primarily relied on their own educational background or outside training when it came to confidence in teaching this content. For example, one teacher shared:

Yeah, I’ve been playing with [NGSS] ever since [it] came out. When I did my [master’s program], it was during the time that NGSS had taken full hold. So, everything that I produced for that particular program was all based on NGSS. So, [I’m] fairly familiar with it.

On the other hand, another control group teacher reported that she was not comfortable teaching NGSS:

No…the training that I received on the standards was years ago. We weren’t really implementing and everything… with COVID, got kind of [got] put on hold… I’m trying to align books from 2006 with NGSS and we’re on our way to buying new curriculum, but we’re still looking and trying and arguing amongst ourselves [over] which publisher we want to use as a department and stuff like that.

In addition, control group teachers reported challenges related to curricular materials: they were unaligned with NGSS, were outdated, or lacked consistency with those used by colleagues teaching the same subject within their school. Teachers attributed these challenges to budget issues and COVID-related challenges, reporting that they also sourced learning materials on their own from various resources to help students grasp and apply concepts. For example, when asked about his curricular materials, one teacher from the control group shared that the classroom textbooks were from 2008 and were not NGSS-aligned. He described:

Our textbook is Biology, Prentice Hall from 2008. Is it aligned with NGSS? No. We’ve been trying to get an NGSS for, I don’t know how many years now, but publishers were lagging a bit when it came to adopting the NGSS standards into the textbook.

He also shared that teachers at his school often generate their own curricular materials from other non-textbook resources. This teacher’s reflection highlighted the non-standardized nature of curriculum use among teachers:

Quite a bit of my curriculum is self-generated or beg, borrow, and stolen from other Bio teachers, including [name of another teacher at school].

Similarly, another teacher reflected that her classroom resources were combined from a variety of different self-generated resources, and that her classroom textbooks were also not updated. When asked about her classroom curriculum, she shared:

[I use] any resources that I can pull in that seem to help the kids really grasp the concepts and be able to apply them from all different walks – whether it be something I saw in a conference, or something I found on YouTube, or something that a teacher saw somewhere and shared with me and I researched into – I mean, beg, borrow, and steal, anything that I can find that helps my students, I will [use]. We currently use California Princeton Hall by Pearson and it’s a 2006 edition, so it’s an old edition that’s out of print…

One control group teacher who had been using out-of-date textbooks shared that her school was able to acquire new textbooks in the 2022–2023 school year but that she still needed to supplement the curriculum as “the book is horrible.” She explained:

I supplement a lot of the materials that this book provides that I don’t like or… inadequate, so I use stuff from the prior book or stuff from when I taught at [another school] or from just my own knowledge. I also get science news magazines, and so I’ll use those as the real-world exposure that’s [legitimate] and not just something they found on a blog. And yeah, I definitely have to supplement most of the stuff. This book is horrible. I wish we would’ve previewed it before we purchased this.

Finally, another teacher in the control group shared that the textbook set she uses differs from the textbook set used in the classroom of another teacher teaching the same course. She shared that the two teachers decided among themselves at the beginning of the school year to divide the sets: “You use the blue [set of textbooks]. I’ll use the brown [set of textbooks].” This highlights a less consistent use of curricular materials, even among teachers of the same course at the same school.

5.2.2.3 Student engagement and real-world applications

Another key success of implementation shared by treatment teachers was that the hands-on nature of the LbyM curriculum promoted student engagement and supported students in understanding complex ideas through real-world applications. For example, one teacher in the treatment group shared that due to the curriculum’s hands-on nature, students were required to be more engaged than they might otherwise have been in other lecture-style classes. She stated:

You cannot sit in this class and be a blob. You can get away with that in other classes that just lecture, but in this class: you got to get up, you got to get out of your seat, you got to plug things in. You got to do something. So, you can’t be lazy in this class. In other classes where you’re just sitting and listening to me or [where] another teacher lectures the whole time, you can be a blob… you can get away with doing nothing. Here, you cannot. So that’s why even with the [difficulty in classroom management], I’ve got such high engagement.

Another treatment teacher noted that LbyM helped him venture beyond his comfort zone to incorporate more hands-on activities that required students’ practical planning and application of concepts, something that was previously lacking in traditional science classes. He shared:

[The LbyM activities] were really something that they could do hands-on, and they could do planning, and they could apply it, which is something that’s lacking in traditional science classes… Instead of being too bookish, [the LbyM curriculum] made me go out of my comfort zone and try to incorporate more hands-on activities with my students. And hopefully they will learn from those…

This teacher also shared that the hands-on curriculum could help students create connections to relatable daily situations such as programming.

The pros [of the curriculum would be] a lot of hands-on activity. Students will be very, very busy. There’s no dull moment. It’s something that they could easily relate to in every daily situation. It can be translated into something that they could do later, basically programming.

Similarly, another treatment teacher elaborated on the real-world applications that students could draw from the LbyM curriculum, sharing:

It is very related to real world phenomena because for example… for electricity, I was telling my students that the basics that we’re learning in Ohm’s law is something that’s also basic for… if you want to become an electrician later. What else? Electricity, and then also with the sensors, and I was trying to tell them that it could be… what we’re actually doing is we’re making our own real world measuring equipment; it’s very, very crude but we’re learning more of the process rather than getting the actual measurement.

6 Conclusion

Hands-on by design, the LbyM curriculum supported teachers in embedding interactive science-learning activities that enlisted students’ active engagement while building teacher self-efficacy. While implementing the LbyM curriculum, teachers shared that students demonstrated high degrees of proactive learning, applying science concepts to an extent beyond what might be seen in traditional lecture-oriented classrooms. Statements by control teachers, on the other hand, expressed ambivalence about their business-as-usual science curricula’s adequacy and currentness, consistency of application across classrooms, alignment with NGSS, development of college and career readiness skills, and support for teacher learning.

While the study was impacted by the deep disruptions to teaching and learning of the COVID-19 pandemic—which illuminated the unique challenges of rural schools, such as transportation, at-home supervision, and digital access issues that were encountered in the study—it highlights a number of key components for an early high school-level science curriculum aimed at addressing gaps in STEM learning in rural settings for improved equity. For example, a growing body of research demonstrates the importance of place in teaching STEM in rural schools (e.g., Ruday and Azano et al., 2019; Azano et al., 2021; Moffa and McHenry-Sorber, 2018). A place-based curriculum centers on students’ immediate community and lived experience and incorporates local resources. LbyM focused on place by building up students’ ability to make sense of local phenomena in the form of applying computational thinking and coding skills, as well as collecting, analyzing, and interpreting data to develop solutions to problems related to their lives. In addition, LbyM supported teachers in facilitating field experiences and opportunities for their students to learn about STEM college and career opportunities in the local community; research suggests that students tend to pursue careers they have experienced in their local context (e.g., Elam et al., 2012). Teachers who implemented the LbyM reported that focusing on real-world applications increased student engagement and self-efficacy.

At the same time, LbyM focused on supporting teachers and building their self-efficacy through professional learning and sharing—a critical aspect of improving STEM education in rural schools. For example, participating in a professional network such as the LbyM’s virtual Networked Improvement Community for computational-based science learning can help rural STEM teachers feel less isolated and access resources beyond their setting (e.g., Azano et al., 2021). In addition, the LbyM teachers received professional learning targeted to their rural settings to support them in centering student-driven, personalized learning connected to students’ lived experiences and to the STEM resources in the local community. This included facilitating local career and technical educational experiences for students and promoting hands-on, student-initiated scientific investigations to develop solutions to interdisciplinary real-world problems.

Finally, LbyM built teacher self-efficacy around delivering NGSS content. It also improved teacher confidence in developing their students’ computational thinking, experiment, research, and coding skills. Emerging research suggests that targeted professional learning can prepare and support teachers to effectively incorporate NGSS content in the classroom (e.g., Christian et al., 2021). Teachers who implemented LbyM reported that the professional learning opportunities built their capacity and increased their confidence around NGSS, as well as CSTEM skills such as modeling and data analysis. A nascent body of research also highlights the importance of a curriculum that effectively incorporates NGSS SEPs and CCCs to adequately support teaching and learning (e.g., Harris et al., 2022). Teachers who implemented the LbyM curriculum reported that it built the NGSS knowledge and skills of both teachers and students.

Study findings highlighted some opportunities for enhancing the LbyM curriculum and instructional approach. For example, the student academic self-management and school-family communication challenges that came to light during the study present the opportunity to incorporate resources to support home-school connections as the curriculum is implemented. Further, treatment teachers noted that students’ prerequisite scientific knowledge was often insufficient for effective engagement with the curriculum. This finding suggests an opportunity to accommodate variations and gaps in learners’ foundational knowledge by creating options for differentiated instruction and learning pathways within the LbyM curriculum. Additional “buffer” units similar to Unit 0 on foundational computer skills could be created for teachers to incorporate as needed to scaffold LbyM content while more advanced students complete Going Further activities.

Thus, opportunities for improving LbyM for deeper student impact include increasing family communication and family involvement through building relationships and developing opportunities for partnership and shared accountability with families. This would include regular in-person meetings with families as well as ongoing communication and follow-up around shared learning goals. In addition, challenges highlighted in the study around students’ foundational science knowledge should be addressed by building scaffolds and differentiated instruction activities into the LbyM curriculum and providing teacher development around implementing these effectively. Modifications such as these—along with further implementation and impact evaluation of LbyM in a normal, unimpacted context and with full fidelity—would likely shed additional light on how to best meet early high school CSTEM teaching and learning needs in rural settings.

6.1 Limitations

While this study provides valuable insights into the challenges influencing the implementation of LbyM in rural schools and the opportunities for improving it, it was underpowered to detect a significant difference in student science achievement between the treatment and control groups. It was likely due to the COVID-19 pandemic, which prevented the study team from recruiting more schools, teachers, and their students. It also caused data loss. Furthermore, the treatment teachers did not meet all LbyM components as expected (component 2, curriculum implementation, was not met). Lack of expected implementation might dilute the treatment’s impact on student learning outcomes. Future research could address these limitations by employing various strategies to recruit more schools or providing more support to teachers to implement a curriculum that meets the program’s expectations.

Data availability statement

The raw datasets generated for this article are currently unavailable due to restrictions imposed by the WestEd Institutional Review Board (IRB). However, requests for access to the datasets can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by WestEd has been issued Federal Wide Assurance number #00001734 from the Department of Health and Human Services. The WestEd IRB identification number is IRB00006866. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation in this study was provided by the participants’ legal guardians/next of kin.

Author contributions

LL: Writing – original draft, Writing – review & editing. C-WH: Writing – original draft, Writing – review & editing. CM: Writing – original draft, Writing – review & editing. KL: Writing – original draft, Writing – review & editing. EC: Writing – original draft, Writing – review & editing. SY: Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This study has been funded by a U.S. Department of Education‘s Education Innovation and Research Grant (grant number U411C180146).

Conflict of interest

LL, C-WH, CM, KL, EC, and SY were employed by the research agency WestEd. All study design activities, random assignment, data collection, analysis, and interpretation of the impact are independently conducted by WestEd. The authors declare that the research was conducted in the absence of any commercial or financial relationship that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Akiba, M., and Liang, G. (2016). Effects of teacher professional learning activities on student achievement growth. J. Educ. Res. 109, 99–110. doi: 10.1080/00220671.2014.924470

Crossref Full Text | Google Scholar

Allison, (2002). Quantitative Applications in the Social Sciences (Book 136) - Missing Data (1st ed.). Thousand Oaks, California, USA: SAGE Publications.

Google Scholar

Avery, L. M. (2013). Rural science education: valuing local knowledge. Theory Pract. 52, 28–35. doi: 10.1080/07351690.2013.743769

Crossref Full Text | Google Scholar

Azano, A. P., and Callahan, C. M. (2021). “Innovations in providing quality gifted programming in rural schools using place-conscious practices” in Rural education across the world: Models of innovative practice and impact. eds. S. White and J. Downey (Singapore: Springer), 91–106.

Google Scholar

Bacher-Hicks, A., Goodman, J., and Mulhern, C. (2020). Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time. J. Public Econ. 193:104345. doi: 10.1016/j.jpubeco.2020.104345

PubMed Abstract | Crossref Full Text | Google Scholar

Bohrnstedt, G., Kitmitto, S., Ogut, B., Sherman, D., and Chan, D. (2015). School composition and the black-white achievement gap. NCES 2015–018, National Center for Education Statistics. National Center for Education Statistics. Available at: https://eric.ed.gov/?id=ED560723 (Accessed September 12, 2024).

Google Scholar

Bouck, E. (2004). How size and setting impact education in rural schools. Rural Educ. 25, 38–42. doi: 10.35608/ruraled.v25i3.528

Crossref Full Text | Google Scholar

Charmaz, K. (2014). Constructing grounded theory. (2nd ed.). Thousand Oaks, California, USA: SAGE Publications.

Google Scholar

Christian, K. B., Kelly, A. M., and Bugallo, M. F. (2021). NGSS-based teacher professional development to implement engineering practices in STEM instruction. Int. J. STEM Educ. 8:21. doi: 10.1186/s40594-021-00284-1

Crossref Full Text | Google Scholar

Darling-Hammond, L., Hyler, M. E., and Gardner, M. (2017). Effective teacher professional development, Learning Policy Institute. Learning Policy Institute. Available at: https://eric.ed.gov/?id=ED606743(Accessed September 12, 2024).

Google Scholar

Davis, J. C., Rupasingha, A., Cromartie, J., and Sanders, A. (2022). Rural America at a glance: 2022 edition. EIB-246. U.S. Department of Agriculture Economic Research Service. Available at: http://www.ers.usda.gov/publications/pub-details/?pubid=105154 (Accessed September 12, 2024)

Google Scholar

Dong, N., and Maynard, R. (2013). PowerUp!: a tool for calculating minimum detectable effect sizes and minimum required sample sizes for experimental and quasi-experimental design studies. J. Res. Educ. Effect. 6, 24–67. doi: 10.1080/19345747.2012.673143

Crossref Full Text | Google Scholar

Duschl, R. (2008). Science education in three-part harmony: balancing conceptual, epistemic, and social learning goals. Rev. Res. Educ. 32, 268–291. doi: 10.3102/0091732X07309371

Crossref Full Text | Google Scholar

Elam, M., Donham, B., and Soloman, S. R. (2012). An Engineering Summer Camp for 760 Underrepresented Students from Rural School Districts. Journal of STEM Education: 761 Innovations and Research. 13, 35–44. Available at: https://www.jstem.org/jstem/index.php/JSTEM/article/view/1619 (Accessed December 09, 2024).

Google Scholar

Engzell, P., Frey, A., and Verhagen, M. D. (2020, October 29). Learning loss due to school closures during the COVID-19 pandemic. doi: 10.31235/osf.io/ve4z7 (Accessed December 09, 2024).

Crossref Full Text | Google Scholar

Gallay, E., Flanagan, C., and Parker, B. (2021). Place-based environmental civic science: urban students using STEM for public good. Front. Educ. 6:693455. doi: 10.3389/feduc.2021.693455

Crossref Full Text | Google Scholar

Gibbs, R. (2005). “Education as a rural development strategy” in Amber waves: the economics of food, farming, natural resources, and rural America (USA: U.S. Department of Agriculture Economic Research Service).

Google Scholar

Gruenewald, D. A., and Smith, G. A. (Eds.) (2014). Place-based education in the global age: Local diversity. Milton Park, Abingdon, Oxfordshire, UK: Routledge.

Google Scholar

Harris, C. J., Feng, M., Murphy, R., and Rutstein, D. W. (2022). Curriculum materials designed for the next generation science standards show promise: Initial results from a randomized controlled trial in middle schools : WestEd.

Google Scholar

Hawkins, M. (2014). “STEM education in the middle school classroom” in Exemplary STEM programs: Designs for success. eds. R. E. Yager and H. Brunkhorst (Arlington: NSTA Press).

Google Scholar

Heaster-Ekholm, K. L. (2020). Popular instructional design models: their theoretical roots and cultural considerations. Int. J. Educ. Dev. Using Inf. Commun. Technol. 16, 50–65.

Google Scholar

Hellman, H., Peticolas, L., and Cominsky, L. (2024). Teaching for tomorrow: science literacy in the classroom. Sci. Teach. 91, 31–35. doi: 10.1080/00368555.2023.2292333

Crossref Full Text | Google Scholar

Johnson, J., Showalter, D., Klein, R., and Lester, C. (2014). Why rural matters 2013–2014: The condition of rural education in the 50 states, Rural School and Community Trust. Rural School and Community Trust. Available at: https://eric.ed.gov/?id=ED556045 (Accessed September 12, 2024).

Google Scholar

Johnson, M. D., Sprowles, A. E., Goldenberg, K. R., Margell, S. T., and Castellino, L. (2020). Effect of a place-based learning community on belonging, persistence, and equity gaps for first-year STEM students. Innov. High. Educ. 45, 509–531. doi: 10.1007/s10755-020-09519-5

PubMed Abstract | Crossref Full Text | Google Scholar

Lee, J. A., and Barnes, A. R. (2015). Predominately White institutions: transition programs to address academic underpreparedness and experiences of discrimination. Transl. Issues Psychol. Sci. 1, 401–410. doi: 10.1037/tps0000043

Crossref Full Text | Google Scholar

Maldonado, J. E., and De Witte, K. (2022). The effect of school closures on standardised student test outcomes. Br. Educ. Res. J. 48, 49–94. doi: 10.1002/berj.3754

Crossref Full Text | Google Scholar

Marksbury, N. (2017). ‘Monitoring the pipeline: STEM education in rural U.S’, Forum Public Policy Online. Available at: https://eric.ed.gov/?id=EJ1173822.(Accessed September 12, 2024).

Google Scholar

Moffa, E., and McHenry-Sorber, E. (2018). Learning to be rural: lessons about being rural in teacher education programs. Rural Educ. 39, 26–40. doi: 10.35608/ruraled.v39i1.213

Crossref Full Text | Google Scholar

Monk, D. H. (2007). Recruiting and retaining high-quality teachers in rural areas. Futur. Child. 17, 155–174. doi: 10.1353/foc.2007.0009

PubMed Abstract | Crossref Full Text | Google Scholar

Papert, S. (1972). Teaching children thinking. Program. Learn. Educ. Technol. 9, 245–255. doi: 10.1080/1355800720090503

Crossref Full Text | Google Scholar

Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. New York, NY, USA: Basic Books.

Google Scholar

Ruday, S., and Azano, A. P. (2019). Arguments that matter: a place-based approach to teaching argument writing to rural. J. Teach. Writ. 34, 1–23.

Google Scholar

Ryoo, J. J., and Calabrese Barton, A. (2018). Equity in STEM-rich making: pedagogies and designs. Equity Excell. Educ. 51, 3–6. doi: 10.1080/10665684.2018.1436996

Crossref Full Text | Google Scholar

Seiter, L., and Foreman, B. (2013). “Modeling the learning progressions of computational thinking of primary grade students” in Proceedings of the ninth annual international ACM conference on international computing education research (New York, NY, USA: Association for Computing Machinery), 59–66.

Google Scholar

Semken, S., and Freeman, C. B. (2008). Sense of place in the practice and assessment of place-based science teaching. Sci. Educ. 92, 1042–1057. doi: 10.1002/sce.20279

Crossref Full Text | Google Scholar

Shute, V. J., Sun, C., and Asbell-Clarke, J. (2017). Demystifying computational thinking. Educ. Res. Rev. 22, 142–158. doi: 10.1016/j.edurev.2017.09.003

Crossref Full Text | Google Scholar

Sobel, D. (2004). Place-based education: connecting classrooms & communities. Great Barrington, MA, USA: The Orion Society.

Google Scholar

von Hippel, P. T. (2007). 4. Regression with missing Ys: an improved strategy for analyzing multiply imputed data. Sociol. Methodol. 37, 83–117. doi: 10.1111/j.1467-9531.2007.00180.x

Crossref Full Text | Google Scholar

Vossoughi, S., Hooper, P. K., and Escudé, M. (2016). Making through the Lens of culture and power: toward transformative visions for educational equity. Harv. Educ. Rev. 86, 206–232. doi: 10.17763/0017-8055.86.2.206

Crossref Full Text | Google Scholar

Welch, C. J., Dunbar, S. B., and Fina, A. D. (2018). Technical summary for form F of the Iowa assessments : Iowa Testing Programs, University of Iowa.

Google Scholar

What Works Clearinghouse. (2022). What works clearinghouse procedures and standards handbook, version 5.0. Available at: https://ies.ed.gov/ncee/wwc/Handbooks

Google Scholar

White, I. R., and Thompson, S. G. (2005). Adjusting for partially missing baseline measurements in randomized trials. Stat. Med. 24, 993–1007. doi: 10.1002/sim.1981

PubMed Abstract | Crossref Full Text | Google Scholar