Kathryn T. Stevenson1*M. Nils Peterson2
Sarah J. Carrier3
Renee L. Strnad4Ryan A. Olson5
Rachel E. Szczytko6- 1Department of Parks, Recreation and Tourism Management, NC State University, Raleigh, NC, United States
- 2Fisheries, Wildlife and Conservation Biology Program, Department of Forestry and Environmental Resources, NC State University, Raleigh, NC, United States
- 3Department of Teacher Education and Learning Sciences, NC State University, Raleigh, NC, United States
- 4Department of Forestry and Environmental Resources, NC State University, Raleigh, NC, United States
- 5Muddy Sneakers, Brevard, NC, United States
- 6Asombro Institute for Science Education, Las Cruces, NM, United States
As K-12 audiences represent a major proportion of environmental education (EE) audiences, academics should be an outcome of interest in EE research and evaluation. However, research around links between EE and academic outcomes (e.g., grades, test scores) is scant. Reasons for limited research on EE and academic outcomes may include disinterest in academic outcomes, assertion that academic outcomes are poor measures of learning, and normative biases against publishing null or negative effects within academia. We argue for adoption of a null effects framework for linking EE and academic outcomes. We begin by outlining what we mean by a null effects framework and then suggest reasons why the EE community should adopt it. Specifically, a null effects framework embraces and celebrates research demonstrating no difference in traditional academic outcomes associated with EE curricula and more traditional classroom instruction. We describe key aspects of operationalizing a null effects framework, including use of key statistical procedures (e.g., measuring power), and changes in peer review associated with emphasizing measures of evidence beyond hypotheses testing and p-values. We conclude by describing how this approach matches EE objectives, strengthens links to academic outcomes without being bound by them, avoids setting unrealistic expectations for EE, and highlights the myriad of non-academic co-benefits offered by EE. As including EE in schools is the best opportunity for reaching the most learners in terms of numbers and diversity, we offer a null effects framework as an approach that can boost adoption of EE where it is arguably needed most.
Introduction
Although environmental education (EE) is a lifelong endeavor, there is a logical nexus with K-12 education. Roth (1969) and Stapp (1969) both called for education across the lifespan, and this sentiment has been echoed in the Tblisi Delcaration (UNESCO, 1977) and in more contemporary statements on EE (Monroe and Krasny, 2016). Though many initiatives exist for older or multi-generation audiences, children represent a significant audience of EE programming (Ardoin et al., 2017). As early intervention can set individuals on a trajectory of lifelong environmental engagement (Chawla, 1999; Wells and Lekies, 2006), many environmental educators see younger audiences as an opportunity for greatest impact. Further, working with K-12 schools represents an opportunity to reach a wide and diverse audience, as education is compulsory in many countries across the globe.
As K-12 school programming is a major focus in EE efforts, it is not surprising that there has been a recent call for more evaluation in EE, paralleling trends in K-12 education. In the United States, A Nation at Risk (US Department of Education, 1983) marked the beginning of a decades-long emphasis on accountability and testing. Responding to the perception that students were being left behind by public education and that better measurement would help mitigate these trends, subsequent iterations of the Elementary and Secondary Education Act (Every Student Succeeds: 114th Congress, 2015; e.g., No Child Left Behind: 107th Congress, 2002) have required standardized tests at every grade level (Darling-Hammond et al., 2016). Similarly, granting agencies have increasingly emphasized the need for evaluation (Boris and Kopczynski Winkler, 2013). In parallel, and perhaps in response to, these trends, EE scholars and practitioners have called for more rigorous evaluation to improve efficacy of programs (Carleton-Hug and Hug, 2010; Heimlich, 2010) and attend to the culture within schools and requirements of funders (Ardoin et al., 2017). As formal education is a key audience for EE, these calls have included the need for more data supporting links between EE and academic outcomes (e.g., grades, standardized test scores) (Jordan and Chawla, in press).
Increased frequency and rigor of EE evaluation, however, is a double-edged sword, particularly as it relates to academic outcomes. On one side, rigorous evaluation linking EE to academic outcomes has the potential to accelerate adoption of EE in schools. Links to academic achievement is required to justify participation in EE programs (Ernst, 2012; Stevenson et al., 2014) and funders of EE programs demand quantitative proof of efficacy, often linked to academic outcomes (Boris and Kopczynski Winkler, 2013). Some research supports a link between EE and academic outcomes, including in the context of test scores (State Education and Environment Roundtable [SEER], 2000) and science knowledge gains (Ballantyne et al., 2001; Barnett et al., 2006). By and large, this research supports positive associations between EE and test scores when EE is fully integrated into the curriculum (State Education and Environment Roundtable [SEER], 2000). Little research has been published on shorter-term EE programs (e.g., field trips, EE lessons). Of 119 EE articles systematically reviewed between 1994 and 2013, only seven included academic outcomes (Ardoin et al., 2017). When academic outcomes are engaged, as in the case of the statewide Oregon Outdoor School program (Braun, 2019), academic outcomes may be indirectly measured through teacher perceptions of student learning and student self-report measures. The relatively few studies directly measuring academic outcomes may represent a disinterest among EE researchers in academic outcomes in favor of others (e.g., connection to nature), rejection of academic outcomes a measure of meaningful learning, or a failure to report null or negative results (Stern et al., 2013). This latter explanation highlights a challenge posed by increased evaluation of EE in academic contexts—generating research that shows that EE is no better than non-EE strategies at boosting academic outcomes when best teaching practices are used in both contexts. If the EE community is to respond to calls for increased evaluation, a framework may be needed to present those null results in ways that can build the case for EE, rather than undercut it, and contribute to scientific understanding of the contributions of EE to student learning.
In this paper, we describe how a null effects framework for EE research and evaluation in in K-12 contexts works toward these goals. Such a framework shifts the emphasis from reporting positive results (e.g., EE strategies being better than non-EE strategies) to celebrating null effects (i.e., no difference between EE and non-EE strategies). Many readers likely adopt the view that academic measures (e.g., standardized test scores) are poor indicators of student learning that ignore critical factors such as motivation or affective gains (Aydeniz and Southerland, 2012; Tienken et al., 2017). We agree with this view but argue that academic outcomes are clearly a metric of interest to policy makers and funders and are essential to advocate for EE in schools. We begin by describing how to detect null effects and, then, suggesting how they may be useful in the context of EE research and evaluation while avoiding unrealistic expectations. We argue a null effects framework fits the objectives of EE and addresses the context of standardized testing in K-12 education without being bound by it.
What Does it Mean to use a Null Effects Framework for EE and Academic Outcomes?
Although it is possible that null effects around EE and academic outcomes are going unreported, EE researchers may not directly study academic outcomes because doing so creates logistical challenges. For example, many EE programs serve K-12 students but are not affiliated directly with schools. Accordingly, evaluation or research efforts focusing on academic outcomes require seeking permission from schools for access to those measures, a procedure that can be prohibitive. Student-level data (rather than aggregate) is typically difficult to obtain from schools and districts, due to concerns about student anonymity and privacy (Family Educational Rights and Privacy Act (FERPA), 1974). Other options may include working with third party data repositories (e.g., North Carolina Education Research Data Center), but these sources are often associated with high access fees to support the infrastructure needed to maintain the databases. These types of barriers make studying academic outcomes nearly impossible for individual EE programs, but universities and associated research grants could overcome them in some instances by providing funds to access the data and assurance that the work meets ethical standards for human research to secure the necessary permissions.
Focusing on null effects for EE on academic outcomes will require minor statistical changes to some studies and a major shift in perspective. Regarding the former, most well-designed studies include statistical elements required to evaluate the likelihood of null effects, and EE scholars have already called for inclusion of these elements in future research. Specifically, rigorous research designs called for in EE studies require large samples and precise variables (i.e., low variance) to ensure high power of analysis, treatment-control experimental designs, measures of effect size in addition to p-values, and consistent metrics to evaluate across programs (Creswell, 2008; Carleton-Hug and Hug, 2010; Stern et al., 2013). These suggestions were made in the context of identifying attributes of EE that are linked to positive outcomes in the realm of environmental literacy (e.g., environmental sensitivity), but they apply equally to evaluating null effects. Statistically, it is not possible to demonstrate an effect size of zero, but studies with large sample sizes and variables measured with high precision can demonstrate effect sizes so close to zero that differences are socially meaningless (Vaske, 2008). For instance, test scores for treatment (EE) and control (non-EE) groups may have confidence intervals which overlap almost completely, potentially yielding a p-value of 0.95 or higher. Though there are multiple statistical techniques for rigorously evaluating the degree of overlap in confidence intervals and testing for no difference, the statistical testing is a relatively small challenge compared to adopting alternatives to research focused on rejecting null hypotheses (Anderson et al., 2000; Trafimow, 2013). The intense pressure in many disciplines to conduct and publish research that obtains statistically significant effects, interpretable as disconfirming alternative theories can lead to inappropriate conclusions (Trafimow, 2013), stifle creativity (Guthery et al., 2001), and encourage evaluators and stakeholders to cherry-pick data to highlight positive associations with treatments and outcomes of interest (Munafò et al., 2017). This might include a tendency to interpret pre-post-increases in academic measures as attributable to a program when no control group is included or to rely on p-hacking, wherein researchers sift through variables, report only the outcomes and drivers with significant relationships (Head et al., 2015), and, ultimately, underreport results which show no difference between non-EE and EE-based methods in terms of academic growth. A null effects framework would instead encourage more transparent reporting of research without threatening adoption of EE in schools.
Stepping outside a hypothesis testing paradigm requires practitioners, researchers, reviewers, journal editorial boards, and program funders to treat hypothesis testing as one of many ways to evaluate evidence rather than the sole way. Doing so would open the door to research focusing on null effects, rather than disconfirming alternative theories, and allow use of unusual but informative models of portraying data. Fortunately, the EE community already values other ways of evaluation beyond typical hypothesis testing. For instance, EE has a rich history in qualitative work (Palmer, 2002) and has recently begun to use methods such as photo elicitation, art, and story-telling (Flowers et al., 2015; Chanse et al., 2017; Piersol and Timmerman, 2017). A null effects framework for EE and academic outcomes provides opportunities for creative use of diverse metrics of evidence including utilizing qualitative analyses, providing basic frequencies and graphing, using equivalence tests to report null effects, and employing information theoretic approaches linked to Bayesian models (Guthery et al., 2001). One first step toward making this change could involve special journal issues focused on exploring the dynamics of EE treatments on academic outcomes, where null effects are welcome, if not required. Further, outreach efforts through organizations such as the North American Association for Environmental Education (NAAEE) or the Children & Nature Network (C&NN) could work to equip practitioners with the tools they need to clearly communicate how null effects fit into the broader narrative of how EE contributes to student growth and learning.
Why Adopt a Null Effects Framework for EE Evaluation and Research?
A primary reason to focus on null effects of EE treatments on academic outcomes is that null effects are arguably what EE practitioners are trying to achieve. Although EE practitioners and organizations argue for EE to be integrated into the fabric of K-12 teaching, the majority of EE in schools is structured as a supplement to classroom teaching (e.g., field trips, isolated activities: Ardoin et al., 2017). In our experience, EE practitioners in these contexts are not trying to beat K-12 teachers at their own game. Rather, they are working to develop mutual gains where academic or cognitive outcomes are supported while building environmental sensitivity, connection to nature, stewardship, or other outcomes related to environmental literacy. Outcomes including connection to nature are more directly related to EE programs' mission statements and theories of change, but EE practitioners recognize the need to report academic outcomes to appeal to their K-12 stakeholders. For instance, we conducted an informal poll of environmental educators in North Carolina and found that nearly all explicitly linked their programs to academic standards, but few listed academic outcomes as a primary goal of their program, and none expected to generate better classroom outcomes on concepts teachers were already teaching. Instead, practitioners commonly listed connection to nature, natural history knowledge, and environmental behaviors as program goals but identified linking to academic standards as important to communicate the worth of their program to K-12 stakeholders. These responses suggest that a null effects framework for EE and academic outcomes is congruent with practitioner conceptualizations of success.
Not only does a null effect framework better fit EE objectives, it may prevent those objectives from being warped by mission creep linked to an emphasis on standardized testing. The troubling impacts of the increased emphasis on standardized testing on teaching and learning is well documented. These include classes being dominated by test preparation (Jacob, 2002), non-tested subjects like art and social studies being deemphasized or dropped completely (Jorgenson and Vanosdall, 2002; Jones et al., 2003), teacher attrition (Jalongo and Heider, 2006), and higher student stress (Amrein and Berliner, 2002). Further, the emphasis on accountability associated with standardized tests imbeds a level of competition and one-upmanship into attending to these outcomes (e.g., schools being graded on standardized test outcomes) (Jones et al., 2003). This type of competition is arguably antithetical to EE culture (Krasny et al., 2016), and it also represents a trap in which EE programs seeking to help achieve higher academic outcomes are forced to participate in a culture that can sacrifice student learning to improve test scores. A null effects framework for EE and academic outcomes would help EE avoid slipping into this paradigm.
By communicating null effects with respect to academic outcomes, EE evaluation and research can present EE as a way to maintain academic achievement while providing a myriad of co-benefits, particularly when paired with time outdoors. Many teachers and administrators fear that EE may detract from student learning (Ernst, 2009, 2012), and to some degree, this view may even be shared by students (Carrier et al., 2014). A null effects framework for EE and academic outcomes assuages these fears to allow the focus to shift to other benefits. Some of these are congruent with outcomes of interest within formal schooling. For instance, EE has been linked to improved cognitive skills (Stevenson et al., 2013), motivation (Legault and Pelletier, 2000), self-efficacy (Barnett et al., 2006), and social-emotional learning (Carter, 2016), all of which have been highlighted as priority areas in national education legislation and standards in the United States and internationally (Breiting and Wickenberg, 2010; NRC, 2012; NGSS Lead States., 2013). Further, when EE incorporates outdoor instruction, students may realize benefits such as improved attention in and out of the classroom (Kuo et al., 2018; Szczytko et al., 2018), lower stress levels (Wells and Evans, 2003), and improved cognitive and social function (Chawla, 2015). Teachers engaging in EE may find new avenues for connecting with their students (Carrier, 2009b; Ernst, 2009), helping curb teacher attrition plaguing many schools. While not related to academic achievement, all of these benefits are likely of interest to formal educators, and when paired with an acknowledgment of test scores, environmental educators may have more of opportunity to gain access to schools. Other environmental literacy-related outcomes such as improved connection to nature (Cheng and Monroe, 2010), environmental sensitivity (Chawla, 2010), environmental behavior (Ardoin et al., 2015) may be of less direct interest to formal classroom educators but are central to the EE community.
Another reason to consider a null effects framework in academic-related EE research is that it avoids setting unrealistic expectations. Education scholars and practitioners have been perfecting classroom instruction as a tool for conveying knowledge for centuries, and an evaluation model bent on “beating” those methods with new EE curricula simply is not realistic. Further, as mentioned above, EE aims to do much more than convey knowledge. In general, EE will likely not outperform classroom teaching in improving academic outcomes as measured using traditional assessments except in special contexts (see below). EE curriculum (e.g., Project Learning Tree, Project WILD) and approaches (e.g., the NAAEE Guidelines for Excellence: NAAEE, 2010) are rooted in educational theory and research, employing constructivist approaches to teach interdisciplinary concepts (e.g., natural sciences, civic engagement) in authentic settings (e.g., the natural world) (Carrier and Stevenson, 2017). However, classroom approaches and teacher training programs have also benefitted from this same theory and research, in an arguably more focused, long-term, and larger scale manner. Specifically, many environmental educators are natural scientists trained during short-term staff training programs in theory- and research-based pedagogical techniques. These programs are often excellent but cannot compare to the pedagogical training typically required of classroom teachers, which includes a 4-year college degree focused on teaching and months of mentored student teaching. This training ensures classroom teachers are equipped to use the same techniques that make EE high-quality education (e.g., project based learning, experiential learning) (Carrier and Stevenson, 2017). Although EE programming, whether implemented through schools and teachers (e.g., Project Learning Tree activities) or through external programing (e.g., field trips), may be a great complement to what teachers are already doing, teachers following best practices of education should be expected to produce student academic outcomes as well as, or better than, EE programming. Because classroom teachers often have more training than environmental educators in pedagogical techniques, null results can be seen as a measure of a program's success in fostering student learning.
Finally, framing null results as supporting EE in K-12 settings sets up few cases of positive effects as the spectacular successes they are, rather than a mere “better than nothing” finding. Although large treatment effects across the general population are likely unrealistic, in some instances, we may see treatment effects and establishing a norm of null effects helps highlight them. For instance, Muddy Sneakers, a North Carolina EE program, organizes their program around the eight content themes as identified in state standards (e.g., energy, forces and motion, ecosystems). While students participating in the program improved science grades over a control group, overall treatment effects were small with respect to academic outcomes. However, academic gains were higher among students with cognitive, emotional, or behavioral disabilities (Szczytko et al., 2018). Similarly, a pilot evaluation project of Oregon's outdoor schools documented gains in self-assessed measures of academic achievement, but those gains were not as high as gains in non-academic areas (e.g., social-emotional learning, environmental learning) (Braun, 2019). However, the academic gains that were reported were particularly pronounced among girls, students identifying as non-binary or trans-gender, and students with substantial or special needs, particularly among measures of academic self-efficacy (Braun, 2019). Other studies have found socially meaningful differences in academic outcomes in specific curricular contexts (State Education and Environment Roundtable [SEER], 2000) or differential academic benefits for girls (Carrier, 2009a) and African American students (Lieberman and Hoody, 1998). These types of results, paired with overall null results for academic outcomes, could make a strong case that EE is not taking away from instructional time, and making progress on factors that may give a boost to those who need it most (Stevenson et al., 2017).
Conclusion
Linking EE to academic outcomes may be necessary to ensure EE is relevant in K-12 settings, which crucially ensures that EE audiences are broad and diverse. Adoption of a null effects framework for EE and academic outcomes will allow for a narrative framework that can communicate why EE is beneficial to key K-12 stakeholders, including administrators, teachers, and parents. This is particularly true in a policy setting in which funding is dedicated to interventions, such as widespread technology adoption, which have little evidence for academic learning (Kimmons, 2015) but rapidly mounting evidence for coupled downsides (e.g., higher stress: Twenge, 2017), a frame in which the EE community can advocate for adoption of EE programming in schools is warranted. We invite researchers to join in this work by including academic outcomes in their studies, reporting the null results, and beginning to shape the conversation. Similarly, we call on journals to encourage and support this work by promoting it through special issues and calls for papers. Standardized tests and grades may be poor measures of learning (Finn et al., 2014), and accordingly, uninteresting or even unappealing to EE researchers. However, they produce arguably the single most powerful data in the context of allocating resources for K-12 schooling, and the EE community would be well-served to utilize them in ways that support integration of EE into schools.
Author Contributions
KS, MP, and SC contributed to the conceptualization and writing of this manuscript. RLS, RO, and RES contributed to the conceptualization and critical editing of the writing.
Funding
This work was supported in part by Muddy Sneakers and NC Sea Grant (2016-R/16-ELWD-1).
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
We would like to thank the numerous students and teachers who have participated in the research that helped shape our ideas around this topic as well as Steve Braun for his insight into the Oregon outdoor schools project.
References
114th Congress (2015). Every Student Succeeds Act (ESSA). Available online at: https://www.govtrack.us/congress/bills/114/s1177
Amrein, A. L., and Berliner, D. C. (2002). High-stakes testing, uncertainty, and student learning. Educ. Policy Anal. Arch. 10, 1–74. doi: 10.14507/epaa.v10n18.2002
Anderson, D. R., Burnham, K. P., and Thomson, W. L. (2000). Null hypothesis testing: problems, prevalence, and an alternative. J. Wildl. Manage. 64, 912–923. doi: 10.2307/3803199
Ardoin, N. M., Bowers, A. W., Roth, N. W., and Holthuis, N. (2017). Environmental education and K-12 student outcomes: a review and analysis of research. J. Environ. Educ. 49, 1–17. doi: 10.1080/00958964.2017.1366155
Ardoin, N. M., Wheaton, M., Bowers, A. W., Hunt, C. A., Durham, H., Ardoin, N. M., et al. (2015). Nature-based tourism's impact on environmental knowledge, attitudes, and behavior : a review and analysis of the literature and potential future research. J. Sustain. Tour. 23, 838–858. doi: 10.1080/09669582.2015.1024258
Aydeniz, M., and Southerland, S. A. (2012). A national survey of middle and high school science teachers' responses to standardized testing: is science being devalued in schools? J. Sci. Teacher Educ. 23, 233–257. doi: 10.1007/s10972-012-9266-3
Ballantyne, R., Fien, J., and Packer, J. (2001). Program effectiveness in facilitating lntergenerational influence in environmental education: lessons from the field. J. Environ. Educ. 32, 8–15. doi: 10.1080/00958960109598657
Barnett, M., Lord, C., Strauss, E., Rosca, C., Langford, H., Chavez, D., et al. (2006). Using the urban environment to engage youths in urban ecology field studies. J. Environ. Educ. 37, 3–11. doi: 10.3200/JOEE.37.2.3-11
Boris, E. T., and Kopczynski Winkler, M. (2013). “The emergence of performance measure- ment as a complement to evaluation among U.S. foundations,” in Performance Management and Evaluation. New Directions for Evaluation, eds S. B. Nielsen and D. E. K. Hunter, 69–80.
Braun, S. M. (2019). Outdoor School for All: Diverse Programming and Outcomes in Oregon: 2017 Pilot Study Evaluation (PNW XXX). Corvallis, OR: Oregon State University.
Breiting, S., and Wickenberg, P. (2010). The progressive development of environmental education in Sweden and Denmark. Environ. Educ. Res. 16, 9–37. doi: 10.1080/13504620903533221
Carleton-Hug, A., and Hug, J. W. (2010). Challenges and opportunities for evaluating environmental education programs. Eval. Program Plann. 33, 159–164. doi: 10.1016/j.evalprogplan.2009.07.005
Carrier, S. J. (2009a). Environmental education in the schoolyard: learning styles and gender. J. Environ. Educ. 40, 2–12. doi: 10.3200/JOEE.40.3.2-12
Carrier, S. J. (2009b). The effects of outdoor science lessons with elementary school students on preservice teachers' Self-Efficacy. J. Elem. Sci. Educ. 21, 35–48. doi: 10.1007/BF03173683
Carrier, S. J., and Stevenson, K. (2017). “Children experiencing the outdoors,” in Reforming Teaching and Teacher Education, eds E. Kimonen and R. Nevalainen (Rotterdam: SensePublishers), 67–88.
Carrier, S. J., Thomson, M. M., Tugurian, L. P., and Stevenson, K. T. (2014). Elementary science education in classrooms and outdoors: stakeholder views, gender, ethnicity, and testing. Int. J. Sci. Educ. 36, 2195–2220. doi: 10.1080/09500693.2014.917342
Carter, D. (2016). A nature-based social-emotional approach to supporting young children's holistic development in classrooms with and without walls: the Social-Emotional and Environmental Education Development (SEED) Framework. Int. J. Early Child. Environ. Educ. Int. J. Early Child. Environ. Educ. 4, 9–24.
Chanse, V., Mohamed, A., Wilson, S., Dalemarre, L., Leisnham, P. T., Rockler, A., et al. (2017). New approaches to facilitate learning from youth: exploring the use of Photovoice in identifying local watershed issues. J. Environ. Educ. 48, 109–120. doi: 10.1080/00958964.2016.1256260
Chawla, L. (1999). Life paths into effective environmental action. J. Environ. Educ. 31, 15–26. doi: 10.1080/00958969909598628
Chawla, L. (2010). Significant life experiences revisited: a review of research on sources of environmental sensitivity. J. Environ. Educ. 29, 11–21. doi: 10.1080/00958969809599114
Chawla, L. (2015). Benefits of nature contact for children. J. Plan. Lit. 30, 433–452. doi: 10.1177/0885412215595441
Cheng, J. C.-H., and Monroe, M. C. (2010). Connection to nature: children's affective attitude toward nature. Environ. Behav. 44, 31–49. doi: 10.1177/0013916510385082
Creswell, J. W. (2008). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Thousand Oaks, CA: Sage Publications, Inc.
Darling-Hammond, L., Bae, S., Cook-Harvey, C. M., Lam, L., Mercer, C., Podolsky, A., et al. (2016). Pathways to New Accountability Through the Every Student Succeeds Act. Palo Alto, CA. Available online at: http://learningpolicyinstitute.org/our-work/publications-resources/ pathways-new-accountability-every-student-succeeds-act.%0AThis
Ernst, J. (2009). Influences on US middle school teachers' use of environment-based education. Environ. Educ. Res. 15, 71–92. doi: 10.1080/13504620802710599
Ernst, J. (2012). Influences on and obstacles to K-12 administrators' support for environment-based education. J. Environ. Educ. 43, 73–92. doi: 10.1080/00958964.2011.602759
Finn, A. S., Kraft, M. A., West, M. R., Leonard, J. A., Bish, C. E., Martin, R. E., et al. (2014). Cognitive skills, student achievement tests, and schools. Psychol. Sci. 25, 736–744. doi: 10.1177/0956797613516008
Flowers, A. A., Carroll, J. P., Green, G. T., and Larson, L. R. (2015). Using art to assess environmental education outcomes. Environ. Educ. Res. 21, 846–864. doi: 10.1080/13504622.2014.959473
Guthery, F. S., Lusk, J. J., and Peterson, M. J. (2001). The fall of the null hypothesis: liabilities and opportunities. J. Wildl. Manage. 65, 379–384. doi: 10.2307/3803089
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., and Jennions, M. D. (2015). The extent and consequences of p-hacking in science. PLoS Biol. 13:e1002106. doi: 10.1371/journal.pbio.1002106
Heimlich, J. E. (2010). Environmental education evaluation: reinterpreting education as a strategy for meeting mission. Eval. Program Plann. 33, 180–185. doi: 10.1016/j.evalprogplan.2009.07.009
Jacob, B. A. (2002). Accountability, Incentives and Behavior: The impact of high-stakes testing in the Chicago Public Schools. Cambridge, MA. Available online at: http://www.nber.org/papers/w8968%0ANATIONAL
Jalongo, M. R., and Heider, K. (2006). Editorial teacher attrition: an issue of national concern. Early Child. Educ. J. 33, 379–380. doi: 10.1007/s10643-006-0122-y
Jones, M. G., Jones, B. D., and Hargrove, T. (2003). The Unintended Consequences of High Stakes Testing. Rowman and Littlefield Publishers, Inc.
Jordan, C., and Chawla, L. (in press). A coordinated agenda for nature-based learning. Front. Psychol.
Jorgenson, O., and Vanosdall, R. (2002). The death of science? Phi Delta Kappan 83, 601–605. doi: 10.1177/003172170208300808
Kimmons, R. (2015). Online system adoption and K-12 academic outcomes. J. Comput. Assist. Learn. 31, 378–391. doi: 10.1111/jcal.12101
Krasny, M. E., Mukute, M., Aguilar, O., Masilela, M. P., and Olvitt, L. (2016). “Community Environmental Education,” in Essays in Urban Environmental Education, eds M. E. Krasny and A. Kudryavtsev (Ithica, NY: Cornell University Press), 18–26.
Kuo, M., Browning, M. H. E. M., and Penner, M. L. (2018). Do lessons in nature boost subsequent classroom engagement? Refueling students in flight. Front. Psychol. 8:2253. doi: 10.3389/fpsyg.2017.02253
Legault, L., and Pelletier, L. G. (2000). Impact of environmental education program on students' and parents' attitudes, motivations and behaviours. Can. J. Behav. Sci. 32, 243–250. doi: 10.1037/h0087121
Lieberman, G. A., and Hoody, L. L. (1998). Closing the Achievement Gap: Using the Environment as an Integrating Context for Learning. Results of a Nationwide Study. San Diego, CA.
Monroe, M. C., and Krasny, M. E. (eds). (2016). Across the Spectrum: Recources for Environmental Educators, 3rd Edn. North American Association for Environmental Education
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie Du Sert, N., et al. (2017). A manifesto for reproducible science. Nat. Hum. Behav. 1, 1–9. doi: 10.1038/s41562-016-0021
NAAEE (2010). Excellence in Environmental Education: Guidelines for Learning (K.-12). Washington, DC: North American Association of Environmental Education.
NGSS Lead States. (2013). Next Generation Science Standards: For States, By States. Washington, DC: The National Academies Press.
NRC (2012). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, an Core Ideas. Washington, DC: National Academies Press.
Palmer, J. A. (2002). Environmental Education in the 21st Century: Theory, Practice, Progress and Promise. New York, NY: Routledge.
Piersol, L., and Timmerman, N. (2017). Reimagining environmental education within academia: storytelling and dialogue as lived ecofeminist politics. J. Environ. Educ. 48, 10–17. doi: 10.1080/00958964.2016.1249329
Roth, R. E. (1969). Fundamental concepts for environmental management education (K-16). Environ. Educ. 1, 65–74. doi: 10.1080/00139254.1969.10801495
Stapp, W. B. (1969). The concept of environmental education. J. Environ. Educ. 1, 30–31. doi: 10.1080/00139254.1969.10801479
State Education Environment Roundtable [SEER] (2000). The Effects of Environment-Based Education on Student Achievement. San Diego, CA. Available online at: http://www.seer.org/pages/research/educeff.pdf
Stern, M. J., Powell, R. B., and Hill, D. (2013). Environmental education program evaluation in the new millennium: what do we measure and what have we learned? Environ. Educ. Res. 20, 581–611. doi: 10.1080/13504622.2013.838749
Stevenson, K. T., Carrier, S. J., and Peterson, M. N. (2014). Evaluating strategies for inclusion of environmental literacy in the elementary school classroom. Electron. J. Sci. Educ. 18.
Stevenson, K. T., Peterson, M. N., Bondell, H. D., Mertig, A. G., and Moore, S. E. (2013). Environmental, institutional, and demographic predictors of environmental literacy among middle school children. PLoS ONE 8:e59519. doi: 10.1371/journal.pone.0059519
Stevenson, K. T., Peterson, M. N., and Dunn, R. R. (2017). Leveraging natural capital to solve the shared education and conservation crisis. Conserv. Biol. 32, 490–492. doi: 10.1111/cobi.13039
Szczytko, R., Carrier, S. J., and Stevenson, K. T. (2018). Impacts of outdoor environmental education on teacher reports of attention, behavior, and learning outcomes for students with emotional, cognitive, and behavioral disabilities. Front. Educ. 3:46. doi: 10.3389/feduc.2018.00046
Tienken, C. H., Colella, A., Angelillo, C., Fox, M., McCahill, K. R., and Wolfe, A. (2017). Predicting middle level state standardized test results using family and community demographic data. RMLE Online 40, 1–13. doi: 10.1080/19404476.2016.1252304
Trafimow, D. (2013). To disconfirm or not to disconfirm: a null prediction vs. no prediction. Front. Psychol. 4:733. doi: 10.3389/fpsyg.2013.00733
Twenge, J. M. (2017). iGen: Why Today's Super-Connected kids are Growing up Less Rebellious, More Tolerant, Less Happy: and Completely Unprepared for Adulthood. New York, NY: ATRIA books.
UNESCO (1977). “The Tbilisi declaration,”in Intergovernmental Conference on Environmental Education (Tbilisi: USSR), 1–8.
Vaske, J. J. (2008). Survey Research and Analysis: Applications in Parks, Recreation and Human Dimensions. State College, PA: Venture Publishing.
Wells, N. M., and Evans, G. W. (2003). Nearby nature: a buffer of life stress among rural children. Environ. Behav. 35, 311–330. doi: 10.1177/0013916503035003001
Wells, N. M., and Lekies, K. S. (2006). Nature and the life course: pathways from childhood nature experiences. Child. Youth Environ. 16, 1–25. Available online at: http://www.jstor.org/stable/10.7721/chilyoutenvi.16.1.0001
Keywords: academic outcomes, test scores, environmental education, nature-based learning, null effects
Citation: Stevenson KT, Peterson MN, Carrier SJ, Strnad RL, Olson RA and Szczytko RE (2019) Making the Case for a Null Effects Framework in Environmental Education and K-12 Academic Outcomes: When “Just as Good” Is a Great Thing. Front. Commun. 3:59. doi: 10.3389/fcomm.2018.00059
Received: 22 October 2018; Accepted: 12 December 2018;
Published: 07 January 2019.
Edited by:
Jessica Leigh Thompson, Northern Michigan University, United StatesReviewed by:
Shawn Kyle Davis, Slippery Rock University of Pennsylvania, United StatesAna K. Houseal, University of Wyoming, United States
Copyright © 2019 Stevenson, Peterson, Carrier, Strnad, Olson and Szczytko. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Kathryn T. Stevenson, a2F0aHJ5bl9zdGV2ZW5zb25AbmNzdS5lZHU=