AUTHOR=Staus Nancy L. , O'Connell Kari , Storksdieck Martin TITLE=Addressing the Ceiling Effect when Assessing STEM Out-Of-School Time Experiences JOURNAL=Frontiers in Education VOLUME=6 YEAR=2021 URL=https://www.frontiersin.org/journals/education/articles/10.3389/feduc.2021.690431 DOI=10.3389/feduc.2021.690431 ISSN=2504-284X ABSTRACT=

The aim of this paper is to describe an analytical approach for addressing the ceiling effect, a measurement limitation that affects research and evaluation efforts in informal STEM learning projects. The ceiling effect occurs when a large proportion of subjects begin a study with very high scores on the measured variable(s), such that participation in an educational experience cannot yield significant gains among these learners. This effect is widespread in informal science learning due to the self-selective nature of participation in these experiences, such that participants are already interested in and knowledgeable about the content area. When the ceiling effect is present, no conclusions can be drawn regarding the influence of an intervention on participants’ learning outcomes which could lead evaluators and funders to underestimate the positive effects of STEM programs. We discuss how the use of person-centered analytic approaches that segment samples in theory driven ways could help address the ceiling effect and provide an illustrative example using data from a recent evaluation of a STEM afterschool program.