Expanding the Boundaries of Informal Education Programs: An Investigation of the Role of Pre and Post-education Program Experiences and Dispositions on Youth STEM Learning

For generations educators have been supporting children and youth’s science, technology, engineering, and mathematics (STEM) learning through informal education programming. Such programming includes a wide variety of outdoor education programs, camp programs, and increasingly targeted STEM programs run afterschool, on weekends, and over the summer months. However, despite the positive impacts these programs have, few would argue that these programs could not be improved or be designed to better meet the needs of a broader and more diverse population of learners. Arguably, one major flaw in how most educators have approached the design and improvement of these programs—a flaw that permeates almost all informal STEM education efforts–is that either explicitly or implicitly, the focus of educators has been exclusively on what happens during the program itself. Superficially this seems reasonable. After all, the time children/youth are within the temporal and physical boundaries of the program, class, or museum is the time when educators have maximal control over events. However, given what is known about how people learn (National Academies of Sciences, 2018), we argue that this long-standing approach needs to be reconsidered.

Introduction

STEM Learning

Three key ideas underlie our current understanding of how people in general and children in particular learn STEM. The first key idea is that STEM learning is continuous, cumulative, and constructed. The second key idea is that STEM learning is highly personal and driven by individual needs and interests. The third key idea is that STEM learning is always situated within a complex learning ecosystem.

Children learn STEM across time and space, in and out of school, and by using a variety of community resources and networks (National Research Council (NRC), 2009; Stocklmayer et al., 2010; The Organization for Economic Co-operation and Development (OECD), 2012). The result is that STEM learning is rarely an instantaneous event, but rather an unfolding, cumulative process (National Academies of Sciences, 2018). Typically, individuals acquire STEM understanding through a continuous accumulation of experiences from many different sources and times (Lave and Wenger, 1991; Caillot and Nguyen-Xuan, 1995; Korpan et al, 1997; Anderson et al, 2000; Miller, 2010; Brotman et al, 2011). Thus, over a lifetime, individuals construct their knowledge about the world, as well how best to use that knowledge, not from one, but from literally hundreds if not thousands of STEM experiences and exposures. This “truism” relates to applications of knowledge and skills, and to perceptions of identity and self-efficacy. So, for example, one’s self perceptions of how to solve a STEM-related problem are equally a product of one’s prior knowledge and experience and one’s self-efficacy around creativity and STEM (National Academies of Sciences, 2018).

For most of the 20th century the prevailing view was that learning was a generalizable, linear, and predictable accumulation of knowledge. Everyone learned in the same way, and as long as the same information was consistently and appropriately presented, every individual would learn, and each would learn the same things. These ideas have often been described as the “transmission-absorption” model of learning (cf., Roschelle, 1995). However, despite the fact that the general process of learning is comparable in most humans, how these processes affect the products of learning are anything but comparable. Learning is a uniquely individual, idiosyncratic event; no two people learn exactly the same thing in quite the same way (Fosnot and Perry, 2005; National Academies of Sciences, 2018). Equally as important, is motivation for and receptivity to learning. Each learner’s unique interests, needs, prior experience, and motivations primarily drive these factors (Immordino-Yang, 2015; Falk and Dierking, 2018). Interest and its correlate, motivation, have been shown to be crucial drivers of learning, and therefore have become a topic of increasing importance to STEM learning researchers and practitioners (cf., Falk, et al., 2016a; Renninger and Hidi, 2016).

As described in a recent National Research Council’s report on out-of-school learning (National Research Council (NRC), 2015), today’s children learn across their entire lives, in and out of school. The acquisition of important STEM capabilities such as creativity, interest and understanding, as well as more generic abilities such as problem solving are supported by a wide range of in-school and out-of-school educational resources (National Research Council (NRC), 2009; Falk and Dierking, 2010; Nature, 2010; Stocklmayer et al, 2010; National Research Council (NRC), 2015). Collectively, these educational resources can be thought of as comprising a single, large, and complex ecosystem of learning (Traphagen and Traill, 2014; Falk, et al., 2015). The ecosystem concept is well suited to describing interactions between people and their environment including processes for learning and developing new knowledge in a variety of contexts (Jackson, 2013; Falk et al., 2020), as it has become increasingly clear that learning rarely occurs in discrete, bounded moments in time, but more typically, is the consequence of sets of cumulative experiences, across multiple learning platforms, e.g., organized classes, broadcast and print media, digital resources, and informal experiences at places like museums (Falk and Needham, 2013; Barron et al., 2014; Azevedo, 2015).

Despite the growing consensus around these three key understandings (cf., National Academies of Sciences, 2018), virtually all informal STEM education programs largely operate as if historic understandings of learning still pertain. Furthermore, even though children’s learning is continuous and incremental and situated within a larger ecosystem of learning, most informal programs are still designed, either explicitly or implicitly, as if learning begins when the child/youth enters the educational program and ends when they exit the program. Regardless of the knowledge that STEM learning is always driven by an individual’s prior experiences, interests, motivations and dispositions, informal educators continue to assume that participants of the same age all have the same educational starting point. Although educators have consistently sought to improve the design of their STEM education programs, either through better quality lessons, improved facilitation, or creation of more exciting, hands-on approaches, as suggested above, few have fully accommodated these three learning realities. As a consequence, improvement efforts in current informal STEM education programs have led to, at best, only incremental improvements in learner outcomes.

Important Pre-Experience Factors

When educators have considered pre-experience factors that might influence learner outcomes, they have typically focused on demographics. For example, it is long been argued and extensively documented that factors such as race-ethnicity, gender, and socioeconomic status are critical determinants of educational success (e.g., López, 2002; Riegle-Crumb, 2006; Carter, 2012; Becares and Priest, 2015; McGee, 2018). However, there are other, potentially equally important influencing factors. Research suggests that five other factors, all of which are non-demographic, might also influence informal education outcomes. In particular, these are: 1) The number and frequency of out-of-school experiences (e.g., Falk and Needham, 2013; Falk and Dierking, 2018; National Research Council (NRC), 2015; Tai and Maltese, 2010); 2) Parental attitudes and support (e.g., Archer, et al., 2010; Barron et al., 2009; Falk, et al., 2016b); 3) Personality constructs like sociability (often referred to as introversion vs. extroversion) (e.g., Topping, 2005; Noftle and Robins, 2007; Poropat, 2009); 4) Prior experiences (e.g., Archer, et al., 2010; Frenzel et al., 2010); and 5) children/youths’ self-related motivations for participating in an informal experience (Falk, 2009; 2018). In theory, any one of these factors, for example the motivations of why a child might choose to attend an informal experience, could significantly change the trajectory of a child’s long-term learning pathway, but detailed research is lacking.

In other words, the ability of any particular informal STEM education program to influence the short and long-term learning trajectories of children is in theory only partially determined by the quality of what happens during a particular informal program. The fact is, these programs likely make a disproportionate impact on children relative to the time invested, but other factors, over and beyond these programs also matter, including affecting how particular children are likely to benefit from such programs. As argued by Falk and colleagues (Falk and Dierking, 2018; Falk, Koke, Price and Pattison, 2018), no single experience or factor is likely to be the sole causative influence on learning outcomes. For most children/youth, most of the time, all experiences and life factors work synergistically, though one or another may be disproportionately important. The question remains, though, is it possible to determine for any particular educational experience, which of these many factors/variables might be disproportionately important? The exploratory research described here was an effort to investigate the relative impact of a range of pre-entry conditions on the short and long-term outcomes of children participating in a one-week summer STEM camp experience with the hope of better understanding which, if any of these factors most contributed to children/youth’s STEM learning, and if so how this information might be used to advance the quality of informal STEM education.

Materials and Methods

This study was designed to explore the impacts pre-program experiences and dispositions had on the learning outcomes of youth participants in a one-week STEM and invention-focused, summer camp program. The program in question was called Camp Invention^®. We used the outcome variables emphasized by this particular informal education program, specifically creativity, STEM interest and problem solving. We selected as independent variables five possible factors we felt were potentially important and readily measurable: parental support, introversion/extroversion, motivations for attending, the number of prior STEM-related experiences and prior experiences with this particular summer camp program.

Design In order to investigate our research questions, we collected data from “Current Participants” and “Previous Participants.” The Current Participant group was derived from children attending Camp Invention^® during the summer of 2017. The initial intent was to include only campers that were 11–12 years old. However due to low participation rates early in the data collection process, the decision was made to also recruit 10–11 year old youth. We collected data from current participants at three points in time: 1) Time-point one (T1), was an online survey taken two to four weeks prior to the start of camp; 2) Time-point two (T2), was a paper survey administered during the last day of camp; and 3) Time-point three (T3) was an online survey intended to be completed by participants two to three months after the conclusion of Camp.

The Previous Participant group was derived from campers who had attended Camp Invention^® when they were 11–12 years old prior to 2017. Data was collected for four previous Camp Invention^® participant cohorts, those attending Camp in 2016, 2015, 2014, and 2013. We measured the study variables across all groups and timepoints.

Participants

The Current Participant study group was intended to be a longitudinal, within-subjects design study, with the sample composed of individuals tracked over all three time-points. Recruitment for the Current Participant group was performed via email. A survey of 2017 Camp Invention^® program and participant rosters was utilized to target camp locations with the most registered 10–12 year old youth. Parents were emailed a recruitment request outlining the goals of the research, a link to the T1 online survey, and were asked for their written consent to collect data from their child across all three time-points of the study. Parents were instructed to encourage their children to complete the survey on their own, however, they could assist their children if the children requested or required it. There was no time limit for survey completion, however, it would have had to be completed in “one sitting” as a second accessing of the survey link would have potentially initiated a second survey for the same child. Solicitation and data collection for the T1 survey continued until the minimum target sample size of 500 was achieved. A total of 560 10–12 year old youth collectively registered to attend 153 different camps, completed the T1 survey.

The T2 survey was in paper format and was administered by camp staff on the last day of camp. Camp staff were instructed to encourage the children to complete the survey on their own, however, they could assist the children if the children requested or required it. Staff were instructed to allot 15 min for students to complete the survey; though there was no actual enforcement of a time limit on completion. Ten to 12 year old campers who had not taken the T1 survey were invited to complete the T2 survey provided that they had returned a completed parental consent form that was made available on the first day of camp. In total, 991 T2 surveys were completed from 100 camp locations.

The T3 survey was an online survey. The web-link for the T3 survey was emailed to the parents of all campers who completed either the T1, the T2, or both surveys. Again, parents were instructed to encourage their children to complete the survey on their own, however, they could assist their children if the children requested or required it and no time limit was imposed. Of the 196 participants that completed the T3 survey, only 100 had also completed the T1 and the T2. Given the exploratory nature of this study and the greater importance of the delayed post-test, we felt it justified to just focus on the comparison between T1 and T3. Accordingly, we opted to compare the responses of the 560 participants who had completed a T1 survey–pre-camp sample–with the responses of the 196 participants completing a T3 survey–short-term post-camp sample - from a between-subjects design perspective.

To recruit participants in the Previous Participant study, Camp Invention^® participant rosters from 2016, 2015, 2014, and 2013 were utilized to obtain parent email addresses for campers who were 11–12 years old at the time of their Camp participation for each of the four target years. Parents were emailed a recruitment request outlining the goals of the research, a link to the online survey for their child’s cohort, and were asked for their consent and their child’s participation. Parents were instructed to encourage their children to complete the survey on their own, however, they could assist their children if the children requested or required it and no time limit was imposed. Again to maximize statistical power, data was collapsed into a single Past Participant group for the purposes of statistical analyses. A total time-point TX sample was 352 youth. Table 1 summarizes data collection statistics for both the Current Participants and Previous Participants study samples. No significant sex differences were found for any of the time-points.

TABLE 1

TABLE 1. Current and previous participant recruitment and survey completion statistics.

Intervention

Camp Invention^® is a summer day-camp program in which children between the ages of 5 and 12 years engage in hands-on activities that promote STEM interest and participation and the building of 21^st century learning skills such as creativity and problem solving (cf., Trilling and Fadel, 2009), through the lenses of invention and entrepreneurship. Developed by educators, the curriculum aligns with state and national standards. Hundreds of schools and districts across the country host Camp Invention^® programs, with millions of children having participated over the 27 years of its existence. Evaluation studies (e.g., ChangeMaker Consulting, 2014; Kent State University, 2004:; Scarisbrick-Hauser and Hauser, 2009) have reported that children who participate in Camp Invention^® showed significant short-term improvements in the program’s defined goals of increasing children’s creativity, STEM interest, and problem-solving skills.

Dependent Variables

Given that creativity, STEM interest, and problem-solving skills were both widely shared programmatic goals for other informal education experiences, and there was strong, pre-existing evidence that the Camp Invention^® experience resulted in a majority, but likely not all of children in the program achieving some measure of these outcomes, we opted to use these three areas of learning as dependent variables. All variables were measured across multiple time points as designated.

To investigate creativity, we utilized existing, age-appropriate measures for Mechanical Science Creativity and Creative Self-Efficacy. For STEM interest, we utilized existing, age-appropriate measures for Science Relevance, Self-Concept in Science, and Science Interest. Finally, we utilized an existing, age-appropriate measure of Critical Thinking to investigate problem-solving skills.

Mechanical Science Creativity was measured using five items from the Mechanical/Scientific domain of Kaufman’s Domains of Creativity Scale (Kaufman, 2012). All items were captured using a six-point Likert scale ranging from one, “Much less creative” to six, “Much more creative.” Participants were provided the prompt of, “Compared to kids your age, how creative would you rate yourself for the following?” Example items include, “Carving something out of wood or similar material” and “Helping to carry out or design a science experiment.”

Creative Self-Efficacy was measured using seven items from the Short Scale of the Creative Self (Karwowskiet al., 2018). All items were captured using a six-point Likert scale ranging from one, “Strongly Disagree” to six, “Strongly Agree.” Participants were provided the prompt of, “Tell us how much you disagree or agree with each of the statements below.” Example items include, “I think I am a creative person” and “Being a creative person is important to me.”

Science Relevance, Self-Concept in Science, and Science Interest were all measured using items drawn from the ASPIRE survey (DeWitt et al., 2011) and ROSE Questionnaire (Schreiner and Sjøberg, 2004). Science Relevance and Self-Concept in Science items were captured using six-point Likert scales ranging from one, “Strongly Disagree” to six, “Strongly Agree.” Participants were provided the prompt of, “Tell us how much you disagree or agree with each of the statements below.” Example items include, “Science and engineering tell us about how people think and behave” for Science Relevance and “I find Science and engineering harder than most subjects” (reverse coded) for Self-Concept in Science. Science Interest items were captured using a six-point Likert scale ranging from one, “Dislike a lot” to six, “Like a lot.” Participants were provided the prompt of, “How much do you like finding out about the following things either in or out of school?” Example items include, “Mixing materials together to see what happens” and “What it’s like on other planets and exploring space.”

Critical Thinking was measured using six items developed by the research team. All items were captured using a five-point Likert scale ranging from one, “Never” to five, “Always.” Participants were provided the prompt of, “How often do the following things happen?” Example items include, “When facing a problem, I always think of lots of options” and “I keep my mind open to different ideas when planning to make a decision.” Principle Components Analysis results for these five items revealed factor loadings ranging from 0.497 to 0.751 and the single factor solution for the five items explained over 41% of the variance in item responses.

Independent Variables

Next, based upon the literature cited above, we created a group of five “Potential Influence” variables that we hypothesized might have varying influence on these outcome variables:

• Out-of-School STEM Experiences (other than Camp Invention^®)

• Parental Attitudes toward science and engineering

• Sociability (a personality measure of introversion vs. extroversion)

• Reason for Attending Camp (was the motivation related to content/learning or some other reason such as being with friends or parent made them go)

• Prior Camp (Invention) (s) Attended (was a proxy for prior experience and social capital).

As above, all items were based upon pre-existing, validated, age-appropriate measures.

Out-of-School Experiences were measured using five items derived from the Multiple Institute Science Center Effects Study (Falk et al., 2017). All items were captured using a six-point Likert scale ranging from one, “Never” to six, “Almost daily.” Participants were provided the prompt of, “Not including homework of stuff for school, how often do you do the following things outside of school.” Example items include, “Use the internet to search for or learn about science or engineering related topics” and “Read books or magazine articles about science or engineering.”

Youth perceptions of their Parent’s Attitudes were measured using four items from the ASPIRE survey (DeWitt et al., 2011). All items were captured using a six-point Likert scale ranging from one, “Strongly Disagree” to six, “Strongly Agree.” Participants were provided the prompt of, “Tell us how much you disagree or agree with each of the statements below.” Example items include, “My parents want me to become a scientist or engineer when I grow up” and “My parents expect me to do well in school, especially in science.” T1 scores for all youth were divided at the median. Due to the positively skewed results–in general youth scored toward higher numbers on the Likert scale–a median split was used for dividing youth into low and high categories. Individuals with T1 scores less than the median were considered “Low” on these two dimensions while individuals with T1 scores on these two dimensions scoring greater than the median were considered “High.”

Sociability was comprised of three separate items and measured on a five-point Likert scale ranging from one, “Almost never” to five, “Almost always.” The three items were “I like to meet with other people,” “I like to be with others,” and “I like to talk with others.” Collectively, these items came from existing personality measures of introversion/extroversion (Noftle and Robins, 2007; Thompson, 2008). As with perceptions of Parent’s Attitudes, responses were skewed toward higher numbers on the Likert scale. Using a median split, greater sociability (i.e., extroversion for the purposes of this analysis) was designated as “High” while low sociability (i.e., introversion for the purposes of this analysis) was designated as “Low.”

Reason for Attending Camp was a dichotomous variable derived from five potential reasons the participant was or had attended camp. The five potential reasons were, “I want to be with my friends,” “I don’t want to go but my parent/guardian is making me go,” “I want to learn about invention,” “I want to learn about science and engineering,” and “I just want to do something fun during the summer.” Participants were asked to rank the statements in the order that they were most appropriate to them. Ranking a statement with the number 5 indicated that it was the most appropriate, or matching, statement for that participant’s reason for attending camp. Ranking a statement with the number 4 indicated that it was the second most appropriate, etc. Participants who indicated “I want to learn about invention” or “I want to learn about science and engineering” as their most appropriate reason for attending were pooled into one “learning” category, and for the purposes of this analysis, were considered “High.” Participants who chose any of the other three statements as their most appropriate reason for attending were pooled into a “not-for-learning” category, and for the purposes of this analysis, were considered “Low.”

Prior Camps Attended was measured by the number of times a participant had previously attended a Camp Invention^® program. Individuals who had never attended Camp Invention^® before or only attended once were designated as “Low.” Individuals who had attended two or more Camp Invention^® camps before were designated as “High.”

Within the scope of the aforementioned variables, one goal of this study was to find out which children benefitted the most from Camp Invention^® and which children benefitted the least and why. Additionally, we wanted to investigate if the influences of the predictor variables on the outcome variables persisted over time, and if so, for which children, why and over what duration.

Results

An Independent Samples t-Test was conducted to investigate differences in outcome mean scores between time-points T1 and T3 for the Current Participant group. In keeping with the exploratory nature of this research, “significance” was defined as having a probability of 0.1 or less (that the likelihood of a result occurring randomly or by chance was less than one in a ten). NOTE: In tables, a probabilities of <0.05 are indicated in bold; probabilities of <0.1 are indicated in underlined; and probabilities of >0.1 are indicated in italic.

Table 2 summarizes the differences between the T1 aggregate and T3 aggregate outcome variable mean scores. The short-term changes in mean scores in youth attending Camp Invention^® between the 560 T1 (just prior to attending Camp) and 196 T3 (a couple of months subsequent to attending Camp) participants were statistically significant and positive for the measures of Creative-Self Efficacy (t (754) = −2.731, p = 0.007) and Science Interest (t (754) = −3.271, p = 0.001).

TABLE 2

TABLE 2. Aggregate T1 to T3 outcome variable means independent samples t-Test.

An additional Independent Samples t-Test was conducted to investigate differences in outcome mean scores between the T1 aggregate group and the combined Past Participant group samples. Table 3 summarizes the differences between the 560 T1 aggregate and 352 Past Participant’s outcome variable mean scores. There were positive and statistically significant differences in four out of the five scores–Creative-Self Efficacy, Science Relevance, Self-Concept in Science, Science Interest, and Critical Thinking. The lone exception was Mechanical Science Creativity which showed no significant change between the two groups.

TABLE 3

TABLE 3. Aggregate T1 to past participant outcome variable means independent samples t-Test.

Pre-Camp

A series of Simple Regression models were conducted for the Current Participant sample to investigate the difference in predicted Outcome variable scores at time T3 based on High or Low T1 Potential Influence scores. Table 4 summarizes the results of these Simple Regression analyses for Current Participants on each of the six dependent variables–Mechanical Science Creativity, Creative Self-Efficacy, Science Relevance, Self-Concept in Science, Science Interest, and Critical Thinking. All statistically significant differences (bold or italic) in Table 4 represent a positive difference between the High and Low Potential Indicator variables. In other words, the mean scores of the High group are significantly greater than the mean scores of the Low group.

TABLE 4

TABLE 4. Current participant differences in T3 Outcome variable scores as a function of high/low T1 Potential Indicator variables.

Out-of-School Experiences explained a significant proportion of the variance for each of these six statistically significant outcome variables. The following represent the variance explained and effect sizes for each of the six outcome variables: Mechanical Science Creativity (R² = .089, F (1,95) = 9.328, p = 0.003), (β = 0.510, t (95) = 3.054, p = 0.003). Science Relevance (R² = 0.232, F (1,96) = 29.033, p < 0.001), (β = 0.837, t (96) = 5.388, p < 0.001). Self-Concept in Science (R² = 0.118, F (1,94) = 12.629, p = 0.001), (β = 0.625, t (94) = 3.554, p = 0.001). And Science Interest (R² = 0.059, F (1,96) = 5.985, p = 0.016), (β = 0.398, t (96) = 2.446, p = 0.016).

Parental Attitudes explained a significant proportion of the variance for three of the six outcome variables. The following represent the variance explained and effect sizes for each of these three statistically significant outcome variables: Mechanical Science Creativity (R² = 0.053, F (1,96) = 5.374, p = 0.023), (β = 0.391, t (96) = 2.318, p = 0.023). Science Relevance (R² = 0.182, F (1,97) = 21.615, p < 0.001), (β = 0.738, t (97) = 4.649, p < 0.001). And Self-Concept in Science (R² = 0.032, F (1,95) = 3.178, p = 0.078), (β = 0.327, t (95) = 1.783, p = 0.078).

Previous Camps explained a significant proportion of the variance for three of the six outcome variables. The following represent the variance explained and effect sizes for each of these three statistically significant outcome variables: Creative Self-Efficacy (R² = 0.018, F (1,95) = 1.745, p < 0.001), (β = 0.596, t (95) = 3.410, p < 0.001). And Critical Thinking (R² = 0.106, F (1,97) = 11.492, p = 0.001), (β = 0.358, t (97) = 3.390, p = 0.001).

Reason for Attending explained a significant proportion of the variance for three of the six outcome variables. The following represent the variance explained and effect sizes for each of these three statistically significant outcome variables: Mechanical Science Creativity (R² = 0.028, F (1,95) = 4.714, p = 0.054), (β = 0.341, t (95) = 1.955, p = 0.054). Science Relevance (R² = 0.155, F (1,93) = 17.000, p < 0.001), (β = 0.684, t (93) = 4.123, p < 0.001). And Self-Concept in Science (R² = 0.041, F (1,92) = 3.959, p = 0.050), (β = 0.374, t (92) = 1.990, p = 0.050).

Sociability did not explain a significant proportion of the variance for any of the six outcome variables.

As expected, only some Current Participant youth were classified as being in the better performing half (as measured in this study) of the four significant Potential Influence variables. In fact, only 12 of the 560 T1 participants, 2.1%, fell within the “high” categories of Out-of-School Experiences, Parental Attitudes, went to Camp Invention^® for a learning reason, and had previously attended Camp Invention^® at least two or more times prior to attending the current Camp Invention^® program.

Post-Camp

Another series of Simple Regression models were conducted for the Past Participant group to investigate the effects that on-going Potential Influence variables had on Outcome variable scores. As with Current Participants, Table 5 summarizes the results of these Simple Regression analyses for Past Participants on each of the six dependent variables–Mechanical Science Creativity, Creative Self-Efficacy, Science Relevance, Self-Concept in Science, Science Interest and Critical Thinking. As above, statistically significant differences (bold) in Table 5 represent a positive difference between the High and Low Potential Indicator variables. In other words, the mean scores of the High group are significantly greater than the mean scores of the Low group.

TABLE 5

TABLE 5. Past participant difference in predicted outcome variable scores based on high/low predictor variable group.

Out-of-School Experiences explained a significant proportion of the variance for all six of the six outcome variables. The following represent the variance explained and effect sizes for each of these six statistically significant outcome variables: Mechanical Science Creativity (R² = 0.143, F (1,334) = 55.92, p < 0.001), (β = 0.579, t (334) = 6.605, p < 0.001). Creative Self-Efficacy (R² = 0.082, F (1,336) = 29.64, p < 0.001), (β = 0.500, t (336) = 6.101, p < 0.001). Science Relevance (R² = 0.198, F (1,336) = 82.80, p < 0.001), (β = 0.670, t (336) = 8.258, p < 0.001). Self-Concept in Science (R² = 0.112, F (1,336) = 42.32, p < 0.001), (β = 0.651, t (336) = 6.558, p < 0.001). Science Interest (R² = 0.096, F (1,336) = 35.85, p < 0.001), (β = 0.326, t (336) = 4.265, p < 0.001). And Critical Thinking (R² = 0.021, F (1,336) = 7.24, p = 0.003), (β = 0.186, t (336) = 3.019, p = 0.003).

Parental Attitudes also explained a significant proportion of the variance for all six of the six outcome variables. The following represent the variance explained and effect sizes for each of these outcome variables: Mechanical Science Creativity (R² = 0.087, F (1,336) = 31.83, p < 0.001), (β = 0.437, t (336) = 4.855, p < 0.001). Creative Self-Efficacy (R² = 0.107, F (1,336) = 40.06, p < 0.001), (β = 0.352, t (336) = 4.177, p < 0.001). Science Relevance (R²= 0.250, F (1,336) = 111.81, p < 0.001), (β = 0.677, t (336) = 8.377, p < 0.001). Self-Concept in Science (R² = 0.132, F (1,336) = 50.83, p < 0.001), (β = 0.577, t (336) = 5.734, p < 0.001). Science Interest (R² = 0.039, F (1,336) = 13.73, p = 0.002), (β = 0.243, t (336) = 3.146, p = 0.002). And Critical Thinking (R² = 0.018, F (1,336) = 6.39, p = 0.030), (β = 0.135, t (336) = 2.175, p = 0.030).

Previous Camps did not explain a significant proportion of the variance for any of the six outcome variables.

Reason for Attending explained a significant proportion of the variance for five of the six outcome variables. The following represent the variance explained and effect sizes for each of these five statistically significant outcome variables: Mechanical Science Creativity (R² = 0.027, F (1,336) = 9.16, p = 0.002), (β = 0.289, t (336) = −3.093, p = 0.002). Creative Self-Efficacy (R² = 0.011, F (1,336) = 3.83, p = 0.048), (β = 0.174, t (336) = −1.980, p = 0.048). Science Relevance (R² = 0.044, F (1,336) = 15.70, p < 0.001), (β = 0.353, t (336) = −3.992, p < 0.001). And Self-Concept in Science (R² = 0.020, F (1,336) = 7.09, p = 0.008), (β = 0.287, t (336) = −2.686, p = 0.008).

Sociability explained a significant proportion of the variance for five of the six outcome variables. The following represent the variance explained and effect sizes for each of these five statistically significant outcome variables: Mechanical Science Creativity (R² = 0.026, F (1,336) = 8.96, p = 0.049), (β = 0.183, t (336) = 1.976, p = 0.049). Creative Self-Efficacy (R² = 0.039, F (1,336) = 13.56, p = 0.012), (β = 0.215, t (336) = 2.512, p = 0.012). Science Relevance (R² = 0.030, F (1,336) = 10.45, p = 0.002), (β = 0.271, t (336) = 3.097, p = 0.002). Science Interest (R² = 0.030, F (1,336) = 10.51, p = 0.002), (β = 0.245, t (336) = 3.172, p = 0.002). And Critical Thinking (R²= 0.081, F (1,336) = 29.76, p < 0.001), (β = 0.255, t (336) = 4.174, p < 0.001).

As with Current Participants, only some Past Participant youth were classified as falling within the better performing half (as measured in this study) of the five Potential Influence variables. Twenty-two of the 352 TX participants, 6.3%, fell within the “high” categories of Out-of-School Experiences, Parental Attitudes, High Sociability, went to Camp Invention^® for a learning reason, and had previously attended Camp Invention^® at least two or more times prior to attending the current Camp Invention^® program.

Discussion

The goal of this exploratory research was to investigate whether it was possible to identify one or more non-Camp Invention®-related factors/variables–Potential Influence variables–which, in interaction with experiences occurring at Camp Invention^®, might significantly contribute to 10 to 12 year-old youth’s short and long-term changes in positive STEM-learning-related outcomes, in particular creativity, STEM interest, and problem solving. To achieve this, we collected and analyzed data from 10 through 12 year-old youth–with data collected prior to entering Camp Invention^®, and two to three months after the conclusion of this camp experience. We also sampled groups of youth who had participated in this educational camp experience variously 1–4 years previously.

Importantly, as predicted, there was evidence that participation in one week of Camp Invention^® resulted in statistically significant short-term improvements for some participating youth for some of the Outcome variables. In particular, measures of creativity and STEM interests. In the short-term, there was no evidence of significant improvements in problem-solving skills. Over the long-term, e.g., time periods of anywhere to 1–4 years post-Camp Invention^®, there was strong evidence of significant growth in the three topic areas of creativity, STEM interest and problem solving (with the exception of Mechanical Science Creativity).

However also as predicted, although overall youth showed statistically significant improvements in their abilities in most of these three key educational areas, there was a distribution in the data. In other words, some youth showed considerable improvements in each of these three Outcome variables and some youth exhibited only small or no improvement.

Based upon the literature cited above, we hypothesized that a range of other non-Camp Invention®-related experiences, proclivities and factors–Potential Influences–might have contributed to this distribution of outcomes. Specifically, that depending upon either a youth’s pre or post-Camp Invention^® experiences or proclivities, that youth might end up having a more “successful” Camp Invention^® experience than others. Accordingly, we created a series of survey items designed to assess, pre-camp and post-camp, the relative strength/presence of the following five Potential Influence variables:

• Out-of-School STEM Experiences (other than Camp Invention^®)

• Parental Attitudes toward science and engineering

• Sociability (a personality measure of introversion vs. extroversion)

• Reason for Attending Camp (was the motivation related to content/learning or some other reason such as being with friends or parent made them go)

• Number of prior Camp Inventions^® Attended (as a proxy for relevant prior experience and social capital).

Pre-Camp

There was evidence, that all of these Potential Influence variables did indeed have an effect on the educational Outcomes of Camp Invention^®, although the influences were not uniform. Youth who had High levels of prior (non-Camp Invention^®) out-of-school experiences showed greater growth across creativity and science interest educational Outcomes as compared with youth with limited or low prior (non-Camp Invention^®) out-of-school experiences. Sociability did not appear to affect Camp Invention^® educational Outcomes in the short-term, but emerges as an influence over the longer term. The other three pre-camp Potential Influence variables–parental attitudes, reason for attending camp, and prior Camp Invention^® experiences–each influenced some of the six measured educational Outcome variables.

Specifically, the results suggested:

• For both creativity and STEM interest,—youth with considerable prior experience in learning STEM outside of school prior to entering camp appeared to benefit more from the Camp Invention^® experience than did youth with limited or no such prior experiences.

• Youth who entered Camp Invention^® with strong perceived parental support for learning about STEM appeared to be much more likely to show gains in STEM interest, as well as somewhat more likely to show improvements in creativity than did youth with lower levels of parental STEM support.

• Youth who went to Camp Invention^® with the expectation that they would learn more about STEM, inventions, or creativity appeared to be much more likely to show gains in STEM interest, and to a degree, creativity than were youth who went to Camp because of other reasons.

• The knowledge, skills and/or social relationships that previous Camp Invention^® experiences engendered appeared to be particularly important for enhancing creativity and problem solving as evidenced by the fact that youth with multiple, prior Camp Invention^® experiences showed significantly higher gains in these areas than did youth with no or only limited Camp Invention^® experience.

Also, important to note, was that only a very small fraction of youth, 2.1%, were in the better achieving half (as measured in this study) on all four of these key independent variables–Out-of-School Experiences, Parental Attitudes, went to camp for a learning reason, and had attended Camp Invention^® at least twice before–and thus optimally pre-positioned to benefit from the Camp Invention^® experience.

Post-Camp

There was even stronger evidence that all of these Potential Influence variables had a post-Camp effect on the educational Outcomes of interest to Camp Invention^®. The results from this longer-term study suggested that over time, the interactions between these, and no doubt other variables, created strong influences on youth creativity, STEM interest, and problem solving. In the first study, which sampled a few months of a youth’s life, the one-week Camp Invention^® experience represented a relatively large, highly salient “dosage” of experience. In the longer-term study, which sampled on average several years of a youth’s life, the one-week Camp Invention^® experience represented a relatively small “dosage” of experience; albeit likely still a salient one.

Youth who had high levels of non-Camp Invention^® out-of-school experiences consistently showed significantly higher levels of creativity, STEM interest, and problem solving than did youth with limited or low (non-Camp Invention^®) out-of-school experiences. The same was also true for youth with high levels of perceived parental support. In this longer-term sample, more social youth also consistently showed significantly higher levels of creativity, STEM interest, and problem solving than did less social youth.

Specifically, the results suggested:

• Across two key areas of education outcomes related to entrepreneurship–creativity and STEM interest–youth with considerable and presumably on-going experiences in learning STEM outside of school appeared to be better able to build on their Camp Invention^® experiences and sustain their gains in these three areas than did youth with limited or no such experiences.

• Youth who had strong perceived parental support for learning about STEM appeared to be much more likely than youth with low perceived parental support to be better able to build on their Camp Invention^® experiences and sustain their gains in creativity, STEM interest, and problem solving.

• Youth who were more social appeared to be much more likely than less social youth to be better able to build on their Camp Invention^® experiences and sustain their gains in creativity, STEM interest, and problem solving. [NOTE: This is a very provocative finding since it defies the stereotype of the introverted science geek.]

However, the fact that this variable did not emerge as significant in the short-term but did in the longer term makes it difficult to fully explain the role that sociability might be playing here and suggests the need for further exploration in the future.

• Youth who went to Camp Invention^® with the expectation that they would learn more about STEM, inventions, or creativity appeared to be much more likely than were youth who went to Camp because of other reasons to maintain high levels of creativity and a strong STEM interest.

And just as in the earlier study, only a small fraction of youth, 6.3%, were “high” on all five of the Potential Influence variables tested–Out-of-School Experiences, Parental Attitudes, and (high) Sociability, went to camp for a learning reason, and had attended Camp Invention^® at least twice before–and thus likely to optimally benefit from the contributions made by Camp Invention^®.

Limitations

As with all social science research, this study had limitations that need to be acknowledged. The majority of the surveys were conducted online via links emailed to parents. As a consequence, participation was limited to children whose parent/caregiver had listed an email address with Camp Invention^® and, after receiving the email, could access the internet. This clearly had the potential to bias the sample toward higher SES participants. Another potential bias was participant self-selection bias. Although we cannot know for sure whether those who self-selected to participate were disproportionately engaged and positive about their experience, it is fair to assume that this was the case. Given that the bottleneck in data collection was in getting longer-term data, i.e., the delayed post-test (T3) for the current campers and for all years of the long-term retrospective sample of youth (TX), it has to be assumed that some kind of self-selection was present in those who opted to respond to these longer-term surveys. Those who responded may have already been the most engaged and interested youth, though of course we have no way to know this for sure, and even if this was true, it is not clear how this bias would have impacted the major outcomes reported in this exploratory study.

The sample sizes for both data sets were smaller than the research team had hoped for, particularly the final short-term, immediate post-camp experience, thus limiting our ability to make generalizations. Limitations in sample sizes also required us to “lump” all of the long-term retrospective youth into a single population, despite the likely effects that developmental differences might have created, particularly in outcomes like self-concept.

In all studies of this nature, there are assumptions the measures one uses are valid and reliable indicators of the variables being considered, but of course this may or may not be the case. Since, by necessity due to the constraints of collecting data from youth within a free-choice context, the number of items used for each construct needed to be limited to increase the likelihood of survey completion, this too had the potential to reduce both validity and reliability.

Finally, due to the typical constraints of time and money, this effort only investigated a relatively modest number of Outcome variables and Potential Influence variables. There is no reason to believe that the particular variables selected for inclusion in this study represented either the most important outcomes possible from a STEM-related informal education experience nor the only influencing variables likely to result in significant effects, or even, after further study, would emerge as the most important influencing variables.

Implications

The above caveats notwithstanding, this study very successfully accomplished the goals it set out to achieve. From the start, this research was designed to be exploratory. Although the results presented are not definitive, and focused on only a single informal education experience, we feel comfortable stating that the findings are likely indicative of the tens of millions of youth who participate in informal STEM education programs annually around the world. In other words, the goal of this research was to explore the possibility that some learners, by virtue of their prior or subsequent experiences, proclivities, interests, and/or types of STEM-related support at home benefited more from a week-long informal education experience than did other learners. And if so, then it would suggest that these non-programmatic factors/variables are sufficiently important (i.e., have the potential to affect informal program learning outcomes) that informal education staff at this particular program, as well as potentially the staff of other similar types of programs, might want to think about how to accommodate, reinforce, support, and ameliorate these effects.

Based on the review of literature summarized above as well as the data gathered from these studies, we would hypothesize that the most cost-effective ways to improve the educational impact of informal education programs would be for educators to consider making modifications to their educational practices in one or some combination of the following three key areas:

1. Modifications in how children are prepared for participation in a program;

2. Changes to in-program experiences that allow for greater customization of experiences in order to better accommodate the differing needs and experiences of participating children; and

3. Implementation of strategies for proactively and mindfully supporting experiences post-program that both reinforce short-term changes in outcomes such as creativity, STEM interest, and problem-solving skills and also leverage opportunities to support these changes over time.

Below are some possible ideas for the kinds of changes in educational practice this research might suggest.

Modifications in Pre-Camp Preparation

Obviously, programs like Camp Invention^® have only limited ways in which they can change the nature of youth prior to entering an informal education experience, but limited is not the same as none. A few suggested things informal education staff might consider doing are:

• Investing greater time and energy in helping youth think about how participation in their program might extend “learning-related” outcomes. Through pre-program materials, including potentially short YouTube videos, staff should work to reinforce how much fun learning about STEM, creativity and problem solving are likely to be–both during the experience, but equally prior and subsequent to the structured programmatic experience.

• Investing greater time and effort in helping parents know how critical is their support and encouragement for their child’s learning. Obviously, getting a youth to participate in a program is an important indication of that support but so too is supporting youth at other times and even during the program period. Staff should develop and provide parents and caregivers with additional parent-learning tools designed to help parents know how to provide this kind of support on an on-going basis.

• Given that there appear to be additive and synergistic effects of multiple out-of-school STEM-related experiences, informal education providers should continue to find ways to partner with other STEM organizations that offer out-of-school experiences and build ever-greater mechanisms for supporting each other’s efforts.

Greater Customization of In-Program Experiences

One of the legacies of 20^th century educational models was a tendency to create “one-size-fits-all” solutions–one set of programs that all children do. Research such as this suggest that more customized, individualized experiences better accommodate the differing needs and backgrounds of participating children and that individualization can potentially pay important educational dividends. A few possible ideas that informal science educators might consider are:

• Inclusion of a few diagnostic questions on pre-program materials that help alert program educators to dispositions and experiences of entering children/youth so that compensatory programming and opportunities can be developed.

• In general, providing more opportunities for children/youth to have some choice and control over the nature of their actual experiences. For example, although low vs. high sociability did not emerge as a consistently significant factor in determining learning outcomes, it did appear to be potentially an issue for some youth over the longer term. Given the current trend in education toward “group work,” informal educators might want to explore ways to create more opportunities for youth to self-select whether they prefer to work by themselves or in a group, as well as have options beyond a “public presentation” for sharing with others what they have accomplished.

• Since it appears possible that youth with prior out-of-school and prior Camp Invention^® experience disproportionately benefit from the informal experiences, informal education staff might try to think further about the relationship between these types of experiences. Is it that these children have a better ability to navigate the daily activities and schedules? Is it that they feel more comfortable with roles and relationships? Is it perhaps that they already understand something about the educational processes and pedagogical approaches that underlie the specific models used within a particular educational approach? If it could be figured out why these “advantages” seem important, then staff might be able to devise compensatory efforts to allow youth with less experience to more quickly get up to speed.

Supporting Post-Program Experiences

As with pre-program interventions, creating strategies for proactively and mindfully supporting long-term post-program experiences are not easy. Still, the evidence was compelling that on-going out-of-school experiences, parental support and even the bias toward extroverts appeared to reinforce informal education program-generated impacts on youths’ creativity, STEM interest, and problem-solving skills. Some possible ideas for how to support these kinds of long-term engagement include:

• Since the data strongly suggested that multiple out-of-school STEM-related experiences have a significant additive and synergistic effect on youth creativity, STEM interest, and problem solving, informal educators should try to find ways to partner with other STEM-organizations that offer out-of-school experiences and collaborate and co-support each other’s efforts.

• As above, given the evidence that perceived parental support is critical to sustaining the effects of informal education experiences, informal educators might consider how best to communicate this key information to parents and potentially even consider investing energy in supporting programming aimed at parents and care-givers as opposed to exclusively youth-focused programming, as is currently the norm.

• Given the preliminary evidence that multiple, prior experiences contribute to enhanced educational outcomes, particularly in the area of creativity, informal educators should explicitly and proactively communicate this to parents. They should tell parents that the benefits of participating in these experiences are not only significant but that there is evidence that such experiences appear to be additive–multiple experiences have the potential to result in significantly greater outcomes than a single experience. They should also communicate that other comparable experiences are also important, and that the more such experiences their children engage in, the greater is the likelihood that they will become STEM-motivated during adolescence and on into adulthood.

• The evidence for the effect of high sociability on long-term capabilities in creativity, STEM interest, and problem solving was, as noted above, quite provocative and worth thinking about how informal educations might create interventions that support youth who are less social than their peers. Perhaps follow-up experiences designed for individuals rather than groups could be developed and specifically targeted at youth identified as more introverted.

In conclusion, the goal of this hypothesis-generating research was to undertake a study to explore which, if any of a range of possible non-informal education experience-related factors/variables might significantly influence informal education program outcomes. Results suggest that a range of factors/variables do indeed appear to influence outcomes and that if thoughtfully and creatively addressed, might open up possibilities for significantly improved learning outcomes. Results also suggest that although informal education experiences are clearly impactful, currently significantly enhancing outcomes like creativity, STEM interest, and problem solving, there is still considerable room for improvement. Although efforts like the one studied in this particular research appear to be working well for the majority of participants, they appear to be only “optimally” working for a relatively smaller percentage of participants. As always, further research is required, but these findings appear to be sufficiently robust, provocative and actionable to warrant practitioners taking these results to heart and making immediate changes to their practice.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors if requested and justified.

Ethics Statement

The studies involving human participants were reviewed and approved by the Heartland Institutional Review Board, Swansea IL, United States. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author Contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

This research was performed under a contract with the United States Inventor’s Hall of Fame.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We wish to thank and acknowledge the on-going cooperation and support of the National Inventor’s Hall of Fame and the staff of Camp Invention. A particular thanks to our collaborators Alaina Rutledge, Jennifer Sitton, and Jayme Cellitioci. Also thanks to input from our colleagues Michael Coe and Lynn Dierking early in the development process of this research.

References

Anderson, D., Lucas, K. B., Ginns, I. S., and Dierking, L. D. (2000). Development of Knowledge about Electricity and Magnetism during a Visit to a Science Museum and Related post-visit Activities. Sci. Ed. 84 (5), 658–679. doi:10.1002/1098-237x(200009)84:5<658::aid-sce6>3.0.co;2-a

CrossRef Full Text | Google Scholar

Archer, L., Dewitt, J., Osborne, J., Dillon, J., Willis, B., and Wong, B. (2010). "Doing" Science versus "being" a Scientist: Examining 10/11-Year-Old Schoolchildren's Constructions of Science through the Lens of Identity. Sci. Ed. 94 (4), 617–639. doi:10.1002/sce.20399

CrossRef Full Text | Google Scholar

Azevedo, R. (2015). Defining and Measuring Engagement and Learning in Science: Conceptual, Theoretical, Methodological, and Analytical Issues. Educ. Psychol. 50 (1), 84–94. doi:10.1080/00461520.2015.1004069

CrossRef Full Text | Google Scholar

Barron, B., Gomez, K., Pinkard, N., and Martin, C. K. (2014). The Digital Youth Network: Cultivating Digital media Citizenship in Urban Communities. Cambridge, Massachusetts, United States‏: MIT Press. doi:10.7551/mitpress/9240.001.0001

CrossRef Full Text

Barron, B., Martin, C. K., Takeuchi, L., and Fithian, R. (2009). Parents as Learning Partners in the Development of Technological Fluency. Int. J. Learn. Media 1 (2), 55–77. doi:10.1162/ijlm.2009.0021

CrossRef Full Text | Google Scholar

Bécares, L., and Priest, N. (2015). Understanding the Influence of Race/ethnicity, Gender, and Class on Inequalities in Academic and Non-academic Outcomes Among Eighth-Grade Students: Findings from an Intersectionality Approach. PLoS ONE 10 (10), e0141363. doi:10.1371/journal.pone.0141363

PubMed Abstract | CrossRef Full Text | Google Scholar

Brotman, J. S., Mensah, F. M., and Lesko, N. (2011). Urban High School Students' Learning about HIV/AIDS in Different Contexts. Sci. Ed. 95 (1), 87–120. doi:10.1002/sce.20405

CrossRef Full Text | Google Scholar

Caillot, M., and Nguyen-Xuan, A. (1995). Adults' Understanding of Electricity. Public Underst Sci. 4 (2), 131–151. doi:10.1088/0963-6625/4/2/003

CrossRef Full Text | Google Scholar

Carter, P. L. (2012). Stubborn Roots: Race, Culture, and Inequality in U.S. And South African Schools. New York: Oxford University Press. doi:10.1093/acprof:oso/9780199899630.001.0001

CrossRef Full Text

ChangeMaker Consulting (2014). 2014 Camp Invention® Evaluation Report. Fresno, CA: ChangeMaker Consulting.Unpublished Technical Report

DeWitt, J., Archer, L., Osborne, J., Dillon, J., Willis, B., and Wong, B. (2011). High Aspirations but Low Progression: The Science Aspirations-Careers Paradox Amongst Minority Ethnic Students. Int. J. Sci. Math. Educ. 9 (2), 243–271. doi:10.1007/s10763-010-9245-0

CrossRef Full Text | Google Scholar

Falk, J. H. (2018). Born to Choose: Evolution, Self and Well-Being. London: Routledge.

Falk, J. H., and Dierking, L. D. (2018). “Viewing Science Learning through an Ecosystem Lens: A story in Two Parts (Pp. 9-30),” in Navigating the Changing Landscape of Formal and Informal Science Learning Opportunities. Editors R. D. Corrigan, C. Buntting, and J. Loughran (Dordrecht: Springer Netherlands).

Google Scholar

Falk, J. H., Dierking, L. D., Osborne, J., Wenger, M., Dawson, E., and Wong, B. (2015). Analyzing Science Education in the United Kingdom: Taking a System-wide Approach. Sci. Ed. 99 (1), 145–173. doi:10.1002/sce.21140

CrossRef Full Text | Google Scholar

Falk, J. H., Dierking, L. D., and Staus, N. L. (2020). The Use of Ecological Concepts in the Social Sciences: Measuring the Productivity, Durability & Resilience of Learning Ecosystems. Ecol. Conservation Sci. 1 (3), 555563. doi:10.19080/ECOA.2020.01.555563

CrossRef Full Text | Google Scholar

Falk, J. H., Dierking, L. D., Swanger, L. P., Staus, N., Back, M., Barriault, C., et al. (2016a). Correlating Science center Use with Adult Science Literacy: An International, Cross-Institutional Study. Sci. Ed. 100 (5), 849–876. doi:10.1002/sce.21225

CrossRef Full Text | Google Scholar

Falk, J. H., and Dierking, L. D. (2010). The 95% Solution: School Is Not where Most Americans Learn Most of Their Science. Am. Scientist 98, 486–493.

Google Scholar

Falk, J. H. (2009). Identity and the Museum Visitor Experience. Walnut Creek, CA: Left Coast Press.

Falk, J. H., and Needham, M. D. (2013). Factors Contributing to Adult Knowledge of Science and Technology. J. Res. Sci. Teach. 50 (4), 431–452. doi:10.1002/tea.21080

CrossRef Full Text | Google Scholar

Falk, J. H., Pattison, S., Meier, D., Bibas, D., and Livingston, K. (2017). The Contribution of Science-Rich Resources to Public Science Interest. J. Res. Sci. Teach. 55 (3), 422–445. doi:10.1002/tea.21425

CrossRef Full Text | Google Scholar

Falk, J. H., Staus, N., Dierking, L. D., Penuel, W., Wyld, J., and Bailey, D. (2016b). Understanding Youth STEM Interest Pathways within a Single Community: The Synergies Project. Int. J. Sci. Educ. B 6 (4), 369–384. doi:10.1080/21548455.2015.1093670

CrossRef Full Text | Google Scholar

Fosnot, C. T., and Perry, R. S. (2005). “Constructivism: A Psychological Theory of Learning,” in Constructivism: Theory, Perspective and Practice. Editor C. T. Fosnot 2nd ed. (New York: Teachers College Press), 8–38.

Google Scholar

Frenzel, A. C., Goetz, T., Pekrun, R., and Watt, H. M. G. (2010). Development of Mathematics Interest in Adolescence: Influences of Gender, Family, and School Context. J. Res. Adolescence 20 (2), 507–537. doi:10.1111/j.1532-7795.2010.00645.x

CrossRef Full Text | Google Scholar

Immordino-Yang, M. H. (2015). Emotions, Learning, and the Brain: Exploring the Educationalimplications of Affective Neuroscience. New York: W. W. Norton.

Jackson, N. J. (2013). Learning Ecology Narratives‏Retrieved Lifewide Learning, Educafion And Personal Development E-Book. Available at: http://www.lifewideebook.co.uk/uploads/1/0/8/4/10842717/chapter_c4.pdf((accessed March 18, 2020).

Google Scholar

Karwowski, M., Lebuda, I., and Wisniewska, E. (2018). Measuring Creative Self-Efficacy and Creative Personal Identity. Int. J. Creativity Problem Solving 28, 45–57.

Google Scholar

Kaufman, J. C. (2012). Counting the Muses: Development of the Kaufman Domains of Creativity Scale (K-DOCS). Psychol. Aesthetics, Creativity, Arts 6 (4), 298–308. doi:10.1037/a0029751

CrossRef Full Text | Google Scholar

Kent State University (2004). A Report on the Evaluation of the National Inventors Hall of Fame’s Camp Invention Program. Kent, OH: Bureau of Research & Training Services, Kent State University.

Korpan, C. A., Bisanz, G. L., Bisanz, J., Boehme, C., and Lynch, M. A. (1997). What Did You Learn outside of School Today? Using Structured Interviews to Document home and Community Activities Related to Science and Technology. Sci. Ed. 81, 651–662. doi:10.1002/(sici)1098-237x(199711)81:6<651::aid-sce3>3.0.co;2-h

CrossRef Full Text | Google Scholar

Lave, J., and Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. New York: Cambridge University Press. doi:10.1017/cbo9780511815355

CrossRef Full Text

López, N. (2002). Hopeful Girls, Troubled Boys: Race and Gender Disparity in Urban Education. New York: Routledge.

McGee, K. (2018). The Influence of Gender, and Race/ethnicity on Advancement in Information Technology (IT). Inf. Organ. 28 (1), 1–36. doi:10.1016/j.infoandorg.2017.12.001

CrossRef Full Text | Google Scholar

Miller, J. D. (2010). Adult Science Learning in the Internet Era. Curator 53 (2), 191–208. doi:10.1111/j.2151-6952.2010.00019.x

CrossRef Full Text | Google Scholar

National Academies of Sciences (2018). How People Learn II: Learners, Contexts, and Cultures. Washington, DC: The National Academies Press.Engineering, and Medicine

Google Scholar

National Research Council (NRC) (2015). Identifying and Supporting Productive STEM Programs in Out-Of-School Settings. Washington, DC: National Academy Press.

National Research Council (NRC) (2009). Learning Science in Informal Environments. Washington, DC: National Academy Press.

Noftle, E. E., and Robins, R. W. (2007). Personality Predictors of Academic Outcomes: Big Five Correlates of GPA and SAT Scores. J. Personal. Soc. Psychol. 93, 116–130. doi:10.1037/0022-3514.93.1.116

PubMed Abstract | CrossRef Full Text | Google Scholar

Poropat, A. E. (2009). A Meta-Analysis of the Five-Factor Model of Personality and Academic Performance. Psychol. Bull. 135 (2), 322–338. doi:10.1037/a0014996

PubMed Abstract | CrossRef Full Text | Google Scholar

Renninger, K. A., and Hidi, S. E. (2016). The Power of Interest for Motivation and Engagement. New York: Routledge.

Riegle-Crumb, C. (2006). The Path through Math: Course Sequences and Academic Performance at the Intersection of Race-Ethnicity and Gender. Am. J. Educ. (Chic Ill. 113 (1), 101–122. doi:10.1086/506495

PubMed Abstract | CrossRef Full Text | Google Scholar

Roschelle, J. (1995). “Learning in Interactive Environments: Prior Knowledge and New Experience,” in Public Institutions for Personal Learning. Editors J. Falk, and L. Dierking (Washington, DC: American Association of Museums), 37–51.

Google Scholar

Scarisbrick-Hauser, A., and Hauser, B. (2009). Camp Invention 2009 Program Evaluation. H.A. Praxis Solutions, Unpublished Technical Report. Alexandria, VA.

Google Scholar

Schreiner, Camilla., and Sjøberg, Svein. (2004). Sowing the Seeds of ROSE. Background, Rationale, Questionnaire Development and Data Collection for ROSE (The Relevance of Science Education) - a Comparative Study of Students’ Views of Science and Science Education. Oslo: Dept. of Teacher Education and School Development, University of Oslo. www.ils.uio.no/english/rose.Acta Didactica 4/2004Available from:

Stocklmayer, S. M., Rennie, L. J., and Gilbert, J. K. (2010). The Roles of the Formal and Informal Sectors in the Provision of Effective Science Education. Stud. Sci. Edu. 46 (1), 1–44.

CrossRef Full Text | Google Scholar

Tai, R., and Maltese, A. (2010). Eyeballs in the Fridge: Sources of Early Interest in Science. Int. J. Sci. Edu. 32 (5), 669–685.

CrossRef Full Text | Google Scholar

The Organization for Economic Co-operation and Development (OECD) (2012). PISA in Focus 18: Are Students More Engaged when Schools Offer Extracurricular Activities? Paris: OECD.

Thompson, E. R. (2008). Development and Validation of an International English Big-Five Mini-Markers. Personal. Individual Differences 45 (6), 542–548. doi:10.1016/j.paid.2008.06.013

CrossRef Full Text | Google Scholar

Topping, K. J. (2005). Trends in Peer Learning. Educ. Psychol. 25 (6), 631–645. doi:10.1080/01443410500345172

CrossRef Full Text | Google Scholar

Traphagen, K., and Traill, S. (2014). How Cross-Sector Collaborations Are Advancing STEM Learning. Los Altos, CA: Noyce Foundation.

Trilling, B., and Fadel, C. (2009). 21st Century Skills: Learning for Life in Our Times. San Francisco, CA: Jossey-Bass.