Stephanie Couch
Kelli M. Paul
Michelle Sullivan3- 1Lemelson-MIT Program, School of Engineering, Massachusetts Institute of Technology, Cambridge, MA, United States
- 2Center for Research on Learning and Technology, Indiana University-Bloomington, Bloomington, IN, United States
- 3Wellesley Centers for Women, Wellesley College, Wellesley, MA, United States
- 4Gevirtz Graduate School of Education, University of California, Santa Barbara, Santa Barbara, CA, United States
Invention Education (IvE), a form of problem-based learning, presents new challenges for educational assessments in public schooling because traditional assessments were designed to evaluate learning in singular disciplines. This study explores the challenges and possibilities for assessing new knowledge and capabilities acquired through students’ engagement with multiple disciplines through IvE. Guided by constructivist and sociocultural theories, as well as an understanding of IvE principles and practices derived from the literature on IvE, we examine the phases of work within a national IvE program for high school students and educators. We then examine ways existing assessments align with the work at each stage of the IvE process. Findings from this case study underscore the need for a flexible assessment system with multiple measures (e.g., disciplinary knowledge and practices, skill inventories, etc.). The system must account for variations in learning contexts, individual and collective achievements, and varying lengths of time students engage in IvE.
1 Introduction: overview of the invention education assessment challenge
Invention, or the process by which humans devise and produce something that is new, novel, useful, and unique (Couch et al., 2019), is central to the creation of technological advancements that increase productivity, foster economic growth, and improve lives by meeting human needs. Educators’ deliberate efforts to develop in pre-college students the capabilities that are essential for the creation of inventions (Kuznets, 1962) constitute a distinctive form of problem-based learning known as Invention Education (IvE). IvE practices are being integrated into the teaching of individual subjects or content areas (such as science or history) in public schooling, beginning in the early years and continuing through high school. Some invention educators are making the process of finding a problem and inventing its technological solution the central focus of instruction. They draw on knowledge, skills, and practices from different disciplines as needed for activities within each phase of the process. This type of IvE pedagogy presents new challenges for educational assessment because most assessment instruments currently being used to show proof of learning, especially in the United States, focus on singular disciplines such as English Language Arts or mathematics.
Traditional tools used to assess core academic subjects measure disciplinary ideas, practices, and concepts that are age- and grade-appropriate, based on predetermined learning outcomes for the given subject. When the invention process is foregrounded, some learning outcomes align with the activities in each phase of the invention process and therefore can also be predetermined and assessed. However, additional learning outcomes associated with the nature of the problem students choose as their focus, or the solution students design and build, can only be known and assessed at the end of the process—after the prototype has been produced. Student outcomes embedded in the design of traditional assessments are based on expectations of what can be learned when all time within a class period or course is dedicated to the singular discipline. Educators shifting their teaching practices to the transdisciplinary work of the invention process need new assessment systems. These systems should reflect students’ engagement with ideas, practices, and concepts drawn from many different disciplines and fields of study. The assessments also need to reflect the amount of time students can devote to learning discrete disciplines during the time allotted for the process.
The disconnects between traditional assessments and those needed for IvE led us to consider the theories of learning embraced by educators taking up IvE, as well as the principles needed to guide a new assessment system that is more conducive to this transdisciplinary approach to problem-based learning. This paper presents an initial set of principles to guide IvE assessment practices and demonstrates their potential applicability to learning opportunities afforded by one national IvE offering in which students engage in year-long invention projects. This telling case (Mitchell, 1984) makes visible the opportunities for learning within each phase of the invention process (i.e., what can be assessed), differences in whose work needs to be assessed (i.e., individual versus the work of a team), and differences in the timelines for when learning and development is assessed.
1.1 Key features of invention education
Our analysis of how educators could assess IvE began with an effort to define what opportunities for learning are being afforded to students. Insights into what counts as IvE pedagogy in the United States are informed by a consensus document produced in 2019 by 39 educators, education researchers, and educational program providers with expertise ranging from the early years of schooling through college (Invention Education Research Community [IvERC], 2019). The consensus document defines IvE as “a deliberate effort to engage learners in the identification of problems and the design and development of new, novel, useful, and unique technological solutions (i.e., inventions) that contribute to the betterment of society (Committee for the Study of Invention, 2004; Couch et al., 2019)” (Invention Education Research Community [IvERC], 2019). Notions of what counts as IvE pedagogy are also informed by articles in an edited volume produced by Finnish educators and researchers in 2023, entitled Invention Pedagogy—The Finnish Approach to Maker Education (Korhonen et al., 2022, p. 304). Elements common to descriptions of IvE pedagogy, represented in publications by the researchers from the United States and Finland, include:
• A problem-finding or problem-defining stage;
• A real-world problem arising from the needs of others;
• Teamwork and collaboration within and beyond the team;
• Mentors and others from the larger community beyond the school or classroom;
• Iterative and recursive learning and design cycles;
• Open-ended inquiry to solve real-world problems;
• Embracing learning from failure and uncertainty as students build invention prototypes;
• Milestones along the way to scaffold student learning;
• Prototyping and creating a potential solution to a real-world problem;
• Educators as guides, mentors, or coaches who learn alongside students; and
• Culminating events and competitions in which students share their work with others.
2 Pedagogy that fosters learning across all levels of Bloom’s taxonomy
Relevant literature pertaining to IvE can be found in studies conducted by creativity researchers. These studies may use different terminology for teaching and learning through inquiry and open-ended problem solving since the work extends beyond invention to other forms of creativity and inventiveness. In this study, we adopt the notion that inventing is a type of creative act, and that the terms “create” and “invent” can be used interchangeably. This perspective guides our understanding of inventing as evoking knowledge processes and fostering thinking skills that activate all levels of Bloom’s taxonomy, including creating—the highest-order thinking skill (Sawyer, 2019; Bloom, 1956). The reinterpreted version of Bloom’s taxonomy re-presented in Figure 1 (Armstrong, 2010), which added “create” to the top of the pyramid, articulates the types of skills that are developing when learners are engaged across all levels through creative problem-solving and inventing.
Figure 1. Revised version of Bloom’s taxonomy.
The lowest tier of Bloom’s taxonomy pertains to the recall of facts and basic concepts. Facts and concepts are transmitted in classrooms through teacher talk, ideas shared by guest speakers or peers sharing what they know, textbooks or other written materials, and ideas conveyed through signs and symbols. Educators may ask students to demonstrate understanding of concepts or facts verbally or through written or computer-based assignments and assessments. Students may engage in problem-based learning in which they apply the knowledge or information to problems that align with the concepts being taught. Students may also be asked to engage in analysis or to apply knowledge by justifying a decision or something they have written. At the highest level of the taxonomy, students create or produce a new or original work, which could include a physical device or contraption (or in some cases an algorithm) that is intended to be a useful technological solution that may ultimately qualify for a patent. IvE fosters learning across all levels of Bloom’s taxonomy as students engage with ideas and concepts from multiple disciplines. The work needed in each phase of the IvE process structures the learning process.
One example of how IvE is enacted in a classroom setting and ways it supports learning across all levels of Bloom’s taxonomy is a lunch box invention project adopted by a middle school classroom (Zhang et al., 2019). In this activity, students engaged with science and engineering concepts related to heat transfer as they worked to create thermodynamic lunch boxes. Students demonstrated their skills in creating a new or original work applying transdisciplinary knowledge that is aligned with Bloom’s cognitive learning tiers. At the lowest tier (“remember”), students recalled basic concepts of thermodynamics (e.g., heat transfer, conserving energy). They discussed problems and solutions related to thermal energy transfer in the context of food safety and transportation (“understand” tier). They examined the thermal properties of various materials to explore the design and components of lunch boxes that keep food and drinks cold in everyday life (“apply” tier). Students analyzed thermal resistance of various materials and ranked the best conductors and insulators (“analyze” tier). They proposed lunch box designs and critiqued each other’s designs to come up with a revised plan that satisfies the demands of potential users and their budget (“evaluate” tier). Finally, evaluating user-centric demands, they built a lunch box prototype and tested a Peltier cooling unit to explore using thermoelectric effect to remove heat from a system (“create” tier) that demonstrated their capability to develop a new solution for a socio-scientific phenomenon. Hence, the IvE practices of the students aligned with Bloom’s cognitive learning tiers, demonstrating content knowledge of energy, problem-solving for cost effectiveness, analytical ability for social issues, and communication and collaboration skills with the teacher and peers.
2.1 The enactment of invention education as guided improvisation
Creativity researcher and psychologist Keith Sawyer (2019) describes the robust evidence base for how open-ended, inquiry-based teaching—such as that required for IvE—supports the development of higher-order thinking skills and deeper learning. He refers to this approach to teaching as “guided improvisation.” In contrast, he uses the term “instructionism” as a counter-state, such that the over-emphasis on the teaching of facts condemns student learning to “shallow knowledge,” or the types of knowledge practices appearing at the lowest level of Bloom’s taxonomy. Sawyer (2019) provides examples to demonstrate quantitative outcomes from improvisational instruction in which students develop creative learning skills that are inclusive of both higher levels and lower levels of Bloom’s taxonomy at the same time (Agarwal, 2019). The improvisational approach to teaching and the use of IvE pedagogical practices have been integrated into K–12 curricula and instruction in ways that fit predefined content standards for particular disciplines (see Ewell et al., 2022; Gale, 2022; Sawyer, 2019; Zhang et al., 2019). In these contexts, including the lunch box example noted above, the problems identified by students and the resulting prototypes created align with the focus on grade-level subject matter. Educators could therefore assess learning using existing measurement tools designed for the relevant grade level and content area. Given prior studies, educators would likely be able to show that, when compared to assessments of students in instructionist classrooms, the use of IvE pedagogical practices supported students’ retention of knowledge (Agarwal, 2019).
As noted earlier, some invention educators are using the invention process as the primary focus of instruction for an extended period, including semester and/or year-long discipline-agnostic courses. The curriculum and expected outcomes require students to draw on knowledge and practices from different disciplines as they build technical solutions to address problems identified in their communities. The development of knowledge in particular disciplines ebbs and flows during different phases of activity as learners engage in the process of inventing. This approach to IvE is referred to as being “transdisciplinary” because it blurs disciplinary boundaries: STEM knowledge is mixed with knowledge from other disciplines and from within different cultural groups in students’ local communities to support students’ efforts to solve complex problems (National Science and Technology Council, Committee on STEM Education, 2022). For example, students draw on knowledge and practices that are common to sociology, anthropology, and the humanities as they explore unmet needs of people in their local communities during the initial problem-finding and definition phase of work. Guided improvisation occurs as teachers become learners alongside their students. Working together, the students and the teacher search for knowledge and develop skills needed to manipulate tools and materials in order to design and build prototypes of technological solutions to problems that reflect needs in the community.
Educators who foreground the invention process as the curriculum—rather than predetermined readings, lesson plans, activities, and assessments tied to discipline-specific K–12 standards—create a “bottom-up group process” (Sawyer, 2019) in which learning emerges through students’ experiences. The process becomes the curriculum. Many of the activities within each phase of the process will be the same for each group of students, even as the problem being addressed, and the solution being pursued, by teams change. The stable “base layer” of activities common to the invention process can be mapped to the practices and standards for individual disciplines (STEM and others). Learning goals and curriculum resources can be developed to reflect the process, activities, disciplinary knowledge, and standards that will be an ongoing focus for the educator with each class of students, since the process will remain the same from one class to the next.
Envisioning the invention process as curriculum is complicated by the fact that a universal description of the invention process does not exist. Educators and researchers who participated in the Invention Education Research Community [IvERC] (2019) agreed on the elements of the invention process as noted above, but there is no agreement on whether all elements must be part of instruction for an offering to be considered invention education. Similarly, there is no uniform approach to engaging learners in the phases of activity in a particular sequence that would constitute “the process.” As authors who have engaged in research to understand the work of inventors, and as researchers and teachers of invention practices, we subscribe to a process that includes four phases of activity. These phases are drawn from the literature describing ways patented inventors approach non-routine problem solving (Committee for the Study of Invention, 2004). The four phases depicted in Figure 2 are: (1) identifying and defining a problem; (2) conducting inquiries and identifying, listening to, and learning about what matters to end users; (3) designing solutions; and (4) building and testing physical prototypes (Aulet, 2013; Estabrooks and Couch, 2018; Middendorf, 1981; Shavinina and Seeratan, 2003; Wagner, 2012). The process concludes by asking learners to go public with their work at a culminating event The full invention process cycle is presented in Supplementary Table 1.
Figure 2. Four phases of the invention process.
Learners’ progression through the four phases of the invention process is non-linear and iterative, requiring learners to revisit earlier work and revise their designs based on new information and/or experiences acquired and new questions that arise throughout the invention process (Estabrooks and Couch, 2018; Frigotto, 2018). The non-linear, iterative nature of inventing creates challenges for assessment. Knowledge and learning outcomes likely will differ depending on the specific problem students choose to solve. As such, the one-size-fits-all approach characteristic of traditional standardized assessments in public schools does not align with the nature of the IvE process.
2.2 Pedagogy that incorporates learning through human engagement and discourse
One hallmark of IvE pedagogy is the use of human interactions and discourse to foster learning. The specific types of community collaborators, roles, and levels of engagement may vary from one instance of IvE to the next. Research studies of IvE pedagogy in the United States and Finland describe the role of the educator as being a guide, mentor, or coach who learns alongside students. Mentors and others from the larger community beyond the school or classroom also play a part in the teaching process by sharing their knowledge or perspectives, asking guiding questions, and offering feedback. Ethnographers in education who researched the IvE practices at a high school in the United States studied the multiple actors who engaged in project-based learning processes, focusing on who, what, when, where, how, and with whom individuals participate in IvE (Skukauskaite et al., 2023; Bridges et al., 2015; Green et al., 2020; Green and Bridges, 2018). The researchers found that support was provided at three different levels: local, national, and a combination of both. Local supporters included organizations and individuals in the community, school personnel, family and friends, and technical mentors, all of whom shared their expertise, guided students in their work (rather than doing it for them), and modeled the engineering design process. Examples of support from both the local and national levels included the media, external funding, and a foundation to provide support. For example, the Lemelson-MIT Program (LMIT) provided a grant to the team, professional development for the teacher, an educational framework and resources to structure the work across the year and to hold students accountable for their work, and opportunities to foster human connections beyond the school.
Researchers’ descriptions of the role of these levels of support allude to ways student learning was scaffolded across the year through conversations in which a variety of adults shared knowledge and expertise. The variety of supports made available to students for learning through human interactions present another challenge for using traditional, one-size-fits-all standardized assessments for IvE. While there may be variation in the ideas being made available for learning through discourse in the classroom based on the teachers’ knowledge, the problem is more acute with IvE as learners intentionally engage and learn through discourse from many different types of community informants with specialized insights or knowledge related to the problem or solution being addressed.
2.3 Learning through individual roles and through teamwork
The studies of IvE pedagogy in the United States (Invention Education Research Community [IvERC], 2019; Korhonen et al., 2022) and Finland identify teamwork and collaboration within and beyond the team as being a common element of students’ opportunities for learning. In some instances, students have lead roles or areas of specialization on a team, such as project management, finance, communications, or technical (including coding or robotics, artificial intelligence, and machine learning). The existence of roles within a team indicates that some portions of the class time and opportunities for learning will be devoted to role-specific activities that will vary from student to student, whereas other opportunities for learning will apply to activities common to the entire group or class. Assessment measures will need to reflect these differences in opportunities for learning at the individual, group, and class levels.
Students filling roles such as welding, manipulating, using software skills, or programming when building the prototype could benefit from commonly available assessments selected on an ad-hoc basis, based on the “fit” with the student’s work on the invention project. However, the additional emphasis on collaborative teamwork in IvE requires assessment tools that can effectively measure interdisciplinary collaboration and problem-solving skills developed within student teams. Measures of personal characteristics such as teamwork and collaboration, thinking skills, and habits of mind fostered through students’ individual efforts and their interactions with others present a greater challenge for assessments. These personal attributes and capabilities that are developed through the process, however, are valuable learning outcomes in their own right and reflect the types of competencies a growing number of schools and districts across the United States are looking to foster in their graduates. The use of competencies for assessing graduates may be an indication of renewed interest in measures of student learning that go beyond content-based outcomes. South Carolina was one of the first states in the United States to enact policies supporting statewide use of competencies and associated rubrics for assessing personalized learning (additional information is available at https://ed.sc.gov/about/ccmr/personalized-learning/). As of the writing of this paper, many school districts in South Carolina had opted to participate in this statewide voluntary system. But South Carolina is not alone. Over the past decade, over 17 states and multiple districts have developed and/or adopted a portrait of a graduate for their students (Getting Smart, 2024; Stanford, 2024).
3 Research and theories for assessing IvE learning outcomes
The key features of IvE described in Section 2 make visible different types of experiences that contribute to student learning. Traditional assessment frameworks are designed to evaluate individual performance on pre-determined concepts and ideas within individual disciplines. However, these frameworks are unlikely to capture the full range of experiences that produce learning outcomes within the individual and the group during IvE. A more comprehensive evaluation of students’ individual and collective achievements and capabilities is needed for IvE. Efforts to develop an assessment framework for IvE should reflect the underlying theories of teaching and learning that guide IvE pedagogy. In this section, we present a brief synopsis of three theories that are central to the authors’ notions of IvE and how to assess aspects of students’ development arising from their work as inventors.
3.1 Constructivist theory, active learning, and creativity
IvE’s emphasis on learning through exchanges that take place between the learner, others with expertise, tools and materials, and cultural spaces to support new knowledge structures aligns with constructivist theories advanced by Vygotzsky. Sawyer (2019) builds on constructivist theories advanced by progressive educators such as Piaget, Dewey, Froebel, and Montessori in his framing of active learning. These constructivist approaches are (1) embodied knowledge, in which learners physically interact with external objects; (2) externalized knowledge, where a learner’s unfolding knowledge is visible in an external artifact; and (3) intrinsic motivation, in which learners have an affinity for what they are doing (p. 69). Sawyer connects these approaches to teaching and learning through making, a necessary component of the IvE process during the phase in which students build working prototypes of their inventions. Intrinsic motivation comes into play when learners are afforded opportunities to pitch ideas to their group as to what problem they will attempt to solve and ideas for how to solve it as they collaborate with the individuals to develop a technological solution.
3.2 Constructionism
IvE includes the building of a physical prototype of an invention. During this building phase, students’ learning is enabled by a combination of thinking skills and their use of tools and materials. The concept of constructionism (Harel and Papert, 1991) underpins this learning method, emphasizing that knowledge structures are built through active engagement. The combination of “hands-on” and “minds-on” work fosters the creation of new knowledge and deeper understanding. Constructionism can be contrasted with instructionist notions of learning, whereby knowledge is merely transferred from one person to another (Estabrooks and Couch, 2018).
3.3 Sociocultural and dialogic theories of teaching and learning
In IvE, students engage in team-based learning through interactions and dialogues with peers, teachers, community members, and distant collaborators, as well as through other inscriptions such as text, audio files, artifacts, or observable actions. Discourse fosters new ways of thinking, knowing, being, and doing that are taken up by the group and by the individuals within the group (Estabrooks and Couch, 2018; Putney et al., 2000). Spoken language, therefore, serves as a cultural and cognitive tool that supports student teams as they reason together and co-regulate their activities (Littleton and Mercer, 2013). Dialogic theorist Rupert Wegerif (2015) argues that the process of thinking in dialogic form is visible in the social external world (such as during talks at the culminating events that are often part of the IvE process) and is accompanied by an internal and invisible aspect. The inclusion of mentors and role models that are relatable and/or offer access to cultural capital (Saenz and Skukauskaitė, 2022) also supports learning.
3.4 Implications of the theories for assessment
The three theoretical foundations of constructivism, constructionism, and sociocultural aspects of knowing (or coming to know) reflected in IvE pedagogy are crucial for shaping assessment practices that align with IvE’s collaborative and transdisciplinary nature. Traditional instructionist approaches where outcomes are predetermined and taught through focused attention to particular aspects of a singular discipline can measure some aspects of what is learned through invention. However, IvE’s emphasis on the generation of new knowledge by engaging with multiple disciplines and with the knowledge, attitudes, and beliefs of people through the iterative process of making an invention prototype requires more than a singular measure. Rather, IvE requires ongoing evaluation across the different types of learning taking place at different phases of work during the invention process. Evaluating learning outcomes that are emergent, multifaceted, individual, and group- or team-based, and focused on collaborative problem-solving to co-construct non-obvious physical solutions also presents challenges. These features of IvE require educators to consider assessment systems with multiple tools capable of capturing both individual and team (or group) contributions to the creation of the invention prototype, the properties of the physical prototype, and new ways of thinking and working acquired by the individual across the phases of work.
4 Principles that align with the theories of learning and guide IvE assessment
The theories guiding IvE pedagogy inform the principles we have identified for assessing what is accomplished through students’ work as inventors and creators. Additionally, we recognize the likelihood that radical changes to current assessment practices in the United States would meet with significant resistance. The principles we describe in the sections below take both factors into account. We offer the principles as a guide for invention educators’ decision making about assessment instruments, recommending that educators select and/or assemble the instruments to fit their instructional goals, local conditions, and the opportunities for learning afforded to their students.
4.1 Formative assessments throughout the invention process should complement a summative assessment
In instances where IvE is embedded in a discipline-based course, the course is likely to have pre-determined formative and summative assessments that are used to inform understandings of student progress. However, when the IvE process is used as the curriculum, educators will need to identify the disciplinary knowledge and practices activated within each phase and adapt existing—or develop new—formative and summative assessments. Some types of knowledge and practices directly related to activities associated with each phase of the invention process will always be invoked, regardless of the nature of the problem students choose to address. For example, many activities that are fundamental to the process will align with English Language Arts standards. However, traditional formative and summative assessments of the science or math concepts learned via the project may be more difficult to adapt.
It is common for students to present their prototypes to community members or experts, both mid-project and as part of culminating events at the end of the project. The verbal feedback introduces variability and richness to assessments, compared to traditional standardized assessments used within single-discipline courses. The team presentations can be used as formative and summative assessments of students’ progress while also serving as a resource for the team’s refinement of prototypes and development of collaboration skills. An approach tested in two engineering courses—one serving undergraduates and the other at the graduate level (Wengrowicz et al., 2018)—provides an example of ways invention project presentations could be part of a formative and summative assessment system of projects where students worked in teams. Formative evaluations of individual student performance and team performance were enacted through peer assessments, in which students offered feedback on their colleagues’ projects in accordance with predetermined assessment categories and related criteria. The feedback development process consisted of two parts: in part one, the student evaluator collected evidence for the evaluation; part two involved comparing and assessing four different projects for clarity and understanding, completeness, correctness, and documentation. The summative evaluation for the course included a student-oriented meta-assessment in which everyone was evaluated by the course team (instructors) according to the quality of the feedback that the student offered to their colleagues. The peer assessment scores for members of a team, as well as the ranking of the team’s own project, were combined to produce a final grade (Wengrowicz et al., 2018).
4.2 Assessments should account for variations in the students’ opportunities for learning
The age of the learners, duration of engagement with IvE, and the number of cycles they undergo are critical factors influencing the evaluation of learners’ developmental progressions. Time devoted to IvE studies and the number of years a learner has engaged in IvE (i.e., repeat experiences) are consequential, and frequently students who have been traditionally underrepresented in STEM have not been afforded access to opportunities for learning. Assessments must differentiate between prototypes based on the level of functionality achieved and the stages of the design process students were able to explore. For example, when IvE is integrated into a class period that primarily focuses on a singular discipline, the time allotted to IvE may be brief. In such cases, students may only have the opportunity to produce an inspirational prototype (e.g., a sketch or rudimentary object) that reflects an invention prospect. In contrast, when IvE is the focus of a two-semester course with roughly 120 h allotted to IvE, students may have more opportunity to produce a fully functional prototype that meets the requirements for earning a United States patent. Relatedly, the amount of time students engage in IvE may influence students’ perceptions of IvE. Student experience surveys cited in prior studies have demonstrated that those who have had repeated IvE engagements over multiple years may no longer perceive of the experiences as being novel and significant for their personal growth (Couch et al., 2018). Students underrepresented in STEM who experience IvE in a formal learning setting for the first time are likely to give higher self-assessment ratings of their experiences and personal growth (Couch et al., 2018). Assessment measures thus should consider how repeated IvE experiences may influence non-cognitive outcomes as students begin to see the IvE process as the norm, rather than as something novel.
4.3 Assessment data should serve as a resource to support metacognition and self-regulation
The reason(s) for engaging in IvE can vary widely among programs and learners, influenced by diverse sociocultural contexts in which learning occurs. One learner may engage in IvE with the intent to develop capabilities for their pursuit of a STEM-oriented college or career path, whereas another learner may simply wish to help people with a problem while developing their own problem-solving capabilities. External benchmarks or determinants of what counts as success may not fit learners’ goals and may be counterproductive to recruiting students who do not have an interest in STEM at the outset of their experience with IvE. Similarly, studies from the maker movement (e.g., Vossoughi et al., 2013, 2016), document many different types of agendas, of educators or program providers, including economic and workforce development; addressing educational inequities; and “engaging young people in personally compelling creative investigations of the material and social worlds (Brahms, 2014; Martinez and Stager, 2013); democratizing access to the tools, skills and discourses of power previously available only to experts (Blikstein, 2013; Halverson and Sheridan, 2014); expanding participation in STEM fields through interest-driven, multidisciplinary learning environments (Martin, 2015)” (Vossoughi et al., 2016, p. 210).
Approaches to assessment that consider the goal-focused orientation of the student and the educator (or program provider) increase the likelihood that both will find value in the time and resources expended on assessment. As educators, we are conscious of the scarcity of financial resources in public education. Assessments are costly to develop and costly for educators to purchase, administer, score, and report. Additionally, we are cognizant of the tension between the time needed for learning versus the amount of time devoted to the assessment of learning outcomes. An assessment system that is integrated into teaching and learning processes, while also providing students with data they can use to monitor their own growth and development, maximizes the potential benefits of the funding and time dedicated to assessment.
IvE educators have an opportunity to consider cognition, metacognition, and ways of teaching metacognitive strategies to help students assess their own learning, thereby enhancing students’ capacity for self-regulation and personal growth. Huang et al. (2022) offer examples of STEM programs in which both individuals and teams were assessed through projects that addressed students’ metacognitive skills. Metacognition and self-regulation are important for scientific literacies as learners engage in “locating, selecting, reading, monitoring, and critiquing various information sources (Wang and Chen et al., 2014; Yore and Treagust, 2006)” (Avargil et al., 2018, p. 33). Avargil et al. (2018) argue that “metacognition is a central feature in life-long learning in general and science education in particular, and that metacognitive engagement is key for developing deeper conceptual understanding of scientific ideas (e.g., Anderson and Nashon, 2007; Blank, 2000; Choi et al., 2011; Georghiades, 2004; Koch, 2001; Nielsen et al., 2009; Wang and Chen et al., 2014)” (p. 33). Wengrowicz et al. (2018) made similar arguments about the value of metacognition and self-regulation within the context of engineering education programs that “train students to be able to conceive, design, implement, and operate complex value-added engineering products, processes, and systems in modern, team-based environments” (p. 191).
4.4 Assessment systems should incorporate multiple measures
As researchers, our insights into the specific traits and characteristics of inventors have been limited by our own backgrounds. None of the authors of this paper have direct experience engaging in inventive activity that resulted in the award of a United States patent. However, we draw upon valuable insights from a group of inventors and leading researchers who came together in 2004 to study invention with funding from the National Science Foundation and the Lemelson Foundation, as well as information about the act of inventing and the capabilities needed by inventors as detailed by the Committee for the Study of Invention (2004). The final publication produced by the committee provided detailed descriptions of the disciplinary knowledge required of an inventor, alongside the ways of knowing, ways of doing (or working, such as hands-on skills), ways of thinking (habits of mind), and ways of being (personal characteristics; Committee for the Study of Invention, 2004). The ways of knowing and thinking include a focus on “transgressive cognition,” or mental moves that cross the boundaries of past practice and convention. This involves tying together academic disciplines in unexpected ways and redefining not only means, but often the problem itself. Additionally, it encompasses challenging entrenched beliefs about the limits of the possible (Invention Education Research Community [IvERC], 2019, p. 12). Additional descriptors are shown in Column 1 of Table 1.
Table 1. Four facets of inventors and their capabilities, as identified by the research community.
A 2022 LMIT study of collegiate inventors who won prizes in a national competition for their work as inventors further underscored these inventors’ views of the multifaceted nature of inventive capacity (Kalainoff et al., 2022). Our analysis of these students’ ways of thinking, knowledge, and ways of knowing, being, and doing are summarized in Column 2 of Table 1. It is worth noting that both the committee and students recognized the need for creativity as a way of being. Unlike their recognition of the necessity of creativity, in many instances the specific words the collegiate students used differed from those used by more experienced inventors and researchers on the committee for the 2004 study. However, the ideas they expressed were very similar. For example, collegiate students did not use the words “boundary transgression” when describing ways of thinking as an inventor. Instead, the students noted the need for “interdisciplinary work where you must speak different languages.” In another example, students mentioned overcoming the fear of failure as a way of being, whereas the committee cited the need to be resilient. Differences in the words and ideas expressed by the two groups reinforced our notion that assessments need to incorporate a wide variety of perspectives. Assessments will need to be customizable by the educator, program provider, and the developing inventor(s) themselves as they monitor their own growth and development.
The information presented in this section suggests that assessment of IvE should incorporate multiple measures due to the interconnected nature of disciplinary knowledge and the diverse ways of thinking, knowing, being, and doing that are required as students engage in IvE. Using singular measures detached from these interconnected aspects of inventing would fail to capture the full spectrum of capabilities and learning outcomes taking place as students engage in IvE projects.
5 Applying the principles for assessing IvE: a case study based on LMIT’s InvenTeams
We offer an example of ways the principles we outlined could be reflected in an assessment system that offers multiple measures for determining assessment of an IvE program at the individual student level. The process and activities described in Section 2.2 guide the work of the LMIT Program as they work/worked with educators and teams of high school students over the past 20 years (i.e., since 1994) as part of a national grant initiative known as InvenTeams. InvenTeams are an example of a year-long educational program that foregrounds the invention process and identifies as providing invention education. The phases of work and detailed activities for each phase of LMIT’s year-long invention process are shown in Table 2.
Table 2. Phases of the IvE process and alignment with practices for U.S. K–12 content standards.
The following types of learning and development have been observed by invention educators: disciplinary knowledge, technical skills and career readiness, and personal capabilities (both individual and group). All of these require multiple assessments to measure students’ growth and development.
5.1 Disciplinary knowledge central to each phase of activity
The IvE process in the LMIT example detailed in Table 2 requires students to apply disciplinary knowledge and practices as they carry out prescribed activities during each of four phases of a project cycle. The cycle concludes in a culminating event where students discuss their selected problem and present their invention prototype to a live audience. Because the process and related activities are constant each year, it is possible to determine the types of knowledge activated and the domains of knowledge or practices applicable to individual disciplines. Therefore, students’ work can be mapped with K–12 content standards and practices. Formative and summative assessments created to measure the applicable standards that are part of K–12 schooling could then be applied to learning taking place during each phase of work students undertake. Table 3 illustrates this possibility by providing an example of ways the activities in phase 1 could be aligned with the process and the practices found in the Next Generation Science and Engineering Standards, Common Core Mathematics Standards, and Common Core English Language Arts Standards. While some content can be mapped to existing standards, other disciplinary content knowledge developed through the project cannot be known at the outset of the process, as it will vary according to the problem and/or the solution generated by the work of the team. Clarity occurs in hindsight, after the problem is identified and studied during the school year. A backwards mapping and summative assessment could serve to capture this additional realm of learning.
Table 3. Phase 1 of the IvE process and alignment with practices for U.S. K–12 content standards.
5.2 Technical skills central to each phase of activity
LMIT does not specifically assess students’ technical skills or career readiness as part of the InvenTeam effort. However, aspects of their existing assessments offer opportunities to do so. Researchers on staff have explored whether learners’ work on technological solutions to problems could be aligned with assessments of computer science or computational thinking skills. A self-published study shows that the technologies used by the students in the creation of their prototype change over time as technology evolves (Estabrooks et al., 2019). Predetermining the technology skills to be assessed would be difficult, as the technology should be tailored to the problem that learners are working to solve. Given the nature of the invention process, the solution cannot be envisaged at the outset.
LMIT’s experiences suggest the need for hands-on and other skills developed during the invention project to reflect the problem being solved and the individual’s role on the team (such as finance lead, project manager, communications, and technical team). While all team members engage in all project components, the amount of time spent engaged in learning activities related to each role will vary, making it impractical to expect uniform achievement on a single outcome measure. Therefore, we should not expect students to meet the same benchmarks on a singular assessment, although all students may show some level of growth since there are opportunities for peer-to-peer teaching and learning. To effectively gauge these skills, an inventory of skills assessments could be curated, allowing instructors and students to select the testing instrument most relevant to their project focus and/or students’ individual roles on a team. Different measures may apply at different phases of the process. For example, the problem-finding stage—Phase 1 of the IvE process—could assess Computer Science (CS) practices. In California, the CS practice includes “collaborating around computing” and “fostering an inclusive computer culture.” Students could be guided in their use of computational thinking and CS in this phase and assessed for those practices in ways that are already in use in schools.
Existing career readiness assessments could be modified by educators or programs on pre- and post- experience surveys (e.g., similar items to InvenTeams). The national Career Readiness Standards from the Common Career Technical Core, for example, identify the need for students to “act as responsible and contributing citizens and employees” and to “work productively in teams while using cultural global competence.” These and other Career Readiness standards align with IvE activities and could be applied as part of a multiple-measures assessment system.
5.3 Personal capabilities central to individual roles and team accomplishments
LMIT assesses students’ perceptions of their own personal growth, attributable to participation in IvE through student reflections in Inventor Notebooks and experience surveys with forced-choice and open-ended answers. Experience surveys have probed student reflections on characteristics that align with growth mindsets (Dweck, 2006) and “grit” (Duckworth, 2013), including factors such as persistence and learning from failure. While aligned with ways growth mindsets and grit are assessed, the questions on the LMIT surveys were developed independently of the two ideas and were based on the LMIT staff’s familiarity with inventors and the characteristics described in Table 1 of Section 4.4. Care must be taken when using such measures to account for the quality or meaning of the work to the student, presence or absence of intellectual safety, or other kinds of cultural differences and microaggressions students have navigated while participating in IvE (Vossoughi et al., 2016; Kohn, 2014; Norris, 2014). Students may have very different experiences, and measures of personal characteristics may reflect the students’ orientation to the experience, as opposed to their own capabilities.
LMIT has considered creating portfolios with samples of student work, observation rubrics, and other indications of student performance. These portfolios would demonstrate personal development in relation to pre-defined competencies and accompanying rubrics to capture change through the duration of students’ engagement in IvE programs. The portfolio system would be designed in a way that offers flexibility for the educators and the students to determine both the what and the how of assessment. The portfolio would belong to the student so that it could be used to document development in the variety of contexts that a learner may experience across their educational journey. Competencies, such as the Future9 Competencies developed by redesign1 for use at the high school level, could be aligned with activities at each phase of the InvenTeam project. For example, students could provide evidence for specific Future9 Competencies, such as, “I can identify challenges in the world around me and design ways to address them. I can nurture my relationships and connections with others to build and sustain my community” (reDesign, 2024, p. 6). Competencies like the ones developed by Future9 align well with the phases of work that learners conduct in the IvE process, and evidence of progress toward these competencies—as developed through the IVE process—could be provided via an electronic portfolio evaluated with an associated scoring rubric.
6 Discussion of the theories and principles for IvE teaching, learning, and assessment
The application of the learning theories and assessment principles to the LMIT InvenTeams national IvE program described in the Section 5 case study demonstrates their potential usefulness for designing a new assessment framework. Such a framework will need to be responsive to opportunities like IvE that are transdisciplinary and activate all levels of Bloom’s Taxonomy as learners develop new creations, such as invention prototypes. Table 4 summarizes the interrelatedness of theories of teaching and learning, principles for IvE, and assessment measures. Specifically, constructivism and constructionism guide our understanding that learning can be supported through engagement with people, tools, materials, and texts at each phase of the invention process. What is being learned can be assessed through formative and summative assessments of particular disciplinary knowledge, practices and concepts that are core to the process and repeated over time as students repeat the process. Some of the disciplinary knowledge, practices, and concepts being learned will be unique to the invention problem and solution and therefore cannot be pre-determined. Our emphasis on sociocultural aspects of knowing (or coming to know) are reflected in IvE pedagogy in the ways students draw on their own cultural and community assets and learn through dialogues with community members during the phases of work. Assessment resources that support metacognition and self-regulation in response to what is learned by engaging with others and with the IvE process, such as competency rubrics and pictures of physical objects constructed using tools and materials, can be collected and presented to others in e-portfolios. A comprehensive view of ways a learner develops through IvE can have elements that are aligned with existing standards, and supplemented by other measures that are both feasible and beneficial.
Table 4. Application of theories and principles to the assessment of LMIT’s InvenTeams.
The potential “fit” between the multiple measures we envision for IvE offerings in public schools and the IvE measures that are needed in a variety of other contexts would be an additional benefit of the new approach. Because learners can engage in IvE across a variety of spaces, including formal school settings, makerspaces, libraries, camps, science fairs, invention conventions, museums, and prize programs (Invention Education Research Community [IvERC], 2019), maximum benefit would be derived from assessments that are useful across all contexts for both the learner and the program provider. This consideration led us to explore the potential alignment of approaching IvE assessment using a complexity lens, like the approach used in developmental evaluation (Patton et al., 2015). Patton has identified four key features of developmental evaluation (2016, p. 277), which align and overlap with the principles we put forth in this paper:
1. An organizing framework is developed.
2. Data are layered over time and aligned with the organizing framework.
3. Data collection, reporting, and sensemaking are timed to meet the needs of key stakeholders.
4. Engagement in values-based collaborative sensemaking takes place.
Patton argued that “Developmental evaluation is not a set of methods, tools, or techniques” and that it “contrasts with prescriptive models, which, like recipes, provide standardized directions that must be followed precisely to achieve the desired outcome” (Patton, 2016, p. 290).
Moreover, developmental evaluation is especially useful for contexts characterized by unpredictability, uncertainty, emergence, interdependency, nonlinearity, and feedback loops (Eoyang and Berkas, 1998). Such a complexity lens fits well for the transdisciplinary nature of STEM programs that foreground IvE. To better understand what is going on in complex IvE programs, we need evidence from multiple perspectives to pay attention and take notice of what is happening. The layering of multiple data collection methods, combined with participative, iterative, and collaborative sensemaking, that is appropriate for a developmental evaluation (Kurtz and Snowden, 2003) may also be a useful approach in IvE.
The principles and potential components of the assessment system explored in this paper could serve as the organizing framework that parallels Patton’s approach to accounting for complexity in a system, providing a robust method for assessing the complex nature of IvE programs. The need for multiple measures suggests the importance of considering electronic portfolios as vehicles for aggregating measures and providing possibilities for student reflections about their growth over time. Electronic portfolios can serve as effective tools for collecting and selectively sharing artifacts, while also empowering students to engage in the assessment of their own learning and development. Several examples of the use of portfolios exist. Portfolios have been used in an Introduction to Design course to assess the growth of skills and knowledge (Newstetter and Khan, 1997), many of which overlap with those needed for IvE, such as procedural know-how and conceptual knowledge for problem structuring, decomposition of a problem into phases, and more.
Newstetter and Khan (1997) note that good portfolio assessment requires continued reflection on key issues such as the purpose of the assessment, the tasks to be included in the portfolio, the standards and criteria to be applied, and how to ensure consistency in scoring (Newstetter and Khan, 1997). The use of portfolios in ways that attend to these issues, in combination with technologies that allow for the aggregation of evidence of individuals’ learning and development over time, could be particularly beneficial to education researchers studying the development of humans’ capacity to invent.
7 Conclusion: the discipline-agnostic approach of IvE benefits from multiple measures
Our review of the literature surrounding IvE and creativity indicates the need for a new assessment system that accounts for the shift from teaching and learning that privileges individual disciplines to the transdisciplinary work of finding and developing technological solutions to real-world problems. Transitioning schooling to deeper learning methods that activate knowledge processes at all levels of Bloom’s taxonomy requires us to rethink what counts as learning. More importantly, we must consider new ways of knowing or showing when learning has occurred. We have offered evidence of the need for multiple measures that capture the range of accomplishments by educators, students, and others engaged in the open-ended, inquiry- and problem-based learning approach known as IvE. Flexibility in determining which measures to include would account for differences in the ways IvE is being enacted, different contexts in which it is being made accessible to learners (both formal and informal), and differences in the goal-directedness of the learners, educators, and program providers.
We have outlined several types of measures that could be part of a new assessment system. These include:
1. Portions of standardized assessments for practices aligned with individual disciplines that are activated during the IvE process (e.g., the Science and Engineering, Mathematics, and English Language Arts practices shown in Table 2, Columns 2–4);
2. Skills assessments that fit the focus of individual team projects and participant roles (e.g., Computer Science practices);
3. Career readiness assessments; and.
4. Competencies and related rubrics for a broader array of capabilities and personal characteristics (e.g., the Future9 Competencies).
Agreement among education leaders on a common set of principles, practices, and tools could accelerate the development of a new system of assessment. The IvE field would benefit from a common process by which a variety of assessment measures could be identified, approved, and validated for use. Consistency in scoring portfolios and assessing competencies could be addressed by limiting portfolio artifacts to those with measures that have been registered and approved by a (currently nonexistent) body that would maintain the library of assessments and oversee their use in a variety of academic and non-academic contexts. The skills needed for invention problems and projects likely will vary across contexts, age, and grade levels. Therefore, assessment tools will need to be designed flexibly so that they are responsive to a wide range of contexts likely to exist in future IvE programs.
IvE is not the only field that has identified challenges with using traditional assessment tools common in United States public schools to assess learning. We have explored several ways existing tools and approaches could be utilized to assess what is being accomplished at different stages of the invention process. Educators could draw from these options for immediate solutions while policymakers begin to consider broader changes to the student assessment systems. Future work could examine emerging assessment practices in the global context that might provide additional perspectives and ways to address the needs of educators and learners engaging in IvE.
Author contributions
SC: Conceptualization, Methodology, Resources, Supervision, Writing - original draft. KP: Data curation, Resources, Visualization, Writing - review & editing. MS: Writing - review & editing. SS: Writing - review & editing.
Funding
The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This systematic review is funded in part by the Lemelson Foundation through a grant that supports the operations of the LMIT Program (grant number 21-01968). The Lemelson Foundation is not directly involved in the conduct of the review and will have no input on the interpretation or publication of the study results.
Acknowledgments
The authors would like to acknowledge Fabrizio Gentili and Wendy Nikolai for their diligent efforts that contributed to the completion of this paper.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2024.1306016/full#supplementary-material
Footnotes
References
Agarwal, P. K. (2019). Retrieval practice & Bloom’s taxonomy: do students need fact knowledge before higher order learning? J. Educ. Psychol. 111, 189–209. doi: 10.1037/edu0000282
Anderson, D., and Nashon, S. (2007). Predators of knowledge construction: Interpreting students’ metacognition in an amusement park physics program. Sci. Educ, 91, 298–320.
Armstrong, P. (2010). Bloom’s taxonomy. Vanderbilt University Center for Teaching. Available at: https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ (Accessed June 10, 2024).
Aulet, B. (2013). Disciplined entrepreneurship: 24 steps to a successful startup. Hoboken, NJ: Wiley.
Avargil, S., Lavi, R., and Dori, Y. J. (2018). “Students’ metacognition and metacognitive strategies in science education” in Cognition, metacognition, and culture in STEM education. eds. Y. J. Dori, Z. R. Mevarech, and D. R. Baker (Cham, Switzerland: Springer), 33–64.
Blank, L. M. (2000). A metacognitive learning cycle: A better warranty for student understanding?. Sci. Educ, 84, 486–506.
Blikstein, P. (2013). Digital fabrication and making in education: The democratization of invention. In J. Walter-Herrmann and C. Büching (Eds.), Fablabs: Of machines, makers and inventors. Bielefeld, Germany: Transcript. (pp. 1–21)
Brahms, L. J. (2014). Making as a learning process: Identifying and supporting family learning in informal settings. Unpublished doctoral dissertation, University of Pittsburgh.
Bloom, B.S. (Ed.), Engelhart, M.D., Furst, E.J., Hill, W.H., and Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals; handbook 1: Cognitive domain. New York: David McKay.
Bridges, S., Green, J., Botelho, M.G., and Tsang, P.C.S. (2015). “Blended learning and PBL: an interactional ethnographic approach to understanding knowledge construction in-situ,” in Essential readings in problem-based learning: Exploring and extending the legacy of Howard S. Barrows, eds. A. Walker, H. Leary, C. Hmelo-Silver, and P.A. Ertmer (West Lafayette, IN: Purdue University Press), 107–130.
Choi, K., Lee, H., Shin, N., Kim, S. W., and Krajcik, J. (2011). Re-conceptualization of scientific literacy in South Korea for the 21st century. J. Res. Sci. Teach. 48, 670–697.
Committee for the Study of Invention (2004). Invention: enhancing inventiveness for quality of life, competitiveness, and sustainability. Available at: http://lemelson.mit.edu/search-resources/817 (Accessed June 17, 2024).
Couch, S., Estabrooks, L. B., and Skukauskaite, A. (2018). Addressing the gender gap among patent holders through invention education policies. Technol. Innov. 19, 735–749. doi: 10.21300/19.4.2018.735
Couch, S., Skukauskaite, A., and Green, J. L. (2019). Invention education: preparing the next generation of inventors. Technol. Innov. 20, 161–163. doi: 10.21300/20.3.2019.161
Eoyang, G. H., and Berkas, T. (1998). Evaluation in a complex adaptive system. Circle Pines, MN: Chaos Limited.
Estabrooks, L. B., and Couch, S. (2018). Failure as an active agent in the development of creative and inventive mindsets. Think. Skill. Create. 30, 103–115. doi: 10.1016/j.tsc.2018.02.015
Estabrooks, L., Zhang, H., Perry, A., Chung, A.-M., and Couch, S. (2019). Where’s the computer science in invention? An exploration of computer science in high school invention projects [white paper]. Cambridge, MA: Massachusetts Institute of Technology.
Ewell, E., Haglid, H., Truszkowski, E., Walicki, C., DeMeulder, P., DeMeulder, M., et al. (2022). Invention as a complement to high school chemistry. J. Chem. Educ. 99, 2012–2019. doi: 10.1021/acs.jchemed.2c00135
Frigotto, M. L. (2018). Understanding novelty in organizations: A research path across agency and consequences. New York, NY: Springer.
Gale, J. (2022). Inventing the baby saver: an activity systems analysis of applied engineering at the high school level. J. Pre-College Engin. Educ. Res. 12, 1–19. doi: 10.7771/2157-9288.1312
Getting Smart (2024). Portrait of a graduate gallery. Available at: https://www.gettingsmart.com/whitepaper/the-portrait-model-building-coherence-in-school-and-system-redesign/portrait-of-a-graduate-gallery/ (Accessed November, 1, 2024).
Green, J. L., Baker, W. D., Chian, M. M., Vanderhoof, C., Hooper, L., Kelly, G. J., et al. (2020). Studying the over-time construction of knowledge in educational settings: a microethnographic discourse analysis approach. Rev. Res. Educ. 44, 161–194. doi: 10.3102/0091732X20903121
Green, J. L., and Bridges, S. M. (2018). “Interactional ethnography” in International handbook of the learning sciences. eds. F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, and P. Reimann (New York, NY: Routledge), 475–488.
Georghiades, P. (2004). From the general to the situated: Three decades of metacognition. Int. J. Sci. Educ. 26, 365–383.
Halverson, E. R., and Sheridan, K. (2014). The maker movement in education. Harv. Educ. Rev. 84, 495–504.
Huang, X., Lin, C.-H., Sun, M., and Xu, P. (2022). Metacognitive skills and self-regulated learning and teaching among primary school teachers: the mediating effect of enthusiasm. Metacogn. Learn. 17, 897–919. doi: 10.1007/s11409-022-09297-9
Invention Education Research Community [IvERC] (2019). Researching invention education: A white paper. The Lemelson Foundation. Available at: https://lemelson.mit.edu/node/2511 (Accessed November 2, 2024).
Kalainoff, M.Z., Couch, S.R., and Cima, M. (2022). Lemelson-MIT student prize retrospective. Lemelson-MIT. Available at: https://lemelson.mit.edu/sites/default/files/2022-12/StudentPrizeWinners_Impacts.pdf and https://lemelson.mit.edu/sites/default/files/2022-12/InventorDevelopmentalPathways.pdf (Accessed November 2, 2024).
Koch, A. (2001). Training in metacognition and comprehension of physics text. Sci. Educ. 85, 758–768.
Kohn, A. (2014). The myth of the spoiled child: Challenging the conventional wisdom about children and parenting. Philadelphia, PA: Da Capo Press.
Korhonen, T., Kangas, K., and Salo, L. (2022). Invention pedagogy–the Finnish approach to maker education. Abingdon, Oxon, United Kingdom: Routledge. 304.
Kurtz, C. F., and Snowden, D. J. (2003). The new dynamics of strategy: sense-making in a complex and complicated world. IBM Syst. J. 42, 462–483. doi: 10.1147/sj.423.0462
Kuznets, S. (1962). Inventive activity: Problems of definition and measurement. The Rate and Direction of Inventive Activity: Economic and Social Factors, 1, 19–51.
Middendorf, W. H. (1981). What every engineer should know about inventing. New York, NY: Marcel Dekker Inc.
Mitchell, C. J. (1984). “Typicality and the case study” in Ethnographic research: A guide to general conduct. ed. R. F. Ellen (London: Academic Press), 238–241.
Martinez, S. L., and Stager, G. (2013). Invent to learn: Making, tinkering, and engineering in the classroom. Torrance, CA: Constructing Modern Knowledge Press.
Martin, L. (2015). The promise of the maker movement for education. Journal of Pre-College Engineering Education Research (J-PEER), 5, 30–39.
Nielsen, W. S., Nashon, S., and Anderson, D. (2009). Metacognitive engagement during field-trip experiences: A case study of students in an amusement park physics program. J. Res. Sci. Teach. 46, 265–288.
National Science and Technology Council, Committee on STEM Education (2022). Convergence education: a guide to transdisciplinary stem learning and teaching. Available at: https://www.whitehouse.gov/wp-content/uploads/2022/11/Convergence_Public-Report_Final.pdf (Accessed April 12, 2024).
Newstetter, W.C., and Khan, S. (1997). “A developmental approach to assessing design skills and knowledge,” in Proceedings of the Frontiers in Education 1997 27th Annual Conference. Teaching and Learning in an Era of Change, 2. pp. 676–680.
Norris, M.D., (2014). At risk students and resiliency. [dissertation: paper 62]. [San Bernardino (CA)]: California State University, San Bernardino. Available at: https://scholarworks.lib.csusb.edu/etd/62 (Accessed May 27, 2024).
Patton, M. Q. (2016). “The developmental evaluation mindset: eight guiding principles” in Developmental evaluation: Applying complexity concepts to enhance innovation and use. eds. M. Q. Patton, K. McKegg, and N. Wehipeihana (New York, NY: Guilford Press), 289–312.
Patton, M. Q., McKegg, K., and Wehipeihana, N. (2015). Developmental evaluation exemplars: Principles in practice. New York, NY: Guilford Press.
Putney, L. G., Green, J., Dixon, C., Durán, R., and Yeager, B. (2000). “Consequential progressions: exploring collective individual development in a bilingual classroom” in Vygotskian perspectives on literacy research: Constructing meaning through collaborative inquiry. eds. C. D. Lee and P. Smagorinsky (New York, NY: Cambridge University Press), 86–125.
Saenz, C., and Skukauskaitė, A. (2022). Engaging Latina students in the invention ecosystem. Technol. Innov. 22, 303–313. doi: 10.21300/22.3.2022.4
Sawyer, R. K. (2019). The creative classroom: Innovative teaching for 21st century students. New York, NY: Teachers College Press.
Shavinina, L., and Seeratan, K. (2003). “On the nature of individual innovation” in The international handbook on innovation. ed. L. Shavinina (Kidlington, Oxford: Elsevier Science), 31–43.
Skukauskaite, A., Saenz, C., Sullivan, M., Hull, K., and Rodriguez, J. M. (2023). Networks supporting problem-based invention education. Interdis. J. Prob. Learn. 17:1. doi: 10.14434/ijpbl.v17i1.32560
Stanford, L. (2024). More states are creating a 'portrait of a graduate.' Here's why. Ed Week, 43(13). https://www.edweek.org/policy-politics/more-states-are-creating-a-portrait-of-a-graduate-heres-why/2023/12 (Accessed November 2, 2024).
Vossoughi, S., Escudé, M., Kong, F., and Hooper, P. (2013). “Tinkering, learning and equity in the after-school setting.” in Paper presented at FabLearn Conference, Stanford University, CA.
Vossoughi, S., Hooper, P. K., and Escudé, M. (2016). Making through the lens of culture and power: toward transformative visions for educational equity. Harv. Educ. Rev. 86, 206–232. doi: 10.17763/0017-8055.86.2.206
Wagner, T. (2012). Creating innovators: The making of young people who will change the world. New York, NY: Scribner.
Wegerif, R. (2015). “Technology and teaching thinking: why a dialogic approach is needed for the twenty-first century,” in The Routledge international handbook of research on teaching thinking, eds. R. Wegerif and L. Li, and J.C. Kaufman (New York and London: Routledge), 427–440.
Wengrowicz, N., Dori, Y.J., and Dori, D. (2018). “Metacognition and meta-assessment in engineering education,” in Cognition, metacognition, and culture in STEM education, eds. Y.J. Dori and Z.R. Mevarech, and D.R. Baker (Cham, Switzerland: Springer), 191–216.
Wang, J. R., and Chen, S. F. (2014). Exploring mediating effect of metacognitive awareness on comprehension of science texts through structural equation modeling analysis. J. Res. Sci. Teach. 51, 175–191.
Yore, L. D., and Treagust, D. F. (2006). Current realities and future possibilities: Language and science literacy – empowering research and informing instruction. Int. J. Sci. Educ. 28, 291–314.
Keywords: invention, STEM, education, transdisciplinary, creativity, assessment, problem-based learning
Citation: Couch S, Paul KM, Sullivan M and Sultana S (2024) Assessing learning and development through transdisciplinary problem-based invention education offerings. Front. Educ. 9:1306016. doi: 10.3389/feduc.2024.1306016
Edited by:
Joanna K. Garner, Old Dominion University, United StatesReviewed by:
Yizhou Qian, Jiangnan University, ChinaShiri Gandhi, Eastern Michigan University, United States
Copyright © 2024 Couch, Paul, Sullivan and Sultana. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Stephanie Couch, c2NvdWNoQG1pdC5lZHU=