Parts of this article's content have been modified or rectified in:
Erratum: Developing and assessing pre- and in-service science and engineering teachers' systems thinking and modeling skills through an asynchronous online course
Roee Peretz
Dov Dori2
Yehudit Judy Dori- 1Faculty of Education in Science and Technology, Technion, Israel Institute of Technology, Haifa, Israel
- 2Faculty of Data and Decision Sciences, Technion, Haifa, Israel
- 3The Samuel Neaman Institute for National Policy, Technion, Haifa, Israel
Systems thinking and modeling are two critical 21st-century skills that teachers and educators are expected to impart to students, and students are expected to acquire and master them as part of their preparation to become literate citizens of a society and environment that is becoming ever more complex. Systems thinking is a thought process in which assumptions about interactions among interconnected elements of a system or a phenomenon can help predict the system’s behavior, outcomes, and in the case of human-made artifacts, the value to its beneficiaries. Conceptual modeling involves the simultaneous visual and textual representation of one’s ideas about a phenomenon or system in science or engineering. The qualitative study described here aimed to examine the effect of an online interdisciplinary asynchronous course on the development of systems thinking and conceptual modeling skills among pre- and in-service science and engineering teachers. Engaging in a qualitative case study with an exploratory orientation, we investigated how science and engineering teachers and teacher educators coped with (a) online learning of conceptual modeling and systems thinking using Object-Process Methodology in a food and sustainability context, and (b) developing an online assignment for teaching those skills to their students and assessing them. Research tools included the online assignment that the participants developed, a dedicated rubric for analyzing their assignments, accounting for use of modeling and systems concepts and the integration of sustainability and COVID-19 issues, a variety of thinking skills, visualizations and disciplines, and a mix of closed- and open-ended questions. Additionally, the participants’ reflections were analyzed to characterize their sense of self-efficacy and academic progression. We characterize five teacher-developed assignment cases along with the related teachers’ reflections, which exposed the benefits they had gained from the online course, as well as the systems thinking and modeling challenges they had faced. Analysis of the effect of the course with emphasis on the final task reveals that this approach is effective for developing the systems thinking and modeling skills of the teachers and serves as a catalyst for their professional development. The study offers a methodological contribution by providing a basis for evaluating teachers’ assessment knowledge and skills using a six attributes rubric.
Introduction
As the world is becoming increasingly interconnected, a knowledge-based economy vigorously and ceaselessly drives globalization forward (Ritzer and Dean, 2019). Like any other social system, education systems need to prepare new professionals to tackle complex systems that characterize today’s world and complex problems that arise as these systems, whether man-made or natural, affect humanity and the environment (Vivekanandan and Pierre-Louis, 2020). An example of a complex system is the National Healthcare Service (Checkland, 2000), or the stock market (Amaral and Ottino, 2004). These kinds of systems often combine people, technology, information, stakeholders, enablers, conflicting interests, and external influences. However, a flock of migrating birds or a termite colony are also (natural) complex systems (Amaral and Ottino, 2004). Common to all complex systems is the fact that they consist of a large number of parts or details that interact with each other and with their environment, making the system highly adaptable, and they are not organized based on a known external organization principle (Amaral and Ottino, 2004). Unlike simple or even complicated systems, complex systems are hardly amenable to predicting long-term performance or outcomes. Differences between simple, complicated, and complex systems can be found in detail in Amaral and Ottino (2004).
A thought process for examining interconnected elements within systems to predict their behavior (Dori, 2016), systems thinking is a higher-order thinking skill that is crucial to responding to the challenges and complexities of the 21st century (Sterman, 1994). A complex system, the National Healthcare Service, for example, requires systems thinking for designing or changing it (Checkland, 2000).
Despite its emergence from the complex reality, systems thinking is neither natural nor innate, and often it is even counterintuitive (Gharajedaghi, 2011). Various educational practices must therefore be adopted in order to foster systems thinking. Leading among these practices are systems modeling, interdisciplinary learning, and context-based learning. These are recognized as essential to 21st-century learners according to various educational frameworks in science, technology, engineering, and mathematics (STEM) disciplines (e.g., Partnership for 21st Century Skills, 2004; NGSS Lead States, 2013; Accreditation Board for Engineering and Technology, 2021).
Yoon et al. (2017) discussed the crucial position of professional development programs for teachers. These are imperative to ensure successful instruction about complex systems and increase teachers’ awareness of using models and modeling in science education. Such professional development programs are equally vital for pre-service teachers. The research described in this paper offers an in-depth view of the challenges and opportunities involved in a conceptual modeling-based online interdisciplinary course. We also investigated the learning process pre- and in-service science teachers went through as they were studying this course, as well as their learning outcomes. We focus on imparting the basis for systems thinking skills and their assessment, followed by self-developed assignments as the final stage of the learning process of the course participants.
Theoretical framework
In this section, the theory underlying systems thinking and conceptual modeling is presented with a focus on their relations to STEM education and teachers’ understanding of these skills.
Systems thinking: Definitions and recent research
Scholars have defined systems thinking differently, but many of the definitions share important commonalities. Arnold and Wade (2015) claimed that the breakdown of systems thinking into its main components may facilitate its teaching and assessment, which is consistent with other studies (e.g., Assaraf and Orion, 2005; Lavi et al., 2019). The main systems thinking components include (1) recognizing interconnections, (2) understanding feedback loops, (3) understanding the system’s structure, (4) differentiating stock and flow variables, (5) understanding non-linearity, (6) understanding dynamic behavior, (7) reducing complexity by conceptual modeling, and (8) recognizing different scales of systems (Arnold and Wade, 2015). These components are common to different frameworks for systems thinking (e.g., Assaraf and Orion, 2005; Stave and Hopper, 2007; Lavi et al., 2019). Arnold and Wade (2017) also divided systems thinking into two facets, extending beyond just understanding systems: Gaining insights, which relates to approaching systems from the outside and investigating them from several viewpoints, and conversely using insights, which is, broadly speaking, approaching systems from the inside, such as rearranging their structure. Each facet encompasses a specific set of techniques that may be used in parallel or in series, constantly strengthening each other.
In this study, we define systems thinking according to Dori et al. (2020) as thinking that involves examining the connections and interactions between elements within a system or phenomenon to understand how they function to influence behavior and to determine the value of human-made artifacts for their intended beneficiaries.
Although a considerable number of studies have examined the development and assessment of systems thinking in different STEM education settings (e.g., Assaraf and Orion, 2005; Gero and Zach, 2014; Lavi and Dori, 2019), the teaching and instruction of this important concept are at best mainly implicit, and students struggle when faced with a topic that requires a high degree of systems thinking (Arnold and Wade, 2017; Chen et al., 2019; Talanquer, 2019). Explicit instruction about complex systems and systems thinking concepts can lead to the deepening of knowledge and understanding and the transfer of these concepts among students (Goldstone, 2006; Hung, 2008), as well as teachers (Yoon et al., 2017). Rates et al. (2022) found that explicitly teaching students about complex system concepts was more effective than self-monitoring scaffolding, which is important in its own right. They referred to explicit learning as “ontological scaffolding,” building on the work of Jacobson et al. (2011). In this context, ontology can be defined as the explicit formal specification of the nature and structure of a system, described in terms of categories and relations (Guarino et al., 2009). Chowdhury (2023), conversely, argued that instead of enclosing systems thinking in a framework that is loaded with professional language, considering systems thinking as a cognitive skill might lead to its greater acceptability by a wider target audience. Verhoeff et al. (2018) claimed that given the current research knowledge of how to foster higher-order systems thinking skills and the conflicting curricular considerations, a complete curricular program of systems thinking teaching is not yet possible. Higher-order systems thinking skills include the ability to understand nonlinearities and cyclicity in systems and recognize complex patterns and relationships, and predict future behavior from current systemic interactions (Assaraf and Orion, 2005; Verhoeff et al., 2018). Another key higher-order systems thinking skill is modeling (Stave and Hopper, 2007).
Conceptual modeling and STEM education
Modeling can facilitate the understanding of complex systems, as well as their explicit teaching and assessment (Hung, 2008; Dori, 2016). Modeling languages and methodologies are important for expressing what complex systems do, why and how they do it, and what is required for that purpose. Researching natural systems or designing human-made ones often involves complexities that cause a significant cognitive load on the learner or designer. By modeling, unnecessary complexity can be reduced, while necessary complexities can be expressed and emphasized (Dori, 2016). To engage students in model-based complex systems thinking, their teachers must master it first (Yoon, 2008; Krell and Krüger, 2016), and these practices must be systematically integrated into the curriculum (Rosenkränzer et al., 2017; Talanquer, 2019).
Science and technology education is largely driven by models and modeling (Gilbert et al., 2000). Merriam-Webster (n.d.) online dictionary defines the verb “to model” as “to produce a representation or simulation of (something).” In the same spirit, models are representations or simulations of something—a phenomenon, a system, the desired product, or even an idea or event (Gilbert et al., 2000). Unlike the internally generated mental models (Johnson-Laird, 1983), conceptual models are external, or ‘expressed’ (Gilbert et al., 2000) representations that can be shared within a given group (scientists, engineers, teachers, etc.) and are coherent with the accepted knowledge of that group. Conceptual modeling is of high value to STEM education as it reflects the transition from models that are personal, incomplete, and lacking firm boundaries, to more precise and complete representations of the accepted knowledge (Norman, 1983; Gilbert et al., 2000). However, conceptual modeling is seen from a didactic point of view as complex and difficult to master by learners and teachers (Rosenthal et al., 2019).
As early as 1989, Richard Mayer concluded his article named Models for Understanding with the statement that “One particularly exciting avenue concerns the role of interactive computer graphic simulations as a vehicle for expanding the power of conceptual models” (Mayer, 1989, p. 61), backing the statement by the work of White (1984). The conceptual modeling language and methodology used in the current study is OPM—Object-Process Methodology (Dori, 2016)—which is ISO1 19,450 and has been implemented in a dedicated online modeling platform, as elaborated in the Materials and Methods section. OPM has been researched over the years in educational contexts (Lavi and Dori, 2019; Akiri et al., 2020; Peretz et al., 2023), and it is the most researched model-based systems engineering modeling method (Dong et al., 2022). Due to its simplicity, intuitiveness, and bimodality (Dong et al., 2022), as well as its domain-independent nature, OPM is most suitable for teaching, learning, and assessment of learning in various disciplines (Dori, 1995). However, a qualitative evaluation of an OPM-based learning process has not yet been carried out. In this research, OPM has served to assess teachers’ systems thinking-related knowledge and abilities as they develop learning materials based on inherently interdisciplinary system ideas.
Modeling is an activity that inherently involves various aspects of the modeled system or phenomenon. This may increase the opportunity for interdisciplinary collaboration between STEM disciplines in authentic settings and, in turn, promote successful integration between and outside STEM disciplines (Hallström and Schönborn, 2019). From the opposite perspective, systems thinking may be fostered through interdisciplinary learning processes, as reported by Gero and Zach (2014) and others (e.g., Ackerman and Perkins, 1989). This is especially important in educational environments that are often disciplinary, and when dealing with inherently complex and multifaceted issues, such as sustainability (Riess and Mischo, 2010; Harsaae et al., 2022). Interdisciplinarity is usually defined as the integration and communication across at least two different academic disciplines (Frodeman, 2013). For example, considering waste production and particulate emission, food production requires an understanding of biology, chemistry, and environmental aspects. Engineering and economic aspects need to be considered too in order to comprehend engineered systems such as food production ones. The interdisciplinary approach is already a fait accompli in STEM research and STEM industries, but its application to STEM education is still superficial (Klaassen, 2018).
Teachers’ understanding and teaching of complex systems
The ability of teachers to engage in the instruction of systems thinking through conceptual modeling in an interdisciplinary learning environment depends, among other things, on their ability to assess this kind of learning in a relevant context-based learning setting (Pilot and Bulte, 2006). As assessment knowledge depends on pedagogical-content knowledge, PCK (Avargil et al., 2012; Tal et al., 2021), both the teachers’ instructional abilities and the content knowledge have to be at a sufficiently high level to fundamentally change the teaching of systems thinking and conceptual modeling as standalone disciplines and as part of STEM teaching (Rosenkränzer et al., 2017). Yoon et al. (2018) pointed out that there is a lack of research on what teachers need in professional development activities concerning complex systems.
Research aim and research questions
In our study, we used a case study approach to investigate the knowledge and ability teachers need to develop conceptual modeling-based online assignments after being introduced to basic system concepts through an OPM-based online learning process. Engaging in a qualitative case study with an exploratory orientation, we investigated the following two research questions (RQs):
RQ1: How did online learning of conceptual modeling and systems thinking in the context of food production and sustainability affect the STEM teachers’ performance?
RQ2: What was the level of each of the six attributes—conceptual modeling, systems thinking, sustainability and COVID-19, thinking skills, visual representations, and interdisciplinarity—as expressed in the online assignments developed by the STEM teachers for their students?
Materials and methods
Rather than generalizing to a wider population or testing a hypothesis, case studies aim to provide a rich, detailed understanding of a specific case. In a collective case study, the phenomenon of interest is researched through selected cases that have both similarities and differences. For each case in the study, the researcher considers its key circumstances, particularity, and complexity (Stake, 1995, p. xi). We implemented a descriptive collective case study to examine how pre- and in-service STEM teachers of various backgrounds approached an online, model-based interdisciplinary learning process.
Research context and procedure
The soaring complexity in almost every aspect of life calls for proper preparation to enable teachers to tackle this reality (Rosenkränzer et al., 2017). To this end, we developed an asynchronous interdisciplinary online learning process, containing four food-related modules that aim to both develop and assess systems thinking and conceptual modeling. Upon completing the learning process, the participants were asked to develop their own assignments as the final stage of their learning process. They were instructed to choose a topic related to food production and combine in their assignment a variety of representations and thinking skills. They were also instructed to include at least one aspect of sustainability and one related to COVID-19. All the assignments had to be accompanied by corresponding conceptual OPM models the participants had to create from scratch.
OPM
OPM is a model-based systems engineering methodology and language (Dori, 2016). Conceptual models created in OPCloud, a web-based collaborative software environment for modeling in OPM (Dori et al., 2019), contain only things and links. In OPM, the link types are divided into two main groups: structural and procedural (see the scoring rubric in the Conceptual Modeling subsection regarding link types and their impact on the scoring). A thing may be a process or an object, each representing a basic unit of knowledge, namely a concept, whereas links represent the relations between things (Dori, 2016). OPM includes both graphical and textual modalities, with the latter automatically generated while the former is created by the modeler. OPM models are organized in a hierarchical tree structure using object-process diagrams (OPDs. See Table 1 for acronyms). The system diagram, SD, is the OPD at the highest level of abstraction in this hierarchy, and it can be further refined into a more detailed view called SD1. This more detailed OPD elaborates on the structural, behavioral, and functional aspects of the system described in SD. An example of SD and its corresponding OPL is presented in Figure 1. Although the refinement process can go on to more detailed levels in other diagrams of the model (SD1.1, SD1.2, SD1.2.1, and so on), in this research the refinement did not go beyond SD1. As OPM is bimodal, each element that is added to the graphical representation in the model, OPD, simultaneously generates a corresponding sentence in object-process language (OPL)—a subset of English or any other natural language. This provides for ongoing metacognitive reflection while building the OPM model and not just retrospectively. Consequently, each OPL paragraph of any given OPD expresses textually the same model facts that the OPD expressed graphically (Dori, 2016).
TABLE 1
Table 1. List of Object–Process Methodology-related acronyms.
FIGURE 1
Figure 1. SD level OPD and its corresponding OPL of Chocolate Producing process.
Figure 1 presents the Chocolate Producing process which is part of the fourth module of the learning process. This OPD formed the background for the questionnaire of the fourth module, in which respondents were required, among other questions, to determine the kind of the missing link marked with the letter L. As in any OPD, rectangles represent system objects, ellipses represent processes, and the rounded-corner rectangles inside an object represent states, which are situations at which that object can be. The OPM default colors are blue for processes, green for objects, and gold for states, both in the OPD and OPL. The black words in the OPL are reserved phrases that unambiguously describe in text all the modeled facts. For example, the sentence at the bottom of the OPL paragraph reads “Chocolate Eating consumes Chocolate.” This is the unambiguous textual description of the model fact that is expressed graphically within the red rectangle: the object Chocolate linked to the process Chocolate Eating using a consumption link—the arrow from that object to the process.
Research participants
All pre- and in-service teachers were recruited at the Technion, Israel Institute of Technology. We used critical case sampling, which is a purposive sampling method, in which researchers aim to gain deeper understanding of the process being examined by selecting cases that are of special importance to the study. This guideline enabled obtaining a diverse set of critical cases within the sample (Ritchie et al., 2003).
The participants were both in- and pre-service teachers enrolled in the Technion’s Faculty of Education in Science and Technology. Some participants took part in the research to receive academic credits within a course or a research project, while others took part after receiving a personal request. After excluding those who did not complete most requirements, 12 teachers made up the initial sample, of which nine were women (Table 2), none of whom had previous experience with OPM. A small minority of the 12 participants had superficial and sporadic previous familiarity with basic system concepts, such as function, structure, behavior, and purpose.
TABLE 2
Table 2. The background of the 12 participants that made up the initial sample.
Case studies of five STEM teachers
Of the 12 teachers above, we chose five to form our collective case study, two of whom were men. While case studies in education typically include at most two or three cases, we included five cases to increase generalizability and the likelihood to gain diverse insights. We based our selection on differences in their performance during the study and their different backgrounds, as well as the similarities among them: all five were current or future STEM teachers who completed the online learning process we had developed. One of them (Eli) was a pre-service teacher, while the other four were in-service teachers. All five held a teaching certificate. As shown in Table 2, the five represented different age groups, academic backgrounds, teaching areas, seniority, and levels of teaching.
The online learning process
According to our previous research (Akiri et al., 2020; Peretz et al., 2023), most pre- and in-service STEM teachers lack basic systems thinking and conceptual modeling proficiency. These deficiencies have been noted also by Yoon et al. (2017) and Arnold and Wade (2017). Therefore, it was necessary to provide the teachers with the basics of systems thinking and conceptual modeling to enable them to create for their students assignments that involve these skills. To this end, we developed a food-related four-module online learning process in the form of an elective professional development course for STEM teachers. We chose food-related topics because they are interdisciplinary and complex (Jagustović et al., 2019) while also being highly relevant. The online learning was carried out asynchronously at a pace adapted to the limitations and constraints of each learner. Indeed, the time it took the teachers to complete the learning process ranged from a few days to a few weeks. We hypothesized that following the implementation of the learning process, the teachers would internalize instructional and assessment principles, and not only the content area we explicitly sought to develop, i.e., conceptual modeling and systems.
Besides establishing a basic knowledge of conceptual modeling and system concepts for the participating teachers, their average score of the responses to questions included in the modules provided us with an indication of the systems thinking level of each participant, as elaborated below. Although the learning process included conceptual modeling requirements, we did not use their scores to assess the conceptual modeling level. This is because the modeling requirements were highly structured, so they could not serve as a measure of the teachers’ ability to create model diagrams from scratch, as explained in the section “Developing Online Assignments” below.
Systems thinking
The learning process that preceded the assignment development was based on studying four modules whose content was chocolate production and cod fish value chain. Each module included a questionnaire and a quiz. The score for each questionnaire and quiz was calculated as the percentage of correct answers. Closed-ended questions were automatically scored by using Google Forms, while open questions were checked manually. The content of each module is presented in Table 3. A detailed description of the content included in the learning process appears in Akiri et al. (2020) and Peretz et al. (2023).
TABLE 3
Table 3. The content of the four modules that formed the learning process.
The principle that guided us in designing the process is a gradual increase in the difficulty level. The difficulty level gradually increased from one module to the next, as well as within each module. The questions’ complexity increased gradually, requiring more advanced thinking skills as the modeling principles and systems concepts gradually became more difficult to internalize. For example, while the first module focused on the introduction of OPM entities—processes, objects, and states, in the fourth module learners were exposed to synchronous versus asynchronous processes and their different refinement mechanisms into more detail levels. Previous research we conducted (Akiri et al., 2020; Peretz et al., 2023) suggests that the online modules, which cover cross-disciplinary processes, can serve as a foundation for developing and assessing students’ and teachers’ systems thinking.
Developing online assignments
In the assignments that the participants developed, they had to apply on several levels what they had learned earlier. The first level was modeling the content knowledge they had acquired, i.e., system and modeling concepts, which provided the common basis for all the ensuing assignments. The participants had to integrate into their self-developed assignment conceptual models that they had created. At the second level, the participants had to apply assessment knowledge while integrating a variety of thinking skills and visual representations. Thirdly, they had to integrate interdisciplinary topics, specifically sustainability and COVID-19, into the topic they had chosen to contextualize their assignment.
In analyzing the assignments, we focused on six attributes: (1) conceptual modeling, (2) systems thinking, (3) visual representations, (4) thinking skills, (5) sustainability and COVID-19, and (6) interdisciplinarity. While conceptual modeling and systems thinking were explicitly taught during the learning process, visual representations, thinking skills, and interdisciplinarity were supposed to be internalized implicitly, following the completion of four learning modules that included multiple visual representations, diverse thinking skills, and different disciplines. No explicit or implicit instructions were given in the learning modules on integrating sustainability and COVID-19 issues into the assignments except that this was a requirement for the assignment development. Since the learning process occurred during the pandemic peak period and while the energy crisis was frequently discussed in the news, we were interested in testing the ability of the participating teachers to integrate these issues into each teacher’s broader chosen context.
Table 4 presents the scoring rubric for the six analyzed attributes. As noted, the systems thinking score was based on the learning process rather than on the developed assignments. This is so because the systems thinking learning process was more extensive and included more aspects related to systems thinking than the other five attributes, whose scores were based on the assignments the teachers had developed.
TABLE 4
Table 4. Scoring rubric for the assignments developed by the teachers.
As recommended by Kaczynski et al. (2008), the scores for the six attributes are presented in the Results section as spider charts to enhance qualitative inquiry in instructional settings. This kind of chart generally considers six related attributes, each with a five-point scale, so we made a minor adaptation of the scale to be 1–3 instead of 1–5 to match our assignment assessment rubric (See Table 4).
Conceptual modeling
To assess the participants’ ability to create conceptual OPM models, we explicitly instructed them to include in their self-developed module a complete OPM model with two OPDs: SD (system diagram)—the top-level, abstract system view, and SD1—the first detail level, in which the main process from SD is zoomed into. We asked them to also supplement each OPD with its automatically generated textual OPL sentences, in a structured subset of English. The entire model had to be included even if not all of it was needed for the assignment they had developed. In some cases, the model itself was the answer to a modeling requirement in the assignment. The conceptual modeling score was calculated based on a modified version of the Systems Thinking Assessment Rubric—STAR, developed by Lavi et al. (2019), and presented in Table 5.
TABLE 5
Table 5. Modified Systems Thinking Assessment Rubric (STAR).
Trustworthiness
Following basic criteria for trustworthiness in qualitatively oriented research (Guba and Lincoln, 1982), we took some measures to establish the trustworthiness of our study’s findings. To triangulate our data and enhance the clarity and depth of our findings, we utilized an array of data sources, including questionnaires featuring both open- and closed-ended questions, conceptual models, self-developed assignments, and written reflections. To facilitate the reproduction of our research process by future researchers, even if their findings may vary, a comprehensive description of the research process is provided. All three authors took an active part in the research to minimize the researcher bias, each bringing their own unique perspective to the research. Lastly, to minimize the positive confirmation bias (Nickerson, 1998), we used a pre-registered research design that may reduce interpretation bias, as data collection occurs after the research plan has already been established.
Ethics
All participants had to agree to an informed consent form at the beginning of the learning process. Choosing the “disagree” option in the informed consent section ended the process so that it was not possible to answer any further questions, let alone continue developing an assignment. All the names presented in this article are pseudonyms, and no detail that could identify participants in any way was included. The research process was approved by the Technion’s Behavioral Sciences Research Ethics Committee, Approval #2020–165.
Results
In the first part of this section we present the five cases—assignments, each developed by one of the participants, a pre- or in-service STEM teacher. Each case is presented with segments from its developed assignment and OPM model, along with the respective analysis. The analysis is performed both quantitatively, through the modified systems thinking assessment rubric to assess the OPM model quality, and qualitatively, via a description of the development process. The second part of the Results section presents a collective analysis of the six predefined attributes for each participant, both through spider charts and a table explaining each scoring.
Five selected cases: Focusing on conceptual modeling and systems thinking
We start with the findings relating to systems thinking and conceptual modeling of each case separately. The five participant names below are pseudonyms.
Case 1—Anna: A novice biology teacher
Anna is a novice biology teacher with an academic background—a master’s degree in biotechnology and a teaching certificate in biology. She has 2 years of experience as an assistant teacher in an elementary school. It took her about 3 months to complete the learning process and about a month and a half to develop the assignment.
The topic she chose for her assignment was tofu production and its by-products. She explained that it was not a random choice of the first food production process she encountered online, but a result of personal experience: She makes and sells homemade tofu and okara—soy pulp, the insoluble phase of crushed soybeans that remains after filtering. Therefore, as she stated, she knows the process inside out and feels connected to it. She emphasized that she had chosen this topic because the assignment she was asked to develop seemed complex to her, and her commitment to the process and the final product called for choosing a relevant, familiar topic. She began her assignment by introducing tofu and its production: its geohistorical roots, its production method, its nutritional-chemical composition, and its virtues. She then focused on the different stages of tofu making from a food engineering perspective, starting from raw soybeans, through soy milk, to the pasteurized product ready for marketing. Figure 2 presents selected segments from her assignment. As can be seen in this figure, she included a variety of thinking skills and used diagrams alongside the text. However, the relevance and integration of sustainability and COVID-19 into the tofu context were insufficient.
FIGURE 2
Figure 2. Selected segments from Anna’s tofu production assignment.
The top-level OPD which Anna created focused on okara fermentation. In SD1 it is refined into four subprocesses, but not all of them are related to the okara fermentation. For example, the first subprocess, Cooking, precedes the fermentation process. Her model received a score of 6/10, with points deducted due to insufficient definition of the system’s purpose and little variety in link types. Table 6 provides the scoring of this OPM model with explanations according to each attribute included in the modified STAR.
TABLE 6
Table 6. Scores and explanations for the OPM model created by Anna.
In her reflection, Anna wrote that conceptual modeling helped her to better define boundaries when it comes to complex systems. She had no background in modeling or systems, so at the beginning of learning, she had difficulty finding herself given the unfamiliar territory of system concepts and system-related language. She pointed out that she had “no relevant background at all, […] it was difficult for me to focus on the unfamiliar concepts and unique terminology involved.” The OPM itself, she stated, was understandable as soon as she was aware of the background and terminology. The most difficult concept for her to understand was “defining the various levels and aspects of the system as reflected in the OPM model.” This issue was reflected in her model (Table 5) by not modeling the beneficiary, and by the somewhat vague definition of the benefit. Since human-made systems are usually designed to benefit one or more beneficiaries—the stakeholders who extract value and benefits from the system, the purpose cannot be described in OPM models and modeling in general without including both the system beneficiaries and the benefit it provides. She needed help a couple of times during the learning process in using OPCloud to create the model, and this was provided online by one of the authors.
Regarding the assignment Anna developed, she noted that it required a lot of knowledge of various kinds, and even though she is very knowledgeable about tofu and okara production, searching online for relevant material on the processes was necessary to model them appropriately. The learning process, from her experience, gave her “a clear framework for planning the learning, […] which in turn helped me even more to understand the system I chose for the assignment. The modeling allowed me to better plan and design my assignment and not only to create a better model.”
Case 2—Eli: Pre-service mathematics teacher
Eli is a pre-service mathematics teacher who is currently pursuing his mathematics and computer science bachelor’s degree combined with a mathematics teaching certification. It took him about 2 weeks to complete the learning process and about a month to develop his assignment. His assignment topic was gummy bears production, starting with the technological and food engineering aspects of the process: from mixed ingredients to soft candies ready for packing. Figure 3 presents selected segments from his assignment. The figure might explain why his assignment score was 18/18: Eli included diverse thinking skills—basic alongside advanced ones. He combined sustainability and the pandemic in an integrative and context-relevant way, relating to a variety of disciplines.
FIGURE 3
Figure 3. Selected segments from Eli’s gummy bears production assignment.
The main process he chose to model in the assignment is the Gummy Bears Production. Eli produced a high-quality model in all three major system aspects—function, structure, and behavior, as well as model clarity.
The layout of elements in his model diagrams were perfectly organized, without a single link crossing another and without spelling mistakes or incorrect phrasing in terms of OPM. His awareness of the model readers’ need to understand his model was higher than that of others, as expressed in his reflection regarding suggestions for improvements that the participants had:
“When we convince ourselves that we can think about components [of the system] and communicate them, it’s time to start modeling.” He scored 10/10 for his OPM model, demonstrating a high level of attention to detail, model readability, and accuracy, as well as proficiency with concepts and terms associated with the system. Table 7 provides the analysis of his OPM model.
TABLE 7
Table 7. Scores and explanations for the OPM model created by Eli.
Although Eli got a full score for his model, based on his experience, building OPM models “requires a lot of imagination and it’s hard to attain this skill in such a short time.” When asked how the learning process benefitted him, he wrote that he has “learned that any process can be described by a model, [and that] any process can be displayed by objects, sub-processes, and links. It will benefit me in the future when I try to learn or apply something new. Now that I know how to model any process, it will be easy to understand every subject.”
Eli’s assignment related well to his OPM model. He used multiple representations, disciplines, and thinking skills, and skillfully integrated relevant sustainability and COVID-19 issues. As a result of the learning process, he reflected that “the more processes I learned and the more ideas I was exposed to in the learning process, the more I was able to understand and develop the best assignment I could. […] The number of examples and the variety of questions gave [me] more possibilities both to understand things and later create things.” As for the relevance of the assignment, he stated that “the more the topic related to me, the more I enjoyed it and the more willing I was to take on the challenge.” He also emphasized the importance of sustainability and COVID-19 to the relevance of the assignment, saying: “I really enjoyed that there were things that concerned me and related to my world.”
Case 3—Danielle: Experienced economics and management and science teacher
Danielle has over 10 years of teaching experience, mostly with college students. She has a teaching certificate and also teaches science and technology in high school. She holds a PhD in economics and management, a field she teaches in colleges. It took her about 2 weeks to complete the learning process and about 3 weeks to develop her assignment. The food-related topic she chose was ice cream production, focusing almost exclusively on the engineering aspect. Selected sections of her assignment are shown in Figure 4, demonstrating that while she incorporated a variety of thinking skills, her assignment was at a low level. The model parts she used in her assignment were taken from her low-quality OPM model, which received a score of 4/10. For example, in the middle-upper area of Figure 4, a part of the model is missing. The disciplinary diversity was the lowest among the participants: only technology and environment were discussed.
FIGURE 4
Figure 4. Selected segments from Danielle’s ice cream production assignment.
Danielle had considerable difficulties during the learning process and in modeling as part of developing her assignment. She attended an online training delivered by one of the authors in the middle of the learning process, from which it emerged that she had specific difficulties internalizing the terms and ideas behind conceptual modeling and system concepts. Despite this training, no substantial improvement was evident from the modeling she performed during the learning process to the creation of the model for the assignment she developed. Indeed, she received a 4/10, the lowest score among the five participants. Her model lacked crucial parts: SD included only the main process without any linked objects, which was refined to SD1 with eight subprocesses, but not as a level by itself to provide abstract information about the system. Her SD1 was incomplete and inaccurate. The objects she included were only inanimate enablers, i.e., instruments, and human enablers, i.e., agents. As agent and instrument links indicate enablement rather than transformation, not a single subprocess in SD1 transformed an object. Object transformation is a key concept without which neither the purpose of a system can be described nor its behavior—the way the system changes over time. In her feedback, Danielle wrote that in developing her assignment “the main difficulties were adapting my way of thinking to the model,” and not, as might be expected, adapting the model to her way of thinking. She added that another difficulty in modeling was “transitions between levels of analysis, combined with the interface [i.e., OPCloud] that was not so user-friendly for me.” It is worth noting that the rest of the participants expressed a positive opinion about the user-friendliness of OPCloud, and after the struggles of using it at first, they generally found it convenient.
Apart from the OPM model quality, which was rather low, her assignment was monotonous from both interdisciplinary and instructional points of view. Beyond the engineering aspect of ice cream production, the assignment included only an economic aspect. She referred to the effect of COVID-19 on the consumption of ice cream but did not include it in the model. Due to the low quality of her model, we do not present its analysis as done with the other cases.
Case 4—Ron: Physics high school and science middle school teacher
Ron is a high school physics teacher and a middle school science teacher with over 5 years of experience. His academic background includes a bachelor’s degree in electrical engineering and a master’s degree in science education with a teaching certificate. Relative to other participants, developing the assignment took him a lot of time, about three and a half months after having completed the learning process, which took him about a month.
The assignment he developed was framed by sustainability in the context of using writing paper. As he explained, developing a complex assignment like this requires a topic that he is passionate about, not just one he is familiar with or even knowledgeable about. We approved his selection because the topic he chose is complex enough to form a basis for the OPM model and because it relates to sustainability, an aspect that had to be incorporated into the assignment. Segments from his assignment are shown in Figure 5, where Hebrew parts are translated in red. Particularly prominent in his assignment and evident in Figure 5 is the monotony of the variety of thinking skills: the potential student needs only a basic understanding level to complete the assignment. Consequently, he received the lowest score for the variety of thinking skills attribute (see Analysis of the developed assignments). Additionally, he was the only one who did not include any part of his OPM model in the assignment.
FIGURE 5
Figure 5. Selected segments from Ron’s paper recycling assignment. The left-bottom part was not included in the assignment, but it appears in an appendix along with his model.
The main process in SD of the OPM model Ron created included Paper Recycling, which is refined in SD1 into eight subprocesses. His model received 7/10, with three points deducted for not attaching the model’s OPL and not defining the beneficiary (Table 8). The model, both SD and SD1, was created in Hebrew, his mother tongue, rather than English, as in the other cases. He explained that he could have done it in English, but the attention to detail would not be the same for him in this case. Reflecting on the modeling side of the learning process, he stated: At first, I had the feeling that conceptual modeling is limiting because it puts every occurrence and data into a model of processes and objects. After practice and work, I reached a high level of thinking thanks to the organized structure of the modeling, and thus actually developed thinking processes at a very high level. For me, this is reflected in the instructional aspects of the assignment planning and not only in the modeling requirements [throughout the learning process].
TABLE 8
Table 8. Scores and explanations for the OPM model created by Ron.
Regarding the assignment development, he explained that it took him considerable time to understand how to include in the model educational aspects, not just technical ones. As he explained, the assignment he developed is part of a lesson plan on misconceptions about recycling and sustainability that he had been contemplating for several years. He emphasized the difficulty in translating the already crystalized educational idea into a learning unit based on modeling and systems concepts. Working with OPM, he added, “helped me understand what the appropriate goals for the lesson plan are for 10th graders, and how to apply them.” The main effect that the learning process had on the development of the assignment for him is the gradual increase in the level of difficulty, and accordingly, the construction of the system and modeling ideas that focus on the details first and eventually converge into a complete model.
Case 5—Romy: A novice technology college teacher
Romy holds a bachelor’s degree in industrial engineering and management and textile engineering. In addition, she has a PhD in science and technology education and a teaching certificate in computer science. Her teaching experience includes about 2 years as a teaching assistant and lecturer in teacher training and development programs, and about a year as a teacher at a technology college. It took her about 2 days, much faster than the other participants, to complete the learning process, and another month to develop her assignment. The assignment revolved around the bread making process, from making the dough to packaging the bread for distribution. It began with a video of the entire process, which was followed by a list of 13 steps it includes. Figure 6 shows selected sections of her assignment, strongly highlighting the variety of visual representations that she included in the assignment. The high level of the other attributes is also evident in the variety of thinking skills, the variety of aspects and disciplines, and a successful combination of sustainability and COVID-19 in the context chosen for the assignment.
FIGURE 6
Figure 6. Selected segments from Romy’s bread making assignment.
The main process of the OPM model she designed was Bread Baking and Packing, a model that clearly described each step included in the assignment. The scoring of her OPM model is explained in Table 9. She gained the biggest improvement in modeling from the level of the models she created during the learning process to the level of the model she created in the assignment she developed. She wrote that “The difficulties increased as the lessons continued. In the beginning, it was not difficult at all, but in the end, it was very difficult.” Unlike most of the participants, at the beginning of the learning process she did not encounter the difficulty that others encountered as they needed to acquire a new language and new concepts. However, she thought that “in using [and learning] the OPM, I think more practice is required, starting with simpler systems.” In addition, she “felt that more advanced concepts were missing, and with more learning time this would also be possible, in addition to imparting the basic concepts in a more sustainable way.”
TABLE 9
Table 9. Scores and explanations for the OPM model created by Romy.
Analysis of the developed assignments
Once participants achieved basic systems thinking and conceptual modeling skills, they could seemingly begin to develop an assignment based on system concepts and conceptual modeling. However, the level that was sufficient to create conceptual diagrams according to specified guidelines may not be sufficient to independently create a complete model from scratch.
We wanted to test their ability to integrate conceptual models and system concepts into context-based learning rather than teach isolated facts. We also wanted to test their ability to combine in their assignment different thinking skills, information representations, and aspects of the topic they chose. Following the rubric for the assignments presented in the Materials and Methods section, we start with presenting the assessment charts of each of the five assignment cases. A common and effective way to present a qualitative assessment of learning and skills is spider charts, also known as radar charts, as shown in Figure 7. Explanations for the scoring of the six attributes for each chart are presented in Table 10. The assignment development process of the five participants is described qualitatively in the previous section. In this section, we present quantitatively the weaknesses and strengths of each participant for each of the six attributes. At the same time, both in the spider charts (Figure 7) and in Table 10, the performance of all five participants is presented along each attribute. This section thus complements the previous one, expanding on insights derived from the results.
FIGURE 7
Figure 7. Spider charts of the five teacher assignments showing for each teacher the total score and scores of each of the six attributes.
TABLE 10
Table 10. Assignment scoring explanations for all five cases, six attributes for each.
Discussion
This study describes a new online learning process based on conceptual models, system basics, and food contexts, at the end of which participants are required to develop an assignment for their current or future students based on conceptual models in the spirit of the learning process they had gone through. Once the fundamentals have been acquired through the learning process, they had the needed content knowledge to develop their assignments. In analyzing the five assignment cases included in the study, we assessed the participants’ performance in two aspects: (1) conceptual modeling and systems thinking, and (2) integration of conceptual models, systems thinking, visual representations, and interdisciplinarity, as well as sustainability and COVID-19 topics into their assignments.
The study adopted the collective case study approach, combining qualitative and quantitative methods aimed to discover new insights. In the next sections, we respond to the two research questions. The first question deals with the way STEM teachers cope with online learning of conceptual modeling and systems thinking in the context of food production and sustainability. The second question focuses on these teachers’ online assignments that they developed for their students. This RQ is addressed in the last section of the Discussion: “Teachers’ conceptual modeling and system concepts knowledge while developing learning materials.”
Conceptual modeling and systems thinking competence of STEM teachers
Most of our participants with an educational background who performed the online learning process we had developed did not have a background in conceptual modeling and systems thinking. Regardless of their teaching experience, none of the five assignment case authors selected for this study had a relevant background, and therefore it would not be unreasonable to infer that the knowledge and skills with which the participants began to develop their assignment were acquired as a result of the learning process they had previously completed. During the learning process, participants gained varying levels of knowledge and understanding of conceptual modeling and systems thinking. For most of them, this was the first introduction to these concepts, as evidenced by their reflections.
Explicit teaching of systems thinking: Using ontological scaffolding
Talanquer (2019) presented a year-long systems thinking-oriented undergraduate general chemistry pilot course. As a result of the course, students were reportedly able to identify processes, interactions, and components in given systems, and also to produce related explanations and build arguments, but only after significant scaffolding and prodding. Our objective was to get educators acquainted with system fundamentals, such as function, structure, behavior, and purpose, by using conceptual modeling in OPM and using them to compose assignments for their current or future students. To this end, we provided the learners with an ontological framework that involved teaching system concepts and modeling them in context. This is in line with other studies, such as those of Verhoeff et al. (2008), Yoon (2008), Yoon et al. (2018), and Rates et al. (2022), who emphasized the importance of explicitly instructed ontological knowledge regarding system concepts and ideas. Engaging in conceptual modeling activities before and during the assignment development was the practical part of the participants’ learning process. This became possible after participants had acquired the ontological foundations of OPM that enabled them to start engaging in OPM-based conceptual modeling. We agree with Rates et al. (2022), who argued that complex systems can easily be misunderstood and misconceptualized when learners have wrong ontologies. As we saw, at the beginning of the learning process, teachers often lack basic systems-related ontological knowledge. This may significantly impede learning that is based on systems thinking, and even more so when such knowledge has to be combined with assessment knowledge, as was the case in our study. One participant, Danielle, who failed to achieve mastery of basic system concepts, experienced major difficulties throughout the entire process, especially in developing her assignment. Arnold and Wade (2017) also argued that while ontological scaffolding is indeed important (e.g., Jacobson et al., 2011; Rates et al., 2022), the inclusion of overly complex system terms and concepts may withhold their instillation among educators. Arnold and Wade (2017) called for a more approachable language for those lacking profound systems knowledge, one accessible outside the systems community. We agree with this because we believe that making systems thinking and conceptual modeling accessible calls for combining different approaches: primarily an explicit systems ontology (e.g., Rates et al., 2022), against the background of treating systems thinking as a cognitive skill like any other skill, which is not reserved only for the systems community (Chowdhury, 2023).
Systemic knowledge acquisition versus systemic knowledge application
Based on our findings, we view the conceptualization of Arnold and Wade (2017), gaining systemic insight vs. using systemic insight, as an appropriate description of the learning process we have developed and implemented. First, the participants acquired basic knowledge that allowed them to gain systemic understanding and communicate the knowledge acquired in systems science language. Then, as the scaffoldings were gradually removed, they could use this knowledge to create models on their own as a central artifact of their self-developed student assignments. As reflected in the last learning module score, most of the participants gained a sufficient level of modeling and proficiency in system concepts that enabled them to develop the assignments.
Modeling engineered systems in combination with natural phenomena
Only one assignment—paper recycling—involved a model that focused on an engineered system with environmental, natural objects undergoing a transformation—deforestation and energy consumption. The low level of integration of natural aspects into human-made engineered systems in the participants’ assignments despite the instruction to include a sustainability aspect in the assignment content calls for further research to understand why this posed a challenge and how such integration can be fostered.
Teachers’ conceptual modeling and system concepts knowledge while developing learning materials
Developing assignments based on conceptual models presented the participating teachers with a complex task. This complexity was reflected throughout the process in two main aspects. First, the need to create models from scratch to represent their system of choice required a significant leap from the learning process, where the modeling requirements were more structured. Second, the participants had to combine instructional knowledge, assessment knowledge, content knowledge, and systems and modeling principles into a coherent, thought-provoking, and cohesive assignment. The focus shifted from teachers acquiring systems thinking and modeling content knowledge in the learning process to practicing creativity in composing assignments that we strive to endow them with so they apply it in their educational work. Indeed, teachers often need to combine principles from different fields to reach quality learning materials. We thus join the recommendations for meaningful curricular integration of systems thinking and conceptual modeling in teacher training and professional development (e.g., Krell and Krüger, 2016; Rosenkränzer et al., 2017; Yoon et al., 2017). However, one aspect that has not been sufficiently researched and promoted is interdisciplinary domain-independent assessment knowledge-based teacher training and professional development. A number of researchers view assessment knowledge as a separate construct from PCK (e.g., Avargil et al., 2012; Tal et al., 2021). As a separate construct, assessment knowledge requires developed PCK as a prerequisite, so developing PCK may also directly or indirectly benefit assessment knowledge (Avargil et al., 2012). Therefore, assessment knowledge-focused training can better equip teachers with the necessary assessment knowledge and skills. Further research may clarify the differences between assessment knowledge-focused versus PCK-focused training in the context of complex systems and conceptual modeling. The learning modules we developed were designed to provide implicit guidelines to leverage instructional principles that make up both PCK and assessment knowledge. These guidelines can be used in follow-up studies.
Conclusion
The cases described and analyzed qualitatively and quantitatively in this paper indicate that the learning process presented in this study may significantly benefit teaching complex phenomena and processes in STEM disciplines. Teachers’ engagement in learning complex systems basics in conjunction with conceptual modeling can be facilitated by putting them in the context of real-life complex phenomena that are relevant to the learner. Based on the desired learning outcomes, teachers can apply systems thinking and modeling methods and techniques they had acquired in their professional development to teach system principles and explain complex phenomena. Adequate accessible modeling methods and tools, such as OPM on the OPCloud platform, are key to endowing teachers and later their students with systems and modeling concepts whose abstract nature is alleviated by grounding them to concrete visual models that are interpreted textually on the fly.
The rubric enabled tracking different levels of assessment knowledge through six different attributes, but additional quantitative research is needed to further validate this conclusion. This rubric can be used to assess a large number of participants in teacher training and professional development programs, not only for summative assessment but also for formative one, as argued by Panadero and Jonsson (2020). The rubric provides for monitoring learners’ progress during task performance to identify individual strengths and weaknesses. Formative assessment is of paramount importance in context-based learning professional development programs for teachers (Pilot and Bulte, 2006). The spider charts can also be applied as part of formative assessment for tracking the learning process, not just for summative assessment and not only for research purposes but also for educational assessment in practice (Kaczynski et al., 2008).
Limitations and further research
The participants’ perceptions, experiences, and contexts in the cases described in this article are described in detail, so other researchers can assess whether the conclusions arising from our findings can be applied to other circumstances, times, and frameworks. Yet, the low degree of generalizability that characterizes case studies calls for quantitative follow-up studies that would validate the findings on large samples.
The focus of this research was on human-made systems and processes, but science teachers are more into teaching natural systems and phenomena. This is especially true for biology, the field from which general systems theory emerged, as well as for chemistry and physics, where many complexities can be simplified by engaging in conceptual modeling. Being domain independent in its nature, OPM “opens system modelling to the entire scientific, commercial, and industrial community” (Dori, 2016, p. 376). That is, human-made and natural systems and phenomena alike can be modeled using OPM. Although OPM has been investigated in relation to natural sciences education (e.g., Lavi and Dori, 2019), it was almost exclusively for the purpose of assessing systems thinking and not for developing such thinking or for integrating it into educational content developed by the teachers themselves. The potential implications of this study may be extended through the inclusion of natural phenomena rather than or in addition to engineered systems in further studies.
Most of the participants in this study, as well as in previous studies that applied a learning process similar to the one described in this paper (e.g., Peretz et al., 2023), demonstrated the acquisition of adequate systems thinking and modeling foundations. However, more advanced concepts and ideas such as feedback loops and conditional links are very relevant to many topics that science teachers are expected to teach. Since the completion of the current research, we have developed three additional modules that have been added to the learning process, containing more advanced approaches to systems engineering with OPM, such as decision nodes, conditional links, feedback loops, and logical operators. Future research should examine how educators without a relevant background cope with similar but more advanced learning and assignment development processes. Including the advanced modules is expected to expose the full potential of this learning process, and consequently, improve teachers’ and students’ modeling and understanding of complex systems. Being a formal conceptual modeling methodology and language, OPM includes recognized, unambiguous definitions of key system principles and their modeling. Further research will be needed to determine if OPM can be used to develop advanced systems thinking among teachers, similar to the process that has been taking place in systems engineering courses at the Technion.
With the exception of one, the models created by the participants in their self-developed assignments were significantly more elaborate and extensive than those created during their learning process, where they did not get to choose the domain and context of their interest. Further quantitative research is needed to determine if this finding can be generalized, and if so, what is the strength of the relationship between the quality of the model and the relevance of the chosen topic.
Finally, a follow-up study on the participants who completed the learning and development processes at a future point in time will provide a longitudinal view of the extent to which systems thinking and conceptual modeling skills developed by the participants are retained over time.
Contributions
This study integrates systems thinking and conceptual modeling skills, sustainability, interdisciplinary thinking, instructional knowledge, and assessment knowledge into student assignments developed by educators. This unique combination has provided for monitoring and documenting difficulties, challenges, and opportunities that arose in the process, opening the door for possible implementation in STEM education beyond research. The theoretical knowledge gained through this research can help to better design teacher training and professional development programs to cater to the growing need for systems thinking and modeling skills as emerging 21st century skills that teachers need to acquire and impart to their students. Knowing what to focus on in the development of teachers’ knowledge and how to do it is a step towards a more competent education system that is ready for the changing, complex challenges we face in all areas of life. The study offers a methodological contribution by providing a basis for evaluating teachers’ assessment knowledge and skills using the six-attribute rubric, subject to further validation in follow-up studies. After being established as a reliable and valid tool, this rubric will allow measuring progress in professional development and teacher training programs of a cross-disciplinary nature with activities focused on systems thinking and conceptual modeling in diverse contexts. As reflected in some of the cases in the research, modeling and systems thinking not only helped in gaining content knowledge and skills for teaching science, but also in structuring and contemplating teaching planning. This is an interesting aspect that we did not expect before starting the research, and establishing it may provide important added value to our research and science education research in general.
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
Ethics statement
The studies involving human participants were reviewed and approved by the Behavioral Sciences Research Ethics Committee of the Technion—Israel Institute of Technology. Approval number 2020–165. The participants provided their online informed consent to participate in this study.
Author contributions
All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.
Funding
This research was funded partially by the EIT Food 2020 Project #20124—TRACOD—Model-based Tracking of Cod and Other Fish Value Chain for Consumer Confidence Boosting and Food Engineers Education, and by the Irwin and Joan Jacobs Graduate School, Technion, Israel Institute of Technology.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Footnotes
1. ^International Organization for Standardization.
References
Accreditation Board for Engineering and Technology, (2021). Criteria for accrediting engineering programs, 2020–2021 [WWW document]. Available at: https://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-programs-2020-2021/ [Accessed January 26, 2022].
Ackerman, D., and Perkins, D. N. (1989). “Integrating thinking across the curriculum and learning skills” in Interdisciplinary curriculum: Design and implementation. Association for Supervision and Curriculum Development. ed. H. H. Jacobs (Alexandria, VA: Association for Supervision and Curriculum Development), 77–96.
Akiri, E., Tal, M., Peretz, R., Dori, D., and Dori, Y. J. (2020). STEM graduate students’ systems thinking, modeling and scientific understanding-the case of food production. Appl. Sci. 10, 1–23. doi: 10.3390/app10217417
Amaral, L. A. N., and Ottino, J. M. (2004). Complex networks: augmenting the framework for the study of complex systems. Eur. Phys. J. B. 38, 147–162. doi: 10.1140/epjb/e2004-00110-5
Arnold, R. D., and Wade, J. P. (2015). A definition of systems thinking: a systems approach. Procedia Comput. Sci. 44, 669–678. doi: 10.1016/j.procs.2015.03.050
Arnold, R. D., and Wade, J. P. (2017). A complete set of systems thinking skills. INCOSE Int. Symp. 27, 1355–1370. doi: 10.1002/j.2334-5837.2017.00433.x
Assaraf, O. B. Z., and Orion, N. (2005). Development of system thinking skills in the context of earth system education. J. Res. Sci. Teach. 42, 518–560. doi: 10.1002/tea.20061
Avargil, S., Herscovitz, O., and Dori, Y. J. (2012). Teaching thinking skills in context-based learning: teachers’ challenges and assessment knowledge. J. Sci. Educ. Technol. 21, 207–225. doi: 10.1007/s10956-011-9302-7
Checkland, P. (2000). Soft systems methodology: a thirty year retrospective. Syst. Res. Behav. Sci. 17, S11–S58. doi: 10.1002/1099-1743(200011)17:1+<::aid-sres374>3.0.co;2-o
Chen, Y. C., Wilson, K., and Lin, H. S. (2019). Identifying the challenging characteristics of systems thinking encountered by undergraduate students in chemistry problem-solving of gas laws. Chem. Educ. Res. Pract. 20, 594–605. doi: 10.1039/c9rp00070d
Chowdhury, R. (2023). Holistic flexibility for deploying systems thinking as a cognitive skill. Syst. Pract. Action Res., 1–23. doi: 10.1007/s11213-022-09626-8
Dong, M., Lu, J., Wang, G., Zheng, X., and Kiritsis, D. (2022). “Model-based systems engineering papers analysis based on word cloud visualization.” in: 2022 IEEE International Systems Conference (SysCon). IEEE. pp. 1–7.
Dori, D. (1995). Object-process analysis: maintaining the balance between system structure and behaviour. J. Log. Comput. 5, 227–249. doi: 10.1093/logcom/5.2.227
Dori, D. (2016). Model-based systems engineering with OPM and SysML, Model-Based Systems Engineering with OPM and SysML. Springer.
Dori, D., Kohen, H., Jbara, A., Wengrowicz, N., Lavi, R., Levi Soskin, N., et al. (2019). OPCloud: An OPM integrated conceptual-executable modeling environment for industry 4.0, in: OPCloud for industry 4.0. pp. 243–271.
Dori, D., Sillitto, H., Griego, R. M., McKinney, D., Arnold, E. P., Godfrey, P., et al. (2020). System definition, system worldviews, and systemness characteristics. IEEE Syst. J. 14, 1538–1548. doi: 10.1109/JSYST.2019.2904116
Frodeman, R. (2013). Sustainable knowledge: A theory of interdisciplinarity. Hampshire, UK: Springer.
Gero, A., and Zach, E. (2014). High school programme in electro-optics: a case study on interdisciplinary learning and systems thinking. Int. J. Eng. Educ. 30, 1190–1199.
Gharajedaghi, J. (2011). Systems thinking, in: Systems thinking: Managing chaos and complexity: A platform for designing business architecture. Elsevier, London, UK.
Gilbert, J. K., Boulter, C., and Elmer, R. (2000). “Positioning models in science education and in design and technology education” in Developing models in science education. eds. J. K. Gilbert and C. J. Boulter (New York, NY, US: Springer Science+Business Media), 3–17.
Goldstone, R. L. (2006). The complex systems see-change in education. J. Learn. Sci. 15, 35–43. doi: 10.1207/s15327809jls1501_5
Guarino, N., Oberle, D., and Staab, S. (2009). “What is an ontology?” in Handbook on Ontologies. eds. S. Staab and R. Studer (Springer-Verlag Berlin Heidelberg).
Guba, E. G., and Lincoln, Y. S. (1982). Epistemological and methodological bases of naturalistic inquiry. Educ. Commun. Technol. 30, 233–252. doi: 10.1007/BF02765185
Hallström, J., and Schönborn, K. J. (2019). Models and modelling for authentic STEM education: reinforcing the argument. Int. J. STEM Educ. 6, 1–10. doi: 10.1186/s40594-019-0178-z
Harsaae, M. P., Østergaard, T., and Bang, A.-L. (2022). “Systems thinking and Interdisciplinarity in disciplinary design education”. In International Conference on Engineering and Product Design Education, London South Bank University. London, UK.
Hung, W. (2008). Enhancing systems-thinking skills with modelling. Br. J. Educ. Technol. 39, 1099–1120. doi: 10.1111/j.1467-8535.2007.00791.x
Jacobson, M. J., Kapur, M., So, H. J., and Lee, J. (2011). The ontologies of complexity and learning about complex systems. Instr. Sci. 39, 763–783. doi: 10.1007/s11251-010-9147-0
Jagustović, R., Zougmoré, R. B., Kessler, A., Ritsema, C. J., Keesstra, S., and Reynolds, M. (2019). Contribution of systems thinking and complex adaptive system attributes to sustainable food production: example from a climate-smart village. Agric. Syst. 171, 65–75. doi: 10.1016/j.agsy.2018.12.008
Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness. Cambridge, Massachusetts, USA: Harvard University Press.
Kaczynski, D., Wood, L., and Harding, A. (2008). Using radar charts with qualitative evaluation: techniques to assess change in blended learning. Act. Learn. High. Educ. 9, 23–41. doi: 10.1177/1469787407086743
Klaassen, R. G. (2018). Interdisciplinary education: a case study. Eur. J. Eng. Educ. 43, 842–859. doi: 10.1080/03043797.2018.1442417
Krell, M., and Krüger, D. (2016). Testing models: a key aspect to promote teaching activities related to models and modelling in biology lessons? J. Biol. Educ. 50, 160–173. doi: 10.1080/00219266.2015.1028570
Lavi, R., and Dori, Y. J. (2019). Systems thinking of pre- and in-service science and engineering teachers. Int. J. Sci. Educ. 41, 248–279. doi: 10.1080/09500693.2018.1548788
Lavi, R., Dori, Y. J., Wengrowicz, N., and Dori, D. (2019). Model-based systems thinking: assessing engineering student teams. IEEE Trans. Educ. 63, 39–47. doi: 10.1109/TE.2019.2948807
Mayer, R. E. (1989). Models for Understanding. Rev. Educ. Res. 59, 43–64. doi: 10.3102/00346543059001043
Merriam-Webster (n.d.). Model [WWW Document]. Merriam-Webster.com Dict. Available at: https://www.merriam-webster.com/dictionary/model [Accessed August 5, 2022].
NGSS Lead States (2013). Next generation science standards: For states, By States, Next Generation Science Standards: For States, By States. National Research Council
Nickerson, R. S. (1998). Confirmation bias: a ubiquitous phenomenon in many guises. Rev. Gen. Psychol. 2, 175–220. doi: 10.1037/1089-2680.2.2.175
Panadero, E., and Jonsson, A. (2020). A critical review of the arguments against the use of rubrics. Educ. Res. Rev. 30:100329. doi: 10.1016/j.edurev.2020.100329
Partnership for 21st Century Skills (2004). Learning for the 21st century: A report and MILE guide for 21st century skills.
Peretz, R., Tal, M., Akiri, E., Dori, D., and Dori, Y. J. (2023). Fostering engineering and science students’ and teachers’ systems thinking and conceptual modeling skills. Instr. Sci. 1–35. doi: 10.1007/s11251-023-09625-9
Pilot, A., and Bulte, A. (2006). Why do you “need to know”? Context-based education. Int. J. Sci. Educ. 28, 953–956. doi: 10.1080/09500690600702462
Rates, C. A., Mulvey, B. K., Chiu, J. L., and Stenger, K. (2022). Examining ontological and self-monitoring scaffolding to improve complex systems thinking with a participatory simulation. Instr. Sci. 50, 199–221. doi: 10.1007/s11251-021-09573-2
Riess, W., and Mischo, C. (2010). Promoting systems thinking through biology lessons. Int. J. Sci. Educ. 32, 705–725. doi: 10.1080/09500690902769946
Ritchie, J., Lewis, J., and Elam, G. (2003). Designing and selecting samples. Qual. Res. Method., 77–108.
Rosenkränzer, F., Hörsch, C., Schuler, S., and Riess, W. (2017). Student teachers’ pedagogical content knowledge for teaching systems thinking: effects of different interventions. Int. J. Sci. Educ. 39, 1932–1951. doi: 10.1080/09500693.2017.1362603
Rosenthal, K., Ternes, B., and Strecker, S. (2019). “Learning conceptual modeling: structuring overview, research themes and paths for future research” in ECIS 2019 (Sweden. Stockholm: Stockholm), 1–21.
Stave, K., and Hopper, M. (2007). What constitutes systems thinking? A proposed taxonomy, in: Proceedings of the 25th International Conference of the system dynamics society. Boston, MA.
Sterman, J. D. (1994). Learning in and about complex systems. Syst. Dyn. Rev. 10, 291–330. doi: 10.1002/sdr.4260100214
Tal, M., Herscovitz, O., and Dori, Y. J. (2021). Assessing teachers’ knowledge: incorporating context-based learning in chemistry. Chem. Educ. Res. Pract. 22, 1003–1019. doi: 10.1039/D0RP00359J
Talanquer, V. (2019). Some insights into assessing chemical systems thinking. J. Chem. Educ. 96, 2918–2925. doi: 10.1021/acs.jchemed.9b00218
Verhoeff, R. P., Knippels, M. C. P. J., Gilissen, M. G. R., and Boersma, K. T. (2018). The theoretical nature of systems thinking. Perspectives on S ystems thinking in biology education. Front. Educ. 3, 1–11. doi: 10.3389/feduc.2018.00040
Verhoeff, R. P., Waarlo, A. J., and Boersma, K. T. (2008). Systems modelling and the development of coherent understanding of cell biology. Int. J. Sci. Educ. 30, 543–568. doi: 10.1080/09500690701237780
Vivekanandan, R., and Pierre-Louis, M. (2020). 21st century skills: What potential role for the global Partnership for Education? A Landscape Review, Global Partnership for Education.
White, B. Y. (1984). Designing computer games to help physics students understand Newton’s Laws of motion. Cogn. Instr. 1, 69–108. doi: 10.1207/s1532690xci0101_4
Yoon, S. A. (2008). An evolutionary approach to harnessing complex systems thinking in the science and technology classroom. Int. J. Sci. Educ. 30, 1–32. doi: 10.1080/09500690601101672
Yoon, S. A., Anderson, E., Koehler-Yom, J., Evans, C., Park, M., Sheldon, J., et al. (2017). Teaching about complex systems is no simple matter: building effective professional development for computer-supported complex systems instruction. Instr. Sci. 45, 99–121. doi: 10.1007/s11251-016-9388-7
Keywords: STEM teachers, systems thinking, modeling, online assignments, rubric, Object-Process Methodology—OPM
Citation: Peretz R, Dori D and Dori YJ (2023) Developing and assessing pre- and in-service science and engineering teachers’ systems thinking and modeling skills through an asynchronous online course. Front. Educ. 8:1154893. doi: 10.3389/feduc.2023.1154893
Edited by:
Tom Bielik, Freie Universität Berlin, GermanyReviewed by:
Katherine Miller, University of Pennsylvania, United StatesJosé Cravino, University of Trás-os-Montes and Alto Douro, Portugal
Copyright © 2023 Peretz, Dori and Dori. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yehudit Judy Dori, eWpkb3JpQHRlY2huaW9uLmFjLmls