BeeLife: a mobile application to foster environmental awareness in classroom settings

Introduction: Significant threats to our environment tremendously affect biodiversity and related gains. Particularly wild bees actively contribute by pollinating plants and trees. Their increasing extinction comes with devastating consequences for nutrition and stability of our ecosystem. However, most people lack awareness about those species and their living conditions, preventing them to take on responsibility.

Methods: We introduce an intervention consisting of a mobile app and related project workshops that foster responsibility already at an early stage in life. Drawing on principles from multimedia learning and child-centered design, six gamified levels and accompanying nature-based activities sensitize for the importance of wild bees and their role for a stable and diverse ecosystem. A pilot evaluation across three schools, involving 44 children aged between 9 and 12, included a pre-, post-, and delayed post-test to inspect app usability and learning gains.

Results: Most children perceived the app as intuitive, engaging, and visually appealing, and sustainably benefited from our intervention in terms of retention performance. Teacher interviews following the intervention support the fit with the envisioned target group and the classroom setting.

Discussion: Taken together, the obtained evidence emphasizes the benefits of our intervention, even though our sample size was limited due to dropouts. Future extensions might include adaptive instructional design elements to increase observable learning gains.

Introduction

The impact of climate change and land use on our planet is becoming more and more evident and has a growing impact on our environment and its biodiversity. Intensification of land use is one of the main causes of insect decline (Seibold et al., 2019), especially in private property. The omnipresent concept of a neat and tidy structure reduces the tolerance of wild plants, which grow by themselves and create suitable pollen and nectar sources contributing to insect-friendly habitats. Likewise, naturally occurring nesting places such as dry plant stems, sand beds, old brick walls, or raw wood beams are cleaned up instead of being available for insects. The resulting loss of habitat appears to be a major cause of the decline of many insect species (Foley et al., 2005; Hodgson et al., 2011), including bees (Brown and Paxton, 2009). Bees have a vital part in the food chain, as, for example, 60% of all bird species depend on insects as a food source (Morse, 1971). Being the primary pollinators of most of the world's wild plant species (Winfree, 2010), both honeybees and their “wild sisters,” the wild bees, are contributing significantly to crop yields. In particular, the manifold wild bee species are highly effective pollinators of many crops (Richards, 1996; Heard, 1999; Javorek et al., 2002; Kremen et al., 2002). Consequently, their decline has dramatic consequences for food supply and the stability of our ecosystem. In Germany, for example, roughly 50% percent of all known wild bee species either face extinction or are already extinct (Westrich et al., 2011). Yet only two out of ~20,000 bee species are listed on the global IUCN Red List (Winfree, 2010), which classifies species at high risk of global extinction. Moreover, bee conservation usually talks about honeybees and neglects wild bee species (van Vierssen Trip et al., 2020).

To address the outlined challenges and make society aware of how their private gardening habits might affect our ecosystem at a broader level, we developed a mobile app that contributes to spreading knowledge about wild bees and the importance of counteracting their extinction, thereby laying the foundation for changes in attitude and behavior. Following established models of environmental competence formation (Kaiser et al., 2008), acquiring environmental knowledge can be considered as a prerequisite for subsequent engagement in environmental actions and related alteration of mindset. By embedding the use of the app in classroom settings in so-called project workshops, i.e., related activities in nature such as building a nesting aid together with the teacher and subsequently engaging in observation of residing species, our concept offers broad potential for directly bringing the acquired knowledge to action (Duerden and Witt, 2010). Additionally, each level in the app itself follows an action-oriented approach to strengthen a sense of responsibility in students and provide opportunities to reflect on consequences of their own behavior.

On purpose, we target a young age group – children of ages 9 to 12 – to build a sense of responsibility for our environment already at an early stage in life (Horwitz, 1996). Most apps that focus on environmental issues are developed for adults (Wirzberger et al., 2021) and even here, the topic of wild bees seems to be underrepresented. We developed an app that aims to fill this gap and educate both children and related adults such as parents or teachers about the important contributions of wild bees. To make the app appealing for children and accessible in classroom contexts, it is designed to run on smartphones and tablets. Furthermore, we draw on established design principles from multimedia learning research and methodological approaches from child-centered design to ensure a proper alignment of software features and user requirements.

We evaluated the app in a technical beta test and a conceptual pilot test in participating schools in order to investigate app usability and learning gains in our target group. The resulting evidence shows that users are thoroughly satisfied with the functionality and design of the app. Furthermore, we are able to demonstrate that students can retain what they learned over several weeks.

We consider our pilot results a promising indication that we created an innovative intervention with appeal to the dedicated targeted group by embedding nature experiences in a digital gamified framework. This allows us to impart knowledge about wild bees and lay the foundation for sustainable behavior early on. Emerging implications relate to extending the app with user-adaptive features to further improve sustainable learning gains.

Related work

Using mobile apps in classroom settings is becoming more and more prevalent (Zhang et al., 2015; Ninaus et al., 2017; Harrison and Lee, 2018; Amorim et al., 2023). Furthermore, the interest in utilizing tablets and other mobile devices has grown considerably with increasing technological advances over the last decades (Coovert et al., 2015; Algoufi, 2016). Studies have shown that providing students with devices such as tablets positively impacts their engagement with the taught subject (Henderson and Yeow, 2012). Additionally, tablets can have a positive influence on students' regulative strategies in learning contexts, as they provide easy access to information, support collaboration (McPhee et al., 2013), and hold potential to reduce the gender gap in learning (Henderson and Yeow, 2012). Ultimately, mobile learning for education offers many benefits (Criollo et al., 2021).

Principles of multimedia learning

Decades of research in multimedia learning have demonstrated the benefits of accounting for learners' limited cognitive resources and applying techniques that support the investment of these resources to relevant content (Sweller et al., 2019).

Characteristically, multimedia instructions leverage multiple channels of information by combining, for example, written text, spoken explanations, and static or dynamic images (Mayer, 2014a). Following the dual-channel assumption stated in the Cognitive Theory of Multimedia Learning (CTML) (Mayer, 2014b), available capacities in both the visual and auditory channel can be utilized. In addition, accompanying spoken content with written information avoids negative effects of transient information (Sweller et al., 2019; Castro-Alonso et al., 2021), which learners otherwise would need to keep active in working memory to ensure continued access.

Further benefits arise from segmenting information into small, well-structured learner-paced units (Rey et al., 2019). By giving learners the opportunity to control the speed of presentation, sufficient time to process the multimedia instruction is ensured.

Expanding the cognitive view on education, relevant evidence emphasizes the benefit of content eliciting positive emotions, both in terms of higher learning performance and reduced cognitive load (Schneider et al., 2018). One approach of incorporating positively connoted design elements relates to the principle of anthropomorphism. This involves the addition of human-like features, such as faces or personality traits to animals or object (Bartneck et al., 2008). Schneider et al. (2019) conclude that adding anthropomorphic features results in learning gains and increased intrinsic motivation. Such an approach further allows instructional designers to present information in an age-coherent way, i.e., choose a child-like appearance, language, and color-scheme of animated pedagogical agents when designing learning environments for younger children. Existing research also underlines benefits in terms of learning gains (Beege et al., 2017).

A closely related, powerful mechanism to foster learning gains – in particular for highly affine target groups such as children – is gamification (Landers, 2014; Armstrong and Landers, 2017), which refers to the embedding of game-related elements in non-game contexts. A meta-analysis by Sailer and Homner (2020) confirms that incorporating such design elements results in positive effects on cognitive, motivational as well as behavioral learning outcomes.

According to Salen and Zimmerman (2004), games consist of an artificial conflict, a formal rule system, and a quantifiable outcome. A casual game taxonomy has been outlined by Trefry (2010), which divides games according to their underlying mechanics into matching, sorting, seeking, managing, hitting, chaining, constructing, bouncing/tossing/rolling/stacking, and socializing. Characteristically, in casual games rules and goals must be clear and the concepts must link to familiar content and themes. Additionally, the player must be able to quickly reach proficiency in game play (Trefry, 2010). Regardless of the specific game mechanics, games are fundamentally about learning and often employ inherent reward structures that provide players feedback on their performance (Salen and Zimmerman, 2004). In educational contexts, feedback is considered to be crucial to improve knowledge. In her often-cited review, Shute (2008) describes formative feedback as information communicated to the learner with the intention to provide effective and efficient insights into individual learning processes. Formative feedback is usually provided immediately in response to a user's actions, and it can take many different forms. These range from simple verifications, if a given answer was correct or wrong, to more elaborate hints on underlying concepts, often in the form of questions that do not necessarily include the correct answer. Whereas, immediate feedback is helpful to rapidly foster acquiring new knowledge and skills, a delayed presentation is beneficial to strengthen long-term gains (Mason and Bruning, 2001).

Child centered computing

When tailoring an application to a specific target group, involving the relevant stakeholders in the development process is crucial. For developing software that targets children, Druin (2002) proposes a framework to ensure to adequately taking their opinions and abilities into account. Across the software development life cycle, children can take on different roles:

• User: Identify needs and challenges by observing how children use a technology.

• Tester: Offer children the possibility to interact with prototypes across design iterations.

• Informant: Gather opinions and ideas from children, directly involving them into the design process.

• Design partner: Include children in the design team with equal partnership in decision making.

This framework allows software developers to include and consider children's opinions and feedback at different essential stages in the development process. However, it is not only important to include them in development, but also to keep their capabilities in mind when inferring and implementing design choices.

Taking on a developmental psychology perspective, children's cognitive abilities are not yet developed to the level of adults. Piaget (1971a,b) introduces a model that divides children's cognitive development in four stages: Sensorimotor stage, preoperational stage, concrete operational stage, and formal operational stage. The latter two cover children approximately aged 6/7 to 11/12 years and 11/12+ years, respectively. In the concrete operational state children are capable of prelogical thinking (e.g., that quantity is maintained with change of appearance), and the hierarchical categorization of objects into groups and subgroups. The formal operational stage describes children as able to think abstractly and formally in symbols beyond objects in real world. Furthermore, they can engage in scientific thinking such as hypothesizing or logical reasoning.

The described stages of cognitive development and related cognitive abilities can be utilized to derive design recommendations to generate visually and content-wise appealing software applications that encourage playful explorations and interactions. Liebal and Exner (2011) provide such age-appropriate recommendations with regards to screen design, control and interaction, and content. Suggestions for screen design specify how colors and layout should be used and how both visual and auditory design elements should be embedded. Control and interaction deal with using input devices, beneficial interaction techniques, and how navigation, menu, and user support should be included in the software. Lastly, content includes the design of leading and supporting characters in styles typical of children's software, recommendations for language use, vocabulary, text volume, and the clear distinction of parent-specific content from child-specific content.

Sustainability and mobile applications

Considering the broad potential of digital learning opportunities in general and for younger target groups in particular, we find a gap in the current educational landscape. Existing evidence emphasizes the value of digital gamified approaches for sparking children's interest in topics related to environment, nature, and biodiversity (Wu and Lee, 2015). Compared to conventional classroom or outdoor settings, they can provide a consistent and scalable experience, allow complex simulations, or assign user control beyond real world level. However, we often do not find suitable offers particularly for a younger target group. The majority of mobile apps with a focus on environment and sustainability serve to convey information on, for instance, the origin and protection of food, regional shopping opportunities, ecologically valid ingredients, or energy consumption (Brauer et al., 2016). A comparative market analysis of such apps reveals that they mainly target adults (Wirzberger et al., 2021). For children and adolescents across age groups, there is usually a wide range of participatory nature-related opportunities, often in the form of leisure activities or environmental groups. Usually, these do not leverage the potential of digital learning environments. By closer inspection, the topics of insect mortality and habitat protection tend to be underrepresented in such contexts and are also rather lacking in biology curricula in schools (Wirzberger et al., 2021). This brings us to our main research question, how we can leverage insights from the outlined research areas to develop a mobile learning app that has the potential to promote environmental awareness in classroom settings.

Materials and methods

Software development

The first step in our user-centered software development process (Liebal and Exner, 2011) focused on gathering underlying requirements. We conducted focus group workshops and interviews with (prospective) teachers, environmental experts, and an interdisciplinary team of scientists. In this endeavor, we determined both the structure of the conveyed content as well as the design of the embedded bee characters. Initially, we had planned to directly involve children in these workshops as well, which should happen during in-person events at participating schools. Due to COVID-19 and the related challenges schools and families had to cope with, we could not follow our plans and decided to (1) conduct joint workshops with experts across all participating schools online, and (2) incorporate children's perspectives more indirectly by (prospective) teachers' didactic expertise.

The focus group discussions followed a user story mapping process and involved iterative brainstorming on challenges for wild bees and related activities to address these. During the entire process, emerging ideas were captured at a virtual interactive whiteboard, which served as base for a subsequent internal clustering of contents in our project team. Building on the obtained insights, we extracted problem definitions (e.g., lack of knowledge about the characteristics and habits of wild bees, lack of awareness for broader effects of own actions) and translated them into requirements the app should fulfill in order to be effective. These requirements related to, for instance, creating an emotional bond, giving wild bees a face, motivating students to engage with the topics, discovering the secrets of different wild bee species, or addressing parent's mindset via their children. We further categorized the collected fields of interest into a few core topics: biodiversity, nesting habits, enemies and adversaries, nutrition habits, benefits for humans, and behavior toward wild bees (see also Table 1). For each topic, we derived concrete activities in the form of mini games with the possibility to introduce connections to real-world activities. An example would be to build a nesting aid based on different materials, which further encourages students to build their own nesting aid in the school garden and observing the variety of residing species. To implement these mini games on a technical level, we had to break larger activities down into subtasks, for instance, learning about facilitative building material for a nesting place first, and applying these insights later when choosing appropriate material for building the nesting aid. Due to the broad variety of potential activities for each of the extracted core topics, we needed to prioritize tasks into high, medium, and low priority, yet ensure that each topic was adequately represented. The resulting story board served as base for software development and subject for iterative consultation within the project team and with the involved external experts.

Table 1

Table 1. Overview of topics currently covered in our intervention.

Character development

Developing the characters required an iterative exchange between prospective teachers, environmental experts, and a designer. In this interdisciplinary team, we selected a set of 11 wild bee species with particularly salient habits and characteristics, complemented by one honeybee. The prototype reported in this paper has an initial set of six of these characters implemented, whose names relate to the Latin name for the species family: The hornfaced mason bee Mia (Osmia cornuta), the patchwork leafcutter bee Chili (Megachile centuncularis) and her opponent the sharp-tailed bee Coelia (Coelioxis inermis), two male bee characters, the wool-carder bee Mani (Anthidium manicatum) and the large earth bumblebee Terris (Bombus terrestris), as well as the pantaloon bee Hirti (Dasypoda hirtipes), an oligolectic bee species. In the future, the app could be expanded to include more characters and thus bee species would represent a wider variety of bees. The current characters were picked to demonstrate the variety of existing wild bee species and their habits. As a particularly salient distinction, the characters represent different nutritional preferences: oligolectic, meaning that a species restricts themselves to the pollen of a particular plant family vs. polylectic, meaning pollen of many plant families are accepted. Furthermore, different living habitats and nesting behaviors are established.

All bees received additional human-inspired personality traits, ranging from being reserved over bubbly to mean or bossy. These traits were not selected at random, but they relate to biological characteristics of the chosen species, for instance, the sharp-tailed bee Coelia being sly in capturing foreign nests or the patchwork leaf-cutter bee Chili being creative in cutting out pieces of leaf. One of the characters, Coelia, the sharp-tailed bee, takes the position of the antagonist in the app. Another character, the hornfaced mason bee Mia, takes the main role and can be seen as the “hero” of the game. She helps the user navigate the world of the wild bees and her journey features themes like inclusion and friendship. Additionally, the characters are assigned genders (4 female and 2 male characters). Figure 1 shows the character of the hornfaced mason bee Mia, which has a child-like appearance and talks to the user with a high-pitched voice.

Figure 1

Figure 1. Providing elaborative feedback by means of an anthropomorphized character fosters personal connections between children and environmental topics.

Applying the outlined evidence on anthropomorphism in multimedia learning allows children to establish an emotional relationship with the characters (Schneider et al., 2019). It further enables alternative ways to convey information, e.g., by having dialogues between the characters. Following their initial introduction, the characters are present across different levels, providing opportunities to interact with them multiple times. Hence, the initially presented information about each wild bee species is activated in a sustainable manner.

Level design

Currently, the app consists of six levels – each having its own distinct focus – and a main screen. The levels do not only differ in terms of their objective but also regarding the aspect of wild bee life they present. Each level starts with a small animation that keeps the audience engaged and introduces both the involved characters as well as the upcoming challenges in the level. While the story works linearly and is supposed to be followed in order, the levels are available at all times, so teachers can skip levels in their lessons if necessary. Table 1 provides an overview of the scope and reward mechanisms of the included levels. As obvious, they are designed differently and apply the already introduced principles of multimedia and game design in different ways.

Each level focuses on a singular issue, supporting a well-structured presentation of information. The content of the levels is broken down into small, user-paced chunks to prevent cognitive overload and ensure that information is relayed in age-appropriate units and at the right speed. Furthermore, it is crucial to make sure that the user at all times knows what has already been covered (Liebal and Exner, 2011). This is done in two layers: At first, a grid displays all characters as silhouettes and reveals them by selection. During the character's introduction in the first level, a progress bar visualizes how many items (containing either a question or a piece of information) are ahead. Figure 2 showcases how the user is able to control the pace of the flow of information. A button loads the next text segment into the scene and starts a new animation sequence once it is pressed. This approach gives the user the opportunity to control the speed of presentation, ensuring that learners receive enough time to process the multimedia instruction. It further allows users to interact more with the app. To communicate each interaction as clear as possible, buttons have the same shape and appearance. While this example only demonstrates how we incorporated benefits of segmentation in the first level, other levels also leverage this principle in similar ways.

Figure 2

Figure 2. Multimedia instructions include written and spoken text and images to support character introductions. The progress bar at the bottom shows the segmented sequence of information (circle) and quiz elements (square).

As visible in Figure 2, information is conveyed by combining spoken and written text with static and dynamic images, realizing various facilitative design principles of multimedia instructions (Sweller et al., 2019; Castro-Alonso et al., 2021). In the first level, where all characters introduce themselves, a lot of information is conveyed. To support an optimal use of individual cognitive resources, we chose a multimodal presentation approach: When a character introduces themselves, a short text is displayed on the left, in addition to a matching human-voiced audio track. Additionally, each piece of information is visually supported by small animations that prevent the screen from becoming overly static for extended periods of time. In the example shown in Figure 2, Mia, the hornfaced mason bee flies to the right side next to the cherry tree to highlight the topic of the sentence.

User actions result in visual and auditory feedback, which is especially important when users have to answer a question – since a correct input yields a different result than a wrong one. Wrong answers also create immediate elaborative feedback, with a wild bee character providing both the correct answer as well as a related explanation. For instance, in the fifth level, displayed in Figure 3, the user needs to find differences between two images. The images show a picnic in a world with wild bees (left-hand side) and one without them (right-hand side) to highlight the impact of wild bees on our everyday lives. By clicking on a difference, the user is rewarded three points. Additionally, a wild bee character provides an explanation about the selected object. Such formative feedback is capable of improving the user's learning process significantly (Shute, 2008).

Figure 3

Figure 3. Information on the impact of wild bees on human's lives is conveyed in a playful way with a search image and related explanations by an anthropomorphized character.

Casual game mechanics were also considered in the level design of our app. These were of particular interest, because their high familiarity in the target audience matched the requirement of child-centered design. Each level aims to implement at least one of these principles. For example, the second level deals with building a nesting aid, therefore implementing the design principle of constructing, whereas the fifth level is a picture game, building on the mechanism of seeking. Subsequently, all level-inherent game elements differ from each other, which prevents the app from becoming monotonous, thereby discouraging users from playing. By implementing an overall score that adds up individual level rewards, the game mechanic of progression (Adams and Rollings, 2006) is incorporated as well.

In each level, the user can receive up to 30 points, depending on their performance. The overall sum of points achieved is displayed after each level, allowing users to monitor both their performance in the current level and their overall progress in the app. Due to the resulting summative feedback, users are encouraged to approach each level with the ambition to obtain as many points as possible (Ninaus et al., 2017). This sparks interest in interacting with the app as well as the desire to understand the provided contents in order to solve a level. Since the app is used in classroom settings, keeping track of how well each student is doing, provides a competitive element, which further motivates the users (Aleksic-Maslac et al., 2017; Arce and Valdivia, 2020) and promotes learning gains (Nebel et al., 2016).

Empirical evaluation

To evaluate both user acceptance and gains in environmental awareness, we applied an iterative approach and conducted both a beta test and a subsequent field evaluation. In a first step, we tested the app with a small sample of users at an outreach event and received feedback in direct conversations and via questionnaires. In a subsequent step, we conducted a field evaluation in school contexts to pilot the app's suitability for classroom use with a substantially larger sample size.

Beta test

The first public beta test happened during a public outreach event at the University of Stuttgart. A group of 24 participants across a wide age range – the majority in child-age, a few young and middle-aged adults – tested the app individually. Afterward, they completed a brief self-report questionnaire with ten statements on app usability and visual design, building on items of the meCUE 2.0 inventory (Thüring and Mahlke, 2007). This questionnaire was chosen because it is a widely used instrument in user experience research. As it has been developed for adults, we did also consider other tests like the Smileyometer. Those tests, however, have been developed for pre-school children, who are at a significantly younger age compared to the school-aged target group we envisioned. Thus, we decided to use the meCUE 2.0 and adapted it to our context of use.

For this adaptation, as a first step we slightly changed the wording to better fit with the situation, for instance, by replacing the word “product” with “app.” As a second step, some repetitive items and items that were irrelevant for our app where removed, for instance, statements like “I could not live without this product.” or “By using the product, I would be perceived differently,” which were clearly meant for products with a more permanent use. Hence, they were not applicable for our learning context, in which the app was only used for a short amount of time, for instance, as part of a project week or regular lessons during school hours. By such meaningful shortening, we aimed at avoiding that the questionnaire would be too demanding for young children.

Each statement was rated on a 7-point Likert scale, ranging from 1 (disagree completely) to 7 (agree completely). An overview of all used statements can be found in Table 2. To provide a more child-friendly rating, the last question of the meCUE 2.0 on “How do you experience the product as a whole?” was adopted with a slider in the shape of a stick figure. Here, participants could freely state their overall impression on the app by placing the stick figure between the extreme poles “very bad” and “very good” on a scale ranging from 1 to 10.

Table 2

Table 2. Mean and standard deviation of user opinions on app usability statements and the overall rating of the app.

The results, displayed in Table 2, indicate that participants were able to navigate our app and the included levels easily and intuitively. Overall, the app seemed to be well received and scored high in terms of usability. In more detail, participants mostly rated the app as easy to use, useful, and considered the required operations clear and understandable. The design was also well received. Only a minority described the app as tedious or annoying. Both the statements “The app relaxes me” and “I cannot wait to use the app again.” received high ratings but were rated slightly lower compared to the other statements. Taken together, behavioral observations hinted at deep levels of engagement and both children and parents reported high enthusiasm about using such an app as part of regular classroom education.

From the beta test, we concluded that the app had the potential to be well received by students in our indicated target group and that it was ready to be integrated into the classroom. To support this hypothesis, in the next step we conducted a field evaluation in some of our collaborating schools. The same app version was used in both test stages since the beta test was mainly focused on identifying technical and content-related obstacles for the subsequently planned field evaluations. With the app performing well during the beta test, no changes were required before testing it in the classroom.

Field evaluation

We distributed our app to four classes in three schools, one elementary school and two secondary schools, who participated over multiple weeks. A total of 44 students (55% female) in fourth and fifth grades (41 and 59%, respectively), aged between 9 and 12 years (M = 10.43, SD = 1.00) participated in all stages of the evaluation. A total 82% of them reported owning a smartphone – a surprising insight, given that about one third of them were only in fourth grade - and 43% used electronic devices for gaming at least once per day. Furthermore, 71% of the students indicated to spend time in nature several times per day. Even though a higher number of students participated in the pre-test survey, over the course of the study, substantial dropout occurred due to students' absence in lessons dedicated to completing the included surveys. Because of the longitudinal focus of the evaluation, students with missing data had to be excluded from further analyses, leaving a final sample of 44 students for the analyses.

The evaluation started with a pre-test survey. It was directly followed by the intervention, namely lessons in school that featured the app and related environmental activities in so-called project workshops. One and 3 weeks later, a post- and delayed post-test were conducted, respectively. Thus, the entire evaluation spanned a period of 3 weeks.

The main focus of inspection related to whether students could achieve learning gains related to knowledge about wild bees by using the app – a particular dimension of environmental awareness. We also investigated how the target group evaluated the their experience related to using the app. In addition, we inspected students' motivation (Ryan et al., 1983), affinity for technology (Karrer et al., 2009), their environmental attitudes (Wingerter, 2001), and their intentions to protect biodiversity (Leske, 2009). Both attitudes and behavioral intentions form additional dimensions of environmental awareness.

To evaluate learning gains in terms of knowledge, we designed ten multiple choice questions. They were supposed to capture knowledge about wild bees drawing on school didactic expertise and age-appropriate learning material. As these questions were specifically tailored to our app and related to the presented information on biodiversity, nesting aids, enemies and adversaries, nutrition, benefits for humans, and behavioral implications, we could not rely on existing standardized inventories. Five questions targeted retention performance, i.e., whether students could remember content directly presented in the app, whereas another five questions related to transfer performance, extending to broader ecological relations, and covering both plants and animals. Each multiple-choice question consisted of four answer options that were either correct or wrong. For each correctly selected or not selected answer option in retention questions, students received one point, and for each correctly selected or not selected answer option in transfer question, they received two points for reasons of complexity. Hence, students could achieve a maximum score of 20 points for retention performance and a maximum score of 40 points for transfer performance.

To assess app usability, we again used the ten statements from the beta test in the exact same manner.

Results

Quantitative analysis

Due to violations of the normality assumption in our dependent variables, non-parametric tests were applied to analyze learning gains. As described above, data was collected at three different time points. Thus, a Friedman test (Friedman, 1937) was applied to test for significant improvements in students' knowledge during pre-, post- and delayed post-test. Subsequent pairwise comparisons with Wilcoxon tests (Wilcoxon, 1992) allowed us to inspect an emerging overall significant difference in more detail. We applied such analyses to all dimensions of environmental awareness, i.e., environmental attitude, environmental knowledge, and intention for environmental protection. Subsequently, we only focus on environmental knowledge due to the absence of significant differences in the other dimensions. Here, we performed separate analyses for both retention and transfer performance.

Inspecting the results, we observed a significant improvement in retention performance [χ²(2) = 22.7, p < 0.001, W = 0.258]. Pairwise comparisons indicated significant differences between pre- and post-test (p < 0.001) as well as pre-test and delayed post-test (p < 0.001). Between post- and delayed post-test, no significant difference could be observed (p = 0.477). With scores of M = 10.34 (SD = 1.98) for the pre-test, M = 13.64 (SD = 3.40) for the post-test, and M = 13.14 (SD = 3.90) for the delayed post-test, on average students displayed a significant improvement due to participating in our intervention. The absence of a significant difference between students' retention performance in the post-test and delayed post-test suggests that the students retain what they have learned over a longer duration.

The Friedman test on transfer performance did not show significant differences [χ²(2) = 0.259, p = 0.879, W = 0.003], with scores of M = 25.46 (SD = 5.31) for the pre-test, M = 25.18 (SD = 6.00) for the post-test, and M = 24.77 (SD = 7.05) for the delayed post-test. This indicates that while our intervention is helpful in building knowledge about the topics addressed in the app, transferring these insights to other parts of wildlife did not improve. Both knowledge-related scores are visualized in Figure 4.

Figure 4

Figure 4. Retention (dashed line) and transfer (dotted line) scores for pre-, post-, and delayed post-test. Error bars represent standard deviations.

Taking a closer look at potentially influencing factors for the observed knowledge gains, we found that the improvement in retention performance significantly correlated with self-reported motivation related to the facet of interest (r = 0.40, p = 0.008) and the rating sum score across the statements on app usability (r = 0.64, p < 0.001).

We also found encouraging results related to app usability in this larger sample. Figure 5 displays ratings of each statement, which are similar to the results we obtained in the beta test. Most participants consider our app easy to use (M= 6.43, SD = 1.21) and intuitive, as it is quickly clear to them, how to operate the app (M = 6.21, SD = 1.21), and they think the operation is understandable (M = 6.30, SD = 0.98). Furthermore, the app is considered useful (M = 5.91, SD = 1.34). Regarding the app's visual appearance, many students found it to be creatively designed (M= 5.80, SD= 1.73) and were attracted by the chosen layout (M= 5.55, SD= 1.66).

Figure 5

Figure 5. Scores on app usability, ranging from disagree completely (1) to agree completely (7). Vertical lines within bars indicate median values, bar width represents the range of response values for each item. Error bars indicate variance, while black dots represent outliers.

Both statements relating to whether the app is considered unpleasant received lower ratings. A majority of students stated that the app does not make them tired (M = 3.09, SD = 2.37) and is not annoying to them (M = 2.72, SD = 2.06). However, ratings on the latter statements display an increased variance toward the extremes, meaning that a considerable number of students find the app tedious or annoying.

Likewise, ratings on relaxing effects of app use also scored lower (M= 3.50, SD= 2.03), which indicates a more activating perception of many students. Again, we observe an increased variance in students' ratings. Finally, a decent number of students cannot wait to use the app again (M= 4.73, SD= 2.07), yet we see substantial variance in these ratings as well.

Qualitative analysis

After the end of the intervention, we conducted semi-structured interviews with one teacher per participating class, adding up to four interviews in total. In these interviews, we asked for details about the participating classes (i.e., group size, estimated prior knowledge and digital competences of students, frequency of use of gamified elements and digital media in class prior to our intervention), details about the teachers (i.e., estimated own prior knowledge about wild bees, amount of preparation time for the project), use of the provided materials (i.e., which levels of the app where played, which workshops where conducted, how many hours were used for that), suitability of our materials for the class level, problems, general feedback, and suggestions for improvement.

The interviews confirmed students' low level of prior knowledge about wild bees and the lack of coverage of the topic in class before, as all four involved teachers reported that the topic had not been part of their lessons yet and students had either low prior knowledge or none at all. Prior knowledge of the teachers themselves was also rather low. Two of them reported about previous experience and in-depth knowledge about honeybees specifically but did not feel confident about their knowledge regarding wild bees. The other two teachers reported to be completely new to the topic. In class, the app was used for 2–4 lessons - not taking disproportionate space in the lesson plan. Furthermore, teachers rated the length of the app appropriate, and even considered it still feasible with partially enhanced contents. Included levels were evaluated as appropriate for fourth and fifth grades, with potential extensions to both late third and early sixth grades. This hints on the fact that the app addresses our envisioned target group quite well. The interviews also showed that digital media, such as tablets and computers, are indeed available in schools but are rarely used. When they are used, they tend to be invested in online research. Nevertheless, according to the reports from the teachers, the children were proficient in using tablets, allowing them to operate the app with ease. This is also reflected in the user experience scores. Teachers further reported a few minor technical problems, such as the mixing of the music or the speed of a game character, which in summary did not crucially impair the evaluation process. A non-technical issue appears to be one of the characters, the sharp-tailed bee coelia, which some students described as “creepy.” When asked about the app's benefits, teachers gave a broad range of answers, with character design being the most prominent one. Additionally, interviewees emphasized the motivating effect in lessons, which happened later in the day. Compared to previous lessons in that slot, children were more enthusiastic and engaged when participating in our intervention.

As part of data collection during the intervention, we further asked students about their opinion and impressions via an open text field in the second and third questionnaire. Here, students were prompted to note what they liked, what they did not like, and what else they wanted to say about the app and related activities in class. Most of the feedback we received was positive, with students complimenting, for example, “the creative design,” “the music,” “the bees,” “everything,” “the questions,” “objective and intelligible presentation,” “talking animals,” or “having fun and learning at the same time.” Some students reported small technical problems, such as slow reaction times and erroneous calculation of reward points. Five students stated overall dislike; however, such feedback is difficult to interpret without giving specific reasons. Taken together, students seemed to like the app, with some of them explicitly requesting the app to be extended with additional levels.

Discussion

In this paper, we present a mobile app that teaches students at a young age about wild bees, their impact on the environment, and the threats they are facing. Using the topic of wild bees as an example, our work supports the development of environmental awareness early on.

Our app utilizes the advantages of modern technologies in the classroom and conveys knowledge in a sustainable way while creating an enjoyable experience for the students. It not only seeks to incorporate the advantages of gamification, immediate feedback and multimedia-based learning but combines it with environmental activities that complement the contents of the app.

Due to the interdisciplinary nature of the app, knowledge from many different professions has been embedded into the project. Therefore, experts on biodiversity and species protection as well as teachers were involved in the development process via regular exchanges. This allowed us to incorporate a wide range of expertise on the subject such as wild bees, didactic methods, or user experience, and combine these perspectives in one project.

A beta test at a public outreach event and a pilot evaluation in classroom settings hinted on a positive reception in the target group. Results further indicate that students successfully retained knowledge they acquired from the app, also over a longer period of 3 weeks. However, this did not apply to transfer performance. Hence, even though our intervention was successful in imparting knowledge linked to presented contents, students cannot transfer the obtained knowledge to other areas of wildlife. There is also the question of whether the app itself improved knowledge or whether the emerging discussions in the class triggered by the activities in the app and/or the project workshops fostered this. It could be a promising direction to look at in future research to enhance our intervention in a more fine-grained way.

With the app successfully motivating students as well as improving their knowledge on the topic, the question arises how and if its usage can be integrated in regular school curricula. Teacher interviews showed that they were enthusiastic about the app and appreciated the use of learning apps in their classes. The curricula for fifth and sixth grades also cover the topic of honeybees, allowing our approach to be included easily.

While some teachers and parents were hesitant for reasons of data security, we could clarify that the app does not record or store any personal data. Ultimately, the biggest hurdle during evaluation was the lack of tablets at some of the schools we worked with. Additionally, the rules about who can install apps on the school's tablets highly differ among participating schools. In one case, teachers had access to tablets but did not have permissions to install apps. These bureaucratic hurdles and lack of equipment will supposedly increase difficulties for schools or teachers interested in using our approach in their classrooms.

Implications

The innovative nature of the app for integrating gamified digital media into the classroom has proven to be well received by participating students. As mentioned before, students' proficiency in smartphone use and gaming was quite high, providing the ground for an effective game-based intervention. Tying in with established evidence from multimedia learning research (e.g., Nebel et al., 2016; Sailer and Homner, 2020), gamified approaches not only result in increased learning gains but also an increased motivation to participate in the learning experience. Our findings indeed hint in this direction as well and show effects of motivation on learning gains. Due to the apparent potential of our app for providing an engaging and effective learning experience, we conclude the benefit of a more frequent use of gamified learning approaches on mobile devices in classroom contexts.

The collaboration and constant exchange with involved experts and stakeholders resulted in a successful development process. Even though the related coordination with multiple busy schedules presented some challenges, this approach allowed us to meet expectations and identify missing content at an early stage. Especially the inclusion of teachers and students is key in co-designing likable characters, user-friendly interaction modes and optimizing the flow of information.

In particular, the fact that a few children described the sharp-tailed bee Coelia as “scary” highlights the importance of such an approach. Hence, for embedding technological advances in the actual educational practice, we propose applying a maximally stakeholder-inclusive approach to form an essential prerequisite, since it fosters a more goal-oriented development of educational apps.

As a crucial outcome of including teachers in the design process, the short but very engaging format of our intervention has the potential to promote sustainable changes in classroom environments. Hence, due to the condensed duration of the individual levels, embedding the app in regular lesson formats can be achieved without additional effort. As a result, we conclude that apps that are included in school lectures should present the content in units which take up only a small portion of the class time. On a general notion, such concepts might be applicable to other content outside biology and therefore can offer the possibility to bring more variability into school lessons by different teaching formats.

Taken together, Table 3 summarizes the most important design implications presented in this section, which should be considered in educational technology development to enhance students' learning experience.

Table 3

Table 3. Essential “Do's” in design for education.

Limitations

Even though our intervention was positively received by the target group and resulted in learning gains, results must partially be taken with a grain of salt. During data collection, several factors might have exerted influence. First, teachers involved in our project were highly interested in the topic and motivated to integrate the app into their lessons. In several cases, schools had reached out to us proactively, hence, we face a self-selection bias. Resulting benefits in superior lesson structure might have impacted learning gains of participating students beyond using our app. Consequently, learning might be less successful if the app is distributed at a broader scale and used in class by teachers who are neutral or negative toward the app. In general, teachers' support for using the app might also have had an impact on children's perceptions of app usability. If students had any questions, they could reach out to the teachers and – in the beginning of the evaluation – also the project staff at any time, hence, use-related issues could be resolved more quickly compared to regular unsupported use.

Additionally, we also experienced a quite high dropout rate. Of the 76 students that participated in the initial survey, only 44 complete data sets could be considered for the quantitative analysis for reasons of completeness, resulting in a dropout rate of 42.1%. Looking into potential causes, we observed that some students were not present across all lessons where surveys were conducted, hence, complete data sets were not available for these students. Furthermore, some students did not complete the entire questionnaires, and, in consequence, these data sets could not be included either.

Further, there might have been external influences on the results of the study, which we could not anticipate or capture in full extent. For example, the evaluation was conducted at the end of the school year. Hence, it could have been possible that some students were not motivated sufficiently to participate but already looking forward to their vacations. Additionally, the extremely warm temperatures common at this time of year could have decreased students' motivation and attention, directly impacting the results of the questionnaire.

Inspecting students' environmental attitude and behavioral intentions to protect biodiversity, we did not find significant changes over the course of the evaluation. On the one hand, these results might have emerged partly due to known shortcomings of self-report inventories as well as the short-term focus of inspection. In particular, sustainable effects on protective behavior can only be observed in the long run. On the other hand, our sample already reported high levels of nature-related engagement in the pre-test. Hence, not much potential for significant increase remained.

The topics discussed so far concern the study and resulting credibility of the conclusions. In addition, the app itself faces limitations. Even though it was well received by the target group and resulted in learning gains, not all aspects of environmental education could be covered to full extent. An inherent shortcoming of digital media in general relates to the fact that game experience does not completely resemble real nature experience. Observing wild bees in their natural habitats guided by environmental expertise offers a more diverse immersion compared to the same observation provided by an app. Hence, as part of our entire intervention, we worked with participating teachers to deepen the content presented in the app. For this purpose, we created workshops where students could build their own nesting aids or experience nature of different species. Proceeding further down that route, a contribution to developing biological awareness could result from the characters in the app themselves, by actively encouraging users to go out into nature and hereby acting as a guide to interpreting nature experiences. This might help users to realize the impact of their own actions on wild bees and what these actions imply and facilitate the transfer of what they have learned to the real world.

Fully realizing the advantages of a child-centered approach would also require children to be involved to a greater extent in different roles, as they currently are mainly involved as testers of the technology (Druin, 2002; Liebal and Exner, 2011). Including children in all stages of the development process further highlights downfalls with accessibility and additional unknown factors early on. Similarly, the potential of teachers' perspectives for software development was not fully taken advantage of, partly also due to high coordination efforts resulting from teachers' busy schedules. Iterating on the prototype in a more systematic way would broaden the scope of involvement in future steps.

Future work

Regarding future extensions of our app, several directions could prove fruitful. By using machine learning (ML) techniques, parameters related to content, learning speed or feedback might be adapted to users' needs. New levels could feature ML-simulated characters that react to changes in a player-controlled environment. This might foster a deeper understanding of the relationship between wild bees and changes in their environment caused by humans. ML-assisted simulations in general are effective to demonstrate how food nets work and how complex and delicate our environment is – and what measures can be taken to strengthen it and increase its resilience for the challenges ahead. Furthermore, creative levels, prompting users to draw wild bees with ML-assistance, result in opportunities for users to tell their own wild bee story – and therefore strengthen the personal relationship. Simple speech-to-text features could enable users to have their wild bee character talking and customized animations could give the newly created wild bee characters more depth. This story-telling approach supports users in developing a deeper sense of responsibility for wild bees.

Additionally, strengthening both collaborative and competitive elements that the app offers could be a promising way to increase users' motivation. Building on evidence from multimedia learning research (Nebel et al., 2016), each session in class could create a leaderboard where students' performance is visualized in relation to their peers. Furthermore, those boards could add up to a class grade, which then could be compared to other classes in the same school. Classrooms could also be divided into groups that compete against each other. Specific badges could be used to highlight users that reach the maximum number of points, make few or no mistakes, or group efforts, for instance, a badge awarded to the group when all members have reached a certain number of points. Deepening and refining such gamified components also constitutes a promising direction for future research. One example would be to use the points obtained in the levels as a currency, which enables our users to build their own virtual garden. The garden then is inhabited by wild bee characters and slowly fills with life as the user progresses through the app.

A further direction could involve using geocaching or location-based gaming to connect the content of the app with actual wild bee habitats or wildflower meadows. Challenges such as “Take a picture of three flowers that this wild bee really likes.” could strengthen the connection between real and virtual worlds. Location-based gaming might further open up ways to connect citizen-science methods with the current approach. With interested users regularly checking how many different species are still living at certain locations, protective measures could be taken in a more meaningful manner. Those tasks could be introduced as daily or weekly “quests,” which would deploy yet another game-inspired mechanism. This promising approach would add activities parallel to the main story and would also allow us to incorporate novel activities after a level is finished.

Conclusion

In conclusion, we present an innovative educational technology that consequently builds on design principles from multimedia learning research to unlock a widely neglected environmental topic for a young target group in a formal educational setting. The presented work is well in line with Konrad Lorenz' well-known quote, related to nature we only love what we know, and only protect what we love.

By climbing on his shoulders, we introduce a gamified approach that supports children to find this love for very specific creatures in danger – the “wild sisters” of our commonly known honeybees. Featuring a mobile app for classroom use that we created in an interdisciplinary design process with multiple stakeholders, first evidence indeed hints in a promising direction. Summarizing our obtained results, we created an engaging and appealing learning experience with broad potential for building sustainable knowledge. Extending the existing perspective beyond schools and the currently included topical focus even paves the way for broader societal changes in environmental awareness.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found at: https://osf.io/eqzph/.

Ethics statement

The study and plan for data collection was approved by the Committee on Responsibility in Research (Ethics Committee) at the University of Stuttgart (Az. 22-011). The reported research was conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was provided by the participants' legal guardians.

Author contributions

AS: Formal analysis, Investigation, Software, Visualization, Writing—original draft, Writing—review & editing. OS: Formal analysis, Investigation, Software, Visualization, Writing—original draft, Writing—review & editing. JM: Investigation, Methodology, Validation, Writing—review & editing. MS: Formal analysis, Investigation, Methodology, Writing—original draft, Writing—review & editing. NK: Data curation, Formal analysis, Visualization, Writing—review & editing. GR: Conceptualization, Funding acquisition, Methodology, Resources, Supervision, Writing—review & editing. MW: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing—original draft, Writing—review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. The authors gratefully acknowledge financial support by the German Environmental Foundation (DBU; grant number: 35164/01-43/2) and the University of Stuttgart for conducting the reported research. This publication was funded by the German Research Foundation (DFG) grant “Open Access Publication Funding/2023-2024/University of Stuttgart” (512689491).

Acknowledgments

We thank Judith Baatz, Ankita Ghorpade, and Felix Krieglstein for their support in character development, user story design, project workshop ideas, and project management during former stages of the research process. We further thank our environmental experts Undine Westphal and Frank Geggus for their helpful advice on biodiversity, species habits, and project workshop activities, and our design expert Marina González for the visual design of the characters and level environments. In addition, we thank all participating teachers and students at our collaborating schools in Gemünden am Main, Neckartenzlingen, and Nürtingen, as well as all participants during the beta test at the University of Stuttgart Science Day 2022.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Adams, E., and Rollings, A. (2006). Fundamentals of Game Design (Game Design and Development Series). Upper Saddle River, NJ: Pearson Prentice Hall, Inc.

Google Scholar

Aleksic-Maslac, K., Sinkovic, B., and Vranesic, P. (2017). Influence of gamification on student engagement in education. Int. J. Educ. Learn. Syst. 2, 8. doi: 10.23919/MIPRO.2018.8400145

Crossref Full Text | Google Scholar

Algoufi, R. (2016). Using tablet on education. World J. Educ. 6, 113–119. doi: 10.5430/wje.v6n3p113

Crossref Full Text | Google Scholar

Amorim, A. N., Jeon, L., Abel, Y., Pape, S., Albuquerque, E. X., Soares, M., et al. (2023). Exploring the use of Escribo Play mobile learning games to foster early mathematics for low-income first-grade children. Comp. Educ. 199, 104759. doi: 10.1016/j.compedu.2023.104759

Crossref Full Text | Google Scholar

Arce, N. H., and Valdivia, A. C. (2020). Adapting competitiveness and gamification to a digital platform for foreign language learning. Int. J. Emerg. Tech. Learn. 15, 194–209. doi: 10.3991/ijet.v15i20.16135

Crossref Full Text | Google Scholar

Armstrong, M. B., and Landers, R. N. (2017). An evaluation of gamified training: using narrative to improve reactions and learning. Simul. Gaming 48, 513–538. doi: 10.1177/1046878117703749

Crossref Full Text | Google Scholar

Bartneck, C., Croft, E., and Kulic, D. (2008). “Measuring the anthropomorphism, animacy, likeability, perceived intelligence and perceived safety of robots,” in Proceedings of the Metrics for Human-Robot Interaction Workshop in affiliation with the 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI 2008). Hertfordshire: University of Hertfordshire, 37–44.

Google Scholar

Beege, M., Schneider, S., Nebel, S., Mittangk, J., and Rey, G. D. (2017). Ageism – age coherence within learning material fosters learning. Comput. Hum. Behav. 75, 510–519. doi: 10.1016/j.chb.2017.05.042

Crossref Full Text | Google Scholar

Brauer, B., Ebermann, C., Hildebrandt, B., Reman,é, G., and Kolbe, L. M. (2016). “Green by app: the contribution of mobile applications to environmental sustainability,” in Proceedings of the 20th Pacific Asia Conference on Information Systems (PACIS 2016). Chiayi: Association for Information Systems.

Google Scholar

Brown, M. J. F., and Paxton, R. J. (2009). The conservation of bees: a global perspective. Apidologie 40, 410–416. doi: 10.1051/apido/2009019

Crossref Full Text | Google Scholar

Castro-Alonso, J. C., de Koning, B. B., Fiorella, L., and Paas, F. (2021). Five strategies for optimizing instructional materials: instructor- and learner-managed cognitive load. Educ. Psychol. Rev. 33, 1379–1407. doi: 10.1007/s10648-021-09606-9

PubMed Abstract | Crossref Full Text | Google Scholar

Coovert, S. A., Howard, D. J., Coovert, M. D., and Nelson, R. M. (2015). An evaluation of medical residents utilization of tablet computers. Comp. Hum. Behav. 53, 289–293. doi: 10.1016/j.chb.2015.07.021

Crossref Full Text | Google Scholar

Criollo, C. S., Guerrero-Arias, A., Jaramillo-Alcázar, Á., and Luján-Mora, S. (2021). Mobile learning technologies for education: benefits and pending issues. Appl. Sci. 11, 4111. doi: 10.3390/app11094111

Crossref Full Text | Google Scholar

Druin, A. (2002). The role of children in the design of new technology. Behav. Inform. Tech. 21, 1–25. doi: 10.1080/01449290110108659

Crossref Full Text | Google Scholar

Duerden, M. D., and Witt, P. A. (2010). The impact of direct and indirect experiences on the development of environmental knowledge, attitudes, and behavior. J. Env. Psy. 30, 379–392. doi: 10.1016/j.jenvp.2010.03.007

Crossref Full Text | Google Scholar

Foley, J. A., De Fries, D., Asner, R., Barford, G. P., Bonan, C., Carpenter, G., et al. (2005). Global consequences of land use. Science 309, 570–574. doi: 10.1126/science.1111772

PubMed Abstract | Crossref Full Text | Google Scholar

Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Amer. Statist. Assoc. 32, 675–701. doi: 10.1080/01621459.1937.10503522

Crossref Full Text | Google Scholar

Harrison, T. R., and Lee, H. S. (2018). iPads in the mathematics classroom: developing criteria for selecting appropriate learning apps. Int. J. Edu. Mat. Sci. Tech. 6, 155–172. doi: 10.18404/ijemst.408939

Crossref Full Text | Google Scholar

Heard, T. A. (1999). The role of stingless bees in crop pollination. Ann. Rev. Ent. 44, 183–206. doi: 10.1146/annurev.ento.44.1.183

PubMed Abstract | Crossref Full Text | Google Scholar

Henderson, S., and Yeow, J. (2012). “iPad in education: a case study of iPad adoption and use in a primary school,” in 45th Hawaii International Conference on System Sciences. Piscataway, NJ: IEEE, 78–87.

Google Scholar

Hodgson, J. A., Thomas, C. D., Cinderby, S., Cambridge, H., Evans, P., and Hill, J. K. (2011). Habitat re-creation strategies for promoting adaptation of species to climate change. Cons. Lett. 4, 289–297. doi: 10.1111/j.1755-263X.2011.00177.x

Crossref Full Text | Google Scholar

Horwitz, W. A. (1996). Developmental origins of environmental ethics: the life experiences of activists. Eth. Behav. 6, 29–53. doi: 10.1207/s15327019eb0601_3

Crossref Full Text | Google Scholar

Javorek, S. K., Mackenzie, K. E., and Vander Kloet, S. P. (2002). Comparative pollination effectiveness among bees (Hymenoptera: Apoidea) on lowbush blueberry (Ericaceae: Vaccinium angustifolium). Ann. Ent. Soc. Am. 95, 345–351. doi: 10.1603/0013-8746(2002)095[0345:CPEABH]2.0.CO;2

Crossref Full Text | Google Scholar

Kaiser, F. G., Roczen, N., and Bogner, F. X. (2008). Competence formation in environmental education: advancing ecology-specific rather than general abilities. Umweltpsy. 12, 56–70. doi: 10.5167/uzh-9249

Crossref Full Text | Google Scholar

Karrer, K., Glaser, C., Clemens, C., and Bruder, C. (2009). “Technikaffinität erfassen – der Fragebogen TA-EG,” [Measuring affinity for technology – the TA-EG questionnaire] in Der Mensch im Mittelpunkt technischer Systeme. Berliner Werkstatt Mensch-Maschine-Systeme (ZMMS Spektrum), Vol. 22, ed. A. Lichtenstein (Berlin: VDI Verlag GmbH), 196–201.

Google Scholar

Kremen, C., Williams, N. M., and Thorp, R. W. (2002). Crop pollination from native bees at risk from agricultural intensification. Proc. Nat. Acad. Sci. 99, 16812–16816. doi: 10.1073/pnas.262413599

PubMed Abstract | Crossref Full Text | Google Scholar

Landers, R. N. (2014). Developing a theory of gamified learning: linking serious games and gamification of learning. Simul. Gaming 45, 752–768. doi: 10.1177/1046878114563660

Crossref Full Text | Google Scholar

Leske, S. (2009). Biologische Vielfalt weltweit und regional erhalten – Einflussfaktoren für Handlungsbereitschaften von Schüler(inne)n der Sekundarstufen I und II [Protecting Biological Diversity World Wide and Regional – Influencing Factors for Action Readiness of Students in Secondary School I and II]. [Doctoral dissertation]. Göttingen: University of Göttingen.

Google Scholar

Liebal, J., and Exner, M. (2011). Usability für Kids [Usability for Kids]. Wiesbaden: Vieweg + Teubner Verlag.

Google Scholar

Mason, B. J., and Bruning, R. (2001). Providing Feedback in Computer-Based Instruction: What the Research Tells Us. Nebraska: University of Nebraska-Lincoln.

Google Scholar

Mayer, R. E. (2014a). “Introduction to multimedia learning,” in The Cambridge Handbook of Multimedia Learning, ed. R. E. Mayer (Cambridge University Press), 1–24.

Google Scholar

Mayer, R. E. (2014b). “Cognitive theory of multimedia learning,” in The Cambridge Handbook of Multimedia Learning, ed. R. E. Mayer (Cambridge: Cambridge University Press), 43–71.

Google Scholar

McPhee, I., Marks, L., and Marks, D. (2013). “Examining the impact of the Apple “iPad” on male and female classroom engagement in a primary school in Scotland,” in ICICTE 2013 Proceedings. Chania: ICICTE ,443–451.

Google Scholar

Morse, D. H. (1971). The insectivorous bird as an adaptive strategy. Ann. Rev. Ecol. Syst. 2, 177–200. doi: 10.1146/annurev.es.02.110171.001141

Crossref Full Text | Google Scholar

Nebel, S., Beege, M., Schneider, S., and Rey, G. D. (2016). The higher the score, the higher the learning outcome? Heterogeneous impacts of leaderboards and choice within educational videogames. Comp. Hum. Behav. 65, 391–401. doi: 10.1016/j.chb.2016.08.042

Crossref Full Text | Google Scholar

Ninaus, M., Moeller, K., McMullen, J., and Kiili, K. (2017). Acceptance of game-based learning and intrinsic motivation as predictors for learning success and flow experience. Int. J. Ser. Games 4, 15–30. doi: 10.17083/ijsg.v413.176

Crossref Full Text | Google Scholar

Piaget, J. (1971a). “Developmental Stages and Developmental Processes,” eds D. R. Green, M. P. Ford, and G. B. Flamer (New York, NY: McGraw-Hill), 172–188.

Google Scholar

Piaget, J. (1971b). “The theory of stages in cognitive development,” in Measurement and Piaget, eds D. R. Green, M. P. Ford, and G. B. Flamer (New York, NY: McGraw-Hill), 1–11.

Google Scholar

Rey, G. D., Beege, M., Nebel, S., Wirzberger, M., Schmitt, T. H., and Schneider, S. (2019). A meta-analysis of the segmenting effect. Edu. Psy. Rev. 31, 389–419. doi: 10.1007/s10648-018-9456-4

Crossref Full Text | Google Scholar

Richards, K. W. (1996). Comparative Efficacy of Bee Species for Pollination of Legume Seed Crops in Conservation of Bees. New York, NY: Academic Press, 81–104.

Google Scholar

Ryan, R. M., Mims, V., and Koestner, R. (1983). Relation of reward contingency and interpersonal context to intrinsic motivation: a review and test using cognitive evaluation theory. J. Pers. Soc. Psychol. 45, 736–750. doi: 10.1037/0022-3514.45.4.736

Crossref Full Text | Google Scholar

Sailer, M., and Homner, L. (2020). The gamification of learning: a meta-analysis. Edu. Psy. Rev. 32, 77–112. doi: 10.1007/s10648-019-09498-w

Crossref Full Text | Google Scholar

Salen, K., and Zimmerman, E. (2004). Rules of Play - Game Design Fundamentals. Cambridge, MA: Massachusetts Institute of Technology.

Google Scholar

Schneider, S., Dyrna, J., Meier, L., Beege, M., and Rey, G. D. (2018). How affective charge and text–picture connectedness moderate the impact of decorative pictures on multimedia learning. J. Edu. Psy. 110, 233–249. doi: 10.1037/edu0000209

Crossref Full Text | Google Scholar

Schneider, S., Häßler, A., Habermeyer, T., Beege, M., and Rey, G. D. (2019). The more human, the higher the performance? Examining the effects of anthropomorphism on learning with media. J. Educ. Psy. 111, 57–72. doi: 10.1037/edu0000273

Crossref Full Text | Google Scholar

Seibold, S., Gossner, M. M., Simons, N. K., Blüthgen, N., Müller, J., Ambarli, D., et al. (2019). Arthropod decline in grasslands and forests is associated with landscape-level drivers. Nature 574, 671–674. doi: 10.1038/s41586-019-1684-3

PubMed Abstract | Crossref Full Text | Google Scholar

Shute, V. J. (2008). Focus on formative feedback. Rev. Edu. Res. 78, 153–189. doi: 10.3102/0034654307313795

Crossref Full Text | Google Scholar

Sweller, J., van Merriënboer, J. J. G., and Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Edu. Psy. Rev. 31, 261–292. doi: 10.1007/s10648-019-09465-5

Crossref Full Text | Google Scholar

Thüring, M., and Mahlke, S. (2007). Usability, aesthetics and emotions in human–technology interaction. Int. J. Psy. 42, 253–264. doi: 10.1080/00207590701396674

Crossref Full Text | Google Scholar

Trefry, G. (2010). Casual Game Design: Designing Play for the Gamer in All of Us. London: CRC Press.

Google Scholar

van Vierssen Trip, N., MacPhail, V. J., Colla, S. R., and Olivastri, B. (2020). Examining the public's awareness of bee (Hymenoptera: Apoidae: Anthophila) conservation in Canada. Cons. Sci. Pract. 2, e293. doi: 10.1111/csp2.293

Crossref Full Text | Google Scholar

Westrich, P., Frommer, U., Mandery, K., Riemann, H., Ruhnke, H., Saure, C., et al. (2011). “Rote Liste und Gesamtartenliste der Bienen (Hymenoptera, Apidae Deutschlands [Red List and overall species list of bees (Hymenoptera, Apidae) in Germany],” in Rote Liste gefährdeter Tiere, Pflanzen und Pilze Deutschlands, Band 3: Wirbellose Tiere (Teil 1), Naturschutz und Biologische Vielfalt 70 (3), eds M. Binot-Hafke, S. Balzer, N. Becker, H. Gruttke, H. Haupt, N. Hofbauer, and M. Strauch (Münster: Landwirtschaftsverlag), 373–416.

Google Scholar

Wilcoxon, F. (1992). “Individual comparisons by ranking methods,” in Breakthroughs in Statistics, eds. S. Kotz and N. L. Johnson (New York, NY: Springer), 196–202.

Google Scholar

Winfree, R. (2010). The conservation and restoration of wild bees. Ann. New York Acad. Sci. 1195, 169–197. doi: 10.1111/j.1749-6632.2010.05449.x

PubMed Abstract | Crossref Full Text | Google Scholar

Wingerter, C. (2001). “Allgemeines Umweltbewusstsein [General environmental awareness.],” in Zusammenstellung sozialwissenschaftlicher Items und Skalen. doi: 10.6102/zis208

Crossref Full Text | Google Scholar

Wirzberger, M., Bornemeier, J., Kampel, S., Álvarez Serrano, M. G., Ullmann, L., and Rey, G. D. (2021). Umwelt trifft App: Verbindung virtueller und realer Welten in der Bildung für nachhaltige Entwicklung – Kurzbericht [Environment meets app: connecting virtual and real worlds in education for sustainable development – short report]. Umweltpsy 48, 123–132. Available online at: https://puma.ub.uni-stuttgart.de/bibtex/2adf4cdac2b9370c7ca24db78a47df2bf/mariawirzberger

Google Scholar

Wu, J. S., and Lee, J. J. (2015). Climate change games as tools for education and engagement. Nat. Clim. Change 5, 413–418. doi: 10.1038/nclimate2566

Crossref Full Text | Google Scholar

Zhang, M., Trussell, R. P., Gallegos, B., and Asam, R. R. (2015). Using math apps for improving student learning: an exploratory study in an inclusive fourth grade classroom. TechTrends 59, 32–39. doi: 10.1007/s11528-015-0837-y

Crossref Full Text | Google Scholar