Santiago Ruiz-Navas
Pajaree Ackaradejraungsri
Sandra Dijk- 1College of International Management, Ritsumeikan Asia Pacific University, Beppu, Japan
- 2HHL Leipzig Graduate School of Management, Leipzig, Germany
Introduction: Gamification can support the practical application of Inclusive Teaching. However, gamification literature reviews to implement Inclusive Teaching are scarce or not existent. Therefore, we conducted a scoping review of gamification literature reviews to identify what themes are covered and specifically if Inclusive Teaching has been explored.
Method: The scoping literature review comprises network and content analyses of gamification literature reviews retrieved from the Web of Science. We analyzed a multimode network of papers and keywords and used their eigenvector centrality to identify themes. The content analysis comprised of a human and automatic tagging process to identify each paper’s discipline/context.
Results: We mapped the themes explored in 125 gamification literature reviews to answer our first research question, what are the areas of knowledge covered by gamification literature reviews? The central topic is gamification and education to increase motivation, followed by gamification itself and understanding the implementation of gamification in various contexts. We identified 12 contexts and the top five frequent were Education, Business, Gamification, and Political Science. From the year-by-year analysis, we separated the themes into four periods: beginning (2014–2015), understanding (2016–2017), focus 2018 and focus and emergence (2019–2022). Regarding our second research question, how is the topic of Inclusive Teaching explored in gamification literature reviews? We did not find literature reviews about gamification to support Inclusive Teaching in the existing dataset.
Discussion: We report on the benefits of organizing central keywords by quartiles and using multimode networks to support scoping reviews; and disadvantages and advantages of using literature reviews as data sources for scoping reviews. We invite researchers to create more gamification literature reviews, to investigate gamification ethics in the light of recent technological developments such as generative models, and to reconnect gamification to the game design elements part of its definition, which goes beyond game elements.
1 Introduction
The concept of Inclusive Teaching (IT) is more than catering to specific students’ traits. The concept of Inclusive Teaching is found in the literature to address issues related to education for persons with or without special needs (Fränkel et al., 2023). However, Inclusive Teaching can also be defined in a broader sense, for example, the definition provided by Hockings (2010): “Inclusive learning and teaching in higher education refers to the ways in which pedagogy, curricula and assessment are designed and delivered to engage students in learning that is meaningful, relevant and accessible to all. It embraces a view of the individual and individual difference as the source of diversity that can enrich the lives and learning of others.” We built this work on this broader definition to help us focus on solutions that facilitate learning for all students.
Recent pedagogical advancements could provide ways to improve the implementation of inclusive teaching. A broader notion of IT helps us to focus on looking for solutions for all “students” but also groups together the challenges, such as balancing the adaptability of the course to the student’s needs with institutional requirements, bridging the gap between student’s learning needs and teacher’s understanding of these, raise awareness between students and teachers of everyone’s commonalities and differences to create safe learning environments (Hockings, 2010). Overcoming these challenges implies going beyond ad-hoc solutions and having stakeholders to understand and connect with the “students” needs. Alternatives to tackle these challenges have been proposed, such as universal design, student-centered learning, and technology implementation for IT (Lawrie et al., 2017). However, these alternatives bring positive results, but with recent developments in games and gamification research (e.g., adaptative gamification, transformative games), new options have become available and reasonable.
Gamification can be a channel to spark positive change in students and teachers toward IT. Gamification is defined as “using game design elements in a non-game context,” game design elements (e.g., game elements, mechanics, heuristics, and conceptual models), and non-game context (different from entertainment) (Deterding et al., 2011). Gamification has been used in education and other contexts to help people reach specific objectives, usually for their benefit, such as learning (Dikmen and Bahadir, 2022) or developing healthy habits (Trinidad et al., 2021). However, the potential of gamification can go beyond that. Gamification is related to game design, and as game research evolves, so does the knowledge available for gamification research. Recently, games and gamified activities have been designed to create experiences that challenge preconceptions and generate positive change in players (Rusch and Phelps, 2020) that transfer to their daily lives (Morrill and Culyba, 2019). Researchers reported these experiences valuable to generate awareness of topics such as cognitive bias (Legaki et al., 2021), SDGs (Souza et al., 2020), how to contribute to solving climate change (Fernández Galeote et al., 2021) and understanding gender inequality (Ortega-Sánchez and Gómez-Trigueros, 2019; Barrera Yañez et al., 2020). Therefore, we believe that implementing game design elements (gamification) into classroom activities could help to tackle current practical challenges to implementing IT in classrooms.
We investigated bibliometric studies exploring gamification and gamification in education as an initial step in our initial search for answers but did not find gamification literature reviews directly focusing on IT. The reviewed literature provided an overview of gamification research directions and its application. For example, we found that bibliometric studies report information such as overall and year-by-year pictures of how gamification research has evolved, the principal authors, institutions, publication avenues, and collaboration networks; Its applications in fields such as education, healthcare, and business to improve people’s motivation and engagement (Swacha, 2021; Trinidad et al., 2021; Dikmen and Bahadir, 2022; Guerrero-Alcedo et al., 2022; Nadi-Ravandi and Batooli, 2022; Irwanto et al., 2023). Furthermore, we can find information about gamification, such as its application, design, and effectiveness. Moreover, these studies call for more empirical research and data collection to validate gamification’s effectiveness, including more theoretical frameworks for gamification practice and implementation (Bozkurt and Durak, 2018). The studies propose research paths such as adaptative gamification, ethics of gamification, and understanding the emotional impacts of gamified experiences (Martí-Parreño et al., 2016; Trinidad et al., 2021).
The reviewed studies used methods such as content analysis and various bibliometric techniques such as descriptive and citation and word co-occurrence network analyses. Furthermore, the most common data sources used for literature reviews were Scopus and Web of Science, with Google Scholar often used to complement analysis. However, we could not find studies covering gamification to foster IT. Therefore, we propose doing a scoping review of literature reviews about gamification to confirm these initial findings and set up for a more detailed study on gamification to foster IT. Furthermore, this study includes two methodological novelties: the use of multimode network analysis and literature reviews as its primary data source.
We have two objectives: (1) to identify what research about gamification has or has not been reviewed so far and (2) to precisely identify the presence of literature reviews related to gamification to support inclusive teaching and learning. To achieve these objectives, we explore journal and conference papers indexed in the Web of Science using a mix of content and network analysis to support the summarization of the available literature and answer the following two research questions:
I. What are the areas of knowledge covered by gamification literature reviews?
II. How is the topic of Inclusive Teaching explored in gamification literature reviews?
2 Method
To answer our research questions, we will conduct a scoping review of literature reviews about gamification. Several techniques to summarize evidence include meta-analysis or systematic, scoping, rapid, and narrative reviews. We selected a scoping review, also called a mapping review, because our objective is to provide a general map of available evidence (Munn et al., 2018); in this case, evidence is the available literature reviews about gamification. Furthermore, maps of science as Scoping reviews are used in various disciplines to map available evidence about a topic or clarify concepts, for example, maps of science (Rafols et al., 2010), bibliometric studies about the application of technology (Kajikawa et al., 2007; Shibata et al., 2010; Navas et al., 2012; Yuan et al., 2021), or to clarify concepts such as emerging technology (Avila-Robinson and Miyazaki, 2011; Rotolo et al., 2015) and scoping reviews (Pham et al., 2014). Moreover, recent research has explained that maps of science can be considered as methodological components of literature reviews. Therefore, we proposed a mixed method approach inspired by the scoping review guidelines reported by Tricco et al. (2018), the methodological components of scoping reviews reported by Pham et al. (2014), Kraus et al. (2022) and science mapping methods (Rafols et al., 2010).
3 Search strategy
We selected lexical query as our search strategy, which consists of creating keywords relevant to the topic to review (literature reviews about gamification). When searching for papers to do a literature review, one can select various search strategies, such as lexical queries, citations, and specialized journals. However, the lexical query search strategy has been reported to be better at extracting relevant papers than other search strategies (Huang et al., 2011) and is broadly used in scoping reviews (Pham et al., 2014). Therefore, we selected lexical query as our primary search strategy.
3.1 Lexical query
We created the lexical query using words and their synonyms extracted from the definition of literature reviews and gamification.
• Literature reviews: “a study that analyzes and synthesizes an existing body of literature by identifying, challenging, and advancing the building blocks of a theory through an examination of a body (or several bodies) of prior work (Post et al., 2020).
• Gamification: defined as “using game design elements in non-game context” (Deterding et al., 2011).
We looked for the words of the lexical query only in paper titles. We did not search in abstracts or full text because the phrase “literature review” can be used by researchers to describe reading multiple papers about a concept or theory to inform research, but not in the sense of doing systematic literature reviews or scoping reviews. Upon conducting a preliminary evaluation of our lexical query via the Web of Science, encompassing titles, keywords, and abstracts, it has come to our attention that the precision of paper retrieval significantly enhanced when exclusively utilizing titles.
3.2 Eligibility criteria
We rejected papers not aligned with the definitions of literature reviews and gamification, for example, reviews of non-literature items such as software, apps, or tools, were excluded. Furthermore, we excluded books and chapters, on the premise that books contain knowledge interpreted or based on what is published in journal articles and conference papers (University Libraries, 2023).
We selected, all years, publications in English, publications by any country and only journal articles, conference papers and review articles.
3.3 Information sources
The Web of Science was the database selected to obtain the papers. There are several other options, such as dimensions (Hook et al., 2018), Semantic Scholar (Kinney et al., 2023), OpenAlex (Priem et al., 2022), and Scopus. Among these options, the Web of Science and Scopus include many multidisciplinary, international, and peer-reviewed journals and conferences. Furthermore, they have strict inclusion criteria to select the journals they index, giving a sense of quality assurance of their indexed content (Pranckutė, 2021). However, both Science and Scopus index also have limitations, such as limited coverage of published literature and possible bias in the process of human curation of the indexed content.
3.4 Search
Considering words in the definitions of literature review and gamification, a round of skimming search results in titles in the Web of Science, looking for literature reviews and the restrictions previously mentioned, our selected lexical query is:
TI = ((review* NEAR/6 (literature OR research) OR “paper reviews” OR “state of the art” OR “review article” OR (A NEAR/2 review*)) NEAR/6 gamification) Excluding books chapters Refining for articles, conference, and early access.
The lexical query was run on 07/26/2022.
3.5 Selection of papers
The selection of papers for the scoping review involved two steps: an initial exploratory revision and a subsequent final revision.
3.5.1 Exploratory revision
Two business school undergraduate students in the role of Research Assistants (RAs) were asked to work as a group to read the abstracts and titles of the papers obtained from the previous step of lexical query. The RAs were trained in the inclusion and exclusion criteria of the study and were shown a few examples of what types of papers complied with the requirements and which did not. They were asked to annotate the papers as accepted or not, explain their decision, and report any difficulties or challenges presented in this process to the authors in charge of the second step.
3.5.2 Final revision
Using the annotations and reports provided by the RAs, the authors further revised the set of papers and gave a final judgment or accepted or rejected the evaluated papers.
4 Data items and charting process
For each of the papers, we extracted three metadata fields from the Web of Science, title, abstract, and publication year; and one from the content analysis, context.
The data charting process consisted of two processes: (1) document metadata (e.g., title, abstract, and publication year) extraction from the Web of Science and (2) content analysis to obtain the document’s context. Furthermore, we added a network analysis step to complement the final analysis.
4.1 Content and network analysis
We used content and network analysis to find evidence that could help us answer our two research questions. Content analysis is a commonly used technique to identify patterns in literature reviews. We implemented content analysis to identify each paper’s context and find evidence to help us answer what fields are covered by gamification-related literature reviews. Furthermore, we used network analysis to provide a general and year-by-year overview of the papers and words that describe the dataset of literature reviews about gamification. We use these two analyses’ results to answer our research questions. For specific details about the operationalization of the content and network analyses (see Supplementary Annex 1).
4.2 Content analysis
The process to get the context of each paper consisted of three steps: an exploratory context annotation, an automatic context annotation, and a final context annotation. We understand the context of a paper as “a broader discipline that studies the topic described in the paper,” for example, for any context annotation referring to health, such as exercise, mental health, or e-health, we would select the broader discipline of health. Moreover, we would select education as a central theme for papers covering topics such as e-learning, training, higher education, and child education.
• Explorative context annotation: Two RAs were given training to extract the “context” of a paper from its title and abstract and were asked to report inconsistencies and difficulties in the annotation process to inform the final annotation stage.
• Automatic context annotation: We added a process to automatically assign disciplines to each document by using word similarity measures between each document’s abstract and title to the first paragraph of the subdisciplines listed in the Wikipedia List of Disciplines website. Using Wikipedia for the scientific process has its advantages and disadvantages; for example, it is not recommended to be used in Scientometric analysis since its content and structure are constantly updated and thus can harm the reproducibility of studies (Arroyo-Machado et al., 2022). However, on the positive side, Wikipedia is easily accessible. It can be used as a source of established knowledge to support decision-making or to train models that support decision-making. For example, it is part of a recent LLMs (Large Language Models) training dataset (Schaul et al., 2023). It has been used to support concept disambiguation (Mihalcea, 2007) and to identify technological convergence (Kim et al., 2019).
4.3 Descriptive network analysis
We selected network analysis to complement the content analysis to provide a better overview of the topics covered by literature reviews related to gamification. Network analysis is widely used in Scientometrics and literature reviews to create maps of science. There are diverse types of maps of sciences that can be created using networks, for example, using citations as links and documents as nodes (Rafols et al., 2010) or using words or concepts as nodes and co-occurrence as links (Callon et al., 1991). Furthermore, we can identify node measures that help to summarize in detail the topics of a specific map of science, such as node centrality using networks.
4.4 Multimode networks
Multimode networks show relationships among more than one type of node or mode, hence the name. A multimode network allows researchers to analyze the relationships of more than one element visually. Also, it is possible to create projections to create single networks for each mode type, which can complement the initial analysis (Wasserman and Faust, 1994). Therefore, we decided to implement a multimode network of papers and words to support our exploration of what topics are and are not covered by current literature reviews about gamification.
4.5 Node eigenvector centrality
We used nodes’ eigenvector centrality to support our descriptive analysis. When mapping science using networks, various network measures such as degree and centrality are used to characterize the network and support analysis. These measures help identify the role of nodes concerning the other nodes in the network. Among these measures, eigenvector centrality is often used to identify nodes in a network that should be given special attention (Newman, 2018). The eigenvector centrality of a node is a number proportional to the sum of the centralities of all the nodes it has a connection with; it is often called a measure of influence or power (Bonacich, 1987; Newman, 2018). For example, a leading influencer with many collaborations with other influencers on YouTube would have a high eigenvector centrality in a network of YouTube influencers. In the case of a network of papers linked by word co-occurrence, the papers with the highest eigenvector centrality would have the highest number of central keywords in their content.
4.6 Division of keyword centrality into quartiles
keyword-based distributions such as frequency, term frequency, inverse document frequency and degrees, are often skewed and Euclidian averages to provide a balanced partition. Furthermore, word-based analysis tends to have an extensive number of keywords that cannot be reported to the readers. Some researchers use the Zipf’s law (Ryland Williams et al., 2015) to set thresholds of top n words or phrases to report their word analysis. However, thresholds when used without consideration for the size or the information that words can provide about the dataset (Genzel and Charniak, 2002) can give readers and analysts an incomplete vision of the dataset. Therefore, we used percentiles, specifically divided the keyword eigenvector centrality into four or quartiles to provide readers and analysts a wider view of the keywords that were most (x > Q75th), average (Q75th<x > Q50th) and in the periphery of the networks (Q50th<x > Q25th and x < Q25th).
4.7 Analysis
We provide two levels of descriptive analysis by using network analysis, an overview of all literature reviews about gamification and a discriminated analysis year by year. We present each multimode network’s and projection’s most central words and documents. Furthermore, we described each network’s contents using the compiled information about the central papers and documents in the network and reading the abstract of the papers in them.
5 Synthesis of results
We combined the results of the content analysis and the network analysis to answer our research question. We analyzed a network comprising all years and the networks of each year. For each network, we investigated the contexts of the papers, their most central papers, and keywords to provide a general description of the themes covered by the gamification literature reviews from 2014 to 2022.
6 Results
We ran the lexical query on 12th January 2023 and obtained 135 papers from 2014 to 2022. Figure 1 presents the identification, eligibility, and included number of papers flowchart.
Figure 1. Included papers flowchart. Source: authors.
All selected papers had at least the title or the abstract. In Figure 2, we present the number of papers per year.
Figure 2. Papers by year in our dataset. Source: authors.
6.1 General content analysis
The five most frequent contexts addressed by the reviewed gamification literature reviews were Education, Business, Gamification, Health, and Political science. Figure 3 shows a bubble plot of the context frequency by year.
Figure 3. Bubble plot of context per year. The circle’s radius represents the total frequency, and the colors represent each context. The color assignment is blue for Education, light orange for Business, green blue for Gamification, orange for Health, dark pink for Political Science, brown for Psychology, pink for Mobile, gray for Engineering, yellow for Computer science, light green for Disaster Emergency, light blue for Crowdsourcing and purple for Library. Source: authors.
We provide a general description of the top five contexts:
• Education: Describes literature reviews related to gamification in learning, education in any context such as academic disciplines (accounting, business, mathematics, and STEM), or training, for example, in the training of nurses or medical doctors.
• Business: We used this label in literature reviews exploring gamification in business or work contexts to improve workers’ motivation, such as in software engineering and quality control teams and manufacturing, and improve customers’ engagement with the firm’s brand, mobile apps, and IoT services.
• Gamification: Literature reviews exploring gamification, for example, bibliometric studies about gamification research, looking into the fundamental theory of gamification, gamification design frameworks, and adaptative or tailored gamification.
• Healthcare: This context describes literature reviews about using gamification to improve patients’ motivation and engagement in healthcare contexts such as elderly care, mental health, nutrition, physical activity, and chronic conditions management.
• Political Science: Described literature reviews exploring the uses of gamification to improve user engagement in civic services such as e-government and democratic participation.
6.2 Multimode network analysis
In this section, we present the analysis of a multimode network for all years (2014–2022) and the summary of the analysis of the networks for each year.
6.2.1 Network properties (2014–2022)
This network’s total number of papers and keywords was 125 and 156, respectively. Figure 4 shows this network, and Table 1 lists the top five central papers and keywords.
Figure 4. Multimode network of papers and keywords for all years (2014–2022). The light blue are paper nodes, the light green are keyword nodes. * Indicates the network’s most eigenvector central nodes, the word “motivation” and paper 21, both with bigger radio size. Source: the authors.
Table 1. Top five eigenvector central paper and keywords, with keywords organized by quartiles (2014–2022).
From the papers titles of the most central papers reported in Table 1, we can see that in general terms and in agreement with the contexts reported in Figure 3, Education is the focus of gamification literature reviews, with five out of five central papers focused on this theme. From the central keywords reported in Table 1, we can see that in:
• Q3: The central theme of this network relates to gamification and education to increase motivation (motivation, game, education).
• Q2: This quartile details what is thematic in gamification, such as adaptative gamification, and gives information about a common context explored in papers workplace (work). Furthermore, it shades more detail into what is behind the word “motivation,” for example, user engagement (user engagement). Moreover, this quartile provides information about the methods used in the literature reviews (bibliometric and systematic study).
• Q1: This set of words expands on the themes where literature reviews about gamification have been done, such as software engineering, service research, and teaching process (software engineering, service, teaching, and process).
• Q0: We found in this quartile information about the methods, such as citations (citations), which are often used in bibliometric studies. Furthermore, this quartile provided additional detail on what was explored in the central theme of pedagogies and technologies to support education and analyzing students’ personalities to design gamification that works for them (pedagogy, personality).
6.2.2 Year-by-year networks’ properties
This section presents the summary of analyzing the central papers and keywords in each year network and their network size in terms of number or documents and keywords. In the annex you can see a finer explanation of each networks’ central papers and keywords. We present in Figure 5 a stacked bar chart of the top central keywords per year with centrality data. Since some years did not have centrality data, we added a narrative summary in Table 2.
Figure 5. Comparative stacked bar chart showing the top five central keywords per year (2017, 2019–2022). The keywords not added to the analysis such as context, use, study and gamife, were removed from the dataset when drawing the graph. Source: the authors.
Table 2. Summary of each year’ network central themes and a note on the thematic change over time.
In Table 2, we can see that the central themes covered by the literature reviews in this research were the application of gamification in various contexts and more prominently in education.
6.3 Literature reviews related to Inclusive Teaching
We found eight reviews directly (1) and indirectly (7) addressing Inclusive Teaching. Directly: The work of Mubin and Wee Ann Poh (2019), which made a review on the use of gamification to support autist children’s development of interaction skills. Indirectly: we found seven works related to adaptative or tailored gamification. The common purpose of these works was to understand the relationships between the user’s profiles/characteristics, the game elements used in the gamified activities and the outcomes of these activities. Furthermore, their objective is to understand how the elements of gamified activities can be adapted/modified to match the user’s characteristics to improve outcomes. We argue that these seven studies are indirectly related to Inclusive Teaching, because the authors do not mention Inclusivity or Inclusive Teaching per-se in their works. However, their main path of inquiry is to cater user’s different characteristics and in various case for learners’ characteristics (four out of six). Therefore, we consider their findings to be relevant to support Inclusive Teaching even if the concept is not explicit in their text. In Table 3, we present a summary of the contributions of these works.
Table 3. List of literature reviews directly or indirectly related to Inclusive Teaching.
6.4 Integration of content analysis and network analysis
From the content and network analysis results, we identified that most literature reviews explored two types of themes: research about gamification itself and its application. We called these two types “gamification” and “gamification in.” Table 4 summarizes the themes explored by gamification literature reviews from 2014 to 2022.
Table 4. Summary of the content and network analysis.
From the information in Tables 1, 2, 4, Figure 2, and analyzing the paper’s abstracts, we divided the analyzed years into four periods: The beginning (2014–2015), Understanding (2016–2017), Focus (2018), and Focus and emergence (2019–2022).
• The beginning (2014 to 2015): Gamification literature reviews did not have a central pattern; the themes varied between applying gamification in different contexts and understanding gamification research.
• Understanding (2016 and 2017): The central theme of literature reviews during this period was to explore the application of gamification to motivate people (users, workers, patients) in various contexts such as Education, Business, and Health, and understanding of gamification design (what game elements and mechanics work).
• Focus (2018): The central thematic changes from reviewing the applications of gamification to motivate people in various contexts to implementing gamification in education to motivate people, educational themes such as adding gamification to pedagogical activities and educational software. Understanding education as teaching and learning in and outside normal academic contexts, for example, learning academic topics such as mathematics, accounting, and software engineering or teaching routines and mindset change toward activities considered tedious, such as reducing household waste, keeping healthy routines; and increasing civil, workplace and brand engagement. Additionally, research was done to map gamification research’s state of the art and further understand its design.
• Focus and emergence (2019–2022): The focus on gamification in education to motivate remains, and more contexts to apply gamification and new concepts related to gamification itself are explored. Literature reviews explored more diverse contexts such as political science, disaster emergency, and computer science. Furthermore, this year’s papers investigated adaptative or tailored gamification, showing a possible future path for gamification research.
7 Discussion
7.1 What are the areas of knowledge covered by gamification literature reviews?
From the network analysis results and the summaries presented in Tables 1, 2, 4, we understood that most of the gamification literature reviews analyzed are related to the words, education, and motivation, which, in general terms, describe literature reviews exploring how gamification is applied to improve people’s (students, workers, users, patients, children) learning in various contexts, such as healthcare, business, and accounting.
The four periods we suggested to summarize the topics covered by the gamification literature reviews over time are product of our interpretation of the network analysis and their central words and papers. What we would like to highlight from the summary is the gradual transition from literature reviews covering gamification and its application in a few contexts, then the increase of context that are being explored to the significant focus on education and finally a sustained focus in education, increased variety of context covered and emergent research trends such as adaptative/tailored gamification.
7.2 How is the topic of Inclusive Teaching explored in gamification literature reviews?
From our results showed in Table 3, we concluded that no available gamification literature reviews explore the specific theme of IT. We found some literature reviews that touched upon IT’s components, such as proposing a gamification framework to support interaction skills building for autistic children (Mubin and Wee Ann Poh, 2019) and the idea of a tailored or adaptative gamification (Hallifax et al., 2019; Rozi et al., 2019; Tomé-Klock et al., 2020; Khakpour and Colomo-Palacios, 2021; Bennani et al., 2022; Denden et al., 2022; Oliveira et al., 2023). These studies contribute to making education more inclusive by revising literature trying to understand how the outcomes of gamified activities can improve by modifying the game elements used to match the users’ specific characteristics. However, these studies did not mention specifically IT as its leading path of inquiry.
7.3 Methodological insights
We highlight three insights: the benefits of organizing central keywords by quartiles and using a multimode network (papers and words) to support thematic qualitative analyses, and advantages and disadvantages of using literature reviews or summaries of evidence as data sources for scoping reviews:
• Showing the top five eigenvector central keywords and organizing them by quartiles helped to analyze the nature of the network as top central keywords, average central keywords, and peripherical keywords. This view helped the analyst to understand and classify the content of the abstract of the papers of each year into a summarized history, which was in line with the structure of the multimode network, the content of the papers, and the keywords. Thus, it could be a helpful tool to report and support the analysis of word networks of small sizes, such as the ones in this study, between 1 to 125 nodes (The logic behind the number is that showing five items per division adds up to 20, which is close to 30% of the 125 nodes, thus close to a “meaningful” sample size). In the case of networks with more nodes, this approach could still be helpful if the analysts use appropriately sized random samples of the keywords belonging to each quartile.
• Using a multimode network (papers and words) facilitated our qualitative understanding of the themes covered by the identified networks. Having the most central words and papers (titles) at hand allowed us to create a general picture that made sense after reading the abstract and supported the summaries of gamification literature reviews and thematic coverage provided in this study. Using paper titles in analysis aligns with a previous result (Hecking and Leydesdorff, 2019) that suggests that adding papers into word analyses can improve thematic identification.
• We report the advantages and disadvantages of using literature reviews as our data source compared to previous gamification bibliometric studies. On the positive side, we obtained similar results when describing general trends in gamification research; for example, the results of our content analysis were similar to the domain analysis by Trinidad et al. (2021), we both reported as central contexts “education,” and “healthcare;” The network analysis of the general network reported as a central topic “gamification in education to motivate” similar to Harman et al. (2014), Bozkurt and Durak (2018), Trinidad et al. (2021), and Guerrero-Alcedo et al. (2022). In contrast, we could see the disadvantages of using only literature reviews in the year-by-year analysis. Using Trinidad et al. (2021) results as a reference, we can see that they captured in their first period 2011–2013 trends that we did not see in our analysis and only appeared in our dataset later. For example, in 2011–2013, Trinidad et al. (2021) reported virtual reality and machine learning in 2014–2016. These themes were invisible to our centrality analysis and only became apparent in our dataset later—virtual reality in 2018 (see Supplementary Table S3), and machine learning in 2020 (see Table 3). We could not “see” these themes because of the low number of literature reviews touching on these topics. In conclusion, using literature reviews as a data source is relevant for understanding the general trends of a research topic in line with the objective of scoping reviews. However, if the review’s objective is to provide detailed trends over time, such as year-by-year analysis, it is better to use all available literature as a source.
7.4 Future research
We propose three paths for future research:
• Generate more literature reviews about gamification, its use, and its impact in more contexts and specific areas. We reported there are themes in which we did not find literature reviews even for the central theme of education, for example, the use of gamification to foster IT, understanding the impacts of gamified curriculums, and recompilation of best implementation practices to evaluate or implement gamification in education, among others. There are even more opportunities in peripherical themes such as citizen science research and adaptative or tailored gamification.
• Further research on the ethical implications of using gamification to modify behaviors and adaptative and tailored gamification. Some studies are in this direction (Hyrynsalmi et al., 2018). We agree because technological advances such as Large Language Models and machine learning algorithms to create recommendations tailored to users bring about social and ethical risks with their implementations (Weidinger et al., 2021; Achiam et al., 2023). Thus, we must think about managing these risks and making the best of these technologies to create gamified activities that positively impact people’s lives.
• We suggest to actors interested in gamification to look beyond game elements and see how game design elements support their objectives. Games research evolves quickly, revealing novel insights into game elements such as mechanics, psychology, technology, and their combination. Therefore, keeping up with game research can provide enriching insights for gamification practice; for example, when looking for means to gamify your classroom, look at the present (what gamification research reports) and at the possible future (what game designers are doing that could be helpful for your objectives). For example, the use of generative AI to procedurally generate or adapt game components (physical or digital) to match the user’s interaction with the game (Hendrikx et al., 2013) or implementing elements of transformative games design (evoking emotions to challenge mindsets) (Rusch and Phelps, 2020).
7.5 Limitations of our methods
Our lexical query faces a challenge reported by Pham et al. (2014), the terminology used to describe literature review still needs to be standardized. Therefore, we could have expanded our scope from literature reviews to studies summarizing evidence around gamification by adding terms such as mapping, scoping, systematic, and mini reviews.
We did not group similar words when creating our networks; an evaluator can argue that this decision can lead to weaker eigenvector centralities for similar words, for example in the case of the words gamify, gamifie, and game, one can say that these represent the same idea because of their “similarity” and unify these words as one, thus increasing this set of word’s centralities. However, we decided to leave them as they were because we could not systematically and reproducibly distinguish the meaning or context of these words to group them. Researchers have proposed using a thesaurus to homogenize similar words in science mapping efforts (Porter and Youtie, 2009; Boyack, 2017). However, thesaurus depends on human annotators to define the relations between words and decide if they can be grouped or not, which is resource intensive, could introduce bias and hurt reproducibility.
We reported a non-valid English word gamifie; this resulted from providing lemmatized text as input to the KeyBert algorithm. We combined two text pre-processing pipelines, one provided by spaCy, and another implemented in the rake package. We pre-lemmatized the text using the spaCy library to standardize the input for the rake and spaCy-based algorithms. However, when we did that, we obtained problematic keywords such as gamifie, gamified, and gamify from the algorithms depending on the KeyBert algorithms (KeyBert and pattern rank).
We used the Web of Science as our unique source of literature. Therefore, we left out literature that is not indexed in this database. As an improvement, future works could use closed databases such as the Web of Science or Scopus in combination with more inclusive datasets such as Dimensions, Open Alex, and Semantic Scholar.
8 Conclusion
We did a scoping review of gamification literature reviews to understand what has been reviewed regarding gamification and to see if literature reviews about gamification to foster Inclusive Teaching existed.
We answer our first research question, from the content and network analysis. What are the areas of knowledge covered by gamification literature reviews? We obtained from the general analysis that the central trend in gamification literature reviews is gamification in education to motivate, followed by gamification research with themes such as its theoretical foundations, understanding its implementation and design, and understanding the implementation of gamification in various contexts. We found that the most frequent contexts for gamification literature reviews were Education, Business, Gamification, Health, and Political science. Furthermore, we proposed four periods that describe the themes covered by the gamification literature reviews: beginning, understanding, focus, and focus and emergence. The literature reviews about gamification started (2014–2015) exploring gamification itself and its applications but with no specific focus; then (2016–2017), a focus on understanding the application to motivate (humans) in diverse contexts appeared, and in 2018 the focus shifted to the use of gamification to motivate (humans) in education, from there on (2019–2022) the focus in education remained, researchers extended the contexts in which gamification applications were studied and the theme of adaptative gamification emerged as a potential path for future research.
The answer to our second research question, how is the topic of Inclusive Teaching explored in gamification literature reviews? After carrying out our analysis and extensively reading the abstracts of the papers in the dataset, we could not identify any literature reviews about gamification in IT; we identified works reviewing themes close to its components, such as using gamification to support autistic children education and the concept of tailored and adaptative gamification which potentially could be used in IT. However, these works did not directly focus on the integrated theme of IT.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors on request, subject to the restrictions that may apply to the data from the Web of Science.
Author contributions
SR-N: Data curation, Formal analysis, Funding acquisition, Methodology, Project administration, Visualization, Writing – original draft, Writing – review & editing. PA: Conceptualization, Data curation, Formal analysis, Investigation, Resources, Supervision, Writing – original draft, Writing – review & editing. SD: Conceptualization, Formal analysis, Resources, Supervision, Validation, Writing – review & editing.
Funding
The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This article was funded by the Center for Inclusive Leadership (CIL) of the Ritsumeikan Asia Pacific University, in the project titled “Exploring gamification theory and practice for cultivating diversity and inclusion in classrooms.”
Acknowledgments
We are thankful to the RAs involved in the project the Satimboeva Rukhsora and Chi Kieu Nguyen, to professor Alcantara Lailani for her guidance in the early stages of the project, special considerations to the CIL support staff for their help and the kind reviewers for their comments to improve this paper.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2024.1306298/full#supplementary-material
References
Achiam, O. J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., et al. (2023). GPT-4 technical report. Available at: https://arxiv.org/abs/2303.08774
Arroyo-Machado, W., Torres-Salinas, D., and Costas, R. (2022). Wikinformetrics: construction and description of an open Wikipedia knowledge graph data set for informetric purposes. Quant. Sci. Stud. 3, 931–952. doi: 10.1162/qss_a_00226
Avila-Robinson, A., and Miyazaki, K. (2011). “Conceptualization and operationalization of emerging technologies: a complementing approach” in Technology Management in the Energy Smart World (PICMET), 2011 proceedings of PICMET’11, 1681–1692.
Barrera Yañez, A. G., Alonso-Fernandez, C., and Fernandez Manjon, B. (2020). “Review of serious games to educate on gender equality” in Eighth international conference on technological ecosystems for enhancing Multiculturality (New York, NY, USA: ACM), 662–668.
Bennani, S., Maalel, A., and Ben Ghezala, H. (2022). Adaptive gamification in E-learning: a literature review and future challenges. Comput. Appl. Eng. Educ. 30, 628–642. doi: 10.1002/cae.22477
Bonacich, P. (1987). Power and centrality: a family of measures. Am. J. Sociol. 92, 1170–1182. doi: 10.1086/228631
Boyack, K. W. (2017). Thesaurus-based methods for mapping contents of publication sets. Scientometrics 111, 1141–1155. doi: 10.1007/s11192-017-2304-3
Bozkurt, A., and Durak, G. (2018). A systematic review of gamification research: in pursuit of homo ludens. Int. J. Game Based Learn. 8, 15–33. doi: 10.4018/IJGBL.2018070102
Callon, M., Courtial, J. P., and Laville, F. (1991). Co-word analysis as a tool for describing the network of interactions between basic and technological research: the case of polymer chemsitry. Scientometrics 22, 155–205. doi: 10.1007/BF02019280
Denden, M., Tlili, A., Chen, N.-S., Abed, M., Jemni, M., and Essalmi, F. (2022). The role of learners’ characteristics in educational gamification systems: a systematic meta-review of the literature. Interact. Learn. Environ. 1–23, 1–23. doi: 10.1080/10494820.2022.2098777
Deterding, S., Dixon, D., Khaled, R., and Nacke, L. (2011). From game design elements to gamefulness: defining “gamification”. Proc. 15th Int. Acad. MindTrek Conf. Envisioning Futur. Media Environ. MindTrek 2011, 9–15. doi: 10.1145/2181037.2181040
Dikmen, M., and Bahadir, F. (2022). Bibliometric mapping of gamification in education. J. Educ. Reflect. 6, 50–67.
Fernández Galeote, D., Rajanen, M., Rajanen, D., Legaki, N.-Z., Langley, D. J., and Hamari, J. (2021). Gamification for climate change engagement: review of corpus and future agenda. Environ. Res. Lett. 16:063004. doi: 10.1088/1748-9326/abec05
Fränkel, S., Sterken, M., and Stinken-Rösner, L. (2023). From barriers to boosters: initial teacher education for inclusive science education. Front. Educ. 8:1191619. doi: 10.3389/feduc.2023.1191619
Genzel, D., and Charniak, E. (2002). Entropy rate constancy in text. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, 2002-July, 199–206.
Guerrero-Alcedo, J. M., Espina-Romero, L. C., and Nava-Chirinos, Á. A. (2022). Gamification in the university context: bibliometric review in Scopus (2012-2022). Int. J. Learn. Teach. Educ. Res. 21, 309–325. doi: 10.26803/ijlter.21.5.16
Hallifax, S., Serna, A., Marty, J.-C., and Lavoué, É. (2019). “Adaptive gamification in education: a literature review of current trends and developments” in European conference on TechnologyEnhanced learning (EC-TEL) (Delft: Springer, Cham), 294–307.
Harman, K., Koohang, A., and Paliszkiewicz, J. (2014). Scholarly interest in gamification: a citation network analysis. Ind. Manag. Data Syst. 114, 1438–1452. doi: 10.1108/IMDS-07-2014-0208
Hecking, T., and Leydesdorff, L. (2019). Can topic models be used in research evaluations? Reproducibility, validity, and reliability when compared with semantic maps. Res. Eval. 28, 263–272. doi: 10.1093/reseval/rvz015
Hendrikx, M., Meijer, S., Van Der Velden, J., and Iosup, A. (2013). Procedural content generation for games. ACM Trans. Multimed. Comput. Commun. Appl. 9, 1–22. doi: 10.1145/2422956.2422957
Hockings, C. (2010). Inclusive learning and teaching in higher education: a synthesis of research. Retreived August, 1–67.
Hook, D. W., Porter, S. J., and Herzog, C. (2018). Dimensions: building context for search and evaluation. Front. Res. Metrics Anal. 3:23. doi: 10.3389/frma.2018.00023
Huang, C., Notten, A., and Rasters, N. (2011). Nanoscience and technology publications and patents: a review of social science studies and search strategies. J. Technol. Transf. 36, 145–172. doi: 10.1007/s10961-009-9149-8
Hyrynsalmi, S., Kimppa, K. K., and Smed, J. (2018). “Gamification ethics” in Encyclopedia of computer graphics and games (Cham: Springer International Publishing), 1–6.
Irwanto, I., Wahyudiati, D., Saputro, A. D., and Laksana, S. D. (2023). Research trends and applications of gamification in higher education: a bibliometric analysis spanning 2013–2022. Int. J. Emerg. Technol. Learn. 18, 19–41. doi: 10.3991/ijet.v18i05.37021
Kajikawa, Y., Ohno, J., Takeda, Y., Matsushima, K., and Komiyama, H. (2007). Creating an academic landscape of sustainability science: an analysis of the citation network. Sustain. Sci. 2:221. doi: 10.1007/s11625-007-0027-8
Khakpour, A., and Colomo-Palacios, R. (2021). Convergence of gamification and machine learning: a systematic literature review. Technol. Knowl. Learn. 26, 597–636. doi: 10.1007/s10758-020-09456-4
Kim, J., Kim, S., and Lee, C. (2019). Anticipating technological convergence: link prediction using Wikipedia hyperlinks. Technovation 79, 25–34. doi: 10.1016/j.technovation.2018.06.008
Kinney, R., Anastasiades, C., Authur, R., Beltagy, I., Bragg, J., Buraczynski, A., et al. (2023). The semantic scholar open data platform. ArXiv:abs/2301.10140. doi: 10.48550/arXiv.2301.10140
Kraus, S., Breier, M., Lim, W. M., Dabić, M., Kumar, S., Kanbach, D., et al. (2022). Literature reviews as independent studies: guidelines for academic practice. Rev. Manag. Sci. 16, 2577–2595. doi: 10.1007/s11846-022-00588-8
Lawrie, G., Marquis, E., Fuller, E., Newman, T., Qiu, M., Nomikoudis, M., et al. (2017). Moving towards inclusive learning and teaching: a synthesis of recent literature. Teach. Learn. Inq. 5, 9–21. doi: 10.20343/teachlearninqu.5.1.3
Legaki, N., Karpouzis, K., Assimakopoulos, V., and Hamari, J. (2021). Gamification to avoid cognitive biases: an experiment of gamifying a forecasting course. Technol. Forecast. Soc. Change 167:120725. doi: 10.1016/j.techfore.2021.120725
Martí-Parreño, J., Méndez-Ibáñez, E., and Alonso-Arroyo, A. (2016). The use of gamification in education: a bibliometric and text mining analysis. J. Comput. Assist. Learn. 32, 663–676. doi: 10.1111/jcal.12161
Mihalcea, R. (2007). “Using Wikipedia for automatic word sense disambiguation” in Human language technologies 2007: The conference of the north American chapter of the Association for Computational Linguistics; proceedings of the Main conference (Rochester, New York: Association for Computational Linguistics), 196–203.
Morrill, B., and Culyba, S. (2019). “Transformational games” in Encyclopedia of computer graphics and games (Cham: Springer International Publishing), 1–5.
Mubin, S. A., and Wee Ann Poh, M. (2019). “A review on gamification design framework: how they incorporated for autism children” in In 2019 4th international conference and workshops on recent advances and innovations in engineering (ICRAIE) (IEEE), 1–4.
Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., and Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med. Res. Methodol. 18, 143–147. doi: 10.1186/s12874-018-0611-x
Nadi-Ravandi, S., and Batooli, Z. (2022). Gamification in education: a scientometric, content and co-occurrence analysis of systematic review and meta-analysis articles. Educ. Inf. Technol. 27, 10207–10238. doi: 10.1007/s10639-022-11048-x
Navas, M. E., Londoño, E., Ruiz, S., and Ruiz, D. (2012). “State of the art of emerging technologies in Colombia,” in 2012 Proceedings of PICMET ‘12: Technology Management for Emerging Technologies, Vancouver, BC, Canada, 358–367.
Oliveira, W., Hamari, J., Shi, L., Toda, A. M., Rodrigues, L., Palomino, P. T., et al. (2023). Tailored gamification in education: a literature review and future agenda. Educ. Inf. Technol. 28, 373–406. doi: 10.1007/s10639-022-11122-4
Ortega-Sánchez, D., and Gómez-Trigueros, I. M. (2019). Gamification, social problems, and gender in the teaching of social sciences: representations and discourse of trainee teachers. PLoS One 14:e0218869. doi: 10.1371/journal.pone.0218869
Pham, M. T., Rajić, A., Greig, J. D., Sargeant, J. M., Papadopoulos, A., and Mcewen, S. A. (2014). A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res. Synth. Methods 5, 371–385. doi: 10.1002/jrsm.1123
Porter, A. L., and Youtie, J. (2009). How interdisciplinary is nanotechnology? J. Nanopart. Res. 11, 1023–1041. doi: 10.1007/s11051-009-9607-0
Post, C., Sarala, R., Gatrell, C., and Prescott, J. E. (2020). Advancing theory with review articles. J. Manag. Stud. 57, 351–376. doi: 10.1111/joms.12549
Pranckutė, R. (2021). Web of science (WoS) and Scopus: the titans of bibliographic information in Today’s academic world. Publica 9:12. doi: 10.3390/publications9010012
Priem, J., Piwowar, H., and Orr, R. (2022). OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts. ArXiv:abs/2205.01833. doi: 10.48550/arXiv.2205.01833
Rafols, I., Porter, A. L., and Leydesdorff, L. (2010). Science overlay maps: a new tool for research policy and library management. J. Am. Soc. Inf. Sci. Technol. 61, 1871–1887. doi: 10.1002/asi.21368
Rotolo, D., Hicks, D., and Martin, B. R. (2015). What is an emerging technology? Res. Policy 44, 1827–1843. doi: 10.1016/j.respol.2015.06.006
Rozi, F., Rosmansyah, Y., and Dabarsyah, B. (2019). “A systematic literature review on adaptive gamification: components, methods, and frameworks” in In 2019 international conference on electrical engineering and informatics (ICEEI) (IEEE), 187–190.
Rusch, D. C., and Phelps, A. M. (2020). Existential transformational game design: harnessing the “Psychomagic” of symbolic enactment. Front. Psychol. 11:571522. doi: 10.3389/fpsyg.2020.571522
Ryland Williams, J., Lessard, P. R., Desu, S., Clark, E. M., Bagrow, J. P., Danforth, C. M., et al. (2015). Zipfs law holds for phrases, not words. Sci. Rep. 5, 1–7. doi: 10.1038/srep12209
Schaul, K., Szu Yu, C., and Tiku, N. (2023). Inside the secret list of websites that make AI like ChatGPT sound smart. Washigntong Post. Available at: https://www.washingtonpost.com/technology/interactive/2023/ai-chatbot-learning/ (Accessed October 6, 2023).
Shibata, N., Kajikawa, Y., Mori, J., and Sakata, I. (2010). “Creating an academic and technological landscape of service innovation: an analysis of the citation network” in 2010 IEEE International Conference on Industrial Engineering and Engineering Management (IEEE), 405–409.
Souza, V. S., de Vasconcelos Marques, S. R. B., and Veríssimo, M. (2020). How can gamification contribute to achieve SDGs? J. Hosp. Tour. Technol. 11, 255–276. doi: 10.1108/JHTT-05-2019-0081
Swacha, J. (2021). State of research on gamification in education: a bibliometric survey. Educ. Sci. 11, 1–15. doi: 10.3390/educsci11020069
Tomé-Klock, A. C., Gasparini, I., Pimenta, M. S., and Hamari, J. (2020). Tailored gamification: a review of literature. Int. J. Hum. Comput. Stud. 144:102495. doi: 10.1016/j.ijhcs.2020.102495
Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K. K., Colquhoun, H., Levac, D., et al. (2018). Preferred reporting items for systematic reviews and Meta-analyses extension for scoping reviews (PRISMA-ScR) checklist SECTION. Ann. Intern. Med. 169, 11–12. doi: 10.7326/M18-0850.2
Trinidad, M., Ruiz, M., and Calderon, A. (2021). A bibliometric analysis of gamification research. IEEE Access 9, 46505–46544. doi: 10.1109/ACCESS.2021.3063986
University Libraries (2023). Primary and secondary sources for science. Inf. Lit. Play. 1 Available at: https://library.albany.edu/infolit/resource/prisci
Wasserman, S., and Faust, K. (1994) in Social network analysis: Methods and analysis. I. ed. M. Granovetter (New York: Cambridge University Press)
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., et al. (2021). Ethical and social risks of harm from language models. ArXiv:abs/2112.04359. doi: 10.48550/arXiv.2112.04359
Keywords: scoping review, gamification in education, Inclusive Teaching, literature review analysis, network and content analyses, gamification implementation
Citation: Ruiz-Navas S, Ackaradejraungsri P and Dijk S (2024) Are there literature reviews about gamification to foster Inclusive Teaching? A scoping review of gamification literature reviews. Front. Educ. 9:1306298. doi: 10.3389/feduc.2024.1306298
Edited by:
Jakub Swacha, University of Szczecin, PolandReviewed by:
José Martín Molina-Espinosa, Monterrey Institute of Technology and Higher Education (ITESM), MexicoIngrid Isenhardt, RWTH Aachen University, Germany
Copyright © 2024 Ruiz-Navas, Ackaradejraungsri and Dijk. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Santiago Ruiz-Navas, cnUyMTAxNGRAYXB1LmFjLmpw
†These authors have contributed equally to this work and share first authorship