Exploring the use of gamification in human-centered agile-based requirements engineering

Gamification has become prevalent in educational settings, particularly in human-centered software engineering. Using gamified learning environments for comprehending software engineering concepts can significantly enhance students’ human factors, including motivation, engagement, and cognitive and critical thinking skills. Software requirements engineering constitutes one of the disciplines in the software development lifecycle. Based on the literature concerning agile-based software requirements engineering, coping with changing user requirements and constant alterations is often challenging. Consequently, both students and software developers must thoroughly grasp agile-based software requirements engineering. Over the last few years, educators and researchers have crafted gamified learning environments for agile-based software requirements engineering activities. Therefore, this paper aims to investigate the effectiveness of agile-based software requirements engineering activities through a gamified design approach. For this purpose, an application, ReGile, was developed and evaluated with 87 second and third-year software engineering students using the M-3 evaluation framework. Overall, the results were promising. Students expressed satisfaction with ReGile’s technological usability. They performed well (72.14%) in answering questions related to software requirements engineering and agile methodology concepts. Furthermore, gender-wise results indicated that male students outperformed female students when using ReGile.

1 Introduction

Gamification has emerged as a successful pedagogical approach for enhancing students’ cognitive skills (Ahmad et al., 2023; Dong et al., 2024), motivation, and engagement (De Porto et al., 2021) in software engineering education (Hermanto et al., 2018; John and Fertig, 2022). Gamification involves integrating game design elements into non-gaming contexts (Deterding et al., 2011) and is becoming increasingly prevalent across various learning domains. It offers four key benefits for students (Hainey et al., 2016): (i) aiding in the acquisition and comprehension of domain knowledge, (ii) fostering the development of cognitive and problem-solving skills, (iii) enhancing motivation and engagement, and (iv) prompting behavioral changes. By incorporating game design elements into a gamified environment, activities become more interactive and enjoyable, potentially increasing engagement in non-gaming settings such as the workplace, classroom, or software and web applications (Garcia et al., 2019). The psychological foundation of gamification lies in self-determination theory (John and Fertig, 2022). According to this theory, learners are motivated when they encounter particular basic needs within a game, including competence, social inclusion, and autonomy (John and Fertig, 2022).

In literature focusing on gamification’s educational aspects, two distinct types emerge: structural and content gamification (Cavalcanti et al., 2018). Structural gamification entails offering rewards, such as badges and points, to students (players) rather than modifying the content. It aims to boost motivation within a course (content) by providing feedback and rewards, thereby gamifying the structure surrounding the content. On the other hand, content gamification involves directly altering and gamifying the content to resemble a game. Various approaches discussed in the literature encourage role-playing for heightened engagement through content gamification design, including designing simulations, crafting narratives, and developing characters (Cavalcanti et al., 2018).

Teaching agile methodologies has become a focal point in software engineering education due to the widespread adoption of agile methods over traditional ones in the software industry (Scott and Campo, 2023). In recent years, educators and researchers have increasingly emphasized the design of gamified, agile-based interactive activities to engage students in learning software engineering (De Porto et al., 2021; John and Fertig, 2022). This emphasis arises from recognizing that, at higher education levels, it is imperative to equip software engineering students with an understanding of common software engineering issues and meaningful learning experiences regarding agile methods used in industry practices (Scott and Campo, 2023). Consequently, there’s a growing body of evidence in the literature utilizing gamification within the software engineering domain to facilitate students in gaining meaningful and engaging learning experiences about agile methods (Rodríguez et al., 2021; Silva et al., 2023).

Software requirements engineering is a crucial step in the software development life cycle, facilitating the design of system requirements documents for knowledge sharing among stakeholders (De and Qusef, 2010). Further, functional and non-functional requirements can use the description of business proceses and, in turn, are a source of data for any system design (Górski, 2020, 2021). However, in agile-based projects, requirements often rely on informally acquired individual knowledge and skills rather than following a formalized mechanism (Alhazmi and Huang, 2020). Moreover, the dynamic nature of changing customer requirements has intensified challenges among developers (Alhazmi and Huang, 2020). Consequently, learning about agile-based software requirements engineering has become increasingly necessary and significant in today’s software industry. However, despite its importance, there has been relatively little focus on agile-based requirements engineering using a gamified approach (Garcia et al., 2019; Silva et al., 2023). Therefore, there is a pressing need to explore environments that provide agile-based requirements engineering activities through a gamified design approach.

To address this need, we have created ReGile, a gamified agile-based requirements engineering application, to facilitate requirements engineering activities using content gamification design. With ReGile, students can receive multiple tasks from various stakeholders and engage in the requirement process steps within a gamified environment, thus learning about software requirement engineering processes. To evaluate its effectiveness, we employed the M3 evaluation framework (Vavoula et al., 2009). The ensuing results and analyses are detailed later in this paper.

2 Related work

In the literature, gamified design in software engineering activities has been shown to enhance participants’ engagement and motivation in performing requirement engineering activities and improve agile-based processes (De Porto et al., 2021). Several applications have been developed to facilitate requirement engineering activities, as outlined in Table 1. REfine (Snijders et al., 2015) is an online gamified application designed to support crowd-centric requirement engineering, incorporating two requirement activities, elicitation and analysis, to engage stakeholders in refining requirements. This application adopts a structural gamification design, targeting six game elements, including roles, resources and points, leaderboards, group forming, exploration, and endorsements. Similarly, iThink (Tebib, 2019) adopts a design science approach to develop gamified industrial requirement engineering activities, also employing a structural gamification design. IThink was evaluated through interviews with software professionals to assess the effectiveness of the gamified environment.

Table 1

Table 1. Relevant gamified requirements applications.

Similarly, the structural gamification design has been applied in other gamified applications, including DMGame (Kifetew et al., 2017), GARUSO (Kolpondinos and Glinz, 2020), and MAF (Piras et al., 2017). In DMGame (Kifetew et al., 2017), software project design decision-makers collaborate to prioritize software requirements. GARUSO (Kolpondinos and Glinz, 2020) is another gamified platform where stakeholders collaborate to support elicitation and analysis processes in software requirement engineering. This research investigates suitable online channels to identify external stakeholders from the organizational reach. Furthermore, this platform utilizes gamification to motivate different stakeholders to participate voluntarily. MAF (Piras et al., 2017) is a research study focusing on organizational studies, wherein University professors engage in a gamified environment to support elicitation and analysis processes in software requirement engineering.

A study examined the effectiveness of a gamified environment, RE-PROVO, to foster participants’ engagement in designing business requirements for legacy system replacements (Alexandrova and Rapanotti, 2020). In this research, participants from various government agencies engaged in requirement engineering activities, including elicitation, analysis, and management. In another study, BakeRE (Delen et al., 2019) is an educational game where University students participated in creating, analyzing, and prioritizing user stories during a software requirements engineering course at Utrecht University. Set in a bakery, students solved cake-related puzzles, following elicitation and analysis processes within the game. Similarly, Biyubi (Garcia et al., 2019) is an educational game where undergraduate computer engineering students at NovaUniversitas University, Mexico, learn about the software requirement elicitation process in a software engineering course. Played in the university’s library, students conducted interviews with stakeholders to elicit requirements. All these applications employed the content gamification design.

However, relatively limited research has been conducted using a gamified design approach on human-centered agile-based requirement engineering activities. In one study (Yilmaz and O’connor, 2016), software development practitioners participated in a technology research center at the Middle East Technical University, Türkiye. This study comprised two phases: transitioning from Scrum to Scrumban and implementing a gamified framework to enhance the software development process. These phases aimed to improve engagement and motivation among software development practitioners during the requirement analysis process. In another example, PRIUS (Silva et al., 2023) is an online gamified application where students from the University of Pernambuco and the Federal University of Pernambuco, Brazil, participated. This application employed Scrum and XP agile approaches to promote participant interactions in prioritizing user stories. Both of these applications employed a structural gamification design.

The relevant research studies indicate that gamified designs significantly enhance requirement engineering activities (Garcia et al., 2019; Kolpondinos and Glinz, 2020; Silva et al., 2023). Most of these applications employ a structural gamified design, as outlined in Table 1. However, there are only a few studies on content-gamified design for requirement engineering activities (Delen et al., 2019; Garcia et al., 2019; Alexandrova and Rapanotti, 2020). Moreover, there is a limited number of studies on the utilization of agile-based approaches in requirement engineering activities through gamified environments (Yilmaz and O’connor, 2016; Silva et al., 2023). To our knowledge, no prior studies have examined content-gamified design for agile-based requirement engineering activities. This gap presents an opportunity for us to explore a gamified learning environment for human-centered software requirement engineering activities in this research.

3 Research aim and research questions

The research aims to develop and evaluate the ReGile application using a content-gamified design approach for conducting human-centered agile-based requirement engineering activities. To achieve this aim, we address three research questions (RQs).

RQ1: How the ReGile application be developed using content-gamified design approach to facilitate human-centered agile-based requirement engineering activities?

In content gamification design, content is transformed into a game-like format, often achieved through designing simulations and creating narratives and characters to promote role-playing and enhance interactivity. To address research question 1 (RQ1), ReGile is developed as a 3D simulation multiplayer application following a content gamification design, where characters facilitate role-playing using narratives within a gamified learning environment. ReGile comprises three main components: requirement engineering process, agile method, and gamification, adapted from existing studies by Garcia et al. (2019) and Silva et al. (2023), as depicted in Figure 1 (see section 4. ReGile: an agile-based gamified requirement engineering application).

Figure 1

Figure 1. Components of ReGile [adapted from Garcia et al. (2019) and Silva et al. (2023)].

RQ2: How does the ReGile application enhance students’ learning experience through its gamified design approach?

The technological usability of an application is crucial in enhancing students’ learning experience when utilizing a gamified design approach. In this context, students evaluated the technological usability of the ReGile application. To address research question 2 (RQ2), an experiment was conducted with university students to assess the technological usability of ReGile. The Micro level of the M3 evaluation framework (Vavoula et al., 2009) was explored through a questionnaire administered after performing agile-based requirement engineering activities using ReGile, as detailed in section 6.1. Micro level evaluation.

RQ3: How effective is ReGile in assisting students to learn about software requirement engineering processes and agile methods through gamification?

To address research question 3 (RQ3), the effectiveness of the ReGile application was evaluated through an experiment involving university students. This evaluation utilized the Meso level of the M3 evaluation framework (Vavoula et al., 2009). At this level, learning scores attained by students during agile-based software requirement engineering activities were calculated, as outlined in section 6.2.1. Students’ learning achievements. Additionally, a gender-wise comparison was conducted to discern differences in learning achievements among all genders. Furthermore, students’ perceptions were explored through semi-structured interviews, focusing on three game behavioral aspects, as detailed in Section 6.2.2. Students’ perceptions. This research question aims to determine the overall effectiveness of the ReGile application in terms of learning achievements and students’ perceptions of using such a gamified application.

4 ReGile: an agile-based gamified requirement engineering application

ReGile is a 3D simulation multiplayer game that visualizes the process of Scrumban, an agile methodology, and the requirement development phase of requirement engineering (RE). Developed on the Unity gaming engine, ReGile also features a multiplayer capability implemented using the PUN 2: Photon Unity Networking package.¹ Additionally, ReGile incorporates a Natural Language Processing (NLP) model, specifically stsb-roberta-base-v2,² which is part of SBERT(Chu et al., 2023), a popular family of deep learning models used for determining semantic similarity between texts. To address research question 1 (RQ1), a content-gamified design approach was employed in developing ReGile. The application comprises three main components: Requirement Engineering Process, Agile Method, and Gamification, adapted from Garcia et al. (2019) and Silva et al. (2023), as illustrated in Figure 1. Further details of these components are provided below.

4.1 Requirement engineering process

This component comprises four phases: Elicitation, Analysis, Specification, and Validation. Each phase contains different tasks and activities the player performs, as shown in Figure 1.

4.1.1 Elicitation phase

In this phase, a player needs to identify the task through stakeholder interviews. Further, a player elicits the requirements from the stakeholders, as shown in Figure 2.

Figure 2

Figure 2. Identify stakeholders for interviews.

4.1.2 Analysis phase

In this phase, the elicited requirements are analyzed, involving the players to classify and prioritize the tasks’ functional requirements. The classification part of requirements includes classifying the requirements into three categories: complete, incomplete, or ambiguous, as depicted in Figure 3. Moreover, the prioritizing task involves assigning a numerical priority to each corresponding requirement according to its importance in the project.

Figure 3

Figure 3. Requirements classification in ReGile.

4.1.3 Specification phase

The players need to specify the non-functional requirements for the project they are working on. These include writing non-functional requirements like Performance, Availability, Reliability, Recoverability, Security, Usability, Data Integrity, and Scalability. The model of SBERT is used to compare the requirements written by the players (i.e., the ideal and accurate requirements of the project) and identifies the similarity of meaning between both. The Use Case and context diagrams are to be drawn by players to represent their project visually, as displayed in Figure 4. Furthermore, the SRS (Software Requirement Specification) is generated at the end of the sprint, as shown in Figure 5. This displays all the work done by players before the present sprint – the requirements elicited, analyzed, and specified together in one document.

Figure 4

Figure 4. Drawing use case diagram in ReGile.

Figure 5

Figure 5. SRS specifying non-functional requirements in ReGile.

4.1.4 Validation phase

The players must go back to the stakeholders to validate the SRS. If the players have a passing score, then the stakeholders will approve the SRS. In contrast, if the passing score is not achieved, the stakeholders disapprove of the SRS and ask the players to replay the game.

4.2 Agile method

This is the second component of the ReGile component diagram, as depicted in Figure 1. In ReGile, Scrumban activities are performed within a fixed period in which an amount of work must be completed, known as a Sprint (Hermanto et al., 2018). The basic workflow of scrumban activities in ReGile is mentioned in Figure 6. Each sprint has four sub-activities namely Sprint Planning, Daily Scrum Meetings, Sprint Review, and Sprint Retrospective. In ReGile, players must traverse all four Sprint activities for software engineering requirements activities. In the sprint planning sub-activity, players are required to decide which work items are supposed to be completed in the upcoming sprint and how the work is to be achieved. In the daily scrum meeting activity, players are involved as attendees to state which work items they worked on yesterday, which are the ones they are doing today, and whether they are facing any obstacles in performing those tasks, as shown in Figure 7. The other two activities are related to review meetings. A Sprint review meeting is held at the end of each sprint with the stakeholders and getting feedback from the stakeholders. However, the sprint retrospective is an internal review meeting among the team members.

Figure 6

Figure 6. Workflow of scrumban activities in ReGile.

Figure 7

Figure 7. Daily scrum meeting in ReGile.

There are a few other elements in the agile method component, as mentioned in Figure 1. Product Backlog (work items required to be completed in an overall developed solution), Sprint Backlog (work items needed to be completed in a sprint), and Kanban Board (shows the progress of the tasks or scores achieved by a player in each sprint), as shown in Figure 8. A Kanban board is used to display the progress of a player throughout the game. Further, it displays all the tasks that need to be completed, the ongoing ones, and those that have been completed in a Sprint. The board visualizes Sprint’s progress and the workload immediately to the players.

Figure 8

Figure 8. Kanban board in ReGile.

Three prominent roles engage in the Scrumban activities (Silva et al., 2023): The Scrum Master (the one who facilitates the Scrum team in the Scrum process), the Product Owner (an individual who works with the Scrum team to prioritize the tasks), and the Scrum Team (software developers, testers, and architects who are responsible for working and delivering the solution). All these roles are designed as non-playable characters (NPCs) in the ReGile. The game’s plot involves two requirement engineers (players), as shown in Figure 9. These requirement engineers (players) are working on a Payroll Management System project in which they perform all the associated Sprint activities using the requirement engineering process, as mentioned in Figure 1. This component tracks the player’s score, which increases each time a player successfully completes an activity. Upon reaching the passing score, players are deemed to have successfully finished the game, and their scores are recorded.

Figure 9

Figure 9. Requirement engineers (players) interact with scrum master in ReGile.

4.3 Gamification

This is the final component of ReGile, focusing on the content-gamified approach for performing requirement engineering process activities. It incorporates various game elements including: Score (the player’s score in the ReGile game), Progress Board (a board showcasing the progress made by a player in the sprint), Avatar (the player’s and non-playable character’s graphical representation), Scrumban activities (the tasks performed using the Scrumban method), Requirement Development Activities (the tasks performed in the Requirement Engineering Process), and Software Engineering Artifacts (collecting and reading several documents; SRS, Vision Scope, Data Dictionary, Gantt Chart, and Stakeholder’s profile) throughout the ReGile game.

5 Methodology

In a learning application, evaluation plays a vital role in assessing its effectiveness (Ahmed and Parsons, 2013). To evaluate the ReGile application, we utilized a part of the M3 evaluation framework (Vavoula et al., 2009), encompassing three levels: Micro, Meso, and Macro. The Micro level focuses on exploring the application’s effectiveness and usability, while the Meso level explores learners’ perceptions and achievements regarding underlying learning activities. In this research, we examined both the Micro and Meso levels to analyze the overall effectiveness of the ReGile application, following adaptations from (Ahmed and Parsons, 2013), as depicted in Figure 10.The Macro level investigates the longer impact of such gamified design approach. At this research stage, it is not possible to evaluate students’ learning peformances and their learning experiences for the longer impact (two-three years time).

Figure 10

Figure 10. Use of micro and meso levels of M3 evaluation framework (adapted from Ahmed and Parsons, 2013).

5.1 Experimental design

In the experimental design, ReGile was assessed using the Micro and Meso levels of the M3 evaluation framework (Vavoula et al., 2009). The Micro level focused on investigating the effectiveness of ReGile in facilitating agile-based requirement engineering activities through gamification. This assessment evaluated four usability aspects: learnability, operability, understandability, and attractiveness (Chua and Dyson, 2004). Table 2 provides explanations for each of these aspects.

Table 2

Table 2. Usability aspects for evaluating ‘ReGile’ (adapted from Chua and Dyson, 2004).

At the Meso level evaluation, students (players) were asked to perform the requirements engineering process through the agile-based gamified application, ReGile. The achieved score out of the total score of the players in this application is considered as students’ learning achievements. The achieved score helps determine whether the players can perform the given tasks in this application. Further, students’ perceptions about using the ReGile application were evaluated through a semi-structured interview. Gamified learning for agile-based requirement engineering activities using ReGile is considered the independent variable of this study, while students’ learning achievements and perceptions are the dependent variables. The evaluation of students’ learning achievements and perceptions determines how well each student has learned and enjoyed during human-centered agile-based requirement engineering activities using ReGile.

5.2 Participants

ReGile was evaluated by 87 students, aged 19 to 21, from the Department of Software Engineering at Bahria University Karachi Campus, Pakistan, with 30 males and 57 females, all voluntarily participating. These students, in their second and third year of the Bachelor of Software Engineering program, possessed basic prior knowledge of agile methodologies and software requirement engineering processes. The experimenters described the study’s purpose to the students, who were then invited to participate and complete individual consent forms detailing the research’s objectives and activity instructions.

5.3 Procedure

Initially, course instructors introduced the research team (experimenters) to the students and explained the purpose of the study, followed by basic instructions on playing the game provided by the researchers. The students were then divided into pairs to engage in the game’s multiplayer feature, with four groups of students participating. Each group was allotted 50 min to play the game and complete the assigned tasks. Students’ learning achievement was assessed based on the scores obtained from correctly answering questions within the game. Subsequently, students completed a five-point Likert scale questionnaire to provide feedback on their overall experience, consisting of eight questions related to usability aspects outlined in Table 3. Additionally, students’ perceptions were evaluated through semi-structured interviews focusing on game behavioral aspects (Yang and Chen, 2020).

Table 3

Table 3. Questionnaire Statements for technological usability of ReGile.

6 Results and discussions

In this study, two levels of the M3 evaluation framework (Micro and Meso) were explored through RQ2 and RQ3. The purpose of the evaluation of ReGile was to investigate its technological usability and the use of ReGile to comprehend human-centered agile-based requirement engineering processes in a gamified learning environment. The results and discussions of each research question (RQ2 and RQ3) are below.

6.1 Micro level evaluation

For answering RQ2, the effectiveness of usability of the application ReGile was investigated through the Micro level of the M3 evaluation framework. For this purpose, the questionnaire consisted of eight statements and was given to students once they had completed all the requirement engineering processes using ReGile. These questionnaire statements address various aspects of technological usability (learnability, understandability, operability, attractiveness) as described in Table 2. A five-point Likert scale was employed, ranging from ‘strongly disagree’ (Ahmad et al., 2023) to ‘strongly agree’ (John and Fertig, 2022). The results of the questionnaire responses are presented in Table 3.

Two statements (S1 and S8) were related to the learnability aspects of technological usability. The responses to the statement ‘S1’ were promising. One of the students mentioned that “no doubt, this application is very easy to use.” In another instance, a student highlighted that “… easy game to play and learn about requirement engineering and sprint. I like this.” The students also communicated positively towards statement 8, which asks about the overall gaming and learning experience of ReGile. For example, one of the students responded, “I have enjoyed this game and learning about agile and requirement engineering processes.” In another instance, a student mentioned “a nice application to play with. Through this, I learn phases of the software requirement engineering process.” These responses were also interpreted similarly by a one-sample t-test against the neutral value of 3.00 (t₈₆ = 10.11, p < 0.01 for Q1; t₈₆ = 6.31, p < 0.01 for Q8).

The sub-characteristic (operability) of the ReGile application was evaluated by asking two question statements, ‘S2’ and ‘S5’ as mentioned in Table 3. The results indicated that most students were satisfied with the application’s ability to guide them through player movements and provide sufficient time to complete all assigned tasks promptly. One sample t-test verified these explanations (t₈₆ = 6.62, p < 0.01 for S2; t₈₆ = 5.85, p < 0.01 for S5). A few responses regarding ‘S2’ were “It is easy to move player in a game” and “… moving player in a game is easy and interesting.” In response to ‘S5’, one of the students mentioned, “the given time for completing the tasks was appropriate.” In another instance, a student indicated “a nice application that gives enough time to do requirement activities.” However, one of the negative comments was received as “the time was not given appropriately for the tasks.” Some students might have faced difficulties in completing tasks on time, possibly stemming from a lack of understanding of the rationale behind the application, which was to assist them in learning about requirement engineering phases and agile method concepts.

In responses to S4 and S7, the results of another sub-characteristic, understandability showed that ReGile was perceived as a straightforward application comprehensible by students (t₈₆ = 5.35, p = 0.01 for S4; t₈₆ = 6.31, p < 0.01 for S7). Most of the students gave positive responses to ‘S4’. Some students indicated it as “a quite simple game to understand. I enjoy playing this” and “…easy game to learn about sprint and software requirements.” However, one of the students highlighted that “it is a good application, but I suggest some more interactive characters may be developed through technology (e.g., augmented reality) so that players may enjoy more.” ReGile was designed to provide learning content related to requirement engineering phases in an environment using 3D characters. The suggestion may be to further enhance the game in the future. The responses to S7 were straightforward. For instance, a student commented, “A nice application that helps me to learn about requirement engineering and agile methodology concepts.” The other student described her experience as “an interactive game in which as a player we are getting tasks from other stakeholders (e.g., Scrum Master). This helps me to learn how sprint and requirement engineering processes can work.”

Statements ‘S3’ and ‘S6’ addressed one of the usability aspects, namely, attractiveness. The responses to ‘S3’ showed that most students felt that the application ReGile has an appealing interface (t₈₆ = 7.63, p = 0.01 for S3; t₈₆ = 4.73, p < 0.01 for S6). One student indicated in such a way that “a nice interface to play a game and learn at the same time. I enjoy it.” Another student highlighted, “no doubt, a good interface to learn about requirement engineering concepts and sprint cycle.” On the other hand, the responses to ‘S6’ were marginally good. There were a few students who submitted their reservations on the vision of the interaction between stakeholders and players, such as “an interactive application can be used. When we are communicating with stakeholders (e.g., Scrum Master) in a game. The messages appear in the screen was not visible” and “… audio may be used to communicate with stakeholders (e.g., Scrum Master) during the game instead of using text messages. It will be more interactive to use this application.” These comments suggest that the application poses challenges to students regarding its attractiveness and the interactivity between players and stakeholders. However, new approaches could be implemented in the future to address these concerns and better satisfy students using ReGile.

In summary, the overall responses received from the students were encouraging, with a positive endorsement rate of 75.3%, indicating that ReGile exhibits good technological usability aspects. However, there is room for further improvement, particularly in enhancing usability aspects in general and attractiveness in particular, in future iterations.

6.2 Meso level evaluation

6.2.1 Students’ learning achievements

To address RQ3, the students’ learning achievements were assessed through ReGile activities, including interactions with stakeholders, scrumban activities, and answering questions, as illustrated in Figure 11. The ReGile application presented students with 10 multiple-choice questions (MCQs) regarding the requirement engineering process and agile methods, as outlined in Table 4. Each student was asked two questions about each requirement engineering phase and agile methodology. The application awarded a score of ‘1’ for each correct answer submitted by a student. A student could achieve a maximum 10 score by answering all questions correctly. The subsequent sections present the results of learning achievements at the Meso evaluation level.

Figure 11

Figure 11. Students are conducting activities using ReGile.

Table 4

Table 4. Questions asked while evaluating learning achievements.

According to the overall results, students achieved an average score of 72.14% across all questions posed. Across different question types, students scored more than 70% correct answers, indicating significant performance across all question categories. The highest score (74.13%) was obtained in the Elicitation question category, while the lowest score (70.69%) was recorded in the agile methodology questions category, as shown in Figure 12. Similarly, in other studies employing a gamified approach, students performed strongly in requirement elicitation and prioritization (Garcia et al., 2019; Silva et al., 2023).

Figure 12

Figure 12. Overall learning score of students at each question type.

Comparing these results regarding gender, male students answered more questions correctly than female students, with a success rate of 76% for male students and 70.5% for female students. Male students excelled in the Analysis question type, achieving a high accuracy rate of 83.33%. In contrast, female students demonstrated more robust performance in the Validation question type, with a success rate of 73.68%, as illustrated in Figure 13. Male students performed well in both the Elicitation and Analysis question types. Similarly, male students marginally outperformed female students in the Specification and Agile methodology question types. However, female students outperformed male students in the Validation question type, as depicted in Figure 13.

Figure 13

Figure 13. Gender-wise learning score of students at each question type.

6.2.2 Students’ perceptions

A semi-structured interview was conducted to gather insights into learners’ perceptions of ReGile for addressing RQ3, posing three questions focusing on game behavioral aspects adapted from (Yang and Chen, 2020), as detailed in Table 5. These aspects encompassed game achievement, game socialization, and in-game assistance, aimed at comprehending students’ learning perceptions and overall gaming and learning experiences with ReGile. Additionally, existing literature underscores the significant role of gamification with agile methodology in understanding software development processes (Yilmaz and O’connor, 2016; Silva et al., 2023). The qualitative content analysis method (Hsieh and Shannon, 2005) was employed to analyze students’ responses, involving categorizing textual data into distinct groups (Weber, 1990). This research utilized conventional content analysis, following an inductive process entailing the identification of grouped categories, designing a concept map, and open coding (Elo and Kyngäs, 2008). Researchers repeatedly reviewed the content through open coding to comprehend observed phenomena and generate distinct categories (Andrews, 1997). Subsequently, distinct categories were identified from students’ responses to each question and grouped into broader categories based on game behavior aspects, with frequencies calculated (Ahmed et al., 2019), as outlined in Table 6.

Table 5

Table 5. Questions asked during semi-structured interviews based on game behavior aspects adapted from Yang and Chen (2020).

Table 6

Table 6. Categories and frequencies related to question responses of game behavior aspects.

In the response to question 1, most students (78.1%) indicated positive feelings after using ReGile. According to them, ReGile is an appealing game where players not only easily engage in a game but also learn about requirement engineering phases and agile methodologies. For instance, some students highlighted “a genuinely nice environment to play in. I enjoy while conducting requirement activities” and “I am so happy to play this game as it helps me to learn about sprints.” However, there were very few students who were unhappy with this game (i.e., 10.3%). One of them explains his concern, “I am not feeling satisfied with this game as the movement of characters was difficult.”

Question 2 deals with the game socialization aspect, which investigates the interactions provided by ReGile to learn about requirements for engineering phases and agile methodologies. As per the results indicated in Table 6, three broader categories were identified from students’ responses interactive game, supporting learning game, and challenging environment for interactions. Of 87 students, 65 (74.7%) agreed that ReGile provides an interactive game environment for socializing while conducting requirement engineering activities. 57 (65.5%) students also highlighted that this game offers good learning support. Few responded as “a nice game that provides relevant information about requirement engineering tasks through different stakeholders” and “it is an interactive and easy game to play. I enjoy it.” In other instances, students mentioned, “This game is well supported in terms of providing learning content, interaction with the scrum master and others” and “the communication between stakeholders was nicely presented.” This indicates that most students were satisfied with the aspect of socialization in the game. On the other hand, 17.2% of students highlighted the issues regarding challenges faced while conducting activities using ReGile. One of the concerns was “a nice interface but description about the tasks were not visible. I think audio recording may be provided for the given tasks.” In a similar vein, a student showed his displeasure about ReGile as “not reasonable guidance provided by the application. I face a lot of difficulty in understanding the movement of characters from one place to another.” These few concerns are good suggestions for further improvement of the game.

In Question 3, one of the game behavioral aspects, in-game assistance, was evaluated in which students answered about the guidance provided by the ReGile application. In response to question 3, 64 out of 87 (73.5%) students were positive about the in-game assistance offered by the ReGile application. This is because an interactive environment was given to students where they can communicate with the Scrum Master and other stakeholders to do requirement engineering activities using agile methodology. One of these students indicated that “instructions are given to us on time.” The other student also mentioned it as a “nicely guided game in which messages are helpful.” However, only 12 students (13.8%) were dissatisfied with the in-game assistance. One of them showed her reservations as “no hints provided by the application while interacting with the stakeholders.” This could be attributed to some students not fully understanding the game’s purpose. ReGile is structured around agile methodology, wherein students engage with stakeholders to receive tasks. Consequently, less guidance may be offered to students in this context. Nonetheless, this feedback could be taken into account for future iterations of the ReGile application.

Overall, the qualitative analysis showed that ReGile has encouraging results in all three-game behavioral aspects: game achievement, game socialization, and in-game assistance. The learners’ perceptions of these aspects were significant as all these aspects received more than 73% positive feedback from students.

7 Conclusions and future work

This research demonstrates the application of the gamified platform, ReGile, to educate students on agile-based requirement engineering activities. ReGile not only offers a gamified environment for student engagement but also facilitates learning about human-centered agile-based requirement engineering processes through interactions with various stakeholders. This approach aims to enhance students’ motivation and enthusiasm in acquiring knowledge about software requirements engineering and agile methodology concepts. Additionally, it provides educators, researchers, and software developers with a framework to design gamified environments for teaching software engineering concepts. This study introduces several contributions previously unexplored in the literature, including using a content gamification design for learning human-centered agile-based requirement engineering activities and the practical implementation of such a gamified learning environment.

This research addresses three main questions. Firstly, the development of the ReGile application, answers the first research question. To tackle the second and third questions, ReGile underwent evaluation at two levels of the M3-Evaluation framework: Micro and Meso. For the second question, the technological usability of ReGile was assessed via a questionnaire at the Micro level. The findings were promising, with 75.3% of students acknowledging its considerable technological usability. Regarding the third question, students’ learning achievements and perceptions were examined. In terms of learning achievements, students performed well across all question categories, scoring over 70% in each. Gender-wise analysis indicated that male students generally outperformed female students, particularly excelling in the Elicitation and Analysis categories. However, female students fared better in the Validation category. Additionally, students’ perceptions of game behavioral aspects were evaluated through a semi-structured interview. Results indicated positive responses (over 73%) across all aspects, suggesting an enjoyable learning experience with ReGile. While the overall results are promising, the study has some limitations. ReGile was only evaluated with students from one university, warranting replication with students from other institutes to validate interpretations. Furthermore, the inclusion of more questions per category may yield different results.

ReGile could evolve into a collaborative gamified environment tailored for conducting agile-based requirement engineering activities in the future. Moreover, there’s potential for the application to create a digital immersive setting for such learning experiences. This would offer insights into utilizing modern technologies within the software engineering domain. Additionally, ReGile could adapt by providing questions aligned with students’ knowledge levels, be it basic or advanced. This adaptation could aid researchers in evaluating learners’ achievements and engagement based on their proficiency levels. While this research delved into two levels of the M3 evaluation framework, future endeavors could explore the Macro level to grasp the long-term impact of employing gamified environments for learning agile-based requirement engineering concepts.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by Ethics Committee, Bahria University, Pakistan. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

AF: Conceptualization, Investigation, Software, Writing – original draft. AS: Conceptualization, Investigation, Software, Writing – original draft. SA: Methodology, Supervision, Writing – original draft, Writing – review & editing. BF: Conceptualization, Project administration, Supervision, Writing – original draft. FA: Formal analysis, Methodology, Writing – review & editing. TL: Formal analysis, Validation, Visualization, Writing – review & editing. ZA: Formal analysis, Validation, Visualization, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Acknowledgments

We acknowledge the support of the faculty members of the Department of Software Engineering who spare students to conduct this research. We also acknowledge the support provided by the Head of the Department in conducting this research.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^https://assetstore.unity.com/packages/tools/network/pun-2-free-119922

2. ^https://huggingface.co/sentence-transformers/stsb-roberta-base-v2

References

Ahmad, F., Ahmed, Z., Shaheen, M., Muneeb, S., and Riasat, R. (2023). A pilot study on the evaluation of cognitive abilities’ cluster through game-based intelligent technique. Multimed. Tools Appl. 82, 41323–41341. doi: 10.1007/s11042-023-15100-x

Crossref Full Text | Google Scholar

Ahmed, S., Javaid, S., Niazi, M. F., Alam, A., Ahmad, A., Baig, M. A., et al. (2019). A qualitative analysis of context-aware ubiquitous learning environments using Bluetooth beacons. Technol. Pedagog. Educ. 28, 53–71. doi: 10.1080/1475939X.2018.1557737

Crossref Full Text | Google Scholar

Ahmed, S., and Parsons, D. (2013). Abductive science inquiry using mobile devices in the classroom. Comput. Educ. 63, 62–72. doi: 10.1016/j.compedu.2012.11.017

Crossref Full Text | Google Scholar

Alexandrova, A., and Rapanotti, L. (2020). Requirements analysis gamification in legacy system replacement projects. Requir. Eng. 25, 131–151. doi: 10.1007/s00766-019-00311-2

Crossref Full Text | Google Scholar

Alhazmi, A, and Huang, S. (2020). “Integrating design thinking into scrum framework in the context of requirements engineering management.” in Proceedings of the 2020 3rd international conference on computer science and software engineering. New York, NY, USA: ACM. p. 33–45.

Google Scholar

Andrews, P. (1997). Secondary teachers’ perceptions of computer availability: a qualitative study. J. Inf. Technol. Teach. Educ. 6, 321–338. doi: 10.1080/14759399700200015

Crossref Full Text | Google Scholar

Cavalcanti, C. C., Filatro, A., and Presada, W. A. (2018). Gamification design for tutor education in an online course. Educ. Temática Digi. 20, 887–904. doi: 10.20396/etd.v20i4.8648223

Crossref Full Text | Google Scholar

Chu, Y., Cao, H., Diao, Y., and Lin, H. (2023). Refined SBERT: representing sentence BERT in manifold space. Neurocomputing 555:126453. doi: 10.1016/j.neucom.2023.126453

Crossref Full Text | Google Scholar

Chua, BB, and Dyson, LE. (2004). “Applying the ISO 9126 model to the evaluation of an e-learning system.” in: Atkinson R, McBeath C, Jonas-Dwyer D, Phillips R, editors. Proceedings of the 21st ASCILITE Conference. p. 184–90.

Google Scholar

De, L. A., and Qusef, A. (2010). Requirements engineering in agile software development. J. Emerg. Technol. Web Intel. 2, 212–220. doi: 10.4304/jetwi.2.3.212-220

Crossref Full Text | Google Scholar

Delen, M, Dalpiaz, F, and Cooper, K. (2019). “BakeRE: A serious educational game on the specification and analysis of user stories.” in 2019 IEEE 27th International Requirements Engineering Conference (RE). IEEE; p. 369–374.

Google Scholar

Deterding, S, Dixon, D, Khaled, R, and Nacke, L. (2011). “From game design elements to gamefulness.” in: Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments. New York, NY, USA: ACM. p. 9–15.

Google Scholar

Dong, X., Liang, H., Ding, X., and Zhang, Y. (2024). Enhancing children’s cognitive skills: an experimental study on virtual reality-based gamified educational practices. Educ. Inf. Technol. 29, 7569–7594. doi: 10.1007/s10639-023-12075-y

Crossref Full Text | Google Scholar

Elo, S., and Kyngäs, H. (2008). The qualitative content analysis process. J. Adv. Nurs. 62, 107–115. doi: 10.1111/j.1365-2648.2007.04569.x

PubMed Abstract | Crossref Full Text | Google Scholar

Garcia, I., Pacheco, C., León, A., and Calvo-Manzano, J. A. (2019). Experiences of using a game for improving learning in software requirements elicitation. Comput. Appl. Eng. Educ. 27, 249–265. doi: 10.1002/cae.22072

Crossref Full Text | Google Scholar

Górski, T. (2020). Verification of Architectural Views Model 1+5 Applicability. In: Computer Aided Systems Theory – EUROCAST 2019. EUROCAST 2019. Lecture Notes in Computer Science, Eds. R. Moreno-Díaz, F. Pichler, and A. Quesada-Arencibia Springer, Cham. 499–506.

Google Scholar

Górski, T. (2021). The 1+5 architectural views model in designing Blockchain and IT system integration solutions. Symmetry (Basel). 13:2000. doi: 10.3390/sym13112000

Crossref Full Text | Google Scholar

Hainey, T., Connolly, T. M., Boyle, E. A., Wilson, A., and Razak, A. (2016). A systematic literature review of games-based learning empirical evidence in primary education. Comput. Educ. 102, 202–223. doi: 10.1016/j.compedu.2016.09.001

Crossref Full Text | Google Scholar

Hermanto, S, Kaburuan, ER, and Legowo, N. (2018). “Gamified SCRUM Design in Software Development Projects.” in: 2018 international conference on Orange technologies (ICOT). IEEE. p. 1–8.

Google Scholar

Hsieh, H. F., and Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qual. Health Res. 15, 1277–1288. doi: 10.1177/1049732305276687

PubMed Abstract | Crossref Full Text | Google Scholar

John, I, and Fertig, T. (2022). “Gamification for software engineering students - an experience report.” in: 2022 IEEE Global Engineering Education Conference (EDUCON). IEEE. p. 1942–1947.

Google Scholar

Kifetew, F, Munante, D, Perini, A, Susi, A, Siena, A, and Busetta, P. (2017). “DMGame: a gamified collaborative requirements prioritisation tool.” in: 2017 IEEE 25th International Requirements Engineering Conference (RE). IEEE; p. 468–469.

Google Scholar

Kolpondinos, M. Z., and Glinz, M. (2020). GARUSO: a gamification approach for involving stakeholders outside organizational reach in requirements engineering. Requir. Eng. 25, 185–212. doi: 10.1007/s00766-019-00314-z

Crossref Full Text | Google Scholar

Piras, L, Paja, E, Giorgini, P, Mylopoulos, J, Cuel, R, and Ponte, D. (2017). “Gamification solutions for software acceptance: a comparative study of requirements engineering and organizational behavior techniques.” in: 2017 11th international conference on research challenges in information science (RCIS). IEEE; p. 255–265.

Google Scholar

De Porto, D. P., De, J. G. M., Ferrari, F. C., and SCPF, F. (2021). Initiatives and challenges of using gamification in software engineering: a systematic mapping. J. Syst. Softw. 173:110870. doi: 10.1016/j.jss.2020.110870

Crossref Full Text | Google Scholar

Rodríguez, G., González-Caino, P. C., and Resett, S. (2021). Serious games for teaching agile methods: a review of multivocal literature. Comput. Appl. Eng. Educ. 29, 1931–1949. doi: 10.1002/cae.22430

Crossref Full Text | Google Scholar

Scott, E., and Campo, M. (2023). An adaptive 3D virtual learning environment for training software developers in scrum. Interact. Learn. Environ. 31, 5200–5218. doi: 10.1080/10494820.2021.1999985

Crossref Full Text | Google Scholar

Silva, D, Lencastre, M, Pimentel, J, Castro, J, and Lira, L. (2023). “Applying gamification to prioritize requirements in agile projects.” in Proceedings of the 38th ACM/SIGAPP symposium on applied computing. New York, NY, USA: ACM. p. 1498–1507.

Google Scholar

Snijders, R, Dalpiaz, F, Brinkkemper, S, Hosseini, M, Ali, R, and Ozum, A. (2015). “REfine: a gamified platform for participatory requirements engineering.” in: 2015 IEEE 1st international workshop on crowd-based requirements engineering (CrowdRE). IEEE; p. 1–6.

Google Scholar

Tebib, M. E. (2019). “Gamifying requirements engineering for better practice” in Joint proceedings of REFSQ-2019 workshops, doctoral symposium, live studies track and poster track (Germany: Essen), 1–6.

Google Scholar

Vavoula, G., Sharples, M., Rudman, P., Meek, J., and Lonsdale, P. (2009). Myartspace: design and evaluation of support for learning with multimedia phones between classrooms and museums. Comput. Educ. 53, 286–299. doi: 10.1016/j.compedu.2009.02.007

Crossref Full Text | Google Scholar

Weber, R. (1990). Basic content analysis. Thousand oaks California, United States of America: SAGE Publication.

Google Scholar

Yang, J. C., and Chen, S. Y. (2020). An investigation of game behavior in the context of digital game-based learning: an individual difference perspective. Comput. Hum. Behav. 112:106432. doi: 10.1016/j.chb.2020.106432

Crossref Full Text | Google Scholar

Yilmaz, M., and O’connor, R. V. (2016). A scrumban integrated gamification approach to guide software process improvement: a Turkish case study. Technicki vjesnik 1, 237–245. doi: 10.17559/TV-20140922220409

Crossref Full Text | Google Scholar