Design recommendations for XR-based motor rehabilitation exergames at home

Lorenz, Emanuel A.; Bråten Støen, Andreas; Lie Fridheim, Magnus; Alsos, Ole Andreas

doi:10.3389/frvir.2024.1340072

ORIGINAL RESEARCH article

Front. Virtual Real., 22 January 2024

Sec. Virtual Reality in Medicine

Volume 5 - 2024 | https://doi.org/10.3389/frvir.2024.1340072

This article is part of the Research Topic Human-Centered Solutions and Synergies across Robotic and Digital Systems for Rehabilitation View all 13 articles

Design recommendations for XR-based motor rehabilitation exergames at home

Emanuel A. Lorenz¹*

Andreas Bråten Støen²

Magnus Lie Fridheim²

Ole Andreas Alsos²

¹Motion Capture and Visualization Laboratory, Department of Computer Science, Faculty of Information Technology and Electrical Engineering, Norwegian University of Science and Technology, Trondheim, Norway
²Department of Design, Faculty of Architecture and Design, Norwegian University of Science and Technology, Trondheim, Norway

Introduction: Acquired brain injuries pose significant societal and individual challenges worldwide. The adoption of XR technologies presents an opportunity to enhance current rehabilitation procedures. However, a comprehensive understanding of the specific requirements of different user groups in XR-based rehabilitation remains incomplete. Our objective was to identify design recommendations for designers and researchers of XR-based exergames for motor rehabilitation for lower-limb motor recovery at home.

Methods: After initially conducting a mini-literature review and brief market analysis, we used a human-centered design process, interviewing central stakeholders to understand their perspectives and using thematic analysis to identify recurring themes and insights related to XR-based rehabilitation.

Results: The resulting eight key themes for integrating XR-based exergames into acquired brain injuries (ABI) rehabilitation were safety, flexibility, efficacy, usability, technology, motivation, ownership, and social factors.

Conclusion: By addressing technical and user-oriented demands, our resulting design recommendations aid designers in developing meaningful XR-based rehabilitation exercises.

Introduction

Rapid developments in entertainment technologies have made immersive gaming based on extended reality (XR) increasingly accessible and enjoyable for the general public. However, these technologies also present huge opportunities for other domains, such as medical rehabilitation. In the field of rehabilitation, an increasing number of people suffer from acute brain injuries, which pose individual and societal challenges associated with support and treatment. Consequently, there is a strong demand for novel technological solutions. Integrating the widely available XR-based technologies in rehabilitation processes has the potential to facilitate and promote it. However, the individual requirements on XR-based technologies of all involved user groups still need to be better understood and, therefore, need closer examination. Thus, this article explores those individual requirements for developing user-centric XR-exergames in motor rehabilitation.

Injuries to the brain can result in various long-lasting disabilities due to the organ’s complexity. Those disabilities range from indiscernible symptoms, as the brain can compensate for some damage, to a combination of movement, sensory, emotional, and cognitive disabilities (Castor and El Massioui, 2018). Consequently, individuals affected by brain injuries often face significant difficulties performing daily activities independently and may experience social isolation (Demakis, 2007).

Acquired brain injuries (ABIs), including strokes and traumatic brain injuries (TBIs), are prevalent conditions, with a combined 81 million cases occurring each year (Dewan et al., 2018; Lindsay et al., 2019). Besides the individual tragedy ABIs cause, the economic burden on society for TBIs alone is estimated to be US$ 400 billion (as of 2017) globally (Maas et al., 2017), underlining the importance of more cost-effective rehabilitation procedures in the future.

Although stroke and TBIs differ in pathology and population, they share similarities regarding the resulting neurologic disorders and the subsequent rehabilitation procedure. Mainly the injury’s size, location, and severity are determinants of the experienced disabilities (Castor and El Massioui, 2018).

In summary, ABIs are a common and complex pathology with grave consequences for a single individual and a considerable socio-economic impact. Thus, the therapy process for ABI patients to restore lost functionality and reintegrate them into society is of high priority. However, this process is complicated due to the inherent complexity and pathology of the brain.

Central to the therapy of ABIs is the brain’s inherent capability to adapt and reorganize to compensate for some structural damages and regain lost functions. This process is also known as neuroplasticity. In the best case, neuroplastic processes can lead to a spontaneous recovery after an injury (Hatem et al., 2016). Nevertheless, this process requires external assistance and guidance to rehabilitate from related disabilities sufficiently. In traditional rehabilitation, the direct interaction between therapist and patient is indispensable throughout all rehabilitation phases. Despite its importance, this approach becomes economically impracticable with growing patient numbers and a decreasing healthcare workforce. Consequently, there is an ongoing effort to develop novel technologies to relieve therapists and improve the rehabilitation process. However, most research in motor rehabilitation focuses on improving upper limb functionality, and younger age groups with unique needs, capabilities, and interests are often overlooked (Rudberg et al., 2020; Holloway et al., 2022).

Only a few days after receiving ABI, the patient usually starts with intensive rehabilitation at a hospital or other medical facilities for several weeks. The patient is cared for there by a multidisciplinary team of medical professionals (Turner-Stokes, Sykes, and Silber, 2008). Physiotherapists and occupational therapists play a crucial role in the rehabilitation of motor and sensory impairments. Physiotherapists treat fundamental disabilities of movement, balance, and coordination, whereas occupational therapists assist in relearning higher-level task-specific functions (Govender and Kalra, 2007; Studer, 2007).

After regaining basic abilities, the patient is moved to outpatient units to provide regular supervised therapy while living at home. The training is transferred to the patient’s home, where the patients themself are responsible for following the advised training regime (Cullen et al., 2007; Young and Forster, 2007; Maas et al., 2017). The recovery often stagnates during the later stages of the rehabilitation process (sequela stage). Additionally, the training intensity usually decreases as prolonged, frequent supervised training is not economically viable. Besides the absence of motivational support, the patient receives less corrective feedback in this phase, leading to maladaptive neuroplastic changes and potentially reversing previous improvements (Maas et al., 2017).

Exergames in physical rehabilitation are a type of serious game that aims to facilitate motor rehabilitation through physical play, other than pure entertainment. Over the last few years, they have become a valuable tool for rehabilitation, as the automatization of the training relieves healthcare providers and facilitates home rehabilitation. Those games can guide a training exercise for motor rehabilitation and sometimes give feedback on execution quality (Rüth et al., 2023). To do so, the game input must reliably track the patients’ movements and consider the user’s specific needs and goals. Standard tracking devices are camera systems, balance boards (e.g., Wii fit), and accelerometers (e.g., VR headset and controller) (Gómez-Portes et al., 2021; Rüth et al., 2023).

Ongoing research investigates various technologies that can supplement or improve the current rehabilitation process. Especially promising are extended reality (XR) systems, such as virtual reality (VR), augmented reality (AR), and mixed reality (MR), due to their improved usability, accessibility, and ubiquitousness over the last few years. With the help of a head-mounted device (HMD), the users of such systems can run various applications that allow for the experience of a fully immersive virtual world environment. Sensors integrated into the HMD track the user’s head movement and can be supplemented with additional controllers, body trackers, headphones, or other feedback devices (Mathew and Pillai, 2020).

In this study, we aimed to discover how XR-based exergames can be employed for motor rehabilitation and how this can be sustainably incorporated into the rehabilitation ecosystem. This was done using human-centered design (HCD) approaches and methods to uncover the target user’s needs and requirements associated with motor rehabilitation using co-creation. Besides incorporating current research findings on XR-based exergaming for motor rehabilitation and commercial solutions, we interviewed stakeholders, such as subject matter experts, healthcare professionals, and patients. Based on the various knowledge streams, we developed design recommendations that can assist Human-Computer Interaction designers in understanding the respective stakeholders’ needs and in developing future XR-based lower limb rehabilitation applications with a particular focus on in-home treatment.

The article is organized as follows: First, we summarize the results of a brief review of the current state-of-the-art XR-based rehabilitation technology. Next, we present findings and resulting themes from interviews with subject matter experts, therapists, and ABI patients. These findings are presented as a general patient journey and exemplified through a specific patient scenario. We then discuss our findings and present design recommendations before concluding the article.

Materials and methods

Mini-literature review on XR-based rehabilitation technologies

As the first step in this study, we researched existing knowledge on XR-based rehabilitation technologies and frameworks gained through an explorative literature review (Adams et al., 2007) and market analysis. This allowed us to understand and frame the problem using different viewpoints.

A mini-literature review was undertaken, accessing the databases of Google Scholar, PubMed, and Elsevier were searched with the search query in February 2021:

“Extended reality (all denominations) AND (* brain injury OR stroke) AND rehabilitat* AND (exergam* OR *reality OR serious gam*) AND rehabilitation AND lower limb OR (lower limb OR balance OR posture OR gait) AND home AND (rehabilitat* OR train* OR exergam* OR * reality).”

Further, only peer-reviewed English articles published after 2004 were included. For articles related to general rehabilitation, no time threshold was set. Relevant additional literature found during extraction was added, too. After scanning the abstracts, evaluating the full article, removing duplicates, scoping, and systematic reviews, the final selection amounted to 25 publications. We identified four reoccurring themes across the literature—efficacy, motivation, ownership, and technology—and used those to structure the brief review presented below.

Interviews with subject matter experts, therapists, and patients

As part of the study to discover how XR-based exergames can play a part in home rehabilitation for lower limb function and how this can be incorporated into the rehabilitation ecosystem, we interviewed Subject Matter Experts, therapists, and patients. This way, we could identify gaps and differences in the parties’ perceived realities. The questions for patients aimed to capture descriptive experience instead of normative recitals. This approach came with a preconceived motive to obtain emotional responses to challenge perceptions from the therapist interviews. For example, therapists made claims that sometimes contradicted the patients’ experiences. At the same time, the questions sought to uncover experiences with logistical and systemic factors, such as specific interactions with healthcare institutions.

Interviewers

The interviews and subsequent data analysis were carried out by two authors, who were in their final year of graduate studies in industrial design engineering at the Norwegian University of Science and Technology. They received supervision from the other two authors, one of whom was pursuing a Ph.D. in medical technology with a specialization in neurorehabilitation, while the other was an associate professor in design with a focus on human-computer interaction and industrial design.

Interviews with subject matter experts

We separately interviewed three Subject Matter Experts (SME) about the challenges of ABI rehabilitation. One of the interviewees was a researcher in motor rehabilitation post-stroke and a practicing physiotherapist, and two were researchers in exergames for motor rehabilitation and former physiotherapists. All three SMEs had an average research experience in motor rehabilitation and exergaming of 8 years (SD: 3.5 years).

The physiotherapist was interviewed about 1) how to interact with patients, 2) how to measure rehabilitation progress, 3) how home rehabilitation works, and 4) their thoughts about XR in rehabilitation.

The two experts on exergames were interviewed about different topics related to exergames, such as 1) exergames used in rehabilitation, 2) XR used in rehabilitation, 3) how to test exergame prototypes with patients, 4) how to measure progression, and 5) how to design for motivation.

Interviews with therapists

The themes we identified during the literature analysis and interviews with the SMEs formed the basis for creating interview guides for both patients and therapists. These guides helped us explore these themes in depth and gather insights from those involved in rehabilitation.

Three female and two male therapists, who were recruited via rehabilitation centers, were interviewed. Two were occupational therapists, and three were physiotherapists. They had an average experience of 18 years (SD: 8.4 years) working in their fields.

The purpose of the interviews was to get a foundational understanding of how therapists work with patients with ABIs. Form and frequency of feedback and follow-up, goal setting, and communication were the focus of our probes. We were also curious about the challenges therapists encounter in handling logistical and technological aspects of their work. Lastly, the therapists spoke of their experiences and thoughts on XR and exergames and if they were familiar with the related concepts and practices.

Interviews with ABI patients

The themes from the therapist interviews informed our subsequent patient interviews. Our goal during patient interviews was to gain insights into the individuals’ personal experiences and pathways.

We interviewed three male ABI patients. The average time passed since injury was 14 months (SD: 3.9 months), and they were all in their 50 s or 60 s. Two of them were from the same facilities as the therapists, recruited with the help of their care personnel. One ABI patient reached out through an open post published on Facebook. All participants taking part in the study were living in Norway. They had all received an information letter about the details of the study before recruitment. A summary of interview themes and subjects was sent to the patients beforehand for preparation, expanding on information from the initial information letter.

As ABI patients’ cognitive capacity and capability can be limited, the interviews had to be adapted to avoid unnecessary strain. Therefore, an upper time limit of 30 min was set following the advice of therapists.

The patients were interviewed about 1) their injury, 2) rehabilitation details, 3) rehabilitation motivation and challenges, and 4) their thoughts about using technology and XR in rehabilitation.

Research ethics, data protection, and analysis

The study was approved by the Sikt—Norwegian Agency for Shared Services in Education and Research (reference number: 250396) and adhered to the declaration of Helsinki. The participants were informed they could refrain from the study without any consequences. Due to the ongoing COVID-19 pandemic, all the interviews were conducted and recorded in Microsoft Teams, then transcribed and summarized in short-form notes.

The pandemic also constraint the number of included study participants, due to the high evaluation cost, considering the targeted interview groups. Although their number might seem too low to obtain relevant statistical power, based on the mathematical model of Nielsen and Landauer (1993), their number suffices for heuristic evaluation at the given costs.

After conducting the interviews, the following steps were to transfer the recordings to a secure server (NTNU NICE-1) and transcribe them into text for further processing using NVivo. Here, we applied affinity mapping. We performed a comprehensive thematic analysis following the six-step data analysis process model by (Braun and Clarke, 2012): familiarization, coding, generating themes, reviewing themes, defining and naming themes, and writing up. We familiarized ourselves with the interview data through individual readings. We systematically coded key insights from these interviews in NVivo, marking each interview cue. The initial identification and subsequent reviewing of themes was a collaborative effort within the research team, with the aid of digital tools like Miro, as well as physical whiteboards and sticky notes to facilitate the process. We iterated on the clusters to arrive at meaningful, bounded themes, finally naming them at the point where they appeared mutually exclusive. Final theme names were then used to structure our design recommendations.

It is important to note that our approach to analysis differed between the literature review and SME interviews on the one hand and therapist and patient interviews on the other. The first two informed and structured our research, while the latter constituted the primary data collection and thematic analysis phase.

Results

Brief market analysis

At the time of the data collection (June 2021), the commercial XR-based rehabilitation tools listed in Table 1 were available. This shows that the market has reacted to the availability of XR technologies. Still, as two of the five products have already ceased, the difficulty of providing a functioning and desirable product is underlined. We further gave an overview on their respectively targeted patient group (stroke: 5 out 5, TBI:1 out of 5), whether the XR hardware used was commercially available (2 out of 5) or specially built (2 out of 5) for the application if any supervision is needed and the product thus more applicable for clinical applications (2 out of 5), how the delivery of training is designed, what the feasibility, how the feasibility can be assessed and its viability for home use. That information was based on the manufacturer’s publications.

TABLE 1

TABLE 1. Comparison of commercially available XR exergames for motor rehabilitation, considering the three lenses of innovation (IDEO, 2015). *-marked products were still operational during the time of the market analysis (June 2021) but are not anymore as of now.

Resulting themes of the brief literature review

Efficacy

Naturally, the efficacy of new rehabilitation technologies in facilitating the rehabilitation process is highly prioritized. Either as part of conventional therapy or as a standalone therapy for a part of the treatment, it is shown by various studies that XR exergames can improve motor function and aid in executive skill transfer (Thornton et al., 2005; Broeren et al., 2008; Barcala et al., 2013; Levac and Miller, 2013; Lohse et al., 2013; Choi et al., 2014; Morone et al., 2014; Darekar et al., 2015; Song and Cho Park, 2015; Sekhavat and Namani, 2018; Levac et al., 2019; Maggio et al., 2019). However, there is currently insufficient evidence that XR-exergame therapy is superior to conventional treatment.

Motivation

The patients’ motivation to perform a particular treatment is integral to successful rehabilitation (Egglestone et al., 2009; Lange et al., 2012; Lohse et al., 2013; Choi et al., 2014; Nijenhuis et al., 2015). As exergames are highly motivating, they have been applied extensively in similar domains (Lohse et al., 2013). Transferred onto XR exergames, their inherently motivational gaming aspect increases the patient’s motivation (Lohse et al., 2013; Llorens et al., 2015).

Measuring motivations is done differently in literature, either quantitative, based on surveys, or qualitative, based on statements and themes from interview sessions (Lohse et al., 2013; Nijenhuis et al., 2015).

Besides the patients’ individual needs based on the pathology of their ABI, personal preferences, skills, and goals must be considered to motivate them optimally during exergaming. Subramanian et al. (2019) have shown that different age groups respond differently to motivation factors in exergames. Nevertheless, even within the various age groups, differences must be expected and addressed individually (W. Chen, 2020).

Ownership

For XR exergame treatments, supervision by physiotherapists and occupational therapists is still considered necessary. They facilitate the XR training, provide safety during training, and motivate the patient to perform the exercises correctly (Levac and Miller, 2013; Pirovano et al., 2013; O’Neil et al., 2018; Weber et al., 2020).

As for motivational factors, the patient’s individual medical and personal needs must be considered to give the users ownership. However, in research, the exergame training and the study participants are kept homogenous to allow for comparable results. This, however, does not mirror reality.

Frequent adjustment of game parameters and highly individualized games are essential to provide optimal training for various pathologies and adapt to functional improvements throughout rehabilitation.

Technology

Exergame applications are being more widely adopted due to the growing accessibility, user-friendly interfaces, and affordability of commercial XR systems. However, most studies investigating the usability of clinical VR applications focus on non-immersive technologies, like 2D screens, and only a few studies considered head-mounted immersive XR technologies (Tuena et al., 2020). Generally, exergames are effective in enhancing motor outcomes for clinical and home-based motor rehabilitation (Norouzi-Gheidari et al., 2020; Jonsdottir et al., 2021; J. Chen et al., 2022). However, little data is available regarding head-mounted XR applications (Mekbib et al., 2020; Trombetta et al., 2017; J. Chen et al., 2022).

Besides the inherent motivational aspect of gaming, the game and its hardware’s usability determine its adherence. Difficulties in using XR exergames can lead to aggravation, resignation, failure to complete tasks, and adverse physical and psychological effects (Broeren et al., 2008; Larson, 2011; Levac and Miller, 2013).

Therefore, setup, onboarding, safety, usability, and time management must be considered in future studies. The actual feasibility of a therapy system can be validated for real-world environments and applications.

Resulting themes of stakeholder interviews on XR-based rehabilitation

During the thematic analysis of the stakeholder interviews, several themes related to XR-based exergames for ABI rehabilitation emerged. These are presented below. Quotes from interviewees were translated by the authors from Norwegian to English.

Safety

In most interviews, safety was discussed as a prerequisite for rehabilitation activities. In traditional therapy sessions, a supervising therapist intervenes and manually corrects rehabilitation activities for the safety of patients. However, the ability to perform exercises without supervision was deemed critical by SME1, as it helps patients transition from inpatient to outpatient and further recovery stages. SME1 also noted that game designers frequently overlook the limitations of many ABI patients. According to therapists, seemingly simple actions and movements can be risky during the early phases of rehabilitation. Unmonitored exercises are only a viable option once a certain baseline is reached. Both therapists and SME 1 emphasized the importance of games that promote safe exercise, which can help patients maintain steady progress at home. In addition, therapists highlighted the significance of workload management in ensuring safety.

Although therapists provide patients with detailed exercise plans adjusted to their needs and capabilities, they are only sometimes followed. This became clear during the interviews with patients. All the patients admitted to having been overconfident in their abilities at some point. Two patients experienced burnout due to poor workload management; one suffered a significant injury while performing a movement. Safety from physical damage due to falls was a concern of the parties interviewed, and the mental struggles overwhelming training exercises can cause were mentioned.

Flexibility

SME2 was involved in research about the parameters of gameplay and its influence on the quality of training. The presence of a feedback loop to adjust the parameters was considered necessary. This was nothing new for the therapists: Monitoring, instructing, and correcting the patients while they trained was a natural part of the therapist’s feedback loop. SME2 said emerging technologies—machine learning paired with new and better sensor technology—could automatically provide this feedback loop without a therapist’s supervision. However, SME2 reported, as it currently stands, that this technology seemed overly complex in terms of implementation on a user-friendly, commercially viable platform.

They implied that translating this practice into exclusively technological feedback loops would be difficult. Today, they reported, a mixture of qualitative and quantitative measures is used in rehabilitation centers to evaluate execution and progress in real-time and subsequently adapt the training. They often utilize ad hoc methods, such as mirrors and bathroom scales, if they find them suitable for the patient. These methods are highly interactive and create a mutual understanding of the state of rehabilitation.

Nevertheless, they also acknowledged the extensive selection of commercially available games that can supplement the rehabilitation process, even though these games are primarily designed for entertainment rather than rehabilitation purposes.

Efficacy

To ensure successful rehabilitation, it is crucial to recognize that effects and outcomes are highly individualized, as mentioned by SME Physio. He and the therapists further stated that it is essential to craft roadmaps for patients using qualitative and quantitative measurements that can be compared to an indisputable baseline. This is important, not just during the early phases but throughout the whole rehabilitation process. One of the patients also agreed on that point.

SME2 and the therapists said that exergames used in rehabilitation should avoid introducing maladaptive movements and setbacks in progression. Instead, the games should inhibit erroneous execution and implement an adaptive feedback loop to prevent that. In traditional therapy, constant feedback is either given by a therapist or through tools like mirrors. These aids support the progression of functional improvement and motivation, which is a more critical factor for patients than for therapists.

During subsequent home rehabilitation, secondary care by the patient’s family, relatives, and friends is crucial for further progression, but this depends on providing those with appropriate knowledge and tools, as stated by a therapist. Further, the therapist mentioned that therapy should challenge patients at the right level to avoid stagnation due to overexertion or under challenge.

Inefficient therapy progression can lead to mental struggle, and achieving personal goals is not only motivating but also a measure of efficacy for patients, as stated by the patients.

Qualitative and quantitative measurements should constantly monitor the efficacy of the current training and compare with known baselines and the patient’s personal goals. This helps communication between all stakeholders, adapts the training throughout the rehabilitation process, and avoids erroneous movement.

Usability

The interview participants all agreed that technology used for rehabilitation needs to be easy to use before it can be widely adopted. Current solutions need to fit more into the everyday lives of therapists and patients alongside other necessary clinical activities. This means that priorities must be made, and more essential tasks will take precedence over time-consuming and expensive activities.

Therefore, the usability of the solutions was considered crucial. One requirement for technology used in rehabilitation is that there should be little or no barrier to entry. Many therapists emphasized the need for a simple “one-button” operation to use exergame platforms. For therapists, easy onboarding and use were significant factors in reducing the threshold for adopting new technology and using it in rehabilitation.

Another essential feature of technology used in rehabilitation mentioned by the therapists is lightweight hardware. This can increase mobility and flexibility in the physical environment, making the onboarding process less daunting. The ability to move the system around makes it more useful in both clinical and home settings.

While ease of use is essential for therapists, digitization in healthcare was viewed as favorable overall. All participants were familiar with and had used exergaming platforms in their practice. The participant with the most experience emphasized the importance of a seamless setup for successful home use.

Technology

The introduction of digital tools into rehabilitation routines was received with mixed feelings. Although all parties were somewhat familiar with VR solutions and exergames, most therapists had only tried them, not incorporated them into the rehabilitation. Therefore, the patients were not offered any digital tools for their rehabilitation. Here, however, it must be mentioned that one of the therapists was linked to a rehabilitation facility that is nationally leading in the use of VR technologies in rehabilitation. The patients, on the other hand, reported being interested in incorporating such technologies into their training. The therapists, however, were more conservative in adopting new technologies in their already established routines. According to SME1, it is therefore essential to also consider the users’ technical literacy based on experience and age.

According to the therapist, the main hurdles to introducing technologies are the patients’ limited cognitive ability to use such complex systems and the therapists’ limited temporal resources to get familiar with the systems and set them up. However, their experience is based on commercial hardware, like the Nintendo Wii platform, with games not tailored to their specific needs. The therapist, however, noted that commercial games are more enjoyable, as the focus is on the game experience rather than the therapy outcome.

Technical challenges might arise for the game developers based on the specific wishes of the therapists and patients regarding the design of the exergame. Both tend towards open-air landscapes, incorporating free exploration using multiple interaction modalities, like gait, reaching, and grasping. The patients wished for tasks aligned with their goals, like walking, climbing, and fishing.

Another technological challenge could be the incorporation of adaptive real-time feedback. In response to this challenge, SME2 mentioned using machine learning models incorporated into the game to counter the adverse effects of maladaptive training. Still, for most use cases, this might be too excessive.

Motivation

Keeping patients motivated was the primary concern of the therapists. They suggested that games can help motivate patients to regain capabilities related to their hobbies and interests, even if those goals are unrealistic. Additionally, therapists highlighted the effectiveness of using milestones revolving around these activities, such as fishing or climbing, to increase motivation. Frequent goal-setting meetings were crucial for aligning short- and long-term goals.

Further, the therapists recognized that patients often become passive after transitioning to the home environment. They experienced that passivity often leads to plateauing or declining progress, decreasing motivation, and creating a negative feedback loop. To combat this, patients expressed the desire for the gaming system to promote initiation from the user, while therapists emphasized the importance of games promoting independent activity and exercise.

Patients reported that personal relationships and interactions were the strongest motivational drivers, followed by interests and hobbies. Returning to everyday life or unrestricted mobility was also important but to a lesser extent.

Social factors

Social belonging is a critical human need, so it is essential for ABI patients in their sometimes socially isolated state. A big hurdle mentioned by the therapist is that many patients experience aphasia, which makes communication a challenge. According to the therapists, this can lead to misunderstandings during goal setting, which relies on correct self-perception and communication of training exercises.

However, mutual understanding between the two parties in rehabilitation is paramount for progress and promotes motivation and setting realistic goals and safety. Besides disease-related communication, the patients expressed their trust in the therapists but less in the healthcare system. They understood their high dependency on their caregivers, especially during the early stages of recovery. Patients also mentioned that during the institutional rehabilitation, their interaction with peers helped them accept and understand their current situation and motivated them.

After release from rehabilitation institutions, the therapists reported, the patients often find themselves left alone with their rehabilitation process. Additionally, they can end up in social isolation due to loss of mobility. Most patients, however, can receive regular ambulatory therapy, which helps with the rehabilitation process and the feeling of being cared for. Besides the ambulatory teams, the social circle—including relatives and friends—plays a significant role in the progression of rehabilitation at home as secondary caregivers. However, according to the therapists, the caregivers must have the correct tools and knowledge to be sufficient in this role. If not, their support can be counterproductive and frustrating, as they tend to support the patient, thereby inhibiting his rehabilitation process.

The patients themselves see their social circle at home as a huge motivator and use specific interactions with them as personal goals for their progress. As a suggestion for incorporating social factors into an exergame, patients wished for a multiplayer function.

Ownership

The insights emphasize that ownership and inclusion are the primary motivators in the rehabilitation process of the patients. According to the subject matter expert, the patient’s personal needs and goals are a crucial factor. Therapists’ and patients’ co-development of the rehabilitation roadmaps must start with goal setting. Realistic goal setting is essential for sustained progress, as motivation and preventing disappointment depend on it. All interview parties agreed on this point. The patient’s personal goals, which often align with their previous recreational interests, social interactions with family members, or their complete independence, can be taken as the primary goal. This is then broken down into smaller, more realistic steps. However, the goals still must challenge the patient to obtain functional improvement. This might sometimes be hard for families wanting to support the patients and participate in their rehabilitation.

Further, it was mentioned that to communicate the rehabilitation progress to the patient sufficiently, therapists employ various metrics to illustrate it better. However, sufficient communication can be hindered by the patients’ cognitive disabilities and must be considered during goal setting. The patients stated that poor communication can be confusing and frustrating, especially when thrown into new situations.

Patients were very positive about integrating innovative technologies into their rehabilitation process; one has done so already. Therapists are more hesitant about introducing new technologies because they have established rehabilitation methods and are afraid of the time overhead new technologies might entail.

Exemplary patient journey from incident to recovery

We used customer journey mapping to better picture a patient’s journey from the occurrence of the acute brain injury via different institutions until they are back home (Rosenbaum et al., 2017). Customer journey mapping is a way of visualizing the storyline of every engagement a customer has with a service, brand, or product. We used the visualization method differently: to show the patient’s journey through different institutions and her/his contact with different stakeholders and to map the patient’s condition, goals, pain points, thoughts, feelings, and recovery process to each step in the rehabilitation process. We believe that this can help other stakeholders, like HCI developers, to better understand the, at times, unfamiliar characteristics of this specific user group and, thus, our final design recommendations and their application.

Stages of rehabilitation

The different stages of rehabilitation, from the occurrence of injury, till the patient is back home, are presented in Table 2. It shows the patient’s condition, goals, pain points, thoughts, feelings, involved stakeholders, recovery process, and duration.

TABLE 2

TABLE 2. Patient journey based on the insights gained throughout this study.

Patient scenario

Based on the patient interviews and the patient journey presented in Table 2, we developed the following scenario (Table 3) and storyboard (Figure 1). This aids the understanding of the main stakeholders’ needs and pain points at various stages of their recovery.

TABLE 3

TABLE 3. Summary of a plausible patient scenario using the fictive person, Michael.

FIGURE 1

FIGURE 1. Illustration of the described patient scenario and relevant milestones during the patient’s recovery. Created by Marianne Paulsen; reproduced with permission.

In this study, we aimed to discover how XR-based exergames can be employed for motor rehabilitation and how this can be sustainably incorporated into the rehabilitation ecosystem. As our primary sources, we reviewed existing XR-based rehabilitation technology and used interviews with various ABI stakeholders, such as subject matter experts, therapists, and ABI patients.

Discussion

The study’s key findings were eight themes related to XR-based rehabilitation that emerged from the interviews with ABI stakeholders. In the remainder of the discussion, we discuss how these can be transformed into specific design recommendations for developing XR-based rehabilitation games. The literature concerns 2D virtual reality games, which rely on screens. In our study, however, we focus on fully immersive 3D XR-based exergames using head-mounted displays. Some of the design recommendations, however, are similar.

Another important finding of the study is the patient journey. This condensed overview was used to understand the needs of the main stakeholders, not just via a proxy. The overview shows that most of the rehabilitation takes place in the patient’s home without a therapist present. At this stage in the rehabilitation process, the patient misses the help they have become used to and experiences stagnation in their recovery. Motivational rehabilitation tools are, therefore, of high importance for good progress in recovery.

Safety

Concerns about the safety of new technologies were clearly mentioned in the interviews and related literature and are essential for the acceptance of new rehabilitation tools by all involved stakeholders (Jordan and King, 2011). However, only a few studies are concerned with the safety recommendations of XR provided by head-mounted displays (Jordan and King, 2011; Maggio et al., 2019). Unsafe training procedures can lead to adverse effects like pain, fatigue, dizziness, and falls (Morone et al., 2014; Rüth et al., 2023).

Our results indicated that especially the patient’s disabilities and their lack of ability to judge their capabilities lead to increased risks of injury. As this is already challenging for the patients themself, understanding their limitations and identifying potential risks is even more difficult for non-matter experts, such as game developers. The individuality of the patient’s disabilities and their change over time increase the difficulty for the developers even further when creating safe exergames. Therefore, the possibility of initial individualization and ongoing adaptation to the functional capabilities should be a central part of the game mechanics. Such customizations, together with good onboarding, are thought to increase accessibility via safety (Jordan and King, 2011). This can be done by finding the delicate balance between being challenging enough to promote functional improvement and easy enough to avoid injury and demotivation.

Besides the patient’s current stage, the environment for which the exergames are intended is also an important safety factor. On one hand, do XR technologies allow for the training of potentially dangerous real-life situations, like participation in traffic, in the safety of one’s home (Pietrzak et al., 2014). On the other hand, this could also create other risks. Home training and related exergames are often unsupervised and occur in suboptimal environments. The use of physical safety equipment, such as safety harnesses, handrails, or chairs, could be incorporated to promote safety if they do not disrupt other central design recommendations. Besides using safety equipment, Maggio et al. (2019) indicated that familiarity with the hardware and its control is a crucial element that can convey safety without the need for supervision and increase acceptability. Using such additional safety equipment, immersive VR exergames seem to be considered safe, even for home use (Broeren et al., 2008; Darekar et al., 2015; Tuena et al., 2020). So could the little-discussed application of AR add to safety during exergaming by allowing the patient to keep the relation to the physical space around him, which can avoid collision accidents or be used as additional support.

Flexibility

The rehabilitation process for individuals with ABI is highly individual due to the varying nature of the resulting functional deficits and varying personal preferences. The rehabilitation process must be adapted correctly to ensure better safety, motivation, and functional outcomes (Krishnan et al., 2023). As the XR technologies allow for the simulation of a wide variety of real-life environments, the modification of sensory presentations and feedback, and the adaption of task complexity, they can seemingly fulfill those requirements (Pietrzak et al., 2014). To make such an adaptation viable in clinical practice, creating an extensive library of exercises, easily individualizable, modular games, or applying machine learning algorithms automatically adapting to the patient’s capabilities should be considered (Pirovano et al., 2013; Muñoz et al., 2019). The first two suggestions, however, need external intervention to choose the correct exercises, which can be impractical for home training. Machine learning-based approaches could assess the movement during training and adapt specific game parameters and feedback to adjust it to the patient’s changing needs automatically, as suggested by SME and literature (Osgouei et al., 2020; Tharatipyakul and Pongnumkul, 2023). Although this technology has its first applications in fitness training, its feasibility in such a complex field as rehabilitation is yet to be systematically assessed.

Efficacy

For a clinical intervention, its efficacy is central for its application to be considered. As mentioned in the literature review, XR exergames showed us to be able to improve motor function and are currently considered a supplementary tool for therapy.

Clear and continuously updated therapy road maps that align with personal goals are paramount for a good progression of the rehabilitation process. Constant qualitative and quantitative measurements of the patient’s performance aid both the therapist and the patient in understanding and communicating deviations from a general baseline and intervening promptly if necessary. Exergames could provide this by allowing for the personalization of individual goals and adaptable level design that considers continuous performance measurements via integrated sensor systems and reports and automatically adapts to the exercise difficulty (Osgouei et al., 2020). Giving comprehensible performance reports alongside the game can increase the patient’s feeling of ownership over the rehabilitation process. By constantly pushing the current boundaries of the user’s abilities to a reasonable degree, functional improvement is facilitated, but overexertion should be avoided. Especially as patients tend to overestimate their own ability, detecting eventual overtraining and promoting rest periods can prevent injuries and stagnation of the recovery (Cho et al., 2023).

Further, the game should prevent maladaptive movement, which can cause insufficient rehabilitation outcomes or even injuries. A certain degree of freedom in executing the movement is acceptable, as it provides a more enjoyable gaming experience; however, for good progress, compensatory movement should be discouraged if possible. This can be facilitated by intelligent game design or performance measurements, like movement trajectory deviations (Alankus and Kelleher, 2012).

Technology

Limitations in the technology of current systems also fail to address the needs of therapists and patients. For clinicians, introducing technology in general into rehabilitation routines is met with hesitancy. While patients are interested in incorporating novel technologies into their training, therapists are more conservative in adopting them in their established routines. The main hurdles are the limited cognitive ability of some patients to use complex systems and therapists’ limited resources to get familiar with the systems and set them up. However, these experiences were based on games not tailored for rehabilitation. In contrast, like other at-home rehabilitation technologies, XR has the potential to reduce the resource requirements of motor rehabilitation programs. This can be accounted for by reducing time-consuming transportation, especially for remote patients, and facilitating labor-intensive traditional rehabilitation practices (Pietrzak et al., 2014).

Several aspects must be addressed to scale the use of XR systems in rehabilitation, especially in the home setting. The product design should focus on flexibility and ease of use rather than vast functional capabilities. Accessibility should be the priority, which also applies to the user interfaces, as the end-user might have restrictions or may not even consider trying a new technology. Therefore, changes to commercial XR hardware and setup instructions must be considered, as patients might struggle with the correct attachment of the headset, controllers, and other accessories, as done by some currently available solutions (Table 1). Meanwhile, the underlying technology offers new opportunities that could enhance the enjoyment and efficacy of the gameplay itself. Advances in hardware make personalization mechanisms feasible on smaller systems, reduce costs, and increase gameplay fidelity. Improved connectivity creates opportunities for more seamless social gameplay and remote rehabilitation interventions. No- or low-code platforms could also lower the threshold so game development can become more rapid and widespread, as Baldassarre et al. (2021) demonstrated. Lowering this threshold moves us closer to applying co-creation in the game design process and, thus, applicable applications.

Motivation

The use of exergames in rehabilitation has been extensively studied due to their inherent positive motivational aspect (Hung et al., 2016; Pacheco et al., 2020). Motivation is directly related to increased training adherence and, therefore, functional improvement (Maclean et al., 2002; Kil and Son, 2020). Due to the immersive nature of XR applications, they are seen as inherently motivational and, therefore, are investigated for their use in rehabilitation (Maggio et al., 2019).

In relation to this, our results indicate that by considering the patient’s individual needs, preferences, skills, and goals, their motivation can be optimized during rehabilitation. The primary concern of therapists is keeping patients motivated throughout their rehabilitation and leveraging their hobbies and interests as key drivers. Frequent goal setting and adjustment with the patient is a critical component and serves as a reference point that can be used to monitor and display progress, which is a motivating factor. Therefore, personalized virtual environments and tasks that are constantly adjusted to the patient’s current state, as well as feedback on the patient’s progress and goals, should be included in the exergames.

After transitioning to the home environment, patients can often become passive and preoccupied with activities of daily living, as indicated by our interviews and the literature review. The lack of further goal setting and display of progress, a frequent activity in the inpatient phase, can lead to decreased motivation. To counteract this, patients expressed the desire for gaming platforms to promote initiation from the user, while therapists emphasized the importance of games promoting independent activity and exercise. Small nudges that keep the patient active and motivated can be deciding in maintaining progress. Incorporating the key drivers—i.e., fishing, biking, and other recreational activities—used in the inpatient phase could reinforce this.

Social factors

Social belonging and interaction are crucial for the not seldomly isolated patient group, and the importance of incorporating social play was discussed previously (Alankus et al., 2010). However, in the early stages of rehabilitation, simple communication can be challenging due to aphasia, making it difficult for patients to engage socially. As patients progress and become more independent, other complex social components become increasingly important.

Once patients transition to outpatient care, they may experience social isolation. Patients often perceive their social circle at home as a significant source of motivation and support, with family and friends acting as supplementary caregivers in many situations. In cases where ambulant supervision is infrequent or unavailable, the involvement of family and friends becomes even more vital in the rehabilitation process. Providing them with tools to assist the patient’s progress can be beneficial.

Integrating social factors and mechanics into games can be an effective approach. Cooperative gameplay and competitive multiplayer are motivational mechanics in general and specifically in ABI rehabilitation (Sweetser and Wyeth, 2005). Asymmetric gameplay, shared virtual spaces, and in-game communication tools are examples of mechanics that facilitate social interaction in games. Social game mechanics need to align with general social factors to enhance motivation and increase the likelihood of user engagement and participation.

Ownership

Our research shows that the rehabilitation journey in the aftermath of an ABI involves feeling a loss of agency. This loss is perceived but also literal, as the patient often suffers significant motor and cognitive impairments. The paths of the interviewees, from the hospital to rehabilitation facilities and eventually back home, revealed a narrative of disorganized logistical practices. There was an apparent disconnect between the desires, recommendations, and actual decisions made, causing confusion and disorientation. While the experiences at the rehabilitation facilities were generally positive and productive, the lack of predictability in the process left the patients feeling a diminished sense of control and agency.

Interestingly, the most remarkable accomplishments in terms of logistics were a result of the patients’ independent decision-making. This suggests they could navigate the rehabilitation journey more effectively when given the chance to make independent choices.

Ownership of the activities in the rehabilitation process enhances the sense of agency and engagement, which can significantly contribute to its efficacy (Maier et al., 2019). When patients feel a sense of ownership, they are more likely to be motivated, actively participate, and take responsibility for their recovery.

In a gaming context, fostering ownership is key to patient rehabilitation. It is crucial to give patients a sense of control over their experience and insights into their rehabilitation progress. Thus, by providing in-game customization options, such as adjusting the general game theme and, to some extent, exercises based on their interests and goals, as well as giving understandable metrics on their performance and progression, agency in decision-making can be promoted. Tailoring the game experience to their preferences can increase their engagement and motivation to participate regularly. It is important to note that transparency is paramount in implementing changes like level difficulty and performance feedback to ensure patients experience ownership and avoid confusion and frustration. When done correctly, this feedback loop can reinforce efficacy and progress (Schmid et al., 2016).

Design recommendations

We propose several design recommendations for XR-based rehabilitation games based on the emerging themes, the patient journey, and the discussion above. The design recommendations are presented in Table 4, and their source and rationale are indicated for reference. These design recommendations can guide designers and developers of XR-based rehabilitation systems. While certain themes are universally applicable for all exergames in motor rehabilitation, we argue that certain themes carry greater significance when applied to XR-based rehabilitation games compared to other exergame applications. Specifically, we want to highlight five key themes: safety, usability, efficacy, technology, motivation, and ownership, which diverge in some respects from non-XR-based exergames.

TABLE 4

TABLE 4. Proposed design recommendations for XR-based rehabilitation games. The source and rationale behind each design recommendation are indicated.

Foremost among the considerations of the involved stakeholders is the safety aspect of XR-based systems, as they most notably can impede spatial orientation and increase the risk of injuries, which might be reduced by using AR environments instead of fully immersive VR environments. For all stakeholders involved, safety was of the utmost importance and was pivotal in trying any new solutions. The significance of safety is further emphasized, that besides proving clinical efficacy, new medical applications must prove their safety by complying with national and international regulations (e.g., Directive 2007/47/EC of the European Parliament 2007).

Although the usability of XR technologies has increased significantly over the last few years, they still require a more elaborate and unfamiliar setup than other exergame systems, such as screen-based counterparts. Thus, assistance in setup, calibration, automatic error handling, and general onboarding is essential to appeal to patients and therapists.

Moreover, the XR system offers additional sensors, including position sensors and even hand-tracking capabilities, that can be leveraged to provide realistic virtual interactions and fine-grained performance assessments, increasing clinical relevance.

The realism and customization capabilities inherent in XR technologies enable an elevated degree of personalization within environments tailored to individual users, which the end users specifically emphasized. This, in turn, cultivates heightened adherence and motivation.

Other frameworks and design guidelines have been proposed previously and highlight similar themes crucial for the successful development of exergames. While the majority of these frameworks predominantly target 2D clinical exergames, specific themes are transferable to XR at-home exergames. Further, most frameworks concentrate on adaptation and individualization strategies tailored to the diverse end-user group. This is either achieved through prior intelligent game design that incorporates the user’s capabilities into account or through automatic adaptation algorithms (Hardy et al., 2015; Pirovano et al., 2016; Tadayon et al., 2020). In contrast, the framework by García-Martínez (2015) prioritizes the creation of ownership by providing appropriate feedback on performance and results. Additional recurrent themes were motivation, appropriate use of sensor technologies, the importance of social interactions, the efficacy of the digital intervention, and the sustainability of a solution (García-Martínez et al., 2015; Hardy et al., 2015; Pirovano et al., 2016; Li et al., 2020; Tadayon et al., 2020). Notably, only the framework proposed by Li et al. (2020) focuses explicitly on using head-mounted VR, guiding the design, development, and evaluation of exergames for health in a broader context.

Compared to the previously proposed guidelines, our design recommendations are an important tool for any game developer who wants to understand the central stakeholders beyond mere clinical requirements. Our recommendations extend the scope by prioritizing direct stakeholder interaction to uncover emotional needs. Moreover, by delivering the design recommendations in a brief manner with relevant contextual information, we ensure their accessibility to non-subject matter experts, including game designers or other stakeholders without a clinical background. It is important to note that while our approach facilitates the initial understanding and design phases, it does not replace the necessity for iterative co-design workshops with the central stakeholder. It enhances and accelerates the planning and execution, leading to a more comprehensive, efficient, and sustainable development journey and desirable games for the patients.

Limitations

Several limitations of this study must be acknowledged. Firstly, due to the restrictions encountered during the COVID-19 pandemic, we could only recruit a limited number of interviewees, all of whom were reached exclusively online. While this approach ensured adherence to health guidelines and timely completion of the interviews, it inevitably narrowed the diversity of perspectives included in our research. Furthermore, it is essential to note that specific stakeholders, like family and friends of patients, game developers, and municipalities, who could have contributed different perspectives, were not included in our study. The absence of these perspectives may result in an incomplete picture of the subject matter.

Secondly, our research is geographically limited to Norway. Although this localized focus seems disadvantageous as it limits the generalizability of our findings to a global audience, this can also be seen as the opposite. The potential to highlight the impact of remoteness of the end-user can aid in understanding the challenges of in-home rehabilitation.

Lastly, it is essential to emphasize that further validation and verification of these findings is advised to ensure the robustness and reliability of our research outcomes and aid in a potential framework for designing in-home XR-based rehabilitation games.

Conclusion

Based on contemporary research, existing commercial solutions, and interviews with subject matter experts, healthcare professionals, and patients, we found several important considerations for home-based rehabilitation exergames for lower-limb motor rehabilitation. Guided by the principles of human-centered design (HCD), our objective was to uncover the needs and prerequisites of individuals undergoing motor rehabilitation and turn these into design recommendations for game developers.

In summary, our recommendations address both system-oriented demands, encompassing sensor fidelity, usability, flexibility, and efficacy, and human-centered considerations, including user engagement and motivation, satisfaction, social interaction, patient participation, and ergonomics. Although existing technologies meet the requirements for their use in motor rehabilitation, market players struggle to seamlessly integrate their solutions with the patient’s needs at various stages. Notably, challenges emerge regarding flexibility, usability, patient participation, and social factor integration. Further, XR-based exergames especially lag behind their traditional non-immersive counterparts. These solutions exhibit less certainty in quality, usability, and efficacy, raising concerns about potential adverse effects on motor rehabilitation.

Future research must consider additional stakeholders, notably those responsible for infrastructural facets, such as management and developers. In addition to our current findings, such collaboration is essential to create and evaluate a robust, developer-friendly framework facilitating the development of utilitarian home-based rehabilitation games for lower-limb neurorehabilitation.

Data availability statement

The datasets presented in this article are not readily available because the local data protection office does not allow for the publication of interview recordings. Requests to access the datasets should be directed to emanuel.a.lorenz@ntnu.no.

Ethics statement

The studies involving humans were approved by the Sikt—Norwegian Agency for Shared Services in Education and Research (Reference number: 250396). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

EL: Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing–original draft, Writing–review and editing. AB: Data curation, Formal Analysis, Investigation, Methodology, Writing–original draft, Writing–review and editing, Conceptualization. ML: Data curation, Formal Analysis, Investigation, Methodology, Visualization, Writing–original draft. OA: Supervision, Visualization, Writing–review and editing, Conceptualization, Project administration.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This research was conducted with the support of the Department of Computer Science at NTNU through the Strategic Research Areas funding scheme.

Acknowledgments

We would like to thank all interview partners involved for their time, energy, and valuable insights.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Adams, J., Khan, H. T. A., Raeside, R., and White, D. (2007). Research methods for graduate business and social science students. New Delhi, India: Response Books. doi:10.4135/9788132108498