Interactions with virtual therapists during motor rehabilitation in immersive virtual environments: a systematic review

Crowe, Stephanie Elena; Yousefi, Mamehgol; Shahri, Bahareh; Piumsomboon, Thammathip; Hoermann, Simon

doi:10.3389/frvir.2024.1284696

SYSTEMATIC REVIEW article

Front. Virtual Real., 05 February 2024

Sec. Virtual Reality in Medicine

Volume 5 - 2024 | https://doi.org/10.3389/frvir.2024.1284696

This article is part of the Research Topic The role of perceptual manipulations of XR in neurological rehabilitation View all 5 articles

Interactions with virtual therapists during motor rehabilitation in immersive virtual environments: a systematic review

Stephanie Elena Crowe*

Mamehgol Yousefi

Bahareh Shahri

Thammathip Piumsomboon

Simon Hoermann

School of Product Design, Faculty of Engineering, University of Canterbury, Christchurch, New Zealand

Introduction: Virtual Reality applications for rehabilitation are increasing in popularity. In traditional as well as virtual rehabilitation, the guidance of an occupational therapist through physical and verbal interaction is often required. Several studies have attempted to implement a virtual therapist or assistant in immersive virtual environments for rehabilitation.

Objective: This research aims to systematically review these studies, understanding the therapist’s role and how they are represented in the virtual environment, how the virtual therapist and patient interact and the experience of users. Our goal is to provide guidance for virtual therapist implementations into fully immersive virtual reality environments.

Method: The researchers systematically reviewed the literature on virtual therapists in immersive virtual environments for motor rehabilitation (protocol prospectively registered with PROSPERO ID: CRD42022357369).

Results and Conclusion: Seven studies were identified, with findings showing that virtual therapists are often human-like avatars that guide patients, requiring them to mimic the therapist’s movements. Visual, haptic, or one-direction audio feedback from the therapist to the patient was provided in five studies. The selected studies were all newly developed custom-made systems, with five systems developed using the same game engine. Studies carrying out user testing utilised several methods to understand user experience, evidencing a positive attitude from therapists and motivated and satisfied patients.

Future Research: The roles of virtual therapists for motor rehabilitation needs to be explored further in rigorous empirical studies to gather data on patient and therapist user experience. Considerable potential was identified in the development of virtual therapists and therapeutic alliance in the virtual environment. Future systems should reduce manual input from therapists and include personalisation and individualised patient feedback.

Systematic Review Registration: [https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=357369], PROSPERO ID [CRD42022357369].

1 Introduction

Motor rehabilitation is an essential part of recovery for many, including the elderly, individuals who have suffered a physical injury, and those with an acquired brain injury (ABI). An ABI can be the result of a physical trauma to the head from an external source such as an accident or assault, or non-traumatic causes, such as stroke. Stroke and traumatic brain injuries (TBI) represent the leading medical conditions requiring rehabilitation (Warlow et al., 2011) and can have disabling effects on someone’s day-to-day life. Effects of an ABI include having difficulties with instrumental activities of daily living such as self-care, employment, and leisure activities. Many of these instrumental activities involve movement, coordination, memory, attention, problem-solving, planning, inhibition, cognitive flexibility, and self-monitoring, all of which can become troublesome for these individuals (De Luca et al., 2018). In a more recent study, these instrumental activities were shown to be among the most common areas requiring support three to 5 years after a brain injury (Tate et al., 2020).

Post-acute rehabilitation has been shown to significantly improve motor and cognitive functioning following an ABI (Cullen et al., 2007), with a variety of activities that assess and treat balance, gait and coordination. However, therapist-guided clinical sessions may be limited or be dependent on the provision of healthcare which is often far below the amount that is recommended for optimal recovery (Stewart et al., 2017). Patients are often required to carry out these tasks repetitively, which has been shown to improve their outcomes (Kwakkel, 2006; Kleim and Jones, 2008; Winstein et al., 2016) and can be especially important for patients with neurological disorders as it allows for implicit learning over time (Ertelt et al., 2007; Garrido et al., 2013). However, a common problem among these individuals is a lack of motivation (Lohse et al., 2014). A key criticism of traditional neurological rehabilitation is that it does not accurately reflect what the patient would be doing outside of the clinic and the cognitive functioning that would entail (Rose and Hasselkus, 1996).

Recent virtual reality (VR) interventions can offer patients an “ecological option to assess and stimulate cognitive functionality” (Hayre et al., 2020) at home. Serious games have also been introduced into rehabilitation to encourage patients and have been shown to increase motivation (Rizzo and Kim, 2005). An extended period of rehabilitation has been proven beneficial for patients, including those who are post-stroke (Ballester et al., 2019) and is often beyond the capacity of rehabilitation services. Therefore, telerehabilitation which allows rehabilitation in the home of the patient has begun to be seen as an option. Additionally, evidence suggests that more intense motor rehabilitation can reduce the length of stay in hospital for patients (Cullen et al., 2007). This is where offering a virtual environment (VE) for patients to interact with others is beneficial.

1.1 Immersive virtual reality in rehabilitation

Advancements in medical technology have resulted in an increase in the use and promotion of computer-assisted interventions in rehabilitation. These often comprise a variety of multisensory hardware, motion capture, biofeedback, augmented reality systems, and VR systems. Broadly, there are three primary groupings of VR simulations based on the level of visual immersion: non-immersive, semi-immersive, and fully immersive. In this review, we focus on fully immersive examples of VR for rehabilitation with users wearing a head-mounted display (HMD). Within a therapy context, VR can be defined as “An advanced form of human–computer interface that allows the user to ‘interact’ with and become ‘immersed’ in a computer-generated environment in a naturalistic fashion” (Laver et al., 2017).

With a recent shift in available rehabilitation tools, games for rehabilitation in virtual reality are now becoming more commonplace in motor rehabilitation. Costa et al. (2019) found that interventions utilising exergames can have a positive effect on motor and cognitive functions of patients. Similarly, Monteiro-Junior et al. (2016) reported that interaction between older people and exergames stimulated brain regions related to cognition and encouraged exercise and physical function improvement resulting in neuroplasticity. In more recent years, research has moved from less immersive VR games that used devices such as the Nintendo Wii (2006) and Microsoft Kinect (‘Microsoft Kinect’, 2010) with a screen, to fully immersive experiences with the patients using a HMD.

An analysis by Rizzo and Kim (2005) found that VR offers a safe training environment for the patient and creates an environment that has enhanced ecological validity, while allowing for real-time feedback and self-guided exploration. Furthermore, a more recent paper by the same first author underlined this with various examples and the prediction that VR will significantly impact future healthcare (Rizzo and Koenig, 2017).

Potential challenges to the use of VR in rehabilitation include cyber-sickness and negative therapists’ attitudes, often identified in the past and partially mitigated in recent developments. For example, the risk of cyber-sickness has been highlighted as a drawback that could restrict a patient’s use of VR. However, much of the risk of cybersickness is substantially reduced when patients use current-generation hardware, as Caserman et al. (2021) showed in their recent study. Therapists’ negative perceptions of VR technology have the potential to impact its prevalence and use in the clinical environment too, with some clinicians experiencing technophobia (Sherrill et al., 2022). Nevertheless, the advancement of VR technology towards being part of our daily lives will eventually and organically reduce this in the future.

1.2 Video games in rehabilitation

The potential of gaming factors to increase motivation during rehabilitation was postulated by Rizzo and Kim (2005); Rizzo and Kim (2017). In particular, digital serious games, designed to engage people in physical activity can be used in motor rehabilitation. Those games are often referred to as ‘exergames’. Oh and Yang (2010) reviewed the use and definition of exergames and defined it as “an experiential activity in which playing an exergame or a video game requires physical exertion or movements that are more than sedentary activities and also include strength, balance, and flexibility activities”. In the context of motor rehabilitation, Pirovano et al. (2016) described exergames as systems that “support all primary and secondary goals defined for an exercise”.

VR exergames have increased in use over the last 10 years, with the acceleration of advances in monitoring devices and sensors to track the users’ movements during gameplay. They have commonly been used in rehabilitation for balance, memory, and improvements in mobility and strength. Numerous studies have found that the use of games during motor rehabilitation increases the motivation of elderly people (Subramanian et al., 2020) and patients with cognitive impairment (Park et al., 2020). Similarly, a recent review by Zhao et al. (2020) found that it is possible to improve physical and cognitive functions such as balance, gait, executive functioning and memory through the use of exergaming for motor rehabilitation.

1.3 Patient and therapist alliance

It has been said that “the ultimate goal of VR-based intervention is to enable patients to become more able to participate in their own real environments in as independent manner as possible” (Weiss et al., 2006). Yet, it has been evidenced that therapists play a vital role in maintaining patients’ engagement and ensuring the accurate execution of exercises (Winstein, 1991; van Vliet and Wulf, 2006).

The relationship between a patient and a therapist is a key element in determining the success of therapy (Rosa and Hasselkus, 1996; Cole and McLean, 2003; Taylor, 2020). This patient and therapist relationship is often referred to as the therapeutic alliance or working alliance and is mostly explored in psychotherapy (Horvath and Symonds, 1991). The therapeutic alliance is believed to be determined by three factors; the patient and therapist’s agreement on tasks, agreement on goals, and the interpersonal bond between them (Bordin, 1979). A systematic review by Ackerman and Hilsenroth (2003) found that by noting past therapy success and attending to the patients’ experience they can encourage therapeutic alliance.

Recent studies on therapeutic alliance have explored motor rehabilitation, (Klonoff et al., 2001; Taccolini Manzoni et al., 2018; Bishop et al., 2021), highlighting similarities in how a therapist guides a patient through difficult changes, and bringing focus to the importance of the therapeutic alliance. A positive therapeutic alliance in motor rehabilitation has been found to foster increased productivity in therapy, with long-term effects, resulting in the patients’ ability to maintain employment post-rehabilitation (Prigatano et al., 1994; Klonoff et al., 2001). Studies from Ackerman and Hilsenroth (2003) and Horvath (2001) highlight the importance of including communication techniques when training therapists to foster the therapeutic alliance.

It is common and important practice for therapists to interact with their patients physically and verbally to provide guidance and feedback. Unfortunately, this is not always possible in remote therapy. Considering the COVID-19 pandemic, Simpson et al. (2021) carried out a review that investigated the therapeutic alliance during remote psychotherapy, finding that research supports this alliance if therapists make adaptations and change etiquette. However, other studies have highlighted that therapists have the perception that video therapy interrupts this important alliance, rating therapeutic alliance lower in video sessions (Reese and Stone, 2005). Hayre et al. (2020) have highlighted this as an upcoming area of research.

Action Observation Therapy can be commonly used during motor rehabilitation, with therapists playing a vital role in performing a movement for the patient to mimic (Mulder, 2007). We can understand from current research on real-world patient and therapist interaction, that therapists often use verbal communication to guide physical exercise (Talvitie and Reunanen, 2002), whilst maintaining sensitivity around feeding back on shortcomings in their performance (Parry, 2005). Moreover, Thomson (2008) highlighted that, in these interactions, it was necessary for there to be a dialogue and negotiation between the patient and therapist, with critical reflection. Similarly, Hunt et al. (2015) stressed that lack of acknowledgement or failure to further explore something the patient said can hinder their rehabilitation process and goal setting.

1.4 Feedback in therapy

As part of this therapeutic alliance, a key element of the therapist’s role is to offer feedback to their patient during rehabilitation. Therapists may attempt to facilitate learning using verbal, physical or visual guidance. Giving feedback that is positive and instant has been shown to influence patient confidence and compliance (Sveistrup et al., 2004; Harris and Reid, 2005), with one study by Wille et al. (2009) highlighting that patient function and engagement can increase with instantaneous feedback. Verbal feedback, when used to encourage patients, has been shown to be effective in supporting patient performance and endurance during tasks (Baltzopoulos et al., 1991). Similarly, Campenella et al. (2000) also found that visual feedback, by showing users progress with bar graphs and torque curves, provided motivation and reinforcement during exercise and rehabilitation, with enhanced performance. Verbal encouragement has also been found to augment the performance of participants during motor endurance tasks (Bickers, 1993). Providing encouragement, feedback about performance, suggesting changes, changing difficulty level, are all examples of feedback for explicit and implicit learning.

Motor learning is often divided into implicit and explicit learning systems, with feedback frequently categorised according to these learning systems. Halsband and Lange (2006) defined explicit learning as the “conscious recollection of previous experience”, and implicit learning as “an unintentional, non-conscious form of learning characterised by behavioural improvement”. Explicit feedback in real-world therapy can be system-generated signals such as alarms or sounds, and it can be communicative acts by the therapist such as verbal instructions, non-verbal demonstrations, and physical touch for guidance. Implicit feedback is provided through motor mimicry, such as during Action Observation Therapy.

Since 2002, studies have begun investigating the effectiveness of feedback in the VE. Like real-world therapeutic feedback tactics, VEs will often implement verbal, visual or tactile feedback (Merians et al., 2002; Reid, 2002). In a more recent study, utilising both VR, a Virtual Therapist (VT) and a gait robot, Hamzeheinejad et al. (2021) found that explicit feedback resulted in improved patient performance. The authors attributed the positive outcome to the synchronisation between the patient and the virtual therapists in the virtual environment.

The way feedback is provided in the environment has been linked with the level of presence a person experiences, i.e., the subjective sensation of a person being there in a virtual place. Schüler et al. (2015) conceptualised presence and offered guidance for the implementation of virtual rehabilitation environments. They attributed three key types of feedback to three dimensions of presence. Focusing on motor rehabilitation after stroke, they linked movement representation with spatial presence, performance feedback with involvement, and context information with realness. Hence, it is important to note the potential impact a VT providing visual and verbal feedback can have on therapeutic alliance and presence in a VE.

1.5 Virtual therapists

Several VR serious games have simulated in-clinic patient-therapist interaction in the VE, allowing therapy sessions to be held in the home setting of the patient. The VT can also be known as a virtual coach, trainer, or guide. VTs are commonly observed as human-like avatars in the VE. According to Ding et al., a “Virtual Coach” refers to a coaching program or device that guides users through tasks to prompt positive behaviour or assist with learning new skills.

Though not included in this review, there are several less visually immersive examples of systems where VTs are integrated into the VE (Morales-Rodriguez and Pavard, 2007; Hoang et al., 2017; Triandafilou et al., 2018; Amrani and Achour, 2019; Yu and Xiong, 2019). These examples include the use of optical tracking devices such as the Microsoft Kinect to track one or both of the therapists and patient’s movement, however, the activities are presented on displays such as TV screens and desktop monitors rather than with a HMD.

1.6 This research

An extensive search was carried out to find review papers on VTs for motor rehabilitation and did not yield any results, hence, to the best of our knowledge, this paper is the first systematic review on this topic. However, we found several reviews in other related fields which address the use of VR in motor rehabilitation (Skjæret et al., 2016; Laver et al., 2017; Elor and Kurniawan, 2020; Tuena et al., 2020; Høeg et al., 2021; Piech and Czernicki, 2021; Xie et al., 2021) and virtual assistants (Islas-Cota et al., 2022; Kyrlitsias and Michael-Grigoriou, 2022). These may be useful for the reader, but we do not include them in this review paper since our focus is on VTs in fully immersive VE for motor rehabilitation.

To date, no systematic review has examined the use of VTs in immersive VR for motor rehabilitation. Evidence suggests that VTs can aid in the current trend of remote telerehabilitation, to support therapists in their ever-increasing workload and to ease pressure on health services whilst offering longer and more consistent periods of rehabilitation to patients. As such, illuminating the use of VTs and the experience of users is essential.

1.7 Research questions

This review focuses on VR use for motor rehabilitation, and the interaction between the VT and the patient in the immersive VE. The aim of this review is to answer these key research questions:

• Do the interactions and activities performed by the therapists align with the key features known to strengthen the therapeutic alliance?

• How are virtual therapists represented and how do they perform rehabilitation in the virtual environment?

• What devices are used to allow for interaction between therapist and patient in the virtual environment?

• How are VTs represented in the VE and what activities do they carry out?

• What is the users’ overall experience when interacting with a virtual representation of a therapist in the virtual environment during rehabilitation?

2 Method

2.1 Protocol

This review paper was synthesised from the literature using a systematic literature review process. The review protocol was developed based on the PRISMA-P framework (Moher et al., 2009) and then registered at the PROSPERO portal (CRD42022357369).

2.2 Search strategy

For this paper, some automated search indexers were used to retrieve records using a search string formulated based on the research questions. Databases were searched by a single researcher.

The databases used were Scopus, Engineering Village, ACM, PubMed, Taylor & Francis, IEEE Xplore, Sage, Cochrane, Frontiers in VR, Google Scholar and GRAY. The preliminary search of databases was undertaken in July 2022 and rerun in February 2023 for the final analysis. A Population, Intervention, Comparison and Outcomes (PICO) framework was used to develop the search strategy using keywords and synonyms of virtual reality, rehabilitation, and virtual therapist.

TITLE-ABS-KEY (vr OR “virtual reality”) AND (rehab* OR therapy OR “virtual rehab*” OR “remote rehab*” OR “telerehab*” OR “tele-rehab*”) AND (“virtual therapist” OR “virtual assistant” OR “virtual agent” OR “virtual coach” OR “virtual trainer” OR “remote therapist*”) AND NOT (psycho*) AND NOT (*phobia).

No date restrictions were considered regarding the year of publication since the use of a VT for motor rehabilitation is relatively new. No language restrictions were applied; however, all search terms were written in English only.

2.2.1 Inclusion criteria

Studies that have developed, designed, or prototyped a system for virtual reality for motor and neuromotor rehabilitation were to be included. They must be peer-reviewed published journal articles published anytime.

These studies could be in any context including hospitals, clinics, and the home. To be included, the virtual reality experience was to be visually fully immersive, requiring users to wear a HMD. The representation of the therapist should be visual. Measures of primary interest were reports of users’ experience, an example of a system that is developed or in the prototype phase, and a visual representation of the therapist in the virtual environment that can interact with the patient user.

2.2.2 Exclusion criteria

Studies were excluded if they were a review article, book chapter, conference poster or workshop. Additionally, they were not to include the use of augmented reality, 3D screens and monitors, mobile phones or tablets as main devices used, studies were included if these were used in addition to VR HMD. Studies not focused on rehabilitation with human participants were excluded. Studies in the areas of psychotherapy, phobia treatment, or speech therapy were not included.

3 Results

3.1 Study selection

Records were downloaded into Rayyan web-based software (Ouzzani et al., 2016), and duplicates were removed. Two reviewers independently screened the titles, abstracts, and full texts of select records. All disagreements between the two reviewers were discussed and a third reviewer resolved any conflicts.

The preliminary search of databases was undertaken in July 2022 and rerun in February 2023 for the final analysis. An overview of the screening and study selection process can be seen in Figure 1. A total of 437 records were found of which seven studies were included for analysis in this review, once they passed the exclusion criteria defined in the research protocol. 212 records were found in the preliminary search, with 13 new reports found when the search was rerun. Of the seven selected studies, six were included in the preliminary search, and one was included in the rerun. The following variables were extracted from the selected studies: 1) participant characteristics, 2) study design, 3) intervention 4) activity, 5) interaction type, 6) visual representation, 7), feedback, 8) user experience, 9) hardware and software, and 10) general outcome.

FIGURE 1

FIGURE 1. Overview of the screening and study selection process.

A total number of seven studies on VTs in motor rehabilitation were found. The selected studies were published between 2018 and 2022, four were sourced through Scopus: Afyouni, Murad and Eina (2020), Stanica et al. (2020), Georgiadis et al. (2021), and Hamzeheinejad et al. (2021). Three were from Google Scholar: Mihajlovic et al. (2018), Moldoveanu et al. (2019) and Sobota et al. (2022).

Details of the selected studies are presented in Table 2, ordered by publication date. To generate this table, one review author extracted data from the included studies, and the second review author double-checked and updated the data based on the source studies if required. Disagreements centred around the suitability of studies that did not have a virtual therapist represented as a human-like avatar, but via alternative methods such as butterflies guiding the patient (Mihajlovic et al., 2018). These disagreements were resolved by discussion with the co-authors.

It should be noted that two of the studies outlined in Table 2 have prior studies, relating to the same system, that appeared in the search. These were Rehabot (Afyouni et al., 2017; Afyouni et al., 2019a; Afyouni et al., 2019b, Afyouni et al. (2020) and TRAVEE (Caraiman et al., 2015; Ferche et al., 2015; Ferche et al., 2017a; Ferche et al., 2017b; Lupu et al., 2017; Ferche et al., 2018; Lupu et al., 2018; Moldoveanu et al., 2019; Petrescu et al., 2020).

3.2 Quality appraisal

The original method for quality assessment, proposed in our protocol published in PROSPERO, was ROB 2 (Sterne et al., 2019). ROB 2 is a revised Cochrane risk of bias assessment tool for assessing the risk of bias and the quality of articles, however, upon performing the quality assessment the authors decided that a more suitable assessment would be the 10-item Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Qualitative Research. This was because the type of studies selected used qualitative methods in exploring the users experience of their systems.

Responses to these questions posed in the JBI Critical Appraisal consisted of “yes (Y)”, “no (N)”, “unclear (U)” or “not applicable (NA)”. The first and second author assessed all studies against this JBI checklist. Disagreements were resolved through discussion and referral to other authors. Studies were not excluded based on the checklist; instead, results were used to identify weaknesses in the literature and considered when synthesising the research. Results are shown in Table 1.

TABLE 1

TABLE 1. JBI critical appraisal.

Q1 (Is there congruity between the stated philosophical perspective and the research methodology?) was answered with “NA” for all studies due to the nature of selected studies, a philosophical approach is not taken in these studies. These are clinical studies, utilising qualitative, and often also quantitative methods, to understand the efficacy of a VR system and the users experience. Regarding studies that did not carry out any user testing for their system, “NA” was used in questions 1, 2, 3, 4, 5, 8, 9 and 10.

3.3 Participant characteristics

Table 2 outlines the key characteristics of the included studies in this review. It states points on participant characteristics such as age range and injury type, study design, intervention, activity carried out, interaction between patient and therapist in the virtual environment, the visual representation of the VT and VE, feedback provided to the patient, the user experience of patient and therapist and the hardware and software used.

TABLE 2

TABLE 2. Characteristics of included studies.

The total number of patients across the seven selected studies was 105. One study included both therapists and patients as participants: Afyouni et al., 2020; four studies involved only patients: Mihajlovic et al.; Moldoveanu et al. (2019); Stanica et al. (2020); Hamzeheinejad et al. (2021) and no studies included only therapist participants. Two studies did not test their system with any users: Georgiadis et al. (2021) and Sobota et al. (2022).

Patient participants varied from healthy individuals to those with varying health conditions, no study involved solely healthy participants, two studies had a mix of healthy participants and those with impairments (Mihajlovic et al., 2018; Stanica et al., 2020) and three studies had solely participants with impairments (Moldoveanu et al., 2019; Afyouni et al., 2020; Hamzeheinejad et al., 2021). Impairments included participants with gait deficits, neurological pathologies including stroke and diabetic neuropathy, and neck pain.

3.4 System activities

The activities in each game varied, however, all of the selected studies were based on the premise of Action Observation Therapy, with the user watching and mimicking the therapists’ actions or movements whilst receiving feedback. The activities varied from walking for gait rehabilitation (Stanica et al., 2020; Hamzeheinejad et al., 2021), upper limb movement for motor control (Moldoveanu et al., 2019; Sobota et al., 2022) and full body movement for motor rehabilitation (Moldoveanu et al., 2019; Georgiadis et al., 2021). One study included gamified elements such as collecting coins and avoiding obstacles (Afyouni et al., 2020). However, the majority were solely based on instruction given by the therapist and the user was to watch and repeat (Moldoveanu et al., 2019; Georgiadis et al., 2021; Hamzeheinejad et al., 2021; Sobota et al., 2022).

Stanica et al. (2020) developed a game that began with a tutorial of mimicking, however, once the patient had progressed with certain movements, they were advanced to playing mini-games including carnival games (hitting targets, ball directing and whack-a-mole), boxing and lower body games (football and dancing). Only one study mentioned personalised input of movement instructions and activities (Georgiadis et al., 2021), with the other selected studies having pre-set movements input, without that specific patient in mind. It should be noted, that in two studies (Afyouni et al., 2020; Stanica et al., 2020) the difficulty level is increased or decreased based on the progress of the patient, i.e., how often they can replicate what the VT is doing or successfully complete the task set to them.

3.5 Therapist representation

3.5.1 Visual appearance

Reviewing the visual appearance includes the body parts’ appearance and clothing. Across six selected studies, VTs are represented as human-like avatars. One study presents the therapist as a blue butterfly that floats through the VE. Three studies show VTs as human-like avatars in plain clothing (Moldoveanu et al., 2019; Afyouni et al., 2020; Hamzeheinejad et al., 2021; Sobota et al., 2022), two studies present therapists in sportswear (Stanica et al., 2020; Georgiadis et al., 2021) and one in medical wear (Stanica et al., 2020). One study has the therapist wearing sunglasses (Moldoveanu et al., 2019), while two studies have the avatar mostly facing away from the patient with the back of their head in view (Afyouni et al., 2020; Hamzeheinejad et al., 2021). Sobota et al. (2022) only show the therapist’s arm. Interestingly, only one study ensured that the patient had a clear view of the joints on the VT (Georgiadis et al., 2021). In Stanica et al. (2020) the VT is presented as a somewhat transparent avatar ahead of the patient user, allowing the user to see through the VT avatar. Other studies presented the VT as non-transparent. None of the selected studies allowed for the personalisation of the therapist in the VE and only one study allowed the therapist to alter the appearance of the patients’ avatar (Moldoveanu et al., 2019).

This patient personalisation allowed for change of gender, age, weight, hair, skin colour, clothes, and hairstyle. In addition, Stanica et al. (2020) mentioned that based on user feedback, it would be beneficial to allow users to personalise their avatar with height and limb dimensions.

It should be noted that patients were also visually represented as human-like in five out of seven studies (Moldoveanu et al., 2019; Afyouni et al., 2020; Stanica et al., 2020; Hamzeheinejad et al., 2021; Sobota et al., 2022), and were not represented at all in two studies (Mihajlovic et al., 2018; Georgiadis et al., 2021). Only one study allowed the user to personalise their own avatar (Moldoveanu et al., 2019).

3.6 Patient and therapist interaction

VTs are often used in the VE to help readjust tempo, to outline mistakes, give feedback to the patient and demonstrate the actions that are to be taken by the patient. In many cases, the VT is at a short distance from the patient in the VE, demonstrating the movements or actions to be completed. The discourse between patient and therapist is only as instructions or feedback sent from the therapist to the patient, these systems are without a dialogue between the two actors. For example, the VT in Sobota et al. (2022) can change the parameters of objects and alter environments, whilst Hamzeheinejad et al. (2021) states that the therapist in their system can alter angles for exercises based on patient goals and abilities. Feedback is shown by way of emoji reactions in Hamzeheinejad et al. (2021), animated butterflies changing when movements are correct in Mihajlovic et al. (2018), and by written feedback in the VE in the study from Afyouni, Murad and Finea (2020). VTs are often used in the VE to help readjust tempo, to outline mistakes, give feedback to the patient and demonstrate the actions that are to be taken by the patient. In all games, the VT and patient do not physically interact in the VE, with the VT frequently at a short distance from the patient in the VE, demonstrating the movements or actions to be completed.

Control of the VE is also one-directional, with the VT in Sobota et al. (2022) able to change the parameters of objects and alter environments, whilst Hamzeheinejad et al. (2021) states that the therapist in their system can alter angles for exercises based on patient goals and abilities.

3.6.1 Therapist input and control

In three studies (Afyouni et al., 2020; Stanica et al., 2020; Georgiadis et al., 2021), the movements of the VT (when shown as an avatar) were pre-recorded utilising different recording devices. The Rehabot system (Afyouni et al., 2020) used a Microsoft Kinect 2 to record the appropriate postures and allowed therapists to assign personalised exercises for each patient. Therapists were able to select the joints to be focused on using a model of a human body anatomy and then utilise the Kinect sensor to record the appropriate postures.

The study from Hamzeheinejad et al. (2021) focussed on gait and required recording of the patient using a Lokomat (‘Lokomat^® - Hocoma’, 2001). Contrary to the aforementioned studies, in this system the therapists’ movements were not recorded to insert into the virtual environment. Instead, they were driven in the system by walk-animation cycles which were adapted for difficulty by the therapist. The difficulty would be varied walking speeds and cycles and changing the environment to increasingly complex ones. Similarly, Mihajlovic et al. (2018) did not require input from therapists via recordings. In this system, the patient was led by a butterfly that is moving in the VE for neck rehabilitation. These butterfly movements were set to guide the patients’ neck rotations in Euler angles, with these angles able to be altered by a therapist to align with patient goals and abilities.

In the study from Sobota et al. (2022), the authors gave a descriptive overview of a product in the initial stages of design and development, hence, there are limited details on therapist input or interaction. It does however explain that control is “owned by the therapist” and that they trigger animations for both their and the patients’ arms in the VE, only after attempting to provide stimulus from Mirror Boxing or a BCI-RAS robotic arm, referred to as the RoboArm. They were able to change parameters such as different virtual objects, and different environments, setting different start and end positions and changing time intervals for patients.

Last, in the TRAVEE system (Moldoveanu et al., 2019) therapists can select which exercises are to be included in any given rehabilitation session via a session definition form with a list of pre-set tasks. However, it is not clear how the VT motions and exercises are captured and stored in the system. In this system, the therapist can also start, pause, stop, or skip exercise sessions and can grade each patient at the end of each session, with session time allocated per patient. In addition, the therapist in this system can change several characteristics of the virtual patient including gender, age, weight, hair and skin colours, clothes, and hairstyle, which the authors claim, “increase the immersion of the patient in the VE” (Moldoveanu et al., 2019).

3.6.2 Data capture and personalisation

Though the therapists’ movements and voices are often pre-recorded in these studies, four systems show therapists able to access a dashboard to view the patients’ progress and patient profile (Moldoveanu et al., 2019; Afyouni et al., 2020; Stanica et al., 2020; Sobota et al., 2022). Profiles included records of progress and demographic information such as height, weight, and condition. Therapists were able to monitor patient progress via autogenerated reports or databases with stored progress data. In all three systems, they used this interface to configure elements, including inputting personalised prescriptions or exercises to be carried out, suggesting update gestures based on performance and specifying which exercises should be carried out based on the patient’s ability and severity of their condition.

3.7 Feedback in the virtual environment

As highlighted in the introduction to this paper, adequate feedback is essential for successful training and rehabilitation. Across the seven selected papers, there were three main types of feedback offered to the patient, these included visual, audio, and haptic feedback. Visual feedback was often presented as playful elements such as emoticons, or gamified elements such as scores. These are repeatedly used to indicate to the patient that their movements match, or do not match those of the VT. Audio feedback from VTs is often recorded by a therapist and gives instruction to the patient and congratulates them if they successfully completed tasks. Last, haptic feedback was presented as vibrations or impulses sent through the hardware to patients, this usually indicated if the patient has matched the position or movements of the VT.

3.7.1 Visual feedback

A key point of visual feedback in the selected studies is that of the patient being represented in the VE as an avatar, with their movements mirrored (Moldoveanu et al., 2019; Afyouni et al., 2020; Stanica et al., 2020; Hamzeheinejad et al., 2021), with one study allowing the user to personalise their own avatar (Moldoveanu et al., 2019) and one study representing the therapist with a butterfly to guide the patients’ gaze (Mihajlovic et al., 2018). Personalisation in these games allowed the user to adapt their gender, age, weight, hair and skin colour, clothes, and hairstyle to help improve their level of immersion. Stanica et al. (2020) mentions in their study that based on user feedback, it would be beneficial to allow users to personalise their avatar with height and limb dimensions. Interestingly, Moldoveanu et al. (2019) utilised brain-computer interface (BCI) technology so that the avatar executes imagined movements by the user.

In an attempt to mimic the traditional feedback of a therapist pinpointing which limb to move, Hamzeheinejad et al. (2021) displayed points of information close to the virtual VTs legs and feet as an indication to the user. In bringing the real world into the VE, Hamzeheinejad et al. (2021) put blue bars in the VE to mimic the real bars that the patient was holding on to. They ensured that during gameplay, therapists could give users visual feedback through social gestures such as a slight turn of the head. They also developed happy and sad emojis to signal to the user that they matched the gait of the VT or not and offered additional explicit feedback as arrows down where the feet of the patient’s avatar would be. Mihajlovic et al. (2018) emitted particles from the butterfly in the VE if the user was correctly following its movements. It continued to emit until the completion of the exercise and the butterfly disappeared.

In one game, visual text was shown to the user when they needed to change a movement to match the VT more accurately. Terms would be presented across the bottom of the screen such as “ElbowRight move up by 101°” (Afyouni et al., 2020). Relatedly, Stanica et al. (2020) found in user testing that there was a need for messages to be displayed on the screen when patient movement is performed correctly. This was to help the patient-user, supporting their lack of proprioception as a result of not being accustomed with the first-person perspective. The system presented by (Sobota et al., 2022) also gave therapists the functionality, via their own dashboard, of displaying short text messages in the VE in real time to allow communication from the therapist to the patient.

One study used gamification to motivate and provide feedback to the user. Afyouni et al. (2020) occupied the VE with attractive or repellent objects such as coins or static objects to avoid, the user can see totals on their dashboard to understand their progress. One study from Georgiadis et al. (2021) did not detail any feedback measures that were given to the user.

3.7.2 Audio feedback

Audio was utilised in two systems to motivate and instruct the user (Stanica et al., 2020; Hamzeheinejad et al., 2021). Hamzeheinejad et al. (2021) utilised physiotherapists pre-recorded auditory walking instructions including “Good! Nice big steps! Keep it going!”, “The force dependency is reduced. You need to try harder”, “Let’s go! Keep moving the feet more forward,” “Push with the forefoot and put the knee to the front and up. Let’s go” to motivate and instruct the patients. These were pre-recorded by a physiotherapist in German. Similarly, Stanica et al. (2020) tasks and instructions were given verbally to the user, but also added music, hints, and jingles. The auditory examples in this study are not clearly exhibited, there is only a mention of it without examples given.

3.7.3 Haptic feedback

Haptic feedback was included in systems in two of the selected studies (Moldoveanu et al., 2019; Stanica et al., 2020). Stanica et al. (2020) provided haptic feedback through impulses from hardware devices including an armband and the game controllers themselves. They mention that there is haptic feedback from the controllers when the user interacts with something and there is an armband to provide feedback on targeted limbs with various levels of intensity.

Similarly, Moldoveanu et al. (2019) utilised hardware to provide haptic feedback that targeted the limbs involved in rehabilitation. Vibrations were applied to certain points on the hand of the user in response to them matching the VTs’ movements. These vibrations were implemented into the system to allow the user to understand if they had performed the execution correctly. In addition, when it detects that a patient is unable to match the OTs exercise movements, it activates a robotic glove for active support. This study found that vibrational stimulation had a positive effect on motor control in patients with lower muscular strength and paraplegia. If no movement was detected, this glove assumes that the user has no movement in their hand and carries our passive range of motion (ROM).

3.8 Virtual environment design

A variety of virtual environments could be seen across the selected studies, including outdoor and nature environments and the inside of a virtual home. Outdoor environments consisted of grassland, forest, stream land, beach, farm, and desert (Hamzeheinejad et al., 2021) and general woodland and a lake (Stanica et al., 2020). Mihajlovic et al. (2018) also utilised a “classic” environment that included a sky and cloud surrounding, however, they did not focus on the nature aspect. Two studies used outdoor and nature-type environments (Stanica et al., 2020; Hamzeheinejad et al., 2021), while Moldoveanu et al. (2019), Georgiadis et al. (2021), and Sobota et al. (2022) used indoor environments.

Moldoveanu et al. (2019), Georgiadis et al. (2021), and Sobota et al. (2022) developed an indoor home-like environment. Mihajlovic et al. (2018) also utilised a realistic environment, replicating a hallway in their faculty building to resemble a place the user was already familiar with. Stanica et al. (2020) offered the user a rehabilitation centre environment. All these studies presented a similar environment to that of the real world, however, Afyouni et al. (2020) provided a game environment that was animated and provided the user with several gamified elements.

3.9 User experience

Only three of the seven selected studies investigated user experience and sought feedback from participants (Mihajlovic et al., 2018; Afyouni et al., 2020; Hamzeheinejad et al., 2021). Two studies asked for user feedback from patients only (Mihajlovic et al., 2018; Hamzeheinejad et al., 2021), and one asked both patients and therapists (Afyouni et al., 2020). Only one study investigated the impact that a virtual therapist has on the patient in the virtual environment (Afyouni et al., 2020), they asked participants to rate on a Likert scale for the statements “the virtual assistant guides me to do the right posture” and “the adaptive virtual assistant adapts posture difficulty, and this makes me feel more confident”. These results showed that the patients felt, on average, positive about the virtual assistant giving guidance, but gave a low average score for feedback regarding their confidence when the virtual assistant adapted the posture difficulty.

These studies used varied methods to gather user feedback, including quantitively measuring patients’ success and efficiency in exercises and qualitatively requesting feedback. All three studies used a combination of quantitative and qualitative methods to capture participant experience, with questionnaires and surveys most commonly used. Qualitative methods included the use of specialized experience evaluation questionnaires with Likert scales (Mihajlovic et al., 2018; Afyouni et al., 2020) the addition of the System Usability Scale Questionnaire (Brooke, 1996) in the study by Afyouni, Murad and Einea (2020). Questionnaires were also used to evaluate user satisfaction and task load, utilising Likert scales and user satisfaction evaluation questionnaires for rehabilitation systems (USEQ) (Gil-Gómez et al., 2017), the intrinsic motivation inventory (IMI) (McAuley et al., 1989), and the NASA task load index (TLX) (Hart and Staveland, 1988) to assess task load. The only questionnaire used that did not include a Likert scale was an experience questionnaire given to patient participants in Hamzeheinejad et al. (2021), asking for comments on their experience, and using LimeSurvey as their digital survey tool; quantitative methods included task time and efficacy, for example, representing the learning curve of the user and improvements.

Regarding patient experience, it was found that they perceived the environment to be motivating (Mihajlovic et al., 2018), they were more satisfied and perceived more enjoyment than traditional rehabilitation, and it was found that when performing personalised and prescribed exercises they felt that the system was engaging and effective (Afyouni et al., 2020). Studies that did investigate user experience, found that patients believed the varied VEs offered within the systems improved their experience and therapists could see the benefits of such systems for patients, with improvements necessary for ease of use and personalisation by therapists. Performance measures were also included in understanding the user experience, with findings showing that the users accuracy increased where there was more engaging, effective and personalised prescribed therapies (Afyouni et al., 2020) and explicit feedback ensuring that patients progressed towards independence, displaying walking dependency and a clear reduction in support needed in Hamzehheinejad et al. (2021).

In one study, therapists were asked about their perceived value of usability, adaptiveness, efficiency, enjoyability, fatigue, efficiency, level of motivation, and mental effort needed to use the system (Afyouni et al., 2020). Results demonstrated that therapists were positive regarding the role that RehaBot could play in telerehabilitation, and they demonstrated a high level of confidence when using the system. They reported that the learning curve for therapists was an important factor that could affect performance in the initial session, however, this could be improved by offering more training for both therapists and patients. The recommendation made by therapist participants was to make changes to the admin dashboard and re-design the way that alerts and instructions are displayed on the patient’s dashboard.

3.10 Hardware and software

3.10.1 Interactive devices

Three types of HMDs were used in the selected studies, these included the HTC Vive (Afyouni et al., 2020; Stanica et al., 2020; Hamzeheinejad et al., 2021), the Oculus Rift (Mihajlovic et al., 2018; Moldoveanu et al., 2019) and the Oculus Quest 2 (Georgiadis et al., 2021). A Lokomat, (‘Lokomat^® - Hocoma’, 2001), a fixed gait rehabilitation exoskeleton robotic device, was used by Hamzeheinejad et al. (2021) to provide intensive physiological gait rehabilitation for severely impaired neurological patients. The Lokomat was used to walk whilst immersed in the VE with a HTC Vive HMD. Several other devices were also used alongside the patients’ VR experience, including EEG and EMG in the study from Moldoveanu et al. (2019) and the Xiaomi Mi Fit 3 Smart (fitness) bracelet (Stanica et al., 2020).

3.10.1.1 Tracking devices and sensors

Two types of optical tracking devices were used to capture patient motion, these were Kinect (Moldoveanu et al., 2019; Afyouni et al., 2020) and Leap Motion (Moldoveanu et al., 2019). These sensors allowed for real-time monitoring of patients’ gestures and activities. Afyouni, Murad and Einea (2020) used the Kinect 2 for upper limb tracking, highlighting its usefulness for full body tracking. However, they highlighted that the Kinect 2 often produced noisy data when involving the tracking of two or more joints at once. They employed smoothing and noise reduction filters and created an algorithm to smooth the data from the Kinect. They underlined the specific issue of being unable to accurately capture movements if limbs were (partially) occluded, such as when one limb is crossing over another - a challenge that all optical tracking systems face. Contrarily, Moldoveanu et al (2019) focussed solely on the rehabilitation of the arm and used both Kinect and Leap Motion, with each device used to monitor the motion of different parts of the arm and hand of patients. By employing this algorithm, they were accurately able to provide feedback on patient movements. In Stanica et al. (2020) the Myo Gesture Control Armband was used, which was placed on the user’s forearm and included motion sensors for establishing orientation, rotation and acceleration, as well as eight EMG muscles sensors.

3.10.2 Virtual environment development

Unity 3D game engine was used in six of the seven selected studies to develop the VE with all VEs custom designed. It should be noted that Sobota et al. (2022) used A-frame. Mihajlovic et al. (2018) used Unity in their main VE, but also used Unreal Engine to produce their second highly realistic environment. The INREX-VR developed by Stanica et al. (2020) was described as a PC VR application developed in C# using Unity3D game engine. The authors stated that the INEX-VR application was highly modularised, meaning that there was an allowance for new virtual scenes or neurorehabilitation exercises. Similarly, Afyouni, Murad and Einea (2020) noted that the Unity game engine could be easily modified to support other 3D motion-capturing devices like the Intel Real Sense devices and input from HMDs. Georgiadis et al. (2021) noted that their key reason for using Unity was Unity’s Mecanim system, which they used to create animations for the VT.

3.10.3 Virtual therapist development

In the development of virtual therapists, there were often two parts: first, using sensing to input the real-life therapist movements, and second, using software to develop an avatar to visualise these recorded movements. In this section, we discuss the development of the avatars, as the sensors for therapist movement have been mentioned in Section 3.6.1 titled ‘Therapist Input and Control’.

One study utilised MakeHuman (2014) to create the patient avatars (Moldoveanu et al., 2019), an open-source tool that allows for the creation of 3D humanoid characters. This software provided a user-friendly interface and a wide range of customisation options. Three studies did not specify how they developed their avatars; however, they developed their environment in Unity.

Unity’s Mecanim animation system (‘Unity’, 2023) was used in another study (Georgiadis et al., 2021). Mecanim is a powerful and flexible animation system for 3D character animation in Unity. It provided tools for creating and managing complex animation systems, including support for inverse kinematics, blending of multiple animation clips, and retargeting of animations. This made it a versatile tool for creating realistic movements for the VTs.

Adobe Fuse (‘Adobe’, 2014) was used to develop VTs in two studies (Hamzeheinejad et al., 2021; Moldoveanu et al., 2019). This software allowed for the creation of detailed 3D models of human characters, with a wide range of customisation options for features such as facial features, body shape, and clothing.

4 Discussion

The analysed literature evidenced that the concept of VTs being applied in the VE for motor rehabilitation is still in its infancy. Although, there was no restriction in the search strategy, we could only find studies published from 2018 to 2022, emphasising how recent these advances are.

All studies have used similar tools and methods in the design and development of their systems, with Unity Engine (‘Unity’, 2023) used in five of the seven selected studies and most captured real-life movements and characters to facilitate virtual therapist design and feedback. Other programs used included Adobe Fuse CC, MakeHuman and Rokoko Skeleton. Sometimes, systems were interlinked with other hardware such as gait tracking sensors and smart fitness bracelets to allow for extended data gathering on patient performance.

Visually, virtual therapists are almost always represented as human-like avatars, but with no personalisation of the visual appearance possible in the selected studies. These human-like VTs were often placed in front of the patient in the VE, requiring the patient to mimic their movements or follow them for gait and neck exercises. In other systems, the therapist could be seen sitting next to patients or opposite them. When presented in front of the patient as a human-like avatar, the VTs in some systems only showed their back, in other games the face of the VT could be seen. In one of the selected studies, therapists had the ability to personalise elements of the patient avatar including gender, age, weight, hair and skin colours, clothes and hairstyle (Moldoveanu et al., 2019). The virtual environments varied from nature scenes to replications of real-life environments such as office buildings and therapy rooms. One study did offer an alternative, where therapists were able to control butterflies to guide patients (Mihajlovic et al., 2018).

The key role of VTs has been clearly identified in all the selected studies as a virtual character that the patient is asked to mimic. Therapists and patients did not interact physically in any of the virtual environments and systems, often with them close to one another but not touching. Interactions in all studies are represented as the therapist being alongside, opposite or in front of the patient, performing movements to be imitated. Interaction is, however, seen in the form of feedback; visual, auditory, and haptic. Feedback is delivered to patients in six of the seven systems, this is often in the form of text, images, and audio, with visual feedback such as changes in the user interface if the patient is moving or mimicking correctly. For example, Mihajlovic et al. (2018) system shows trails emitting from virtual butterflies when the user accurately follows them. Across all studies, the patient is expected to mimic or follow the therapist while receiving performance feedback. During this time, the therapist can change the difficulty level according to the patient’s progress.

All studies discussed feedback given in the VE with the aim for most of these systems to be in the home, facilitating telerehabilitation. However, one study mentioned feedback being offered by a real therapist at the time of therapy, outside of the virtual environment (Sobota et al., 2022). Feedback is often either pre-recorded or delayed, and in the case of audio and visual feedback, the real therapist pre-recorded their voice for standard feedback phrases and for written visual messages. However, feedback in the systems was produced based on the progress of the patient, mainly if they have matched the movements of their VT. Feedback could also be visual with emojis, visual indications with gamified elements and written phrases used as a response to patient progress. Audio recordings were triggered to reward the patient on their progress or advise how they can improve, and haptic feedback used to indicate to patients if they had successfully matched therapist movements. In four systems, therapists were able to review the output reports or profiles and dashboards of patients in the system, allowing them to provide feedback based on results and amend future exercises and goals accordingly (Moldoveanu et al., 2019; Afyouni et al., 2020; Stanica et al., 2020; Sobota et al., 2022). Only one study offered real-time feedback, which was given through short text messaging (Sobota et al., 2022).

Currently, in the VE’s reviewed, there is no back and forth dialogue, only one directional feedback from the virtual therapist. As highlighted by Thomson (2008), back and forth interactions are necessary for there to be a negotiation between the patient and therapist, with critical reflection. Similarly, Hunt et al. (2015) emphasised the importance of therapists being able to further explore something that a patient has said to influence goals and outcomes. Therapists have an element of control, showing in most studies that they have a dashboard that allows them to change the difficulty or personalise movements based on the patient’s rehabilitation goals and abilities.

The selected studies used a variety of methods, mostly quantitative, to measure the user experience, with only one paper including therapist responses in addition to patient responses. Table 1 emphasises this variation by evidencing that seven of the eight selected papers did not adequately represent participants and their voices. Also, the primary focus of each study was on remote rehabilitation (telerehabilitation), with only three studies stating that they carried out user testing in a clinical setting (Moldoveanu et al., 2019; Afyouni et al., 2020; Hamzeheinejad et al., 2021). Participant numbers varied from eight to 30 in the selected studies, indicating small groups of participants. These participants also varied in age from 18 to 87 and impairments ranged from healthy participants, to participants with neck pain, gait deficits and stroke. In addition to this, as can be observed in Table 1, the authors in five papers did not locate themselves culturally or theoretically with four of seven not addressing their influence on the research. Participant information in all studies was sparse, with most not communicating participant characteristics, which is in line with findings from a review by Laver et al. (2017). This review found that low numbers of participants and diverse populations resulted in low quality results in papers investigating VR for stroke rehabilitation.

When asked about their experience in the systems, patients seemed satisfied and motivated by the VEs presented to them and therapists were mostly positive about them. The selected papers did not show a clear standard on how researchers can study the users experience, with each study choosing a variety of different methods. Through these studies, it was found that patients believed the varied environments were improving their experience and they perceived them as more motivating (Mihajlovic et al., 2018), however, no further detail was given on what it is about the environment that they felt motivated by. From the selected studies, it can be determined that patients were more satisfied and perceived more enjoyment than traditional rehabilitation, and where the system was personalised and had prescribed therapies, they felt that it was engaging and effective. However, only one study investigated the impact that a virtual therapist has on the patient in the virtual environment (Afyouni et al., 2020). It should be noted, that an overview of the selected studies not only yields a variety of patient ages and characteristics, but it also shows a variation of different types of activities dependant on the patients’ impairment. Overall, patients seemed satisfied and motivated by the VEs presented to them, and therapists who were asked for feedback shared their positive attitude toward a system that utilises a VT (Afyouni et al., 2020).

It is unfortunate that only one study sought therapists’ feedback on their system. Tatla et al. (2015) highlighted the interest of therapists in helping to shape the development of these systems, with Schmid et al. (2016) finding that clinicians believed that the success of VR was dependent on their patients’ particular goals, and if those can be met with VR as a tool. Something which is important to the therapeutic alliance, ensuring that there is agreement on aims and tasks. Hence, it is surprising how little personalisation is available to both the therapist and patient regarding the visual look of their avatar, and the lack of ability to quickly make changes to the interactions and/or environment for the patient.

5 Limitations of this review

The scope of this systematic literature review is relatively limited due to the focus on immersive VR and motor rehabilitation. In addition, given the heterogeneity in terminology used in this somewhat new field of research, we suspect that, despite our best effort, we may have missed some studies in our literature search. We acknowledge that a meta-analysis was not conducted. This would have been impossible due to the heterogeneity of the outcome measures and interventions used across the seven studies. To minimise the likelihood of missing relevant research, we also reviewed reference lists of included studies but found no additional papers to those already found by the systematic review process. The authors also found it necessary to exclude the terms “psycho*” and “phobia*”. Although much work on VTs lies in this research area, these studies were not applicable to this review since they did not include the correct populations and interventions. Nevertheless, we found a number of studies utilising an HMD and immersive environments.

6 Conclusion and future directions

The aim of this review was to systematically review the literature on VTs in immersive VR environments for motor rehabilitation, with the goal of providing guidance for future research for VT implementation in fully immersive VEs. The review explored the therapist’s role and how they are represented in the VE, how the therapist and patient interact in the VE, activities carried out, and the experience of patient and therapist users. Seven articles met the inclusion criteria.

The roles of VTs in motor rehabilitation present a rich field for exploration. The current body of research has begun to scratch the surface of this potential, but there are several areas where more focused attention could yield valuable insights.

Firstly, the user experience of both patients and therapists with VTs needs to be more thoroughly investigated. While initial studies have shown positive attitudes and satisfaction, a more nuanced understanding of these experiences could inform the design and implementation of VTs. For instance, qualitative studies could explore the subjective experiences of patients and therapists, including their perceptions of the benefits and challenges of using VTs. The selected studies use a variety of methods in collating users experience, but as highlighted by Brassel et al. (2021), it is important that future research into ABI rehabilitation should consider a stepwise approach to VR development, utilising co-design studies with end users to larger controlled trials. These approaches, along with surveys and interviews could be used to gather data on both patient and therapists experience, providing a rich source of information to improve VT systems.

As part of the user experience, no studies investigate the alliance between the patient and therapist. Future studies should seek a more detailed understanding of how the interaction between the patient and therapist, the appearance of the VT, and the types and frequency of feedback, effect the therapeutic alliance during rehabilitation. It would be interesting to understand if the patient seeks a bond with the VT, and if agreement on aims and tasks during virtual rehabilitation influence their relationship with the VT.

Secondly, the development of VT systems that require less manual input from therapists is an important area for future research. This could involve the use of machine learning algorithms to adapt the VT’s guidance to the patient’s specific needs and progress. Such systems could potentially reduce the workload for therapists, making the rehabilitation process more efficient. In addition, with new technology such as the Oculus Quest 3, therapists could utilise this hardware in clinic and provide real time feedback and interaction with the patient. Using Quest 3 can also allow the therapist to not only play multiplayer with the patient, but utilise its augmented reality functionality to be both in the VE and able to see patient movements in the real-world. This use of augmented reality can enhance the immersive experience of the VT, potentially increasing patient engagement and motivation.

Thirdly, the personalisation of VT systems is another promising direction for future research. Currently, most VTs provide generic guidance that is not tailored to the individual patient. However, personalised feedback could potentially enhance the effectiveness of motor rehabilitation. For instance, VTs could be designed to adapt their feedback based on the patient’s performance, providing more challenging exercises for patients who are making rapid progress, and more supportive guidance for patients who are struggling. Also, none of the selected studies discuss their reasoning for how they visually represent the therapist, future research could explore different visual representations of therapists. Also, many of the systems in the selected studies use verbal cues, however, for patients who have aphasia as a result of their ABI, it could be useful to implement more visual cues, similar to the butterfly trails and emojis, to provide helpful feedback without reliance on verbal cues as aphasia can cause difficulty understanding speech.

Finally, research by Piumsomboon et al. (2022) and Thanyadit et al. (2022) provides insights into the process of disengaging users from immersive virtual environments, transitioning them back to the real world, and enhancing asynchronous shared-space demonstrations with spatial-temporal assistive toolsets. These could be relevant for the development of VT systems, as they could inform the design of mechanisms for transitioning patients between different stages of the rehabilitation process and for enhancing the effectiveness of asynchronous therapy sessions. For instance, VTs could incorporate visualisation and interaction techniques to smoothly transition patients from intensive rehabilitation exercises to less demanding activities, or to end the therapy session. Furthermore, VTs could incorporate spatial-temporal assistive toolsets to enhance the effectiveness of asynchronous therapy sessions.

In addition, research by Zhang et al. (2022) and Hart et al. (2021) provides insights into the concept of shared control of virtual avatars and manipulating avatars for enhanced communication in extended reality. These could be particularly relevant for the development of VT systems, as they could inform the design of mechanisms for allowing therapists to control multiple VTs simultaneously and for improving communication between patients and therapists. For instance, therapists could control multiple VTs at the same time, allowing them to supervise multiple patients simultaneously. Furthermore, VTs could be designed to manipulate avatars in a way that enhances communication between patients and therapists, potentially improving the effectiveness of the therapy sessions. In conclusion, while the use of VTs in motor rehabilitation is a promising area of research, there is still much to learn. Future research should aim to deepen our understanding of the user experience, understand how design and interactions in the VE and VT can foster therapeutic alliance, develop more autonomous and personalised VT systems, and explore the integration of VTs with other technologies.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

SC: Conceptualization, Methodology, Data curation, Formal analysis, Writing–original draft, Writing–review and editing. MY: Data curation, Writing–review and editing. BS: Writing–review and editing. TP: Writing–review and editing. SH: Conceptualization, Methodology, Supervision, Writing–review and editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Ackerman, S. J., and Hilsenroth, M. J. (2003). A review of therapist characteristics and techniques positively impacting the therapeutic alliance. Clin. Psychol. Rev. 23 (1), 1–33. doi:10.1016/S0272-7358(02)00146-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Adobe (2014). Adobe. Available at: https://www.adobe.com/wam/fuse.html (Accessed: August 11, 2023).

Google Scholar

Afyouni, I., Einea, A., and Murad, A. (2019a). “Rehabot: gamified virtual assistants towards adaptive telerehabilitation,” in ACM UMAP 2019 Adjunct - Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, 305–306.

Google Scholar

Afyouni, I., Murad, A., and Einea, A. (2019b). “Towards an adaptive gaming framework for telerehabilitation,” in ACM International Conference Proceeding Series, 1465–1470.