Bhing-Leet Tan
Jing Shi
Suyi Yang2
Hannah Loh- 1Health and Social Sciences Cluster, Singapore Institute of Technology, Singapore, Singapore
- 2Occupational Therapy Department, Institute of Mental Health, Singapore, Singapore
- 3Department of Psychiatry, New York State Psychiatric Institute, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, United States
Objectives: Virtual reality and augmented reality have been used in psychosocial rehabilitation for adults with neurodevelopmental disorders in recent years, to provide functional training in a scaffolded and appealing manner. This systematic review attempted to evaluate (1) how virtual reality or augmented reality technology was deployed, when used as an intervention for adults with neurodevelopmental disorders; and (2) how virtual or augmented reality-assisted psychosocial rehabilitation programs impacted on the functional domains of community living, employment and social participation.
Methods: The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines was adopted and a search of publications between June 2012 and June 2022 was carried out. The target groups were adults with schizophrenia/schizoaffective disorders, autism spectrum disorder, intellectual disabilities and attention deficit hyperactivity disorder. Interventions that targeted at least one functional domain were included.
Results: The database search generated 1,267 records and 38 studies met the inclusion criteria. Three studies utilized augmented reality while the rest utilized virtual reality. The virtual scenarios were displayed in various ways, such as head-mounted displays, computer screens, mobile devices and cave rooms. A few studies also used features such as speech recognition, eye tracking and motion-capture device to provide real-time feedback to participants during rehabilitation. Eleven studies reported interventions that addressed community living, 15 studies addressed vocational skills and nine studies trained participants in social skills or social cognition. Three studies sought to improve quality of life using virtual scenarios to expose participants to various situations. Majority of these studies reported preliminary promising results, with improvement in the functional domains addressed. However, several studies had small sample sizes and many single-arm pretest-posttest studies were assessed to be of serious or critical risk of bias.
Conclusion: Virtual reality and augmented reality are deployed in various ways to augment psychosocial rehabilitation for adults with neurodevelopmental disorders. Most interventions target skills training or strategy learning in the areas of community living, work and social participation. Preliminary positive findings of their effects on functional performance were reported. Larger and robust studies using ecologically valid outcome measures will be needed to establish their effects on real-world functional outcomes.
Systematic review registration: identifier: CRD42022335443.
Introduction
Psychosocial rehabilitation is a process of facilitating persons with psychiatric disabilities to reach their optimal level of functioning in their natural settings, as well as to enhance their sense of empowerment and quality of life (1). Adults with neurodevelopmental disorders such as intellectual disabilities (ID), autism spectrum disorder (ASD), and schizophrenia can benefit from psychosocial rehabilitation to maximize their participation in community living, work and leisure (2, 3). This is often done through skills training, task adaptations, environmental modifications, advocacy and educating stakeholders (2, 3).
Technology and assistive devices have often been used in psychosocial rehabilitation, due to their ability to provide structured graded training to facilitate scaffolding, as well as their interesting graphics to motivate service users (4, 5). In the recent decade, augmented and virtual reality have enabled rehabilitation practitioners to provide a more immersive training environment. Virtual reality (VR) enables stimulation of the senses by transporting the users to a realistic three-dimensional environment, while allowing users to interact with objects in the simulated environment (6). Augmented reality (AR) differs from VR in that the virtual three-dimensional graphics and images are superimposed on the real environment, such that the users are still able to perceive their surroundings (7). Both VR and AR present as viable options for rehabilitation practitioners to conduct skills training in a less time-consuming and costly way, as there is no need to set up a real training environment and the virtual consumables can be reused. In a pilot study on an AR-enabled vocational training program for persons with intellectual and developmental disabilities (IDD), participants also reported that they could experiment with new behaviors within the virtual environment in a self-paced and less anxiety provoking manner (8).
In view of these benefits, practitioners have harnessed the capabilities of VR and AR as adjunctive interventions for children and youths with ID, ASD, and attention-deficit hyperactivity disorders (ADHD). Many of these interventions are conducted in special education settings and positive findings have been found (6, 9). However, the use of VR and AR in psychosocial rehabilitation for the adult population with IDD is less established. As for adults with schizophrenia, reviews have been conducted on the use of VR and AR in the treatment of psychiatric symptoms such as hallucinations and delusions (10–12). In recent years, however, there is also an emerging interest in the use of VR and AR in psychosocial rehabilitation for schizophrenia, to improve outcomes in personal, functional and clinical recovery. It would be useful to review the effectiveness of such psychosocial rehabilitation programs in facilitating service users to achieve valued goals in life.
Therefore, this systematic review attempted to evaluate a variety of psychiatric rehabilitation programs that utilized augmented or virtual reality for adults with neurodevelopmental disorders, to understand how they address the functional domains of community living, employment and social participation. The review focused on two questions:
1. How was the virtual reality or augmented reality technology deployed, when used as an intervention for adults with neurodevelopmental disorders?
2. How did virtual reality and augmented reality-assisted psychosocial rehabilitation programs impact on the functional domains of community living, employment, and social participation?
Treatment approaches, mode of delivery and outcome measurements used in these studies were examined. In addition, their effects on addressing proximal outcomes such as cognitive performance, social skills, vocational skills or independent living skills were discussed.
Methods
Study design
This systematic review was registered with PROSPERO (ID: CRD42022335443) and was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines (13). Findings of this review were presented in a narrative approach.
Eligibility criteria and search strategy
The target diagnostic groups of this systematic review were neurodevelopmental disorders, which included schizophrenia/ schizoaffective disorders, autism spectrum disorder (ASD), intellectual disabilities (ID) and attention deficit hyperactivity disorder (ADHD). In some studies, intellectual disabilities and autism spectrum disorders were grouped together and known as intellectual and developmental disabilities (IDD). At least half of the study participants had to be diagnosed with these conditions and co-morbid psychiatric and/or medical conditions were included. In addition, at least half of the research participants were above 18 years of age. Other eligibility criteria were showed below.
Inclusion criteria:
• Publications in English language.
• Publications from June 2012 to June 2022.
• Types of publications: randomized controlled trials, non-randomized experimental studies, quasi-experimental studies, pretest–posttest intervention, mixed methods studies, pilot trials, and feasibility studies.
• Interventions included the use of virtual reality or augmented reality.
• Interventions addressed at least one functional outcome (self-care/daily living, community living, work/education, leisure, and social participation).
Exclusion criteria:
• Non-English publications.
• Types of publications: case reports, single case designs, study protocols, reviews, and conference papers.
• More than half of the research participants were below 18 years of age.
• Fewer than half of the research participants had a diagnosis of schizophrenia/schizoaffective disorders, autism spectrum disorders, intellectual disabilities or attention deficit hyperactivity disorders. Studies solely on formal or informal caregivers of these persons were also excluded.
• Virtual reality or augmented reality not used.
• Interventions did not target functional outcomes.
Articles included in the systematic review were identified through a computer-based search of the following databases: CINAHL, PubMed, PsychInfo, IEEE Xplore, ACM Digital Library, and Web of Science. Subject headings were used when available. Expanders and equivalent subjects were also applied. The first two authors conducted the database searches.
The search terms and Boolean operators used were: [(“virtual reality” OR “augmented reality” OR “VR” OR “AR”) AND (“psychiatric rehabilitation” OR “psychosocial rehabilitation” OR “intervention” OR “train*” OR “rehabilitation” OR “therap*” OR “functional re-training”) AND (“neurodevelopment*” OR “psychiatric disabilit*” OR “intellectual disabilit*” OR “developmental disabilit*” OR “Autism” OR “Autism spectrum*” OR “Asperger*” OR “mental retardation” OR “schizophrenia” OR “schizoaffective” OR “attention deficit hyperactivity disorder*” OR “ADHD”) NOT (“motor disorder*” OR “tic disorder*” OR “Tourett*” OR “learning disab*”)].
We selected literature from the past 10 years, due to the rapid technological advancements of VR and AR in the past decade, which led to changes in the way they were used by clinicians in psychosocial rehabilitation.
Article selection and risk of bias assessments
Five authors (B-LT, CC, HL, DN, and SY) were paired up to independently assess the articles for inclusion or exclusion (B-LT was paired twice), then came together to arrive at a consensus. They also manually removed duplicates in the database searches. In addition, the reference sections of reviews and articles identified from database searches were studied by the first author for relevant citations. All authors then completed the systematic review table of the selected studies and cross-checked each other's risk of bias assessments.
Risk of bias assessments
The risk of bias assessments were performed by the first six authors and cross-checked with each other in pairs. The Cochrane Collaboration's ROB-2 (Risk of Bias version 2) tool was used to assess the risk of bias for randomized studies (14). The effect of assignment to intervention (the “intention-to-treat” effect) was assessed. Biases were categorized into five domains, namely (1) bias due to randomization; (2) bias due to deviations from intended intervention; (3) bias due to missing outcome data; (4) bias due to measurement of outcome; and (5) bias due to selection of reported results. An overall risk of bias indicator (“low risk,” “some concerns,” or “high risk”) was also derived. The authors followed the ROB-2 guidelines to answer signaling questions and used the algorithm to estimate the level of risk for each domain and the overall risk.
The ROBINS-I (Risk of Bias in Non-randomized Studies-of Interventions) was used to assess risk of bias for all the other non-randomized studies (15). Functional performance was used as the main outcome measurement. Biases were categorized into seven domains, namely (1) bias due to confounding; (2) bias in selection of participants; (3) bias in classification of interventions; (4) bias due to deviations from intended interventions; (5) bias due to missing data; (6) bias in measurement of outcomes; and (7) bias in selection of the reported result. Similarly, the authors answered signaling questions and used the ROBINS-I evaluation table to estimate the risk in each domain and the overall risk. The overall risk was reflected as “low,” “moderate,” “serious,” “critical,” or “no information.”
Results
Results of screening and selection of studies
The database search generated a total of 1,267 records, out of which 55 duplicated records were identified by EndNote referencing software and excluded. The remaining 1,212 records were screened manually, and 115 duplicates were further excluded. The rest (1,097 studies) were then assessed for eligibility and a total of 1,062 studies were excluded for not fulfilling the inclusion criteria, yielding 35 selected studies for review. In addition, three studies were identified by the first author through searching the reference sections of articles manually. The final number of studies selected for this systematic review was 38. See Figure 1 for the PRISMA flow diagram.
Figure 1. PRISMA flow diagram.
The selected 38 studies were presented in Tables 1–4, which corresponded to the functional domains that the studies addressed, namely (1) self-care and community living; (2) vocational skills and employment; (3) social skills and social participation; and (4) quality of life. The tables outlined the study design, overall risk of bias, intervention methods, treatment modalities, target population, experimental and control conditions (if any), functional domains targeted, outcome measurements and effects of the intervention. The ROB-2 and ROBINS-I assessments were presented in the Supplementary material. An adapted robvis visualization tool format was used, with traffic light plots of estimated risk for each domain (16).
Table 1. Studies that addressed self-care and community living.
Table 2. Studies that addressed vocational skills and employment.
Table 3. Studies that addressed social skills and social participation.
Table 4. Studies that addressed quality of life.
The deployment of technology
Augmented reality (AR)
Out of the 38 selected studies for review, only three studies used AR in their interventions (7, 19, 23). One study developed independent living tasks videos (ironing, bed making, and setting an alarm clock) and presented them on an AR application installed on an iPad (19). As the study was conducted in a university campus dormitory for adults with ID, laminated targets were set up in the participants' rooms or common areas. During intervention, participants pointed the iPad cameras at laminated targets to activate video models, which appeared as an overlay across the target. This facilitated the learning of daily living skills within their natural environment.
Another study used AR to teach the self-care skill of tooth brushing (23). The three-dimensional motion sensor in the smart toothbrush enabled recognition of the participants' hand movements and provided a visual guide for posture and direction. This form of instructional guidance using visual feedback allowed participants to learn in a more interactive manner.
The third study also used AR as training feedback. In this intervention named ARCoach, AR was used as a vocational task prompting system for food preparation training (7). AR tags were used to represent food items and the system provided sound alerts when participants made errors in the steps of the tasks.
Virtual reality (VR) with head-mounted display (HMD)
The studies utilizing VR also harnessed the technology differently. Six studies immersed the participants in the virtual environment using a head-mounted display (HMD). Bozgeyikli et al. (28) used a VR-based vocational training platform, to train six transferrable vocational skills. Three of the six skill modules (cleaning, shelving, and environmental awareness) were designed to be performed while wearing the HMD while the remaining three skill modules (loading the back of a truck, money management, and social skills) were presented on a 180° curved screen. Facilitation by job trainers were done using a remote-control interface on a separate tablet. Another study by Simões et al. (27) used a VR Oculus Rift headset for participants to experience the full immersion of using public bus transportation. Their anxiety levels were also monitored by a bracelet that captured electrodermal activity.
Three studies used immersive VR to assist in cognitive behavior therapy for persons with schizophrenia, in order to improve their social participation and quality of life. Participants in the Virtual-Reality Based Cognitive Behavior Therapy (VR-CBT) were given guided exposure to four virtual social environments (a street, bus, café, and supermarket) by viewing them through HMDs (47). On the other hand, participants in VR-assisted Therapy (VRT) created an avatar that resembled the source of their auditory hallucinations and engaged in a dialogue with it through Samsung Gear HMD and smartphone (48, 51).
Lastly, one study used HMD to immerse participants in simulated social interactions with an avatar to train theory of mind (53). A smartphone and handheld controller were used alongside to facilitate virtual conversations.
Virtual reality (VR) with cardboard viewer or 3D glasses
In two studies, a VR application was installed on a smartphone and slotted into the Cardboard Viewer. Kuper et al. (33) used this technology to train participants on the skill of wiring an electrical outlet, while Miller et al. (24) simulated the whole process of air travel, from entering the virtual airport, checking in, navigating security, waiting at the departure gate, and boarding the airplane.
On the other hand, the Soskitrain program by Rus-Calafell et al. used 3D glasses and headphones in a brief social skills intervention for participants who had schizophrenia (54). A laptop and a 15.6-inch monitor with stereoscopic view was also used. Participants practiced social interactions with virtual avatars who used their facial expressions to provide feedback. This feature encouraged progressive learning of social skills through positive or negative reinforcement.
Virtual reality (VR) on desktops or laptops
Almost half of the studies (18 studies) deployed VR on desktops or laptops and participants used a keyboard or mouse/joystick to interact within the virtual scenarios. Microphones and headphones were also used in interventions that involved verbal interactions. Several of these studies made use of virtual characters, speech recognition software and artificial intelligence algorithm to create an interactive platform to train skills such as job interviewing and public speaking. In Virtual Reality Job Interview Training (VR-JIT) and Job Interview Training with Molly Porter programs, the human resource manager “Molly Porter” generated interview questions based on the participants' job modification needs and their verbal responses (32, 34–36). A subsequent adapted version added screen reader capabilities and a virtual job coach that provided additional support and feedback (37, 38).
Other studies presented non-immersive VR simulated cities, towns or workplaces on desktops/laptops with large screens, so that participants could learn navigation, vocational skills, community living skills, public speaking or social skills by using keyboard or joysticks (17, 20, 22, 41, 42, 45, 46).
Virtual reality (VR) with other features
Two studies added Brain-Computer-Interface (BCI) features by using electroencephalogram (EEG) cap or headband. Participants in a social attention training program put on an Oculus Rift headset with eye tracking package to watch four animated scenarios and their attention was tracked using an EEG cap with 16 electrodes (43) Participants in another study interacted with avatars in social situations and their attention levels were tracked by a BCI headband, which provided visual or textual feedback on their attention levels (50).
One study added Kinect as a motion capture device, which enabled movements to be mapped onto the virtual space. Participants who were learning navigation and street crossing could see their full-body movements mapped onto the VR city, which enabled them to have an authentic sense of their body within the three-dimensional virtual environment (26).
Two studies used Cave Automatic Virtual Environment (CAVE) rooms to maximize the immersion experience. The first study coined its room as the “Blue Room VR Exposure,” which was a 360-degree seamless screened room with computer-generated images projected onto the walls and ceilings (52). Participants diagnosed with ASD were gradually exposed to their phobic situation (such as getting onto a bus or going into a shop) within this immersive environment without the use of headsets or goggles. Navigation was controlled by a tablet. The second study used a similar fully immersive stereoscopic and wireless six-wall CAVE-like system where participants with ASD moved freely with a wand tracker (44).
Two studies utilized VR driving simulators to train driving skills and executive functioning for adults with ASD (18, 21). Lastly, a study reported on the use of a VR application installed onto tablets to train community living skills such as shopping at a supermarket and preparing a suitcase (25). The program was self-administered at home with caregivers, with a chat tool to contact a day rehabilitation staff member if assistance was required.
Functional domains addressed
The functional domains which were addressed included self-care/community living, employment and social participation. The outcome measurements ranged from in-game performance of functional skills to real-world functional outcomes such as employment rates. Several studies also used observer-ratings of functional performance such as the Personal and Social Performance (PSP) scale and Modified Barthel Index (MBI). In addition, a few studies measured subjective quality of life.
Self-care and community living
Eleven studies reported interventions that addressed self-care and community living. Out of these studies, four of them created a simulated town or facility to train community mobility and navigation. In one study, a VR intervention was conducted by a therapist weekly over 10 sessions to train outpatients with ASD in navigation and street crossing (26). In-game street crossing performance was measured within the VR application. Although the study found no significant reduction in street-crossing errors, navigation became more efficient. In addition, caregivers reported a significant post-treatment improvement in the participants' real-world navigation performance. Another study used VR to train adults with ASD in taking public bus transportation (27). Their anxiety was tracked using electrodermal activity recording throughout the three intervention sessions. In-game measurement on action accuracy (number of correct steps) and post-game debriefing accuracy (getting participants to describe each step of the task) were used. At the end of the intervention, there was a statistically significant increase in debriefing accuracy and a significant reduction in anxiety among the participants. In another study, de la Torre-Luque et al. (22) created a VR environment of a disability center and engaged its service users with ID in daily navigation training of its ground floor facilities over 15 sessions. At post-intervention, the participants were able to perform real-life location tasks with a significant reduction in errors and time taken. Lastly, one small study trained adults with ASD in air travel by creating a virtual airport (24). Speech and language therapists facilitated the weekly 20-min sessions over 3 weeks by training participants on the use of functional language skills, which helped them interact and navigate busy environments such as an airport. Participants showed improvements in functional language skills and were able to accurately describe the sequence of air travel (24).
Two studies targeted the functional skills of driving in adults with ASD using VR driving simulators. The Cognitive Behavioral Intervention for Driving (CBID) consisted of cognitive behavioral strategies to enhance executive functions and generalizing them to driving (18). Intervention was carried out weekly over 10 sessions. Among the 19 participants who completed the intervention, eight obtained a driver's permit and one obtained a driver's license within 2 months. Another study on Virtual Reality Driving Simulation Training (VRDST) explored if outcomes could be different with automated feedback mechanism or eye-tracking device (21). Participants in the Automated VRDST group received real-time feedback via the simulator's computerized voice, instead of being coached by trainers in Standard VRDST. On the other hand, participants in the Eye-Tracking VRDST group wore eye-tracker glasses during driving, from which videos of their eye gazes were produced for the trainer's review and feedback. Results of tactical simulator tests showed no significant differences across the three VRDST groups (21).
The studies that trained participants in Activities of Daily Living (ADLs) were mainly implemented in the participants' natural settings. A study by Panerai et al. (25) found that caregivers of persons with ID could utilize a VR application on tablets at home, to train persons with ID in community living skills such as shopping at a supermarket and preparing a suitcase. In-vivo tests of these daily living tasks showed an increase in the number of correct responses and a reduction in prompts and number of errors. High levels of participant and family satisfaction were also reported (25). Another small-scale study using AR for three adults with IDD living in a school dormitory found that their independent living skills (e.g., ironing, bed making, and setting an alarm clock) improved immediately after intervention (19). Finally, a quasi-experimental study on adults with ID in residential facilities found that compared to traditional tooth brushing training, an AR-facilitated tooth brushing intervention yielded significantly better improvement in Modified Barthel Index (Korean version) scores and Simplified Oral Hygiene Index (23). This intervention was conducted twice a week for 12 weeks by an occupational therapist.
Two studies incorporated cognitive rehabilitation into community living skills training. In Câmara et al.s' single-blind randomized controlled trial of the Reh@City v2.0 intervention, participants with schizophrenia engaged in cognitive training through performing ADLs such as baking cookies at home and buying groceries in the supermarket (20). This was conducted as part of an inpatient psychosocial rehabilitation program. Participants in the comparison Task Generator group engaged in paper and pencil tasks. After 2 months of intervention, the Reh@City v2.0 group did not show significant improvement in quality of life but had significant improvements in overall cognition and immediate recall (20). Another study by Amado et al. (17) used a VR simulated town where outpatients with chronic schizophrenia or schizoaffective disorders were trained by a psychologist and occupational therapist to plan their actions around the town. After 12 interactive 90-min weekly sessions, participants showed significant improvements in Social Autonomy Scale total scores. In addition, there were significant improvements in attention and working memory but not executive functioning (17).
Vocational skills and employment
There were 15 studies which utilized VR or AR to provide vocational skills training or to enhance employment outcomes. Nine out of these 15 studies focused on job interview training. Smith and colleagues implemented the Virtual Reality Job Interview Training (VR-JIT), where a virtual human resource manager “Molly Porter” would ask questions and adopt different demeanor based on an algorithm of difficulty levels and customizable features (34). In a randomized controlled study of adults with ASD, VR-JIT participants demonstrated greater improvement during live standardized job interview role-play performances and self-reported job interview confidence than control participants (34). Participants attended 90% of the five-session lab-based training and found VR-JIT easy to use and enjoyable. A 6-month follow-up study of these participants showed that VR-JIT participants had greater odds of attaining competitive jobs than controls (35). The same intervention also yielded similar benefits for individuals with schizophrenia or schizoaffective disorder (36). Compared to treatment-as-usual, participants who underwent VR-JIT demonstrated better role-play performance and had greater odds of receiving a job offer by 6-month follow-up. More training was associated with shorter waiting time to job offers (36).
Smith and colleagues subsequently adapted VR-JIT and developed the Virtual Interview Training for Transition Age Youth (VIT-TAY), to cater to youths with ASD attending pre-employment transition services (Pre-ETS) in schools (37). Adaptations included a token economy system, three additional learning goals across interview difficulty levels, screen reader capabilities, and social storytelling with the use of video and audio to reduce cognitive load. The VIT-TAY was facilitated by teachers in the high school special education settings delivering Pre-ETS. After 15 sessions of VIT-TAY, participants showed better job interview skills (as measured by Mock Interview Rating Scale) and significant reduction in job interview anxiety than those receiving Pre-ETS only (37). A logistic regression also showed that the VIT-TAY+Pre-ETS group was more likely to obtain competitive employment after 6 months than the Pre-ETS only group (37). A subsequent study comparing VIT-TAY outcomes with processed data from the prior VR-JIT study found that 48.1% of VIT-TAY participants obtained new jobs (n = 168) between baseline and follow-up, which was significantly higher than the 32.7% job attainment rate of VR-JIT participants (38).
Another job interview VR program was the Virtual Interactive Training Agents (ViTA) by Burke et al. (29). In this program, participants with ASD or ID interacted with avatars, which asked 10–12 interview questions, with 144 different scenarios to choose from. This intervention was facilitated by the program implementation team, who worked with the participants and their teachers. A pilot single-arm study found significant improvement in Marino Interview Assessment Scale scores after intervention (29). A subsequent larger study was conducted using within-subjects repeated measures design and included participants with ASD, ID, and ADHD (30). Results showed that ViTA participants had significant improvements in job interview self-efficacy and job interview skills after intervention (30).
Other job interview training programs included JobTIPS, where the clinician assumed the avatar role of an “interviewer” remotely and the intervention group participants with ASD were coached by program staff in theory of mind and given visual supports to demonstrate appropriate job interview skills (40). There were no significant differences in Social Responsiveness Scale scores between JobTIPS and control participants, although JobTIPS participants performed better in aspects of job interview skills (40). Another study used the “Molly Potter” software with both adults with schizophrenia and adults with ASD. There were significant treatment condition effects for the Molly Porter intervention group when compared to waitlist controls, in terms of performance in standardized interview role-plays and self-confidence (32).
There were six studies which used VR or AR to train vocational skills relevant to open or sheltered employment in a vocational rehabilitation program. The Virtual Reality-based Vocational Training System (VRVTS) provided a non-immersive VR boutique environment to train inpatients with schizophrenia receiving vocational rehabilitation services in a psychiatric hospital (41). Participants were trained in vocational skills required of a salesperson, such as work-related social skills, flexibility, and problem solving. Participants randomized to VRVTS reported significantly better self-efficacy in sales-related activities than the control group (41). Another study on the Virtual Reality-based Vocational Rehabilitation Training Program (VR-VRTP) presented virtual work scenarios encountered by a convenience store employee and a supermarket clerk (39). Participants spoke directly into a microphone and their voices were recorded, with a system in place for providing feedback. In this small-scale single-arm pre-post study, outpatients with schizophrenia showed significant improvement in Personal and Social Performance scale scores after 8 weeks of VR-VRTP (39). A study by Bozgeyikli et al. on “Virtual Reality System for Vocational Rehabilitation of Individuals with Disabilities” (VR4VR) employed a closed-loop adaptive VR-based vocational training platform to train six transferrable vocational skills of cleaning, shelving, environmental awareness, loading the back of a truck, money management, and social skills (28). In this quasi-experimental small sample study involving adults with ASD and neurotypical individuals, job trainers reported improvements by ASD participants in all the trained skills, especially in money management, cleaning, and social skills (28).
Three of these six studies utilized technology to train specific vocational tasks. One study used VR to train community-dwelling adults how to wire an electrical outlet in their vocational assistance center or at home (33), while another study used AR as a task prompting system to aid job coaches in training food preparation for community-based supported employment service users with ID (7). Lastly, semi-immersive VR scenarios were used to teach adults with ID in a residential home the work task of sowing zucchini seeds (31). These three studies reported that participants were able to learn the specific tasks well, although no validated measurements were used (7, 31, 33).
Social skills and social participation
Nine studies utilized VR to train social skills or to facilitate social participation. Majority of these studies involved getting participants to interact with virtual avatars in different social scenarios, with feedback provided by the VR platform. For example, a study by Rus-Calafell et al. adopted a VR software called Soskitrain as an adjunct to a brief social skills intervention for participants with schizophrenia (54). The clinical psychologist coached participants to work on the feedback given by virtual avatars based on their facial expressions. At the end of the 16 individual 1-h sessions, participants showed significant improvements in social functioning, social anxiety, avoidance and negative symptoms, with majority of the gains maintained at 4-month follow-up (54). Another study required participants with schizophrenia to complete 12 virtual social missions over a 5-week program (42). Although reduction in psychiatric symptoms were seen, there was no significant change in Social Functioning Scale scores (42).
Three studies used VR to train aspects of social cognition. The Virtual Reality Social Cognition Training (VR-SCT) intervention used different virtual environments and characters to train adults with ASD in social cognitive domains such as emotion recognition and theory of mind (45). Significant improvements in theory of mind, emotion recognition and social skills performance were reported after the intervention (45). Similarly, the VR-based Theory of Mind Intervention (VR-ToMIS) provided an immersive social environment with virtual conversations to train theory of mind in outpatients with schizophrenia (53). The control group was able to access the same VR environment but could not interact with the avatars. Although no significant difference in quality of life was reported between the VR-ToMIS and control group, the VR-ToMIS participants demonstrated significantly greater improvements in theory of mind, pragmatic language skills, visuospatial skills and attention (49). Lastly, a study conducted in a VR CAVE-like environment trained adults with ASD to detect social cues so as to improve social perception (44). In this small-scale study, two out of the three participants reported improvements in social skills.
Two studies paired VR with Brain Computer Interface (BCI) to monitor social attention. In the Brain-Computer Interface for ASD (BCI-ASD) program, the BCI headband provided feedback on participants' attention levels while they interacted with avatars in social activities (50). These participants were adults on the autism spectrum with co-occurring diagnoses who were attending college. This study had a small sample size (n = 4 in each group) and no clinically meaningful change on overall adaptation to college and social adjustment was found, when compared with participants receiving a psychosocial support program (50). On the other hand, a different study which used VR-assisted BCI training for social attention found improvement in adaptive behavior composite on the Vineland Adaptive Behavior Scales but did not observe any change in joint attention among its participants with ASD after seven sessions (43).
There was only one study that used AR to train adults with ASD in public speaking. Participants performed public speaking in front of the VR system and received feedback via the agent's eye movement and nodding behavior (46). When compared to participants who practiced in an empty room, participants in the VR training exhibited lower anxiety and more improvement in self-confidence (46).
Lastly, one study combined VR with Cognitive Behavior Therapy (VR-CBT) to work on social avoidance in persons with schizophrenia (55). Schizophrenia participants with persistent paranoid ideation were given guided exposure to virtual social environments such as a street, bus, café, and supermarket. At follow-up, the VR-CBT participants showed improvements in social functioning and self-stigmatization, but treatment-as-usual participants did not. However, quality of life between the two groups did not differ significantly (55).
Quality of life
Three studies used quality of life as the sole functional outcome measurement. A small study on the use of immersive VR exposure to reduce anxiety in adults with ASD found no significant change in World Health Organization Quality of Life abbreviated (WHOQOL-BREF) scores, but five out of the eight participants reported improvements in day-to-day situations (52).
Two studies on Virtual Reality-Assisted Therapy (VRT) involved getting participants with refractory schizophrenia or schizoaffective disorders to create an avatar that best resembled the source of their residual hallucinations (48, 51). They were then facilitated by a psychiatrist to engage in a dialogue with these avatars. Statistically significant improvement in Quality-of-Life Enjoyment and Satisfaction Questionnaire-Short Form (QLESQ-SF) score was found between baseline and post-intervention (48, 51).
Quality of studies and risk of bias assessments
Study design and sample size
Among the 38 selected articles, 13 were randomized controlled studies, six were quasi-experimental studies and the rest were single-arm studies. The sample size of the randomized controlled studies ranged from 4 to 64 participants, while the sample size of the quasi-experimental studies ranged from 9 to 356 participants. Among the 19 single-arm studies, nine studies had a sample size of fewer than 10 participants. Three of the single-arm studies used repeated measures while the rest adopted the pretest-posttest design.
In addition to the small sample size observed in the randomized controlled studies, it was noted that only four studies had an active comparison intervention that matched the time spent in the experimental condition. The remaining nine studies adopted either treatment-as-usual or waitlist control conditions. As for the quasi-experimental studies, almost all had some form of comparison interventions, but they were not matched to the intensity of the experimental conditions.
Outcome measurements used
A range of outcome measurements was used to evaluate the effectiveness of the interventions on functional outcomes. Six studies used data stored within the VR or AR games as one of the outcome measurements. Eight studies relied solely on self-developed evaluation tools that were not validated. These included questionnaire to job trainers on participants' vocational performance, checklists on number of correct steps of functional tasks, rating grid on appropriate social skills, questionnaire on participants' self-confidence in functional tasks, asking participants to recount important information or sequence of functional tasks etc. One study used the Experience Sampling Method (ESM) to track time spent with others as an objective social participation measure (47). The rest of the studies used validated outcome measurements such as the Social and Occupational Functioning Assessment Scale, Vineland Adaptive Behavior Scales, Social Functioning Scale, Personal and Social Performance scale, etc.
In addition, four studies on job interview training measured real-world functional outcomes such as successful job placements. One of the two driving simulation studies also used the attainment of a driver's permit as an outcome measure. Lastly, two studies included caregivers' reports of the participants' level of independence in community living.
Risk of bias assessments
ROB2 was used to evaluate the 13 randomized controlled studies and it was found that two studies had low risk of bias, seven studies had some concerns and four studies had high risk of bias. There was no particular domain which presented with several high risks.
The remaining 25 non-randomized controlled studies were assessed using ROBINS-I. Two studies were evaluated to have moderate risk of bias, 18 studies had serious risk of bias and five studies had critical risk of bias. Among the studies with serious and critical risk of bias, 19 were in domain of “bias due to confounding,” 11 were in the domain of “bias in measurement of outcomes,” and six were in the domain of “bias in selection of the reported result.”
The details of the ROB-2 and ROBINS-I assessments were presented using an adapted robvis visualization tool and found under Supplementary material.
Discussion
This systematic review presented an overview of how VR and AR were used in psychosocial rehabilitation for adults with neurodevelopmental conditions over the past 10 years. Almost half of these studies utilized VR software programs that were presented on computer or projection screens and participants interacted with virtual characters via keyboard or mouse/joystick. The rest of the VR studies used head-mounted display (HMD), Cardboard Viewer, 3D glasses, CAVE system, Brain-Computer Interface (BCI), or Kinect sensors. Three studies deployed AR in functional training.
In terms of the benefits of VR or AR-assistive psychosocial rehabilitation programs, a majority of the studies reported that participants were able to execute functional tasks more accurately or with more confidence after the interventions. These functional tasks ranged from instrumental activities of daily living, job interview and customer service, to social interactions and public speaking. However, the impact on real-world functional outcomes such as community integration, job tenure and social inclusion was unclear.
Clinical practice implications
Technological advancements in VR and AR present opportunities for clinicians to harness the unique features of VR and AR to augment traditional psychosocial rehabilitation programs. Many functional training programs involve practitioners assessing service users' functional performance through observation, followed by using a variety of approaches and methods to assist service users to perform functional activities to fulfill their valued life roles (56). Evaluating task performance in an objective and reliable manner requires close observation and calibration is a requisite for some observer-rated functional assessments (57–59). There is potential for VR to integrate sensor technology to detect body movements and gestures, videos to capture facial expressions and eye tracking to assess eye gaze, which will provide rich data on service users' functional performance in a detailed and objective manner. This will expedite functional performance evaluation and assist practitioners in setting suitable goals and intervention plans.
Some of the studies in this systematic review also made use of data from speech recognition, eye tracking motion sensors, artificial intelligence algorithms or motion-capture device in providing real-time feedback to participants during their rehabilitation (21, 34, 43). Such interactive elements enabled participants to adjust their behavior, communication and information processing strategies such that they could complete the gamified activities in a more adaptive manner. The study by Cox et al. showed that automated feedback provided by the virtual driving simulator did not differ from trainers' feedback (21). The studies on Virtual Interview Training for Transition Age Youth (VIT-TAY) also showed that a virtual job coach was able to provide additional support and feedback during the job interview training (37, 38). Therefore, it would be useful to establish the sensitivity and precision of VR or AR-enabled feedback systems and explore how these could maximize learning of new functional skills. Integrating artificial intelligence systems within VR or AR platforms may also enhance rehabilitation outcomes by matching learning activities to individual characteristics. It is unlikely for such feedback mechanisms to replace human practitioners. However, such features allow practitioners to train service users in a group and may consequently improve treatment efficiency.
Although HMDs are commonly used for full immersion in a virtual environment, only six studies in this systematic review utilized this mode of implementation. This could be due to concerns about weight and tightness of the headset or motion sickness (8). Therefore, some of the studies in this review utilized low resolution VR cardboard goggles or 3D glasses (33, 54), while others used CAVE-like systems to deliver the immersive effect (44, 52). The distinctive feature of immersive VR is its ability to provide a sense of presence with other people, by simulating real life situations such as standing in a crowded place or working in a noisy environment. Such effects could not be achieved when the virtual environment is presented on a computer screen. It will also not be feasible to set up a busy vocational training site or a public transportation network within a rehabilitation facility. Therefore, it will be useful for practitioners and researchers to harness the unique elements of VR and AR by utilizing their immersive features. The studies that utilized HMD, CAVE and 3D glasses reported generally good acceptance by participants, although there were concerns about sensory sensitivity in adults with ASD (28, 33, 53). There is a need for more research on the risk-benefit ratio of using fully immersive environment and strategies for managing motion sickness for adults with neurodevelopmental disorders.
Addressing functional outcomes
While some studies in this systemic review used AR or VR gamified platforms to teach steps of a functional task using repeated practice (e.g., ironing clothes, taking public transportation, and wiring an electrical socket), others used the technology to coach participants to respond to different real-life scenarios using the strategy approach (e.g., solving a problem at work in a boutique, interacting with job interviewers or strangers). Both the rehearsal/repeated practice approach and strategy approach have their merits and cater to service users with different functional goals. However, the theoretical and clinical framework underpinning many of these studies were not defined, with a lack of clarity on the rehabilitation approach. It will be useful for future VR/AR-assisted psychosocial rehabilitation programs to be grounded in a conceptual framework, to guide the intervention approach.
Although a number of studies in this review implemented VR or AR interventions in a laboratory setting, there were a few studies which integrated the VR or AR interventions within an existing service. Delivering the intervention in the context of an existing program in a natural setting appeared to yield benefits in functional outcomes and could facilitate generalizability. For example, the VIT-TAY intervention was implemented for youths with ASD attending pre-employment transition services (Pre-ETS) in schools (37, 38). Teachers in the schools volunteered to be trained to facilitate the VIT-TAY sessions, which paved the way for implementing this intervention beyond the research period. Participants receiving VIT-TAY not only improved in job interview skills, but also achieved higher rates of competitive employment (37, 38). Therefore, AR/VR interventions with features that bridged ongoing psychiatric rehabilitation programs appeared to maximize outcomes. Similarly, the study by Rus-Calafell et al. involved the use of a VR Soskitrain program as an adjunct to a manualized social skills intervention for persons with schizophrenia (54). Although the study was small-scale and of a single arm pretest-posttest design, participants showed significant improvements in social functioning as measured by the Social Functioning Scale, with gains maintained at 4-month follow-up (54). When service users could apply the skills learned during AR sessions directly to their regular rehabilitation program with clear strategies to transfer learning to the real-world, the impact appeared to be greater.
Combining VR with an evidence-based intervention such as Cognitive Behavior Therapy (CBT) could also yield benefits. Pot-Kolder et al. used four virtual social environments alongside CBT to work on social avoidance in persons with schizophrenia (47). Using Experience Sampling Method (ESM) to track actual time spent with others, the authors found a treatment effect on objective real-world social participation (47). Similarly, the Virtual Reality Job Interview Training (VR-JIT) was also recently incorporated into an Individual Placement and Support (IPS) service (60). IPS is an evidence-based model of supported employment but not all service users achieve positive outcomes (61). Preliminary findings suggested that IPS non-responders benefited from VR-JIT as shown by higher employment rates within a shorter time, compared to non-responders receiving IPS only (60). Therefore, augmenting an evidence-based program with AR or VR sessions appeared to enhance the effectiveness of the program. Further research into this area is warranted.
As described earlier, many studies in this review used in-game performance or self-developed assessments to measure functional outcomes, which may not approximate real-world functioning. Some studies used performance-based assessments such as job interview role-plays with standardized actors and administered by blinded raters. Although such assessments are more structured, their validity and reliability need to be established further. With the recent emphasis on implementation science, idiographic functional outcomes such as the Goal Attainment Scale have gained traction as they may be more effective than standardized measures in capturing real-world changes in individuals (62–64) The Experience Sampling Method (ESM) is another measurement method with high ecological validity, as it utilizes technology such as mobile phones to collect participants' day-to-day report of their experiences (65). This minimizes retrospective recall bias as participants are prompted at different times of the day to report their thoughts, emotions and behaviors on the spot (65). However, considerations need to be made for participants with IDD and schizophrenia, to ensure that they can understand and feel comfortable with this form of self-reporting.
In the field of psychosocial rehabilitation, the recovery movement has brought about the conceptualization of recovery as three domains: clinical, functional, and personal (66). While clinical recovery denotes the general stabilization of mental state, functional recovery constitutes the restoration of occupational, community, and social functioning (67). Personal recovery, however, is the process of experiencing hope, empowerment, identity, connectedness, and meaning (68, 69). As the ultimate goal of psychosocial rehabilitation is to restore function and empower service users to experience personal recovery, it will be pertinent to include measurements of personal recovery when assessing long-term outcomes of VR or AR-assisted psychosocial rehabilitation programs. In addition, it was noted that only two studies in this review included caregivers' evaluation of the participants' functional independence. As many adults with neurodevelopmental conditions are supported by family or formal caregivers, it will be useful to obtain caregiver reports to corroborate self-report findings.
Quality of selected studies
Fewer than half of the studies in this systematic review were randomized controlled studies and majority of these randomized studies used treatment-as-usual or waitlist control conditions. Although there are mixed findings on the value of an active control intervention, having a treatment-as-usual control makes it difficult to rule out placebo effects or determine if differential outcomes are due to the therapeutic elements of the experimental conditions (70–72). Assessors were also not blinded in three of these studies, which could lead to bias in measurement of outcomes.
Among the nine single-arm studies with fewer than 10 participants, some used parametric statistical analyses even though there was no indication that the data was normally distributed. Many of these studies also failed to address confounding variables and presented with bias in measurement of outcomes. To ascertain the effectiveness of AR or VR-assisted psychosocial rehabilitation programs, larger randomized controlled studies with adequate blinding and active comparison groups will be necessary.
Study limitations and recommendations for future research
This systematic review adopted a narrative approach, as majority of the studies had small sample sizes and adopted a single arm pretest-posttest design. More robust and bigger studies are required in this area, so that meta-analyses could be conducted. Studies that delve deeper into the therapeutic components of VR or AR-enhanced psychiatric rehabilitation programs are also necessary. More research can be done on how VR or AR can facilitate strategy learning and enhance the effectiveness of evidence-based programs such as IPS. Robust studies could also be conducted on the use of VR or AR on other therapeutic interventions such as metacognition training (73, 74). In addition, the use of a consistent set of standardized functional outcome measures will enable the pooling of outcome data. With rapid advancements in the use of sensors and artificial intelligence in healthcare, it will also be useful to review their usage in psychosocial rehabilitation in the future.
This systematic review has presented preliminary promising findings on the use of virtual reality and augmented reality to enhance traditional training methods in psychosocial rehabilitation. There is a need for future research to move beyond feasibility trials and address scalability, implementation issues and dissemination across sociocultural contexts (75). As the cost of devices goes down with increased usage, it may also be useful to explore cost effectiveness of VR and AR-assisted interventions in psychosocial rehabilitation.
Conclusion
Virtual reality and augmented reality are deployed in various ways to augment the implementation of psychosocial rehabilitation for adults with neurodevelopmental disorders. Most interventions target skills training or strategy learning in the areas of community living, work and social participation. Preliminary positive findings of their effects on functional performance were presented in this systematic review. Delivering the AR or VR intervention with features that bridged ongoing psychiatric rehabilitation programs appeared to have a positive impact on functional outcomes. In addition, combining VR or AR with an evidence-based intervention could enhance its benefits. There is potential to harness the useful features of virtual and augmented reality and integrate other technology to enhance treatment outcomes. More robust studies using ecologically valid outcome measures will be needed to establish their effects on real-world functional outcomes.
Resource identification initiative
PROSPERO (RRID:SCR_019061).
Data availability statement
The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
B-LT and AM: conception and design. B-LT, JS, SY, HL, DN, and CC: article search and critical appraisal. All authors: manuscript writing and approval.
Funding
This study was funded by the Ministry of Education-Singapore Institute of Technology (SIT) Ignition Grant (R-MOE-E103-F018).
Acknowledgments
The authors would like to acknowledge the assistance of SIT Library in obtaining full texts of the research articles.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2022.1055204/full#supplementary-material
Abbreviations
AR, Augmented Reality; ASD, Autism Spectrum Disorders; ID, Intellectual Disabilities; IDD, Intellectual and Developmental Disabilities; VR, Virtual Reality.
References
1. World Health Organization: Division of Mental Health & WHO Initiative of Support to People Disabled by Mental Illness. Psychosocial Rehabilitation : A Consensus Statement. Geneva: World Health Organization. (1996). Available online at: https://apps.who.int/iris/handle/10665/60630 (accessed September 12, 2022).
2. MacDonald S, Summers SJ. Psychosocial interventions for people with intellectual disabilities and dementia: a systematic review. J Appl Res Intellect Disabilit. (2020) 33:839–55. doi: 10.1111/jar.12722
3. Vita A, Barlati S. The implementation of evidence-based psychiatric rehabilitation: challenges and opportunities for mental health services. Front Psychiatr. (2019) 1:147. doi: 10.3389/fpsyt.2019.00147
4. Ben-Zeev D. Technology-based interventions for psychiatric illnesses: Improving care, one patient at a time. Epidemiol Psychiatr Sci. (2014) 23:317–21. doi: 10.1017/S2045796014000432
5. Lancioni GE, Singh NN, O'Reilly MF, Sigafoos J, Alberti G, Zimbaro C, et al. Using smartphones to help people with intellectual and sensory disabilities perform daily activities. Front Public Health. (2017) 24:5. doi: 10.3389/fpubh.2017.00282
6. Montoya-Rodríguez MM, de Souza Franco V, Tomás Llerena C, Molina Cobos FJ, Pizzarossa S, García AC, et al. Virtual reality and augmented reality as strategies for teaching social skills to individuals with intellectual disability: a systematic review. J Intellect Disabilit. (2022) 2022:174462952210891. doi: 10.1177/17446295221089147
7. Chang YJ, Kang YS, Huang PC. An augmented reality (AR)-based vocational task prompting system for people with cognitive impairments. Res Dev Disabil. (2013) 34:3049–56. doi: 10.1016/j.ridd.2013.06.026
8. Tan BL, Guan FY, Leung IMW, Kee SYM, Devilly OZ, Medalia A. A gamified augmented reality vocational training program for adults with intellectual and developmental disabilities: a pilot study on acceptability and effectiveness. Front Psychiatry. (2022) 13:966080. doi: 10.3389/fpsyt.2022.966080
9. Karami B, Koushki R, Arabgol F, Rahmani M, Vahabie AH. Effectiveness of virtual/augmented reality–based therapeutic interventions on individuals with autism spectrum disorder: a comprehensive meta-analysis. Front Psychiatr. (2021) 12:665326. doi: 10.3389/fpsyt.2021.665326
10. Rus-Calafell M, Garety P, Sason E, Craig TJK, Valmaggia LR. Virtual reality in the assessment and treatment of psychosis: a systematic review of its utility, acceptability and effectiveness. Psychol Med. (1945) 48:362–91. doi: 10.1017/S0033291717001945
11. Valmaggia LR, Latif L, Kempton MJ, Rus-Calafell M. Virtual reality in the psychological treatment for mental health problems: an systematic review of recent evidence. Psychiatry Res. (2016) 236:189–95. doi: 10.1016/j.psychres.2016.01.015
12. Freeman D, Reeve S, Robinson A, Ehlers A, Clark D, Spanlang B, et al. Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol Med. (2017) 47:2393–400. doi: 10.1017/S003329171700040X
13. Page MJ, Mckenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Br Med J. 372:n71. doi: 10.1136/bmj.n71
14. Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. Br Med J. (2019) 366:14898. doi: 10.1136/bmj.l4898
15. Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. (2016) 355:i4919. doi: 10.1136/bmj.i4919
16. McGuinness LA, Higgins JPT. Risk-of-bias VISualization (robvis): an R package and Shiny web app for visualizing risk-of-bias assessments. In: Research Synthesis Methods. New York, NY: John Wiley and Sons Ltd. (2021). p. 55–61. doi: 10.1002/jrsm.1411
17. Amado I, Brénugat-Herné L, Orriols E, Desombre C, dos Santos M, Prost Z, et al. A serious game to improve cognitive functions in schizophrenia: a pilot study. Front Psychiatry. (2016) 7:1–11. doi: 10.3389/fpsyt.2016.00064
18. Baker-Ericzén MJ, Smith L, Tran A, Scarvie K. A cognitive behavioral intervention for driving for autistic teens and adults: a pilot study. Autism Adulthood. (2021) 3:168–78. doi: 10.1089/aut.2020.0009
19. Bridges SA, Robinson OP, Stewart EW, Kwon D, Mutua K. Augmented reality: teaching daily living skills to adults with intellectual disabilities. J Spec Educ Technol. (2020) 35:3–14. doi: 10.1177/0162643419836411
20. Câmara J, Ferreira R, Teixeira L, Nóbrega J, Romeira C, Badia SB, et al. Efficacy of adaptive cognitive training through desktop virtual reality and paper-and-pencil in the treatment of mental and behavioral disorders. Virtual Real. (2021) 2021:6. doi: 10.1007/s10055-021-00559-6
21. Cox DJ, Brown T, Ross V, Moncrief M, Schmitt R, Gaffney G, et al. Can youth with autism spectrum disorder use virtual reality driving simulation training to evaluate and improve driving performance? An exploratory study. J Autism Dev Disord. (2017) 47:2544–55. doi: 10.1007/s10803-017-3164-7
22. de la Torre-Luque A, Valero-Aguayo L, de la Rubia-Cuestas EJ. Visuospatial orientation learning through virtual reality for people with severe disability. Intl J Disabil Dev Educ. (2017) 64:420–35. doi: 10.1080/1034912X.2016.1274022
23. Jeon B, Oh J, Son S. Effects of tooth brushing training, based on augmented reality using a smart toothbrush, on oral hygiene care among people with intellectual disability in Korea. Healthcare. (2021) 9:348. doi: 10.3390/healthcare9030348
24. Miller IT, Miller CS, Wiederhold MD, Wiederhold BK. Virtual reality air travel training using apple iPhone X and google cardboard: a feasibility report with autistic adolescents and adults. Autism Adulthood. (2020) 2:325–33. doi: 10.1089/aut.2019.0076
25. Panerai S, Catania V, Rundo F, Ferri R. Remote home-based virtual training of functional living skills for adolescents and young adults with intellectual disability: feasibility and preliminary results. Front Psychol. (2018) 9:1730. doi: 10.3389/fpsyg.2018.01730
26. Saiano M, Pellegrino L, Casadio M, Summa S, Garbarino E, Rossi V, et al. Natural interfaces and virtual environments for the acquisition of street crossing and path following skills in adults with Autism Spectrum Disorders: a feasibility study. J Neuroeng Rehabil. (2015) 12:1–13. doi: 10.1186/s12984-015-0010-z
27. Simões M, Bernardes M, Barros F, Castelo-Branco M. Virtual travel training for autism spectrum disorder: proof-of-concept interventional study. JMIR Serious Games. (2018) 20:8428. doi: 10.2196/games.8428
28. Bozgeyikli L, Bozgeyikli E, Raij A, Alqasemi R, Katkoori S, Dubey R. Vocational rehabilitation of individuals with autism spectrum disorder with virtual reality. ACM Trans Access Comput. (2017) 10:3046786. doi: 10.1145/3046786
29. Burke SL, Bresnahan T, Li T, Epnere K, Rizzo A, Partin M, et al. Using virtual interactive training agents (ViTA) with adults with autism and other developmental disabilities. J Autism Dev Disord. (2018) 48:905–12. doi: 10.1007/s10803-017-3374-z
30. Burke SL, Li T, Grudzien A, Garcia S. Brief Report: improving employment interview self-efficacy among adults with autism and other developmental disabilities using virtual interactive training agents (ViTA). J Autism Dev Disord. (2021) 51:741–8. doi: 10.1007/s10803-020-04571-8
31. Giachero A, Quadrini A, Pisano F, Calati M, Rugiero C, Ferrero L, et al. Procedural learning through action observation: preliminary evidence from virtual gardening activity in intellectual disability. Brain Sci. (2021) 11:766. doi: 10.3390/brainsci11060766
32. Humm LB, Olsen D, Bell M, Fleming M, Smith M. Simulated job interview improves skills for adults with serious mental illnesses. Stud Health Technol Inform. (2014) 199:50–4. doi: 10.3233/978-1-61499-401-5-50
33. Kuper GE, Ksobiech K, Wickert J, Leighton F, Frederick E. An exploratory analysis of increasing self-efficacy of adults with autism spectrum disorder through the use of multimedia training stimuli. Cyberpsychol Behav Soc Netw. (2020) 23:34–40. doi: 10.1089/cyber.2019.0111
34. Smith MJ, Ginger EJ, Wright K, Wright MA, Taylor JL, Humm LB, et al. Virtual reality job interview training in adults with autism spectrum disorder. J Autism Dev Disord. (2014) 44:2450–63. doi: 10.1007/s10803-014-2113-y
35. Smith MJ, Fleming MF, Wright MA, Losh M, Humm LB, Olsen D, et al. Brief Report: vocational outcomes for young adults with autism spectrum disorders at six months after virtual reality job interview training. J Autism Dev Disord. (2015) 45:3364–9. doi: 10.1007/s10803-015-2470-1
36. Smith MJ, Fleming MF, Wright MA, Roberts AG, Humm LB, Olsen D, et al. Virtual reality job interview training and 6-month employment outcomes for individuals with schizophrenia seeking employment. Schizophr Res. (2015) 166:86–91. doi: 10.1016/j.schres.2015.05.022
37. Smith MJ, Sherwood K, Ross B, Smith JD, DaWalt L, Bishop L, et al. Virtual interview training for autistic transition age youth: a randomized controlled feasibility and effectiveness trial. Autism. (2021) 25:1536–52. doi: 10.1177/1362361321989928
38. Smith MJ, Sherwood K, Ross B, Oulvey EA, Monahan JA, Sipovic JE, et al. Scaling out virtual interview training for transition-age youth: a quasi-experimental hybrid effectiveness-implementation study. Career Dev Transit Except Individ. (2022) 40:205–14. doi: 10.1177/21651434221081273
39. Sohn BK, Hwang JY, Park SM, Choi JS, Lee JY, Lee JY, et al. Developing a virtual reality-based vocational rehabilitation training program for patients with schizophrenia. Cyberpsychol Behav Soc Netw. (2016) 19:686–91. doi: 10.1089/cyber.2016.0215
40. Strickland DC, Coles CD, Southern LB. JobTIPS: a transition to employment program for individuals with autism spectrum disorders. J Autism Dev Disord. (2013) 43:2472–83. doi: 10.1007/s10803-013-1800-4
41. Tsang MMY, Man DWK. A virtual reality-based vocational training system (VRVTS) for people with schizophrenia in vocational rehabilitation. Schizophr Res. (2013) 144:51–62. doi: 10.1016/j.schres.2012.12.024
42. Adery LH, Ichinose M, Torregrossa LJ, Wade J, Nichols H, Bekele E, et al. The acceptability and feasibility of a novel virtual reality based social skills training game for schizophrenia: preliminary findings. Psychiatry Res. (2018) 270:496–502. doi: 10.1016/j.psychres.2018.10.014
43. Amaral C, Mouga S, Simões M, Pereira HC, Bernardino I, Quental H, et al. A feasibility clinical trial to improve social attention in Autistic Spectrum Disorder (ASD) using a brain computer interface. Front Neurosci. (2018) 12:477. doi: 10.3389/fnins.2018.00477
44. Jacques C, Cloutier V, Bouchard S. The “decoding of social interactions in virtual reality” tasks for autism spectrum people: development of an intervention protocol and pilot testing. Ann Rev Cyberther Telemed. (2018) 16:148–52.
45. Kandalaft MR, Daniel DN, Krawczyk C, Allen TT, Chapman SB. Virtual reality social cognition training for young adults with high-functioning autism. J Autism Dev Disord. (2013) 43:34–44. doi: 10.1007/s10803-012-1544-6
46. Kumazaki H, Muramatsu T, Kobayashi K, Watanabe T, Terada K, Higashida H, et al. Feasibility of autism-focused public speech training using a simple virtual audience for autism spectrum disorder. Psychiatry Clin Neurosci. (2020) 74:124–31. doi: 10.1111/pcn.12949
47. Pot-Kolder R, Geraets CNW, Veling W, van Beilen M, Staring ABP, Gijsman HJ, et al. Virtual-reality-based cognitive behavioural therapy versus waiting list control for paranoid ideation and social avoidance in patients with psychotic disorders: a single-blind randomised controlled trial. Lancet Psychiatry. (2018) 5:217–26. doi: 10.1016/S2215-0366(18)30053-1
48. du Sert OP, Potvin S, Lipp O, Dellazizzo L, Laurelli M, Breton R, et al. Virtual reality therapy for refractory auditory verbal hallucinations in schizophrenia: a pilot clinical trial. Schizophr Res. (2018) 197:176–81. doi: 10.1016/j.schres.2018.02.031
49. Vass E, Simon V, Fekete Z, Lencse L, Mária E, Kis B, et al. A novel virtual reality-based theory of mind intervention for outpatients with schizophrenia: a proof-of-concept pilot study. Clin Psychol Psychother. (2020) 28:727–38. doi: 10.1002/cpp.2519
50. White SW, Richey JA, Gracanin D, Coffman M, Elias R, LaConte S, et al. Psychosocial and computer-assisted intervention for college students with autism spectrum disorder: preliminary support for feasibility. Educ Train Autism Dev Disabil. (2016) 51:307–17. Available online at: https://www.jstor.org/stable/24827526
51. Dellazizzo L, Potvin S, Phraxayavong K, Dumais A. Exploring the benefits of virtual reality-assisted therapy following cognitive-behavioral therapy for auditory hallucinations in patients with treatment-resistant Schizophrenia: a proof of concept. J Clin Med. (2020) 9:1–15. doi: 10.3390/jcm9103169
52. Maskey M, Rodgers J, Ingham B, Freeston M, Evans G, Labus M, et al. Using virtual reality environments to augment cognitive behavioral therapy for fears and phobias in autistic adults. Autism Adulthood. (2019) 1:134–45. doi: 10.1089/aut.2018.0019
53. Vass E, Simon V, Fekete Z, Lencse L, Ecseri M, Kis B, et al. A novel virtual reality-based theory of mind intervention for outpatients with schizophrenia: a proof-of-concept pilot study. Clin Psychol Psychother. (2021) 28:727–38.
54. Rus-Calafell M, Gutiérrez-Maldonado J, Ribas-Sabaté J. A virtual reality-integrated program for improving social skills in patients with schizophrenia: a pilot study. J Behav Ther Exp Psychiatry. (2014) 45:81–9. doi: 10.1016/j.jbtep.2013.09.002
55. Pot-Kolder R, Veling W, Geraets C, van der Gaag M. Effect of virtual reality exposure therapy on social participation in people with a psychotic disorder (VRETp): study protocol for a randomized controlled trial. Trials. (2016) 17:25. doi: 10.1186/s13063-015-1140-0
56. American Occupational Therapy Association. Occupational therapy practice framework: domain and process—fourth edition. Am J Occup Ther. (2020) 74(Suppl.2):7412410010p1–87. doi: 10.5014/ajot.2020.74S2001
57. Merritt BK. Validity of using the assessment of motor and process skills to determine the need for assistance. Am J Occup Ther. (2011) 65:643–50. doi: 10.5014/ajot.2011.000547
58. Aubin G, Chapparo C, Gélinas I, Stip E, Rainville C. Use of the perceive, recall, plan and perform system of task analysis for persons with schizophrenia: a preliminary study. Aust Occup Ther J. (2009) 56:189–99. doi: 10.1111/j.1440-1630.2007.00725.x
59. Nott MT, Chapparo C, Heard R. Reliability of the perceive, recall, plan and perform system of task analysis: a criterion-referenced assessment. Aust Occup Ther J. (2009) 56:307–14. doi: 10.1111/j.1440-1630.2008.00763.x
60. Smith MJ, Smith JD, Blajeski S, Ross B, Jordan N, Bell MD, et al. An RCT of virtual reality job interview training for individuals with serious mental illness in IPS supported employment. Psychiatr Serv. (2022) 73:1027–38. doi: 10.1176/appi.ps.202100516
61. Christensen TN, Wallstrøm IG, Stenager E, Bojesen AB, Gluud C, Nordentoft M, et al. Effects of individual placement and support supplemented with cognitive remediation and work-focused social skills training for people with severe mental illness: a randomized clinical trial. J Am Med Assoc Psychiatr. (2019) 76:1232–40. doi: 10.1001/jamapsychiatry.2019.2291
62. Ruble L, McGrew JH, Toland MD. Goal attainment scaling as an outcome measure in randomized controlled trials of psychosocial interventions in autism. J Autism Dev Disord. (2012) 42:1974–83. doi: 10.1007/s10803-012-1446-7
63. Edbrooke-Childs J, Jacob J, Law D, Deighton J, Wolpert M. Interpreting standardized and idiographic outcome measures in CAMHS: what does change mean and how does it relate to functioning and experience? Child Adolesc Ment Health. (2015) 20:142–8. doi: 10.1111/camh.12107
64. Herdman KA, Vandermorris S, Davidson S, Au A, Troyer AK. Comparable achievement of client-identified, self-rated goals in intervention and no-intervention groups: reevaluating the use of Goal Attainment Scaling as an outcome measure. Neuropsychol Rehabil. (2019) 29:1600–10. doi: 10.1080/09602011.2018.1432490
65. Cho H, Gonzalez R, Lavaysse LM, Pence S, Fulford D, Gard DE. Do people with schizophrenia experience more negative emotion and less positive emotion in their daily lives? A meta-analysis of experience sampling studies. Schizophr Res. (2017) 183:49–55. doi: 10.1016/j.schres.2016.11.016
66. Tse S, Davidson L, Chung K, Fai, Ng KL, Yu CH. Differences and similarities between functional and personal recovery in an asian population: a cluster analytic approach. Psychiatry. (2014) 77:41–56. doi: 10.1521/psyc.2014.77.1.41
67. Roosenschoon BJ, Kamperman AM, Deen ML, van Weeghel J, Mulder CL. Determinants of clinical, functional and personal recovery for people with schizophrenia and other severe mental illnesses: a cross-sectional analysis. PLoS ONE. (2019) 14:222378. doi: 10.1371/journal.pone.0222378
68. Leamy M, Bird V, le Boutillier C, Williams J, Slade M. Conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. Br J Psychiatr. (2011) 199:445–52. doi: 10.1192/bjp.bp.110.083733
69. Shanks V, Williams J, Leamy M, Bird VJ, le Boutillier C, Slade M. Measures of personal recovery: a systematic review. Psychiatr Serv. (2013) 64:974–80. doi: 10.1176/appi.ps.005012012
70. Au J, Gibson BC, Bunarjo K, Buschkuehl M, Jaeggi SM. Quantifying the difference between active and passive control groups in cognitive interventions using two meta-analytical approaches. J Cogn Enhanc. (2020) 4:192–210. doi: 10.1007/s41465-020-00164-6
71. Datta L. Why an active comparison group makes a difference and what to do about it. J Multidiscip Eval. (2007) 4:1–12.
72. Munder T, Geisshüsler A, Krieger T, Zimmermann J, Wolf M, Berger T, et al. Intensity of treatment as usual and its impact on the effects of face-to-face and internet-based psychotherapy for depression: a preregistered meta-analysis of randomized controlled trials. Psychother Psychosom. (2022) 91:200–9. doi: 10.1159/000521951
73. Drigas A, Mitsea E, Skianis C. Virtual reality and metacognition training techniques for learning disabilities. Sustainability. (2022) 14:1610170. doi: 10.3390/su141610170
74. Cella M, Reeder C, Wykes T. Lessons learnt? The importance of metacognition and its implications for Cognitive Remediation in schizophrenia. Front Psychol. (2015) 6:1259. doi: 10.3389/fpsyg.2015.01259
Keywords: psychosocial rehabilitation, neurodevelopmental disorders, functional outcomes, community living, employment, social participation, systematic review, virtual reality and augmented reality
Citation: Tan B-L, Shi J, Yang S, Loh H, Ng D, Choo C and Medalia A (2022) The use of virtual reality and augmented reality in psychosocial rehabilitation for adults with neurodevelopmental disorders: A systematic review. Front. Psychiatry 13:1055204. doi: 10.3389/fpsyt.2022.1055204
Received: 27 September 2022; Accepted: 25 November 2022;
Published: 14 December 2022.
Edited by:
Mariyana Schoultz, Northumbria University, United KingdomReviewed by:
Roberto Keller, ASL Città di Torino, ItalyAthanasios Drigas, National Centre of Scientific Research Demokritos, Greece
Copyright © 2022 Tan, Shi, Yang, Loh, Ng, Choo and Medalia. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Bhing-Leet Tan, bhingleet.tan@singaporetech.edu.sg