Assessment of Task Demand and Usability of a Virtual Reality-Based Rehabilitation Protocol for Combat Related Traumatic Brain Injury From the Perspective of Veterans Affairs Healthcare Providers: A Pilot Study

The risk of traumatic brain injury (TBI) is significantly higher among Veterans compared to non- Veterans. Access to treatment for TBI and post concussive symptoms is sometimes difficult, because of barriers related to distance, finances, and public safety (i.e., COVID-19 infection). Virtual reality rehabilitation (VRR) offers an opportunity to incorporate a virtual space into a rehabilitation environment. To our knowledge, VRR has not been used to assist Veterans with TBI and related health problems with Instrumental Activities of Daily Living (iADLs). The purpose of this study is to investigate the usability of a novel VRR ADL and iADL training protocols, developed by the Gaming Research Integration for Learning Laboratory (GRILL^®) at the Air Force Research Laboratory, for cognitive rehabilitation for Veterans with a TBI. We deployed a prototype protocol among healthcare providers (n = 20) to obtain feedback on usability, task demand, and recommended adjustments. Our preliminary analysis shows that providers found the VRR protocol involved low physical demand and would likely recommend it to their patients. Although they had some concerns with vertigo-like symptoms from using a digital technology, they believed the protocol would improve iADL functioning and was a good addition to pre-existing rehabilitation protocols. These outcomes provide justification for more impactful studies investigating the effectiveness of this protocol among Veterans with TBI.

Introduction

Since 2001, up to 23% of deployed service members sustained a traumatic brain injury (TBI) while in combat, and there are currently 415,000 within the Veterans Affairs (VA) Healthcare System receiving treatment for a TBI (Apkarian et al., 2004). A TBI is a life-altering condition that can significantly reduce quality of life and increase depression, anxiety, overall pain, headaches, sleep disturbances, and impairment in activities of daily living (ADLs) (Bryson-Campbell et al., 2013). If left untreated, these symptoms can be debilitating, resulting in attenuation in brain structure and function, leading to further disability and lower quality of life.

Healthcare access is limited for certain underrepresented populations including veterans who live in rural areas or have no access to VA based care (Ohl et al., 2018). This finding is mirrored in a recent survey conducted by the United States pain foundation on April 15th, 2020, in which 77.4% of respondents faced barriers to medical care, such as travel, financial distress, and stigmatization, leading to untreated healthcare needs during the pandemic (Chronic Pain and COVID-19, 2020). Additionally, 33.7% of respondents reported in this survey conducted during the early stages of the pandemic that they relied on in-person interaction for Physical Therapy (PT) which was not possible while 20.5% said they had surgery or other appointments that were cancelled (Chronic Pain and COVID-19, 2020). The SARS-CoV-2 pandemic of 2020, commonly known as COVID-19, has exacerbated these issues by exposing potentially high-risk populations to a deadly virus when visiting a clinic or health care center (Chronic Pain and COVID-19, 2020; Ronnie, 2021). These findings illustrate the need for an effective in-home rehabilitation treatment protocol.

Recent reviews including meta-analyses report the utility and benefits of in-home, virtual reality-based rehabilitation (VRR) protocols to treat cognitive, mental, and physical health conditions, while reducing contact with high risk environments, such as a hospital (McLay et al., 2011; Morina et al., 2015; Guest et al., 2019; Alashram et al., 2020; Mirelman et al., 2010; Powers and Emmelkamp, 2008). VRR offers a virtual space for a user to potentially practice a protocol in the absence of clinical services - it provides an ecologically valid environment (Nahin, 2017). Through activation of somatosensory and premotor circuitry systems, the use of a virtual embodiment as an immersive VR technique can influence completion of ADLs in patients, and improve function (Mallari et al., 2019). Established research supports VRR as a treatment strategy for burn pain, acute pain, and experimentally induced pain (Greenleaf, 2017; Hoffman et al., 2000; Hoffman et al., 2006; Hoffman et al., 2007; Carrougher et al., 2009; Greenhalgh et al., 2019). However, its use in cognitive paradigms has been limited. Furthermore, a previous meta-analysis found that patients with PTSD who received exposure therapy through VRR had significantly lower posttraumatic stress disorder (PTSD) and depressive symptoms compared to waitlist controls post treatment (Kothgassner et al., 2019). Additionally, focus groups, assessing a virtual grocery store exposure intervention to treat PTSD-TBI symptoms, found the intervention to be usable, relevant, and acceptable (Levy et al., 2019).

To utilize this technology, stakeholder input is necessary to provide the best and most efficient protocol to the patient. This input is critical, especially for developing unique and innovative in-home healthcare technologies for a population that values self-sufficiency (Hart and Staveland, 1988; Kulich et al., 2020). The Veterans Affairs Healthcare System (VHA) Rehabilitation Service, including polytrauma systems of care, consists of clinicians and providers with distinct expertise to improve the quality of life among veterans with TBI. Because of this expertise, their feedback is essential to ensure deployment of a VRR protocol that is usable and acceptable for Veterans with a TBI.

The VRR protocol is based on the characteristics of the patients seen at VA Polytrauma Service, particularly those patients who are in the polytrauma transitional rehabilitation unit for 4–12 weeks after being an inpatient. These patients are recovering from severe and moderate brain injuries and many also have polytrauma injuries. Therefore, we designed tasks that include problems in executive function, motor control and organization of activities of daily living that are focused in the apartment location.

Thus, the purpose of this preliminary effort was to survey healthcare providers who treat Veterans with TBI on their opinions regarding a VRR protocol for TBI patients in terms of cognitive load, ease of use, and appropriateness of using VR with this population; therefore we collected both qualitative and quantitative feedback from these healthcare providers.

Methods

Inclusion/Exclusion Criteria

The Stanford University School of Medicine and Veterans Affairs Palo Alto Health Care System (VAPAHCS) Institutional Review Board (IRB) approved the project in November 2020. We recruited our participants from the healthcare providers working at one of the five national polytrauma centers located at VA Palo Alto Health Care System (VAPAHCS). Participants were included if 1) they were over 18; 2) were healthcare providers; 3) provided care for Veterans with a TBI; and 4) were willing to dedicate 20 min of their time to complete our protocol. Exclusion criteria included 1) self-reported discomfort or anxiety with closed spaces; 2) vertigo or related conditions that would hinder safety during a VR protocol; and 3) were pregnant.

Gaming Research Integration for Learning Laboratory^® Development Phase

The GRILL^® team developed, performed and tested a VR protocol which included various interactive ADL and IADL applications on-site at the AFRL facility located in Dayton, Ohio just outside of Wright Patterson Air Force Base prior to deployment at the VAPAHCS. This AFRL facility houses a 2000 square foot VR experimentation area where they are able to test, receive feedback, make modifications and re-deploy applications in-house for rapid prototyping. It is also the primary location where they mentor the local area college and high school students selected to participate in the AFRL Summer Research Challenge.

First, they created a virtual model of the VAPAHCS Polytrauma apartment as the target environment to deploy their ADL and IADL applications (Figure 1). Created in the Unreal Engine, this environment allowed the GRILL^® development team to carefully map strategic virtual locations to deploy their applications for the Veterans under treatment. The first application developed was a room cleaning task which included vacuuming, picking up items on a table and returning pillows to a couch. These tasks would be executed in the context of the Polytrauma apartment. The vacuuming task (Figure 2) showed areas of completion and areas that require coverage during execution of the task so Veterans could monitor their progress and manage their time to completion. The intent was to help veterans create a mental plan to complete a task they would be expected to execute independently post treatment. As for picking up items on a table and returning pillows to the couch, the Veteran patients were required to interact and grab items in the VE and move them to their proper location as described in the pre-execution instructions. This task will enable the Veterans to assess the cleanliness of their surrounding area and develop plans to clean and organize it if required.

FIGURE 1

FIGURE 1. The virtual apartment with task summary and breakdown.

FIGURE 2

FIGURE 2. A patient vacuuming the floor.

The second application developed was for medication administration and management (Figure 3). In this iADL application, the patient was required to open the medical vial, remove the appropriate number of medication tabs and re-close the vial (Figure 4). This was considered one of the most critical tasks to enable independent living as Veterans would be required to manage their daily medication in-take while being distracted by normal day-to-day events. The GRILL^® experimented with various ways to simulate user feedback while opening and closing the vial. Audio and physical interactions were considered and implemented as well. The GRILL^® researchers viewed this task as being one that would be most effective for learning if performed in conjunction with another ADL or IADL in which the Veteran would have to stop and resume their activities at various stages of completion.

FIGURE 3

FIGURE 3. A patient beginning the medication task.

FIGURE 4

FIGURE 4. A patient adding pills to the cup.

The final application developed for pilot testing was brushing teeth (Figure 5). This application required the patient to grab the toothbrush, open the toothpaste, apply the toothpaste to the brush, and finally move the brush uniformly over the teeth to ensure they are clean (Figure 6). The same audio and physical feedback challenges emerged when developing this application for pilot testing. The GRILL^® team successfully broke this task down into small manageable step so the Veterans could develop their own repeatable process to perform this task independently. As with the other tasks, this iADL application was developed within the bathroom near the sink in the virtual Polytrauma apartment enabling the Veteran to develop his or her own process to execute this task on a daily basis. This task and the others described in this section were developed to completion and included in the initial pilot testing conducted at VAPAHCS. They were developed as initial proofs of concept for pilot testing and to test this mode of clinical treatment would be effective for veterans with TBI.

FIGURE 5

FIGURE 5. A patient starting the brush teeth task.

FIGURE 6

FIGURE 6. A patient nearly finished brushing their teeth.

Equipment

The synthetic task environment is encapsulated in an executable that requires a laptop or workstation with a recent version of Windows (64-bit) installed. The software is written in Unreal Engine and is compiled by Unreal Engine into a simple executable and associated executable. In addition to the workstation, a SteamVR-compatible or Oculus virtual reality tethered headset, along with the Oculus Software Development Kit (SDK) and associated software is used with the simulation.

Once the prototypes were deemed ready for pilot testing, the applications were sent via secure file transfer to VAPAHCS. In addition, they provided documentation to install and deploy the applications on the head-mounted displays (HMDs) at VAPAHCS. The GRILL^® researchers provided training to the VAPAHCS team on the VR applications. Demonstrations were also provided in real-time as they performed each ADL in VR through a Zoom Conference. On Zoom, the GRILL^® researchers showed the VAPAHCS clinicians how to accurately complete each ADL task, how the tasks were scored and identified potential areas of l difficulty. Various modifications/reiterations were made during these feedback sessions to enable seamless VR usage during the planned clinical pilot tests with the providers.

Deployment Phase

Prior to initiation of the study, participants were asked about their experience with VR technology. Research staff provided brief training on the VR equipment. The HTC VIVE Pro Headset was used for this pilot. Participants were then asked to complete a tutorial to demonstrate their comfort using the VR equipment prior to starting the protocol. Once the provider was comfortable with the equipment, the protocol was initiated.

Surveys

Following completion of the protocol, investigators asked participants to fill out the following surveys:

The NASA-TLX (the Raw Score only); a psychometric test, commonly used for novel medical equipment, to assess self-reported task related cognitive load with subscales on mental demand, physical demand, temporal demand, performance, effort, and frustration on a scale from 0 to 100 (0 = task success; 100 = task failure) (Hart, 2006; Greenhalgh et al., 2020).

General Usability survey; a survey about the impact of the VR was collected via a 10 cm visual analogue survey (0 = Negative Outcome; 10 = Positive Outcome) regarding perceptions on aspects related to device usability. Questions include but are not limited to applicability, safety, ease, and innovation. Participants also indicated what aspect of the technology they liked the most: 1) usability, 2) ease, 3) efficiency, 4) independence, or 5) comfort (Lackey et al., 2016). Finally, the usability survey included an additional section for open ended feedback, where participants provided critiques, and investigators made notes., VR survey; a seven-point Likert scale (1 = strongly disagree and 7 = strongly agree) assessing virtual reality specific statements, such as rehabilitation appropriateness, symptom reduction, pain reduction, cognitive improvement, and potential side effects from the VR itself (i.e., dizziness, vertigo, etc.).

Data Analysis

Data was analyzed using Statistical Package for the Social Sciences (SPSS) V21 (IBM, 2013). Data analytic methods for each survey are listed below:

NASA-TLX: Means and standard deviations were first calculated. Next, a Kolmogorov-Smirnov test was then used to identify if there were significant differences within our participants.

General Usability Survey: Means and standard deviations were calculated for the linear analog data. Frequencies were tallied for questions pertaining to the aspects of the technology that the participants liked most. Thematic analysis was employed for the open-ended responses. Investigators reviewed the open-ended feedback and, using a thematic analysis, identified the top five most common answers regarding “device accessibility, usability, and integration.” We performed this for the “like,” “dislike,” and “suggestion questions”.

Results

We recruited 20 providers for the study. As shown in Table 1, the average age was 39.8 (SD = 10.6), and seven (50%) held a professional degree. Four (28.6%) of our participants were trained as physical therapists with the one other profession being a dietician (7.1%).

TABLE 1

TABLE 1. Provider demographic information.

As shown in Table 2, no significant differences were reported among our participants in terms of task demand, indicating consistent outcomes when using the technology.

TABLE 2

TABLE 2. Raw NASA-TLX Survey results regarding the level of cognitive and physical demand of the VRR protocol designed for Providers who work with Veterans with a TBI.

Results from the general usability survey found a significant difference in innovation (p = 0.04), indicating potential non-normality among innovation results (see Table 3).

TABLE 3

TABLE 3. Survey results conducted in providers regarding feasibility of the VRR protocol for the Veteran TBI patient population.

When asked about the most appealing traits of the virtual reality protocol, 11 (55.0%) of our participants found the usability of the technology to be the most enticing piece of the equipment (See Table 4).

TABLE 4

TABLE 4. General Usability Survey results indicating most appealing features of the VRR protocol for the Veteran with TBI population.

As seen in Table 5 below, the majority of the participants reported very positive feedback on the VR protocol, particularly in terms of its ability to work well with providers’ rehabilitation protocols, felt that it could improve functioning and cognitive skills, and found it engaging. Most providers had no prior experience with VR.

TABLE 5

TABLE 5. Survey Results to all Questions. Question disagree neutral agree.

Open ended questions from the general usability survey revealed commonalities in terms of features that the providers liked and disliked, as well as in their suggestions for protocol refinement (See Table 6).

TABLE 6

TABLE 6. Thematic analysis results from the open ended questions in the general usability survey reveals common likes and common suggested changes.

Discussion

The purpose of this study was receive feedback from practitioner stakeholders regarding the suitability of the GRILL^®’s VRR protocol for deployment among Veterans with TBI for assessment and rehabilitation of cognitive problems. VRR has the potential to treat complex injuries, such as TBI, in a way that is safe and promotes self-sufficiency. However, because it is a relatively new concept and most people have not experienced VR, its feasibility and appropriateness for patients with TBI must be determined prior to implementation. Implementing an inappropriate protocol could negatively impact the recovery of veterans with TBI. Results from our survey of providers, who specialize in treating Veterans with TBI, indicates that our newly developed VRR protocol is appropriate for this patient population and provides a solution to potential barriers related to symptomatology and healthcare access.

Healthcare providers reported relatively low task related demand performing the GRILL^®’s protocol, based on outcomes closer to “0,” meaning task success, than “100,” indicating task failure (Hart, 2006; Greenhalgh et al., 2020). This result indicates that healthcare providers do not believe the GRILL^®’s protocol to be overly demanding for Veterans with TBI. Task related demand was similar across all our participants, based on non-significant outcomes for each domain. Although literature on VR related demand is scarce, previous research on virtual utility-based completion of tasks among Soldiers found relatively low task demand scores outcomes among all six domains, indicating this would not be as challenging to complete as other procedures (Maggio et al., 2019). It should be noted that Mental Demand and Effort were higher than other domains. Such an outcome is to be expected, as VR is a relatively new technology. Previous literature finds that operating innovative medical equipment requires significant mental demand compared to current gold standard treatments (Lackey et al., 2016). Therefore, while this protocol shows promise, additional practice is necessary to reduce the personal stress one experiences during this protocol.

Results of this study highlight the potential clinical use of a VRR protocol for Veterans recovering from a TBI through iADL cognitive retraining. Usability outcomes were the highest among perceptions of the technology’s “innovative” appeal, as well as if the provider would “recommend” the technology to a fellow healthcare provider, or a patient with a TBI. The outcomes align with previous literature, which find general excitement about innovative aspects of Internet of Things (IoT), and VR based protocols, particularly when treating the cognitive and behavioral pathologies in Veterans (Dicianno et al., 2018; Kirsch, 2019). Furthermore, the innovative utility has been shown to work among older adults before, specifically with recalling tasks and specific details (Dicianno et al., 2018; Kirsch, 2019). Such protocols add a safer element to rehabilitation, while offering an entertaining distraction to their symptoms. This correlated with personal feedback from providers, who found that the “gamified” protocol, as well as the practical applications of performing VR based ADLs, would integrate well into current rehabilitation protocols.

Despite promise, healthcare providers listed several concerns with the protocol. Seventeen, or 85%, of our participants listed they were concerned about possible VR related side-effects such as dizziness and vertigo. In addition, providers shared some challenges with the technology, which included experiencing technical difficulties, which did not allow them to successfully complete some of the tasks. The frustration they experienced potentially explains the significant difference in “innovative” outcomes for device usability. One of the primary concerns among VR is motion sickness related to wearing a headset in an immersed space over an extended period of time (Ohl et al., 2018). A previous study found that the human machine interface, as well as the immersion aspect, were chief causes of dizziness when using VR. VR-related dizziness is potentially problematic among a population with symptoms that impact cognitive and perception, such as a TBI. Further research and design should attempt to create an interface that is easy for users to understand, and update the immersive space so the user can navigate without feeling the negative impacts of this medium.

Study Limitations

The following investigation included several limitations. It used a convenience sample of healthcare providers at one healthcare clinic. Veterans were not included in the study. This was intentional as we wanted to affirm the protocol would be safe and effective. However, it does present information bias that will potentially change outcomes if provided to patients, particularly those with significant cognitive and behavioral TBI related symptoms. Further research that garners the same usability feedback from Veterans with TBIs is necessary to validate the results of this study. Additionally, as this was a convenience sample of healthcare providers from one clinic it cannot be assumed the data is representative of the general healthcare provider population. Because the home institution is based in Palo Alto, CA, healthcare providers potentially have easier access to and experience with advanced equipment, such as VRR. Previous research suggests providers from more rural areas have difficulty understanding advanced medical technology (Lackey et al., 2016). Further work is needed to gauge the digital divide, if any, among healthcare providers and clinics in the United States.

As for the technology, the GRILL^®’s software experienced several technical difficulties that prolonged the time to complete the protocol. These glitches potentially impacted perceptions of device usability as providers. Additionally, as seen in the feedback, directions provided in the software were too small for some of our participants. They believed that Veterans recovering from a TBI would have significant difficulties processing the directions, thus hindering the utility of the VRR protocol. Engineers and researchers plan to release a second-generation GRILL^®’s software that addresses provider concerns and technological problems, and as such be ready for release to Veterans.

Conclusion

The purpose of this effort was to collect initial feedback from healthcare providers on the usability and appropriateness of VRR therapies to treat Veterans with TBI. Outcomes of this preliminary effort, which involved healthcare providers, indicated the VRR protocol offered by GRILL^®’s, while promising, needs additional technical development to better align with patient needs and comfort. Next steps include ADL task refinement followed by additional feedback from practitioners with insight into current clinical treatment regimens. Provider’s recommendations such as improving the interface and addressing potential VR related side effects should be addressed prior to including Veterans in a similar study. Once providers deem the technologies to be appropriate and safe for clinical use, an experiment will be conducted with the target population.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Stanford IRB. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

MA, QK, EM, CF, and TR conceptualized the project and wrote the protocol. MG with MA, EM, and QK conducted the protocol and analyzed the data. MG, QK, MA, and CF wrote the manuscript. EM, MT, JC, DA, OH, EM, and QK gave feedback to MA, CF, MG, and TR.

Funding

The Headache Center for Excellence provided resources for the VA Palo Alto staff to carry out this research. VA office of Academic Affairs funded MG for the VA polytrauma postdoctoral Fellowship. GRILL was funded by the Air Force Research Lab (AFRL).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to thank the Headache Center for Excellence at VA Palo Alto, and all the providers in Rehabilitation Service at VA Palo Alto Healthcare System for their participation. We also appreciate the engineers at the GRILL^® lab as well as those at the Naval Postgraduate School.

References

Alashram, A. R., Padua, E., Hammash, A. K., Lombardo, M., and Annino, G. (2020). Effectiveness of virtual reality on balance ability in individuals with incomplete spinal cord injury: A systematic review. J. Clin. Neurosci. 72, 322–327. doi:10.1016/j.jocn.2020.01.037

PubMed Abstract | CrossRef Full Text | Google Scholar

Apkarian, V. A., Sosa, Y., Krauss, B. R., Thomas, S. P., Fredrickson, B. E., Levy, R. E., et al. (2004). Chronic pain patients are impaired on an emotional decision-making task. Pain 108, 129–136. doi:10.1016/j.pain.2003.12.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Bryson-Campbell, M., Shaw, L., O'Brien, J., Holmes, J., and Magalhaes, L. (2013). A scoping review on occupational and self identity after a brain injury. Work 44 (1), 57–67. doi:10.3233/wor-2012-01561

PubMed Abstract | CrossRef Full Text | Google Scholar

Carrougher, G. J., Hoffman, H. G., Nakamura, D., Lezotte, D., Soltani, M., Leahy, L., et al. (2009). The effect of virtual reality on pain and range of motion in adults with burn injuries. J. Burn Care Res. 30, 785–791. doi:10.1097/bcr.0b013e3181b485d3

PubMed Abstract | CrossRef Full Text | Google Scholar

Chronic Pain and COVID-19 (2020). West Hartford, CT: U.S. Pain Foundation. https://uspainfoundation.org/wp- content/uploads/2020/04/COVID19-report.pdf.

Dicianno, B. E., Joseph, J., Eckstein, S., Zigler, C. K., Quinby, E., Schmeler, M. R., et al. (2018). The voice of the consumer: a survey of veterans and other users of assistive technology. Mil. Med. 18311-12, e518–e525. doi:10.1093/milmed/usy033

CrossRef Full Text | Google Scholar

Greenhalgh, M., Landis, J. M., Brown, J., Kulich, H., Bass, S., Alqahtani, S., et al. (2019). Assessment of Usability and Task Load Demand Using a Robot-Assisted Transfer Device Compared With a Hoyer Advance for Dependent Wheelchair Transfers. Am. J. Phys. Med. Rehabil. 98 (8), 729–734. doi:10.1097/PHM.0000000000001176

PubMed Abstract | CrossRef Full Text | Google Scholar

Greenhalgh, M., Blaauw, E., Deepak, N., St. Laurent, C. O. L. M., Cooper, R., Bendixen, R., et al. (2020). Usability and task load comparison between a robotic assisted transfer device and a mechanical floor lift during caregiver assisted transfers on a care recipient. Disabil. Rehabil. Assistive Tech., 1–7. doi:10.1080/17483107.2020.1818137

PubMed Abstract | CrossRef Full Text | Google Scholar

Greenleaf, W. How virtual and augmented reality will change the way we shop Palo Alto, CA: Techno FAQ. (2017).

Google Scholar

Guest, F., Marshall, C., and Stansby, G. (2019). Amputation and rehabilitation. Surgery (Oxford) 37 (2), 102–105. doi:10.1016/j.mpsur.2018.12.008

CrossRef Full Text | Google Scholar

Hart, S. G. (2006). Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 50 (9), 904–908. doi:10.1177/154193120605000909

CrossRef Full Text | Google Scholar

Hart, S. G., and Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Adv. Psychol. 52, 139–183. doi:10.1016/s0166-4115(08)62386-9

CrossRef Full Text | Google Scholar

Hoffman, H. G., Richards, T. L., Van Oostrom, T., Coda, B. A., Jensen, M. P., Blough, D. K., et al. (2007). The analgesic effects of opioids and immersive virtual reality distraction: evidence from subjective and functional brain imaging assessments. Anesth. Analg 105, 1776–contents. doi:10.1213/01.ane.0000270205.45146.db

PubMed Abstract | CrossRef Full Text | Google Scholar

Hoffman, H. G., Doctor, J. N., Patterson, D. R., Carrougher, G. J., and Furness, T. A. (2000). Virtual reality as an adjunctive pain control during burn wound care in adolescent patients. Pain 85, 305–309. doi:10.1016/s0304-3959(99)00275-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Hoffman, H. G., Seibel, E. J., Richards, T. L., Furness, T. A., Patterson, D. R., and Sharar, S. R. (2006). Virtual reality helmet display quality influences the magnitude of virtual reality analgesia. The J. Pain 7, 843–850. doi:10.1016/j.jpain.2006.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Kirsch, B. (2019). Virtual Reality. Ital 38, 4–5. doi:10.6017/ital.v38i4.11847

CrossRef Full Text | Google Scholar

Kothgassner, O. D., Goreis, A., Kafka, J. X., Van Eickels, R. L., Plener, P. L., and Felnhofer, A. (2019). Virtual reality exposure therapy for posttraumatic stress disorder (PTSD): a meta-analysis. Eur. J. psychotraumatology 10, 1654782. doi:10.1080/20008198.2019.1654782

PubMed Abstract | CrossRef Full Text | Google Scholar

Kulich, H. R., Bass, S. R., Griscavage, J. S., Vijayvargiya, A., Slowik, J. S., and Koontz, A. M. (2020). An ergonomic comparison of three different patient transport chairs in a simulated hospital environment. Appl. Ergon. 88, 103172. doi:10.1016/j.apergo.2020.103172

PubMed Abstract | CrossRef Full Text | Google Scholar

Lackey, S. J., Salcedo, J. N., Szalma, J. L., and Hancock, P. A. (2016). The stress and workload of virtual reality training: the effects of presence, immersion and flow. Ergonomics 59, 1060–1072. doi:10.1080/00140139.2015.1122234

PubMed Abstract | CrossRef Full Text | Google Scholar

Levy, C. E., Miller, D. M., Akande, C. A., Lok, B., Marsiske, M., and Halan, S. (2019). V-Mart, a Virtual Reality Grocery Store. Am. J. Phys. Med. Rehabil. 98, 191–198. doi:10.1097/phm.0000000000001041

PubMed Abstract | CrossRef Full Text | Google Scholar

Maggio, M. G., De Luca, R., Molonia, F., Porcari, B., Destro, M., Casella, C., et al. (2019). Cognitive rehabilitation in patients with traumatic brain injury: A narrative review on the emerging use of virtual reality. J. Clin. Neurosci. 61, 1–4. doi:10.1016/j.jocn.2018.12.020

PubMed Abstract | CrossRef Full Text | Google Scholar

Mallari, B., Spaeth, E. K., Goh, H., and Boyd, B. S. (2019). Virtual reality as an analgesic for acute and chronic pain in adults: a systematic review and meta-analysis. Jpr Vol. 12, 2053–2085. doi:10.2147/jpr.s200498

PubMed Abstract | CrossRef Full Text | Google Scholar

McLay, R. N., Wood, D. P., Webb-Murphy, J. A., Spira, J. L., Wiederhold, M. D., Pyne, J. M., et al. (2011). A randomized, controlled trial of virtual reality-graded exposure therapy for post-traumatic stress disorder in active duty service members with combat-related post-traumatic stress disorder. Cyberpsychology, Behav. Soc. networking 14 (4), 223–229. doi:10.1089/cyber.2011.0003

PubMed Abstract | CrossRef Full Text | Google Scholar

Mirelman, A., Patritti, B. L., Bonato, P., and Deutsch, J. E. (2010). Effects of virtual reality training on gait biomechanics of individuals post-stroke. Gait & Posture 31 (4), 433–437. doi:10.1016/j.gaitpost.2010.01.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Morina, N., Ijntema, H., Meyerbröker, K., and Emmelkamp, P. M. G. (2015). Can virtual reality exposure therapy gains be generalized to real-life? A meta-analysis of studies applying behavioral assessments. Behav. Res. Ther. 74, 18–24. doi:10.1016/j.brat.2015.08.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Nahin, R. L. (2017). Severe Pain in Veterans: The Effect of Age and Sex, and Comparisons With the General Population. J. Pain 18, 247–254. doi:10.1016/j.jpain.2016.10.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Ohl, M. E., Carrell, M., Thurman, A., Weg, M. V., Pharm, T. H., Mengeling, M., et al. (2018). "Availability of healthcare providers for rural veterans eligible for purchased care under the veterans choice act". BMC Health Serv. Res. 18, 315–317. doi:10.1186/s12913-018-3108-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Powers, M. B., and Emmelkamp, P. M. G. (2008). Virtual reality exposure therapy for anxiety disorders: A meta-analysis. J. anxiety Disord. 22 (3), 561–569. doi:10.1016/j.janxdis.2007.04.006

CrossRef Full Text | Google Scholar

Ronnie, J. (2021). Using Telehealth to Expand Access to Essential Health Services during the COVID-19 Pandemic, Atlanta, GA.