Is Continued Improvement After Automated Virtual Reality Exposure Therapy for Spider Phobia Explained by Subsequent in-vivo Exposure? A First Test of the Lowered Threshold Hypothesis

Consumer Virtual Reality (VR) technology offers a powerful, immersive medium for scalable dissemination of mental health interventions. Decades of research has shown VR exposure therapy to be efficacious in the treatment of anxiety disorders and that the fear reduction generalizes to real-world stimuli. Many studies also report continued improvement over time, after discontinuing VR use. The lowered threshold hypothesis states that this continued improvement is moderated by lowering the threshold to conduct subsequent in-vivo exposure. The current study is the first to formally test this hypothesis, using data from a recent trial on automated VR exposure therapy for spider phobia, in which participants (n = 49) were followed for 1 year, completing assessments 1 week, 3 and 12 months post-treatment. The assessment included validated self-report of phobia symptoms, a standardized behavioral approach test featuring a real spider, and a questionnaire for self-reporting frequency of in-vivo exposures since last assessment. Number of in-vivo exposures was found to be independently associated with greater symptom decrease in longitudinal outcome models. In sequential structural equation models, immediate post-treatment symptom reduction was associated with subsequent in-vivo exposures, which in turn was associated with continued symptom reduction. However, this applied only to self-reported phobia symptoms (not behavioral avoidance) and no associations were found past 3 months. Our findings offer preliminary, partial support for the lowered threshold hypothesis, suggesting that VR exposure interventions may benefit from including explicit in-virtuo to in-vivo transitioning components.

Introduction

Virtual Reality (VR) refers to technology capable of generating immersive experiences of being present in virtual, computer-generated environments, typically achieved through the use of a head mounted display that withholds the outside world, continuously tracks head rotation (and possibly position), and updates the stereoscopic display as the user looks around and interacts with environment (1). Although recognizable VR technology first appeared in the 1960's, it is only in the last 5 years that consumer VR devices have become common (standalone versions of which now cost only a few hundred USD), alongside mature ecosystems for development and dissemination of VR applications (2). This presents a paradigm shift also for clinical and public health applications of VR (3), numerous successful examples of which have accumulated in the last 25 years (4).

One such example is VR exposure therapy: here, VR technology is used to create and present virtual equivalents of phobic stimuli (e.g., animals, heights, and situations) to users in an immersive way (5, 6), to allow for graded, controlled, and systematic exposure until the fear response habituates and inhibitory learning occurs (7). This treatment format is attractive to both patients (8–10) and therapists (11, 12), is feasible also with adolescents (13)—although considerably less studied with this population (14)—and is associated with low rates of deterioration, i.e., unlikely to have negative effects (15). Recent research has shown that VR exposure can be self-directed (16), and even packaged as automated interventions relying on gamification components rather than a real-life therapist directing treatment (17–20), and even a virtual therapist (21, 22).

As VR continues to grow and establish itself as a consumer technology (23), automated and self-directed interventions have the potential to make a significant public health impact by addressing the large treatment gap (24) and delay in treatment-seeking for anxiety disorders (25), phobias in particular. Although some methodological concerns have been raised (26), meta-analyses have demonstrated efficacy of VR exposure therapy for anxiety disorders (27–29), even when only considering trials featuring in-vivo behavioral approach tests as outcomes (30). This latter finding reveals that a reduced fear response to virtual phobic stimuli generalizes to the in-vivo equivalents that users will ultimately face in real life. This generalization likely explains why many trials report continued symptom improvements at follow-ups 1 to 6 years after completing VR exposure therapy (18, 31–34). Notwithstanding the immediate fear reduction (30), the full clinical effect may come from lowering the threshold for users to subsequently engage in in-vivo exposure opportunities as they appear in everyday life, breaking the vicious circle of avoidance that maintains anxiety disorders (35).

Although the idea that post-treatment, continued exposure predicts continued improvement likely applies equally well to traditional exposure therapy (36), VR exposure therapy is unique in that here, continued exposure entails not only continuing to do what was taught in treatment, but doing it in a different way. Despite having been noted as a limitation of the extant literature (4, 17), the necessary transition from in-virtuo to in-vivo exposure—and implicitly the lowered threshold hypothesis—has received surprisingly little research attention. Indeed, we are not aware of any study that has measured and attempted to statistically model the moderating role of post-VR, in-vivo exposure in explaining long-term continued symptom improvement. Indirect evidence, however, comes from two trials featuring the same automated VR exposure intervention for spider phobia: in one trial (18), participants in the VR arm were given written and oral information explaining the rationale for transitioning to in-vivo exposure in everyday life, as well as instructions for planning, execution, and evaluating exposure tasks; while in the other trial (17), no such written material was provided. In line with the lowered threshold hypothesis, the former trial saw continued improvement during the follow-up period, while the latter did not. In another trial, completing in-vivo exposure exercises during an explicitly framed transition period following a single session of VR exposure therapy for public speaking anxiety, was independently associated with additional symptom decrease (16).

These findings warrant further investigation, since the lowered threshold hypothesis has important implications for the design of automated VR exposure interventions that show such a great public health potential. In the current study, using data from one of the trials on automated VR exposure therapy for spider phobia, and using complementary statistical modeling techniques, we report a first such investigation.

Methods

Ethics

The current study uses data from a randomized non-inferiority trial (18, 37) that was pre-registered (Clinicaltrials.gov: NCT02533310), received ethical approval from the Swedish Ethical Review Authority (2015/472-31 and 2015/1695-32), and had all participants provide written informed consent.

Procedure and Participants

See the published trial protocol and primary report for details (18, 37). In brief, n = 100 participants were recruited from the general public, assessed for spider phobia, provided self-report measures, completed a standardized behavioral approach test (BAT) with a real spider (see Measures below). Included participants were randomized 1:1 to a single, 3 h session of either in-vivo exposure therapy (38), or a novel, automated, gamified VR exposure therapy intervention. Only participants in the VR arm (n = 50) were included in the current study, with all but one completing treatment. At the end of treatment, participants were given standardized oral and written material explaining the rationale for transitioning to in-vivo exposure in everyday life, as well as instructions for planning, execution and evaluating exposure tasks (39) (material available from the authors on request). One week (n = 49), 3 months (n = 45) and 1 year (n = 47) after receiving treatment, participants completed a similar assessment with the same outcome measures. Assessments were intended to be conducted on-site at Stockholm University; however, if a participant could not be scheduled for a visit, a mailed paper or online version of the included self-reports was made available. Across assessment occasions, a total of k = 34 assessments (of k = 191) were returned on paper or completed online, entailing that fewer BAT data points are available for analysis.

Measures

For the current study, we selected the two outcome measures from the non-inferiority trial (18, 37) that showed the largest effect sizes, covering related by separated aspects of spider phobia: a standardized BAT used in previous research (40) providing an objective measure of avoidance behavior, as well as the self-rated Fear of Spider Questionnaire (FSQ) (41) covering a broader set of phobia symptoms. The BAT featured a real spider (a harmless species native to Sweden, ~2 cm in size including legs) and was scored in 13 steps (rated 0−12), with higher scores corresponding to lesser avoidance. The FSQ consists of 18 items, scored 1−7 for a maximum score of 126, with higher scores corresponding to more phobia symptoms.

At the 3 and 12 month follow-up assessments, participants also reported on extent of in-vivo exposure using a custom, standardized form. The form differentiated between planned exposure exercises vs. involuntary exposure to real spiders, and had participants estimate number of occasions since the last assessment. Exact phrasings used were “Approximately how many times have you deliberately and specifically for exposure purposes put yourself in a position where you were exposed to spiders,” and “Approximately how many times have you involuntarily been caught in situations where you were exposed to spiders?” (both translated from original Swedish). The raw reported values showed several outliers; see Figure 1 for distributions split by time points and type (Figure 1A), as well as a correlation scatter plot across time points (Figure 1B). Since there was no a prior reason to expect linear associations across this width of values (making raw values inappropriate), and in lieu of any established way of statistically handling outliers in this type of variable, we opted for an explorative approach wherein the initial analyses were repeated with three common outlier-adjustment alternatives, for each type of exposure and outcome (respectively). These adjustments involved either binarizing (zero or any), truncating outliers (any value >20, threshold chosen due to a sharp decrease across types and occasion as revealed by histograms), or omitting outliers (same threshold used in truncation). For sake of transparency, all models (including ones with the raw variable for comparison) are reported.

FIGURE 1

Figure 1. Distributions and correlations of planned and involuntary exposure occasions across time. (A) Distributions of (raw) reported exposure occasions split by type and assessment point. Dashed line shows chosen outlier threshold. (B) Scatterplots of planned vs. involuntary exposures split by assessment point.

At the 3 month follow-up, the questionnaire also included an additional five questions on: whether they considered themselves to have actively continued to perform exposure exercises (Yes or No); whether they created a maintenance and transition plan as instructed (Yes or No), self-rated compliance to this plan (0−10, from Not at all to Completely); whether they believed that they had been given a good rationale on the need for maintenance (Yes or No); and whether they were given sufficient instructions for how to construct their plan (Yes or No).

Analyses

For our initial test of the lowered threshold hypothesis, we ran cluster-bootstrapped general linear models (GLM) on each outcome (BAT or FSQ scores) and exposure type (planned or involuntary) separately, each with every version of the exposure frequency variable. All models featured a numeric time variable (post-treatment, 3 and 12 month follow-ups, scored as 0, 1, and 2), with exposure frequency as an additional, time-varying predictor (set to zero at post-treatment). Cluster-bootstrapped general linear modeling is a novel statistical technique, robust to normality assumptions, that is especially appropriate when there is risk of miss-specifying a corresponding mixed model (e.g., when few time points are available). Analyses were performed using the ClusterBootstrap R package (42), each with 10,000 repeats.

Second, the sequential and interactive associations captured by the lowered threshold hypothesis was examined by running structural equation models (SEM) on each outcome separately. Both models featured the truncated exposure frequency variable for several reasons: a numeric variable was necessary, results from the first analysis step suggested that this adjustment corrected for any measurement error, and this variable adjustment retained all available data, thereby maximizing power. In a longitudinal sequence, the SEMs used the score difference between two adjacent time points to predict the subsequent number of exposure occasions, which in turn predicted the next score difference, and so on. Covariance between adjacent score differences was also included. Robust Huber-White standard errors were calculated, and missing data estimated using maximum likelihood. Analyses were conducted using the lavaan R package (43).

Results

Self-Reported Adherence

At the 3 month follow-up, 61.4% (n = 27) reported having actively engaged in exposure maintenance; congruently, Poisson regression models revealed that these participants reported more (raw) in-between exposure occasions, both planned (B = 1.71, SE = 0.17, p < 0.001) and to a lesser extent involuntary (B = 0.79, SE = 0.25, p = 0.0018). Compliance groups did however not differ on post-treatment BAT (95% CI of mean difference: −0.45−2.49) or FSQ scores (95% CI of mean difference: −23.84−3.18). Somewhat fewer participants (43.2%, n = 19) reported having created a formal transition/maintenance plan, which in separate Poisson models was significantly associated with planned (B = 0.72, SE = 0.11, p < 0.001) but not involuntary exposure occasions (B = 0.31, SE = 0.22, p = 0.156). Among participants who created such a plan, average compliance was rated to M = 5.64 (SD = 1.95, IQR = 3). All but two participants (of 44) reported receiving a good rationale for exposure transition/maintenance, and 72.8% (n = 32) reported receiving sufficient instructions. Paired t-tests revealed that participants reported similar number of planned exposure occasions at the 3 and 12 month follow-ups (95% CI of mean difference: −7.43−0.73) yet more involuntary exposure occasions at the latter (t₄₀ = −3.28, p = 0.0022), although it should be noted that covered periods are not equal in duration (3 vs. 9 months).

In-vivo Exposures as Time-Varying Predictor

See Table 1 for full results (not including intercept and time predictor for each model). In no model and with neither outcome were involuntary exposure occasions associated with improvement. In all but one model (binarized exposure occasions predictor and BAT outcome) were planned exposure occasions independently associated with improvement.

TABLE 1

Table 1. Independent impact of exposure frequencies in longitudinal models.

Sequential SEM

Since the results above showed that only planned exposures were associated with continue improvement, only this type of exposure was examined in sequential SEMs. There were no associations in any direction between temporally adjacent BAT score differences and planned exposures. However, congruent with the lowered threshold hypothesis, significant associations in the expected direction were found on FSQ scores in the first 3 months: a greater post-pre FSQ score difference (note negative value in case of improvement) was associated with more reported between-assessment exposure occasions at the 3 month follow-up, which in turn was associated with additional FSQ score decrease at this follow-up. However, sequential associations from here on were insignificant. See Figure 2 for full SEM results. A post-hoc analysis revealed that FSQ and BAT score differences did not correlate significantly (pre to post, r = 0.17, p = 0.246; post to 3 month, r = 0.145, p = 0.411), although a strong correlation remained between the two measures at each individual time point (all p < 0.002, r > −0.55), with negligible change in correlation strength over time (r = −0.58 pre-treatment, r = −0.55 at 12 month follow-up).

FIGURE 2

Figure 2. Sequential SEM. (A) Sequential model with BAT outcome. (B) Sequential model with FSQ outcome.

Discussion

Meta-analytic research has shown that fear reduction following VR exposure therapy generalizes also to in-vivo equivalent stimuli; moreover, several studies report that individuals who have completed VR exposure therapy continue to improve in the year(s) that follow. The lowered threshold hypothesis proposes that this post-VR continued improvement is explained in part by individuals now being able to engage in in-vivo exposure in everyday life, breaking the vicious circle of avoidance that define phobias. The current study offers a first test of this hypothesis, the results of which are in partial support of the hypothesis: while reported planned (but not involuntary) in-vivo exposure occasions was independently associated with continued improvement over the course of 1 year (with one exception), a sequential model revealed the hypothesized bidirectional association between symptom decrease and extent of exposure applied only to self-reported phobia symptoms, not behavioral avoidance, and only during the first few months after treatment.

Finding no outcomes associations with number of involuntary exposures is expected given that this measure presumably captures life circumstances (e.g., type of residence and line of work) rather than treatment adherence, and was included to control for this possible source of bias. Associations with planned exposures were however largely robust across analyses and in the expected direction. While these findings need to be replicated in independent samples, our results are consistent with the observed difference in long-term outcomes across the two trials on this particular VR intervention (17, 18), and suggest that it is indeed beneficial to explicitly promote transition to in-vivo exposure after completing VR exposure, and that this can be accomplished with brief oral and written instructions. Even in the broader field of cognitive behavioral therapy (CBT), where numerous studies have examined predictors of relapse (44, 45), there has been very relatively little research on the associations between long-term symptom trajectories and post-treatment adherence to the therapeutic strategies taught and practiced during active treatment. This lack of comparable research also makes it difficult to interpret whether the self-rated compliance of 61% found in the current study is high or not. One rare previous study found that adherence to different CBT components for insomnia varied extensively between components, and that the most reportedly used component (74%) was in fact not associated with improvement, while behavioral strategies like stimulus control and sleep restriction (41% reported use) were (46). While such findings are obviously of great clinical interest, examining this particular research question entails facing several methodological barriers. First, in lieu of automatically collected measures using wearables (47), ecological momentary assessments (48), or even momentary reporting of involuntary spider exposures (49), measurements of adherence and/or exposure need likely be retrospective, at least to some temporal degree. Second, since adherence is typically not a parameter that participants can be randomly assigned to, special statistical techniques are required to estimate the causal effect of adherence in itself (50). Including a second-stage randomization after the VR intervention to either a transition component or not, with explicit adherence targets (frequency of exposures, habituation threshold etc.), would not only allow more advanced and appropriate statistical modeling, but would also resolve the need for outlier adjustment, as was the case in the current study.

Interestingly, binarized exposure counts were not significantly associated with BAT improvements and barely significantly associated with FSQ improvements. This suggests that one exposure occasion is not enough for continued improvement and that the numeric range (covered by the other variable version) is in itself clinically pertinent. A seemingly counter-intuitive finding was that immediate self-rated phobia symptom improvement, but not behavioral approach improvement, predicted number of subsequent in-vivo exposure exercises. This could be explained by individuals with great behavioral improvement seeing no need for further in-vivo exposure, and/or that improved performance for many came at the cost of high distress that discouraged further in-vivo exposure. The discrepancy between models, along with the corresponding weak correlation between improvement metrics (yet stable correlations at each time point) may also be due to a delay in improvement insight: a similar, reversed discrepancy—rapid behavioral improvement without simultaneous decrease in self-reported fear—has been observed in fear memory reconsolidation disruption experiments (51). Although we cannot rule out the influence of demand characteristics (52), the remarkable stability over time of the correlation between the objective and subjective outcome measures, suggests no or negligible effects only. It should also be noted that participants did not have access to prior FSQ nor were provided with any interpretation guidelines; further, since no further treatment was offered at follow-ups, participants had no incentive to exaggerate symptom ratings. Regardless of source of the diverging results, the findings of the current study shows the value of capturing both aspects of phobia presentation.

The sequential model revealed that planned exposure exercises did not influence continued symptom reduction (self-rated phobia symptoms specifically) beyond the 3 month follow-up, yet planned exposure exercises were associated with both types of symptom reduction in longitudinal models (with few exceptions) that included exposure exercises as a time-varying predictor, i.e., assuming equal effect of exposure exercises across time. While the former approach breaks down the presumed process in individual steps, the latter approach examines whether variance in observed outcomes that is not explained by time, is instead explained by frequency of exposure exercises. In addition, statistical power varies between approaches. Of interest, participants reported the same number of planned exposure exercises at the 3 and 12 month follow-ups, despite covering different durations, revealing that this became less common over time. However, over the same 12 months, treatment effects continued to grow (18), suggesting that other factors likely contribute to continued symptom improvement at a later stage. Examining a broader range of predictors of long-term outcomes in VR exposure therapy should be considered an important topic for future research.

Strengths and Limitations

This first study on the lowered threshold hypothesis has several limitations that need to be acknowledged. First, the presence of outliers in the measure of post-VR exposures suggests some degree of measurement error. It should be noted that outlier here is used in the statistical sense, i.e., data points far off the observed distribution. All reported values were however in the plausible range (see Figure 1). Some measurement error with regards to what participants interpreted to constitute an exposure exercise is likely to have been present, and we cannot guarantee that some participants did not e.g., include exposure to television spiders. Future research would likely benefit from using more precise question phrasing. However, findings were largely consistent across outlier-adjustment methods, and many of the expected associations were found regardless, suggesting that these measures at least sufficiently captured what was intended. Second, exposures occasions were self-reported, retrospectively, and the temporal resolution was low. No auxiliary measures like experienced distress during and after each exposure task were collected either, nor was use of other therapeutic techniques (e.g., cognitive exercises) during the follow-up period measured. This should be considered an important goal of future research. Including continued VR exposure using applications specifically designed for at-home use (16, 20) would also allow collection of objective adherence data, yet would of course not cover the transition aspect. Using augmented reality technology to bridge VR and real-world exposure is a possible solution that remains to be explored. Third, the sample size and the lack of a comparison group randomized to not receive maintenance/transition promotion, precludes us from statistically estimating the causal effect of adherence to the transition program; this too should be addressed in future research. Of note, including also the in-vivo arm from the non-inferiority trial would not have address this issue and a comparison with other treatment modalities falls outside the scope of the current study, which focuses on the lowered threshold hypothesis for VR exposure therapy. Fourth, a consumer-targeted version of the same VR treatment was released in-between the 3 and 12 month follow-ups and the latter assessment did not include any question on use of this application. However, given the very limited consumer adoption of VR at this time, and the fact it was only available for one VR platform requiring a specific smartphone to run, the percentage of participants who had resumed VR exposure should be negligible if any.

Strengths of the study include the use of validated outcome measures, although as with any complex behavioral measure, we cannot rule out that equidistance in BAT scoring was suboptimal. Although results were not entirely consistent across outcome measures (see above for possible reasons), the use of two different statistical modeling techniques—with largely congruent findings—lends credibility. Other strengths include a low percentage of missing data at follow-ups, and that both the VR treatment and the subsequent promotion of in-vivo exposure were standardized.

Conclusions

Our findings offer preliminary, partial support of the lowered threshold hypothesis of how Virtual Reality exposure therapy promotes continued symptom improvement. In longitudinal models with time-varying predictors, number of preceding in-vivo exposure occasions was associated with greater symptom decrease. In a sequential model, immediate post-treatment symptom reduction was associated with self-reported frequency of subsequent in-vivo exposures which in turn was associated with continued symptom reduction. However, this applied only to self-reported phobia symptoms, not behavioral avoidance, and was limited to the first 3 months of the follow-up. If these findings can be replicated in independent samples, it appears beneficial to actively promote in-vivo transition in order to maximize long-term effect of VR exposure therapy.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Stockholm Ethical Review Board. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

PL, PD, AM, GA, LR, WH, and PC designed the study. PL, PD, and WH collected the data. PL and PD analyzed the data. PL drafted the manuscript. PD, AM, GA, LR, WH, and PC revised the manuscript critically for important intellectual content. All authors contributed to the article and approved the submitted version.

Conflict of Interest

WH is the founder and chief technology officer of Mimerse, a company specializing in developing VR interventions for mental health. PL has consulted for Mimerse but holds no financial stake in the company.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. Scarfe P, Glennerster A. The science behind virtual reality displays. Annu Rev Vis Sci. (2019) 5:529–47. doi: 10.1146/annurev-vision-091718-014942

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Lindner P. Better, virtually: the past, present, and future of virtual reality cognitive behavior therapy. Int J Cogn Ther. (2020) 14:23–46. doi: 10.1007/s41811-020-00090-7

CrossRef Full Text | Google Scholar

3. Lindner P, Miloff A, Hamilton W, Carlbring P. The potential of consumer-targeted virtual reality relaxation applications: descriptive usage, uptake and application performance statistics for a first-generation application. Front Psychol. (2019) 10:132. doi: 10.3389/fpsyg.2019.00132

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Freeman D, Reeve S, Robinson A, Ehlers A, Clark D, Spanlang B, et al. Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol Med. (2017) 47:2393–400. doi: 10.1017/S003329171700040X

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Botella C, Fernández-Álvarez J, Guillén V, García-Palacios A, Baños R. Recent progress in virtual reality exposure therapy for phobias: a systematic review. Curr Psychiatry Rep. (2017) 19:42. doi: 10.1007/s11920-017-0788-4

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Lindner P, Miloff A, Hamilton W, Reuterskiöld L, Andersson G, Powers MB, et al. Creating state of the art, next-generation Virtual Reality exposure therapies for anxiety disorders using consumer hardware platforms: design considerations and future directions. Cogn Behav Ther. (2017) 46:404–20. doi: 10.1080/16506073.2017.1280843

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Craske MG, Treanor M, Conway CC, Zbozinek T, Vervliet B. Maximizing exposure therapy: an inhibitory learning approach. Behav Res Ther. (2014) 58:10–23. doi: 10.1016/j.brat.2014.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Lindner P, Rozental A, Jurell A, Reuterskiöld L, Andersson G, Hamilton W, et al. Experiences of gamified and automated virtual reality exposure therapy for spider phobia: qualitative study. JMIR Serious Games. (2020) 8:e17807. doi: 10.2196/17807

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Garcia-Palacios A, Botella C, Hoffman H, Fabregat S. Comparing acceptance and refusal rates of virtual reality exposure vs. in vivo exposure by patients with specific phobias. Cyberpsychol Behav. (2007) 10:722–4. doi: 10.1089/cpb.2007.9962

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Guillén V, Baños RM, Botella C. Users' opinion about a virtual reality system as an adjunct to psychological treatment for stress-related disorders: a quantitative and qualitative mixed-methods study. Front Psychol. (2018) 9:1038. doi: 10.3389/fpsyg.2018.01038

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Lindner P, Miloff A, Zetterlund E, Reuterskiöld L, Andersson G, Carlbring P. Attitudes toward and familiarity with virtual reality therapy among practicing cognitive behavior therapists: a cross-sectional survey study in the era of consumer VR platforms. Front Psychol. (2019) 10:176. doi: 10.3389/fpsyg.2019.00176

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Schwartzman D, Segal R, Drapeau M. Perceptions of virtual reality among therapists who do not apply this technology in clinical practice. Psychol Serv. (2012) 9:310–15. doi: 10.1037/a0026801

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Kahlon S, Lindner P, Nordgreen T. Virtual reality exposure therapy for adolescents with fear of public speaking: a non-randomized feasibility and pilot study. Child Adolesc Psychiatry Ment Health. (2019) 13:47. doi: 10.1186/s13034-019-0307-y

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Kothgassner OD, Felnhofer A. Lack of research on efficacy of virtual reality exposure therapy (VRET) for anxiety disorders in children and adolescents. Neuropsychiatrie. (2020) doi: 10.1007/s40211-020-00349-7

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Fernández-Álvarez J, Rozental A, Carlbring P, Colombo D, Riva G, Anderson PL, et al. Deterioration rates in virtual reality therapy: an individual patient data level meta-analysis. J Anxiety Disord. (2019) 61:3–17. doi: 10.1016/j.janxdis.2018.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Lindner P, Miloff A, Fagernäs S, Andersen J, Sigeman M, Andersson G, et al. Therapist-led and self-led one-session virtual reality exposure therapy for public speaking anxiety with consumer hardware and software: a randomized controlled trial. J Anxiety Disord. (2019) 61:45–54. doi: 10.1016/j.janxdis.2018.07.003

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Lindner P, Miloff A, Bergman C, Andersson G, Hamilton W, Carlbring P. Gamified, automated Virtual Reality Exposure Therapy for fear of spiders: a single-subject trial under simulated real-world conditions. Front Psychiatry. (2020) 11:116. doi: 10.3389/fpsyt.2020.00116

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Miloff A, Lindner P, Dafgård P, Deak S, Garke M, Hamilton W, et al. Automated virtual reality exposure therapy for spider phobia vs. in-vivo one-session treatment: a randomized non-inferiority trial. Behav Res Ther. (2019) 118:130–40. doi: 10.1016/j.brat.2019.04.004

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Freeman D, Haselton P, Freeman J, Spanlang B, Kishore S, Albery E, et al. Automated psychological therapy using immersive virtual reality for treatment of fear of heights: a single-blind, parallel-group, randomised controlled trial. Lancet Psychiatry. (2018) 5:625–32. doi: 10.1016/S2215-0366(18)30226-8

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Donker T, Cornelisz I, van Klaveren C, van Straten A, Carlbring P, Cuijpers P, et al. Effectiveness of self-guided app-based virtual reality cognitive behavior therapy for acrophobia: a randomized clinical trial. JAMA Psychiatry. (2019) 76:682. doi: 10.1001/jamapsychiatry.2019.0219

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Miloff A, Carlbring P, Hamilton W, Andersson G, Reuterskiöld L, Lindner P. Measuring alliance toward embodied virtual therapists in the era of automated treatments with the Virtual Therapist Alliance Scale (VTAS): development and psychometric evaluation. J Med Internet Res. (2020) 22:e16660. doi: 10.2196/16660

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Lambe S, Knight I, Kabir T, West J, Patel R, Lister R, et al. Developing an automated VR cognitive treatment for psychosis: gameChange VR therapy. J Behav Cogn Ther. (2020) 30:33–40. doi: 10.1016/j.jbct.2019.12.001

CrossRef Full Text | Google Scholar

23. Bunz U, Seibert J, Hendrickse J. From TAM to AVRTS: development and validation of the attitudes toward Virtual Reality Technology Scale. Virtual Real. (2020) 25:31–41. doi: 10.1007/s10055-020-00437-7

CrossRef Full Text | Google Scholar

24. Alonso J, Liu Z, Evans-Lacko S, Sadikova E, Sampson N, Chatterji S, et al. Treatment gap for anxiety disorders is global: results of the World Mental Health Surveys in 21 countries. Depress Anxiety. (2018) 35:195–208. doi: 10.1002/da.22711

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Wang PS, Angermeyer M, Borges G, Bruffaerts R, Tat Chiu W, DE Girolamo G, et al. Delay and failure in treatment seeking after first onset of mental disorders in the World Health Organization's World Mental Health Survey Initiative. World Psychiatry. (2007) 6:177–85.

PubMed Abstract | Google Scholar

26. McCann RA, Armstrong CM, Skopp NA, Edwards-Stewart A, Smolenski DJ, June JD, et al. Virtual reality exposure therapy for the treatment of anxiety disorders: an evaluation of research quality. J Anxiety Disord. (2014) 28:625–31. doi: 10.1016/j.janxdis.2014.05.010

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Wechsler TF, Kümpers F, Mühlberger A. Inferiority or even superiority of virtual reality exposure therapy in phobias?—A systematic review and quantitative meta-analysis on randomized controlled trials specifically comparing the efficacy of virtual reality exposure to gold standard in vivo exp. Front Psychol. (2019) 10:1758. doi: 10.3389/fpsyg.2019.01758

CrossRef Full Text | Google Scholar

28. Fodor LA, Coteţ CD, Cuijpers P, Szamoskozi Ştefan, David D, Cristea IA. The effectiveness of virtual reality based interventions for symptoms of anxiety and depression: a meta-analysis. Sci Rep. (2018) 8:10323. doi: 10.1038/s41598-018-28113-6

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Carl E, Stein AT, Levihn-Coon A, Pogue JR, Rothbaum B, Emmelkamp P, et al. Virtual reality exposure therapy for anxiety and related disorders: a meta-analysis of randomized controlled trials. J Anxiety Disord. (2019) 61:27–36. doi: 10.1016/j.janxdis.2018.08.003

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Morina N, Ijntema H, Meyerbröker K, Emmelkamp PMG. Can virtual reality exposure therapy gains be generalized to real-life? A meta-analysis of studies applying behavioral assessments. Behav Res Ther. (2015) 74:18–24. doi: 10.1016/j.brat.2015.08.010

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Wiederhold BK, Wiederhold MD. Three-year follow-up for virtual reality exposure for fear of flying. CyberPsychol Behav. (2003) 6:441–5. doi: 10.1089/109493103322278844

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Botella C, Gracía-Palacios A, Villa H, Baños RM, Quero S, Alcañiz M, et al. Virtual reality exposure in the treatment of panic disorder and agoraphobia: a controlled study. Clin Psychol Psychother. (2007) 14:164–75. doi: 10.1002/cpp.524

CrossRef Full Text | Google Scholar

33. Anderson PL, Edwards SM, Goodnight JR. Virtual reality and exposure group therapy for social anxiety disorder: results from a 4–6 year follow-up. Cogn Ther Res. (2016) 41:230–6. doi: 10.1007/s10608-016-9820-y

CrossRef Full Text | Google Scholar

34. Rothbaum BO, Hodges L, Anderson PL, Price L, Smith S. Twelve-month follow-up of virtual reality and standard exposure therapies for the fear of flying. J Consult Clin Psychol. (2002) 70:428–32. doi: 10.1037/0022-006X.70.2.428

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Hofmann SG, Hay AC. Rethinking avoidance: toward a balanced approach to avoidance in treating anxiety disorders. J Anxiety Disord. (2018) 55:14–21. doi: 10.1016/j.janxdis.2018.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Wolitzky-Taylor KB, Horowitz JD, Powers MB, Telch MJ. Psychological approaches in the treatment of specific phobias: a meta-analysis. Clin Psychol Rev. (2008) 28:1021–37. doi: 10.1016/j.cpr.2008.02.007

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Miloff A, Lindner P, Hamilton W, Reuterskiöld L, Andersson G, Carlbring P. Single-session gamified virtual reality exposure therapy for spider phobia vs. traditional exposure therapy: study protocol for a randomized controlled non-inferiority trial. Trials. (2016) 17:60. doi: 10.1186/s13063-016-1171-1

CrossRef Full Text | Google Scholar

38. Hellström K, Öst L-G. One-session therapist directed exposure vs two forms of manual directed self-exposure in the treatment of spider phobia. Behav Res Ther. (1995) 33:959–65. doi: 10.1016/0005-7967(95)00028-V

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Öst L-G. A maintenance program for behavioral treatment of anxiety disorders. Behav Res Ther. (1989) 27:123–30. doi: 10.1016/0005-7967(89)90070-3

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Öst L-G, Salkovskis PM, Hellström K. One-session therapist-directed exposure vs. self-exposure in the treatment of spider phobia. Behav Ther. (1991) 22:407–22. doi: 10.1016/S0005-7894(05)80374-0

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Muris P, Merckelbach H. A comparison of two spider fear questionnaires. J Behav Ther Exp Psychiatry. (1996) 27:241–4. doi: 10.1016/S0005-7916(96)00022-5

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Deen M, de Rooij M. ClusterBootstrap: an R package for the analysis of hierarchical data using generalized linear models with the cluster bootstrap. Behav Res Methods. (2020) 52:572–90. doi: 10.3758/s13428-019-01252-y

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Rosseel Y. lavaan: an R Package for Structural Equation Modelin. J Stat Softw. (2012) 48:1–36. doi: 10.18637/jss.v048.i02

CrossRef Full Text | Google Scholar

44. Wojnarowski C, Firth N, Finegan M, Delgadillo J. Predictors of depression relapse and recurrence after cognitive behavioural therapy: a systematic review and meta-analysis. Behav Cogn Psychother. (2019) 47:514–29. doi: 10.1017/S1352465819000080

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Lorimer B, Kellett S, Nye A, Delgadillo J. Predictors of relapse and recurrence following cognitive behavioural therapy for anxiety-related disorders: a systematic review. Cogn Behav Ther. (2020) 50:1–18. doi: 10.1080/16506073.2020.1812709

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Harvey L, Inglis SJ, Espie CA. Insomniacs' reported use of CBT components and relationship to long-term clinical outcome. Behav Res Ther. (2002) 40:75–83. doi: 10.1016/S0005-7967(01)00004-3

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Sequeira L, Perrotta S, LaGrassa J, Merikangas K, Kreindler D, Kundur D, et al. Mobile and wearable technology for monitoring depressive symptoms in children and adolescents: a scoping review. J Affect Disord. (2020) 265:314–24. doi: 10.1016/j.jad.2019.11.156

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Walz LC, Nauta MH, aan het Rot M. Experience sampling and ecological momentary assessment for studying the daily lives of patients with anxiety disorders: a systematic review. J Anxiety Disord. (2014) 28:925–37. doi: 10.1016/j.janxdis.2014.09.022

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Hart AG, Nesbit R, Goodenough AE. spatiotemporal variation in house spider phenology at a national scale using citizen science. Arachnology. (2018) 17:331–4. doi: 10.13156/arac.2017.17.7.331

CrossRef Full Text | Google Scholar

50. Hesser H. Estimating causal effects of internet interventions in the context of nonadherence. Internet Interv. (2020) 21:100346. doi: 10.1016/j.invent.2020.100346

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Soeter M, Kindt M. An abrupt transformation of phobic behavior after a post-retrieval amnesic agent. Biol Psychiatry. (2015) 78:880–6. doi: 10.1016/j.biopsych.2015.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

52. McCambridge J, de Bruin M, Witton J. The effects of demand characteristics on research participant behaviours in non-laboratory settings: a systematic review. PLoS ONE. (2012) 7:e39116. doi: 10.1371/journal.pone.0039116

PubMed Abstract | CrossRef Full Text | Google Scholar