Ian Chard
Nejra Van Zalk
Lorenzo Picinali- 1Design Psychology Lab, Dyson School of Design Engineering, Imperial College London, London, United Kingdom
- 2Audio Experience Design Group, Dyson School of Design Engineering, Imperial College London, London, United Kingdom
Introduction: Although several trials have demonstrated the effectiveness of Virtual Reality Exposure Therapy (VRET) for reducing social anxiety, there is little understanding about the factors that lead to symptom reduction across different treatment designs. Such factors may include outcome expectancy, therapeutic alliance, presence (perception of being in the virtual environment) and social presence (perception of interacting with others). We report on findings from a pilot trial of VRET targeting social anxiety in people who stutter, and examine the association of these four factors with treatment outcome.
Methods: People who stutter reporting heightened social anxiety (n = 22) took part in the trial after being recruited via online adverts. Remotely delivered VRET was administered to participants in three sessions across three weeks. Each session targeted both performative and interactive anxiety. A virtual therapist helped participants to engage with treatment strategies, whilst also guiding them through exercises.
Results: Findings showed that presence and social presence were both negatively related to changes in fear of negative evaluation between pre- and post-treatment. However, presence, outcome expectancy and therapeutic alliance were positively related to changes in social anxiety symptoms. Furthermore, outcome expectancy and therapeutic alliance were quadratically related to fear of negative evaluation change. Nevertheless, the effect of presence on social anxiety, and the effects of presence and therapeutic alliance on fear of negative evaluation must be interpreted with caution as these were not large enough to reach sufficient statistical power. Therapeutic alliance did not mediate the relationship between outcome expectancy and treatment outcome.
Discussion: These findings suggest that the current VRET protocol affected social anxiety and fear of negative evaluation differently. We discuss how presence may underlie these mixed associations. We also suggest that the unexpected positive effects on social anxiety symptoms may have resulted from insufficient treatment strategies which inadvertently encouraged maladaptive learning.
1 Introduction
Virtual Reality Exposure Therapy (VRET), which involves an individual confronting virtually recreated fear-inducing situations, is increasingly becoming a viable and effective option for treating anxiety disorders. One area of growing interest is using VRET for treatment of social anxiety associated with stuttering. Stuttering is a developmental speech disorder which disrupts the fluent production of speech (American Psychiatric Association, 2013). People who stutter (PWS) are also at a greater risk compared to non-stuttering individuals of developing heightened levels of social anxiety (Craig and Tran, 2014; Iverach and Rapee, 2014). Social anxiety represents “a marked, or intense, fear or anxiety of social situations in which the individual may be scrutinized by others” (American Psychiatric Association, 2013), and is underlain by the expectation of negative evaluation from others (Rapee and Heimberg, 1997). VRET can target this fear of negative evaluation by using avatars in virtual social scenarios that provoke fears of public scrutiny. In the case of stuttering, this might involve the inclusion of stuttering-specific feared situations such as speaking on the telephone, or targeting stuttering-specific processes such as the avoidance of feared words (Iverach et al., 2017). The ability of virtual reality (VR) to create on-demand experiences and reduce the resource burden of in vivo exposure makes VRET a particularly appealing alternative for social anxiety treatment as well as prevention.
There is strong support for the development and implementation of VRET across different anxiety types and disorders (Freeman et al., 2017). Several randomized controlled trials (Klinger et al., 2005; Wallach et al., 2009; Robillard et al., 2010; Anderson et al., 2013; Kampmann et al., 2016a; Bouchard et al., 2017) and three meta-analyses (Kampmann et al., 2016b; Chesham et al., 2018; Horigome et al., 2020) support its use for reducing social anxiety. These results suggest that VRET is either comparable to existing treatments (Klinger et al., 2005; Wallach et al., 2009; Anderson et al., 2013; Kampmann et al., 2016a; Kampmann et al., 2016b; Bouchard et al., 2017; Chesham et al., 2018; Horigome et al., 2020) or at least superior to waitlist control conditions (Robillard et al., 2010; Kampmann et al., 2016a; Bouchard et al., 2017). We recently conducted a small-scale randomized pilot trial of VRET specifically adapted to target social anxiety associated with stuttering (Chard et al., 2023). Despite no observed differences between the VRET and waitlist control group on symptom reduction, VRET was associated with reduced social anxiety between post-treatment and 1-month follow-up. Two other small-scale trials have explored the use of exposure within this group, one using VR (Walkom, 2016) and the other in vivo (Scheurich et al., 2019). Both targeted public-speaking fears relevant to stuttering and observed decreases to social anxiety symptoms. However, neither study statistically analyzed their data.
Despite these promising early findings, there are several knowledge gaps regarding how VRET works and for whom, compared to in vivo exposure treatment. In a recently published review of different techniques adopted in VRET protocols, we examined the comparative value and applicability of different features across contexts (Chard and van Zalk, 2022). We discussed how various visual modalities, headsets, exposure techniques and therapist formats have been applied across protocols. We emphasized the importance of VRET targeting different social fears, particularly verbal interaction. However, the role of VR user experience and treatment mechanisms in influencing treatment outcome is poorly understood. Due to the relative novelty of this treatment modality, it cannot be assumed that the mechanisms through which in vivo exposure causes symptom reduction can be directly applied to VRET. In this study, we will build on our previous work by investigating the association between four factors (presence, social presence, outcome expectancy and therapeutic alliance) and treatment outcome from VRET targeting social anxiety in stuttering. Thus, we seek to understand how VRET can be designed to elicit greater social anxiety symptom reduction in PWS.
1.1 The importance of presence
Presence is an important factor for eliciting normal human behavior and emotional reactions in response to virtual stimuli. It describes the “perceptual illusion” of being physically present and involved within a virtual environment (Lombard and Ditton, 1997). Even though users know they are experiencing a virtual world, this illusion allows them to react to a virtual environment in the same way they would to a real environment (Slater, 2018). Research suggests presence is comprised of two components (Slater, 2009): 1) place illusion, referring to the sense of “being there” within a physical space, and 2) plausibility illusion, referring to the illusion that what the user is experiencing in the virtual environment is actually happening. The resulting sensation is conceptualized as a construct that facilitates the activation of fear in response to a virtual stimulus. There is widespread support for the link between presence and anxiety (Ling et al., 2014), though research suggests it might not be straightforward in the context of social anxiety (Ling et al., 2014; Morina et al., 2014). In particular, one study found that presence was negatively correlated with social anxiety (Felnhofer et al., 2014). The authors suggest that this could be the result of behavioral inhibition in which higher anxiety levels lead to less exploration of virtual environments which in turn reduces presence. Others suggest that presence is necessary for inducing emotional reactions, but there is a trigger point at which it becomes sufficient, indicating a non-linear relationship (Bouchard et al., 2012). The only study, to our understanding, that has examined presence in the context of stuttering, found it was not related to stuttering severity during a virtual job interview (Brundage et al., 2006). Given stuttering severity appears linked to state social anxiety (Ezrati-Vinacour and Levin, 2004), this finding might suggest presence is also less relevant for PWS. Together, these findings indicate that the components of presence may not capture the essential sense of presence required for activating fear in response to social cues.
A separate but related concept that is particularly relevant for social scenarios in VR is social presence. This refers to the “sense of being with another” (Biocca et al., 2003, p. 456). Social presence is elicited within a mediated interaction with another co-present being that appears sentient. Without social presence, this being would be perceived as just an artificial object. By placing emphasis on social cues rather than physical cues, it may better explain how fear of negative evaluation can be elicited in response to virtual verbal and non-verbal cues. Research supports social presence as a facilitator of regular human behavior (Strojny et al., 2020), and emotional reactions in virtual social settings (Pfaller et al., 2021). However, the association between social presence and social anxiety has not yet received much research attention.
The conceptualization of presence and social presence as facilitators of emotional reactions within virtual environments suggests they may influence treatment outcome in VRET. According to the emotional processing theory, exposure relies on the activation of fear structures during exercises to achieve habituation towards feared stimuli (Foa and Kozak, 1986). Presence and social presence have therefore been theorized as key mechanisms that facilitate habituation (Wiederhold and Wiederhold, 2005). However, the lack of clarity over whether presence influences social anxiety implies that presence may not be a core mechanism through which emotional processing leads to social anxiety symptom change. Nevertheless, social presence may play this role if it can facilitate the induction of social fears in VR. The concept of presence is also theorized to be influential for treatment outcome under the inhibitory learning model of exposure (Craske et al., 2008; Craske et al., 2014). This approach uses a broader learning strategy to break down individuals’ fearful expectations rather than promoting fear activation and habituation. The overall aim is to learn that feared scenarios do not consistently result in feared expectations. Nevertheless, it is important that feared stimuli are believable and induce the same reactions when presented virtually so that strategies can be learnt to manage this fear. Greater levels of presence and social presence may facilitate these emotional reactions. More importantly, however, they are expected to improve attentiveness to others’ behaviors and improve believability of events in the virtual scene, reinforcing newly learnt associations with feared social cues.
As expected, evidence for the influence of presence on social anxiety reduction from VRET is not consistent. In one trial of VRET for public speaking anxiety, presence significantly predicted symptom reduction (Reeves et al., 2021). However, another study showed that despite a significant effect of VRET on social anxiety reduction, presence did not moderate this effect (Kahlon et al., 2019). A third study found that whilst presence was not related to treatment outcome, the involvement subfactor was (Price et al., 2011). The authors suggest that a higher level of involvement indicates a greater level of sustained attention during exposure exercises, minimizing distraction from other cues and allowing individuals to focus on extinction learning. Nevertheless, the influence of social presence on treatment outcome is unknown. As social presence is more relevant to virtual social environments, it may also play a greater role than presence in supporting symptom change from VRET. These links require further research.
1.2 The importance of therapeutic alliance
Another factor that could impact treatment outcomes in VRET for anxiety is therapeutic alliance. This refers to the collaborative working relationship between a therapist and individual. It encompasses both the individual’s sense of feeling supported by the therapist, and their faith in the therapeutic process (Luborsky, 1976). Therapeutic alliance is suggested to provide the necessary context for therapeutic strategies to work (Horvath and Greenberg, 1989). As such, therapist-led treatments tend to emphasize alliance in treatment design.
By this broad definition, alliance is conceptualized as an integral mechanism for affecting symptom change across a variety of therapeutic contexts. In the context of speech therapy targeting speech fluency in stuttering, the therapeutic relationship is considered a particularly important factor (Sønsterud et al., 2019). PWS report that a pre-existing therapist relationship facilitates access to psychological treatment alongside speech therapy (Lindsay and Langevin, 2017). However, research is yet to explore the effect of alliance on outcomes from social anxiety treatment amongst PWS.
Research exploring the relationship between therapeutic alliance and treatment outcome in the general population has shown mixed findings, especially in the context of social anxiety treatment. Type of treatment and level of therapist responsibility have been suggested as moderators of the relationship. In two studies, the level of therapist influence and input was used to explain the existence and absence of this relationship in different treatment methods (Jazaieri et al., 2018; Kivity et al., 2021). Similarly, the ability of the therapist to personalize treatment may also determine alliance’s ability to influence treatment outcome. Delivering treatment through a group format is unlikely to result in the same level of tailored goals and exercises as would be expected in individual treatment. Findings from two trials using group cognitive behavioral therapy (CBT) found no association between alliance and symptom change (Woody and Adessky, 2002; Jazaieri et al., 2018), whilst alliance has been linked to symptom change in individual CBT (Kivity et al., 2021). Another study using individual CBT found that higher levels of alliance were associated with improvements to treatment outcome, whilst lower levels of alliance increased the chance of dropping out (Haug et al., 2016). Nevertheless, one study comparing individual and group CBT found alliance was unrelated to symptom change in both treatment formats (Mörtberg, 2014). Taken together, evidence for the influence of alliance on social anxiety reduction from CBT remains inconclusive. The fear of being negatively evaluated may contribute to this by posing a threat to the interpersonal relationship with a therapist (Draheim and Anderson, 2019). As such, alliance may be more challenging to recreate in the context of social anxiety treatment.
In VRET, the therapist may take on different roles including functional operating roles. For the majority of trials examining VRET targeting social anxiety, real therapists with traditional roles have supported treatment delivery (Chard and van Zalk, 2022). In three trials, alliance was shown to be comparable between VRET and in vivo exposure therapy (Anderson et al., 2013; Ngai et al., 2015; Bouchard et al., 2017). In the only study of VRET targeting social anxiety to explore the influence of alliance, findings showed that it was a strong significant predictor of later symptom change (Bouchard et al., 2017). Similar relationships have been shown in VRET targeting other phobias (Miragall et al., 2015). As some of the therapist interaction in VRET happens without eye contact, there are fewer visual cues for this interaction to trigger the fear of negative evaluation. This might therefore provide a better context for alliance to develop, but research relating to alliance in VRET is limited and more is required to understand how it operates.
Remotely delivered and self-guided VRET extends the question of what the therapist role should look like, and what features can be harnessed from traditional treatments. There are no consistent guidelines for how virtual therapists should operate, and they can appear visually or through audio (Provoost et al., 2017). In the only trial of VRET for social anxiety that has experimented with a virtual therapist, findings showed that VRET significantly reduced social anxiety compared to waitlist, though there was no assessment of alliance (Zainal et al., 2021). Because this method eliminates real social exchange that can elicit fear of negative evaluation, virtual therapists may have the potential to elicit higher levels of alliance, which further promotes treatment outcome. However, they also offer less flexibility in providing personalized guidance, and developing a collaborative relationship may be challenging. As the technology improves, VR poses the opportunity to recreate therapist-patient interactions in a setting that is believable, approachable, and potentially less anxiety-inducing. Further research is clearly required to elucidate whether therapeutic alliance is a significant factor within this format of VRET.
1.3 Outcome expectancy and links to therapeutic alliance
Outcome expectancy refers to an individual’s beliefs about how successful treatment will likely be, and the potential benefits that will come if they continue to engage with it. This is related to treatment credibility, which refers to an individual’s belief of how plausible and logical a treatment is. Expectations of a successful outcome are suggested to provide hope and are thus considered necessary for ensuring an individual’s full engagement with treatment (Greenberg et al., 2006). Findings from one meta-analysis found that outcome expectancy was significantly positively related to treatment outcomes across a variety of treatment contexts (Constantino et al., 2011). Strategies that promote positive expectations of treatment are therefore generally promoted, so long as they do not give the individual unrealistic hope.
Therapeutic alliance has been suggested as a mechanism through which outcome expectancy affects symptom change in VRET. Individuals with more positive expectations of treatment may be more inclined to engage with the therapist, which in turn allows therapeutic strategies to work more effectively. A growing body of evidence supports this mediated association, with empirical support from trials of CBT for anxiety disorders (Sauer-Zavala et al., 2018) and depression (Vîslă et al., 2018), amongst others. Nevertheless, preliminary work suggests that therapeutic alliance may not operate as a mediating factor between outcome expectancy and social anxiety reduction. In one study comparing VRET and in vivo exposure therapy for social anxiety, there was no evidence for alliance playing this role in either treatment format (Draheim and Anderson, 2019). As before, fear of negative evaluation may have interfered with the therapeutic relationship, reducing the influence it has on treatment outcome. However, no research has explored whether these relationships exist in the context of self-guided VRET for social anxiety. If therapeutic alliance plays a more influential role in this modality, it may also mediate the relationship between outcome expectancy and treatment outcome. This needs to be explored empirically, however.
1.4 Current study
Based on these identified literature gaps, the aim of the current study is to investigate the influence of presence, social presence, therapeutic alliance, and outcome expectancy within VRET targeting social anxiety in stuttering. We use data from a small-scale randomized pilot trial of VRET targeting stuttering-specific social anxiety that was delivered remotely and guided by a virtual therapist (Chard et al., 2023). The VRET protocol was based on inhibitory learning strategies to promote learning of new associations with feared stimuli presented in exposure exercises. As participants had to work collaboratively with the virtual therapist, we hypothesize that therapeutic alliance will be associated with treatment success. In addition, as therapeutic alliance is predicted to be more influential within self-guided VRET, we also hypothesize that it will mediate the association between outcome expectancy and treatment outcome. Similarly, we expect both presence and social presence to influence symptom change under the inhibitory learning approach used in the current VRET protocol. In particular, social presence is hypothesized to affect treatment outcome given its relevance for interacting with social cues in VR. As PWS are affected similarly by presence and therapeutic alliance, there are no expectations regarding effects in the current sample, though this will be explored.
2 Methods
2.1 Study design
The current study was part of a parallel-group randomized controlled pilot trial, which assessed smartphone-based VRET against a waitlist control condition across a period of 2 months. Full details of the pilot trial have been published elsewhere (Chard et al., 2023). Participants were randomized into the two conditions. The VRET group completed a battery of measures at pre- and post-treatment assessing treatment outcome. After completing the 3-week waitlist period, the control group received VRET and completed the same battery of measures at pre- and post-treatment. As the post-waitlist experience was identical to the treatment condition, the two groups were merged to maximize sample size. All participants completed questionnaires for outcome expectancy, therapeutic alliance, presence, and social presence during the first session of VRET. The study procedure is illustrated in Figure 1.
FIGURE 1. Participant flow diagram illustrating study procedure.
The trial was approved by the Science Engineering Technology Research Ethics Committee at Imperial College London (reference number: 21IC7055). The report is written in line with the Consolidated Standards of Reporting Trials (CONSORT) Statement and the extension for reporting trials on psychological interventions (Montgomery et al., 2018).
2.2 Participants
Twenty two participants were recruited (see Table 1 for demographic statistics). The majority male sample reflects the gender split of PWS within the population (Bloodstein and Bernstein Ratner, 2008). Recruitment finished at the end of the allotted study period, which ran between September 2021—June 2022.
TABLE 1. Participant demographic information.
Participants were recruited through online adverts on the British Stammering Association’s website (Stamma.org), and through local stuttering groups across the United Kingdom. Inclusion criteria were 1) Person who stutters, 2) English-speaking, 3) United Kingdom resident, 4) age over 18, 5) in possession of a smartphone (iPhone/Android) and headphones/earphones, 6) no previous participation in the pilot trial or focus groups for same treatment, 7) no current experience of psychosis/schizophrenia/epilepsy/dementia/amnesia/bipolar disorder/intellectual disability/autism spectrum disorder, 8) no current use of tranquilizers or change in dosage of antidepressants in the past 6 weeks, 9) no current suicidal ideation, 10) no alcohol/substance dependence, 11) no severe cognitive impairment, 12) no treatment for social anxiety within the last year, 13) not participating in any other psychotherapeutic treatments during the study, and 14) no experience of severe cyber/motion-sickness. All participants who completed the pre- and post-treatment assessments (or equivalent for waitlist) received a £10 Amazon e-voucher.
2.3 Procedure
Upon contacting the researcher, prospective participants received the information sheet and consent form. Once the consent form was returned, participants were randomized into either the treatment or waitlist condition. The procedure for the pilot trial has been detailed elsewhere (Chard et al., 2023).
All participants were asked to provide their home address to which a cardboard VR headset could be sent. Once this was delivered, the VRET participants were sent a link to complete the pre-treatment assessment. This consisted of four self-report questionnaires assessing treatment outcome and a behavioral assessment task (not used in the current study). Participants were asked to notify the researcher when the assessment had been completed and were then given access to the first session of VRET. Three weeks later, at the end of the treatment program, participants were asked to complete the same battery of treatment outcome measures. Upon completion, they were given the option to receive an Amazon e-voucher worth £10.
After receiving their cardboard headset, waitlist participants completed the same battery of measures before a 3-week waiting period. At the end of this period, participants completed the measures again before being given access to the first session of VRET. The treatment procedure was identical to the treatment condition and finished with participants completing the battery of measures. Participants were then also given the option to receive an Amazon e-voucher worth £10.
Presence and social presence were assessed during each VRET session, whereas therapeutic alliance was assessed after each session. Outcome expectancy was completed after the first session only. As the study procedure was identical for both groups from the onset of treatment, participants were merged into a single group for the purpose of the current study. Therefore, the pre-waitlist assessment was ignored, and the post-waitlist assessment was treated as the pre-treatment assessment.
2.4 VRET
The VRET sessions were delivered remotely using a VR smartphone app developed in Unity (available on both iOS and Android devices) and a cardboard smartphone-based VR headset (Virtual Real Store Google Cardboard V2). The app offered three degrees of freedom, meaning participants could move their head to look around the 3D scene but could not physically move around it. The phone’s inertial sensors modified the visual and auditory rendering accordingly. The phone displayed two images, which were projected to each eye through the headset, showing where the individual is looking in the scene.
The treatment program consisted of three weekly sessions. The first session began with several psychoeducation exercises during which participants were introduced to the virtual therapist. In a series of pre-recorded 360° video clips, these covered the mechanisms underlying social anxiety, principles of exposure therapy and inhibitory learning, treatment structure and how to use the app. The virtual therapist also asked participants to formulate a goal for what they would like to achieve by the end of the treatment program. A total of 17 clips (13 min) were presented and all had to be completed within the first session and before progressing to exposure.
Each treatment session included three exposure exercises based on the same three social scenarios: ordering a drink at a café, telephone interaction, and public speaking (see Figure 2 for treatment app screenshots). These were adapted to target stuttering-specific social fears (see Supplementary Appendix SA1 for details) and were created using 360° video clips. Exercises varied in difficulty, but level of difficulty did not increase linearly throughout sessions. For the performative speech task, a single video clip displayed an audience in a room. For the interactive café and phone exercises, alternating clips of people talking/asking questions and being silent were used to facilitate turn-taking. Participants progressed through these interactions by clicking a button to indicate the end of their turn and play the next clip.
FIGURE 2. Screenshots of treatment app. (A) Virtual therapist; (B) Public speaking exercise; (C) Café exercise; (D) Telephone exercise. Note: Permission was granted from individuals to be filmed and to use footage in published material.
In this phase of the treatment program, the virtual therapist’s role was to engage participants in the psychological aspects of treatment and prepare participants to utilize these during exercises. Before each exercise, the virtual therapist helped participants define expectations about the exposure scenario and the safety behaviors they might perform in similar settings. They were also offered stuttering-specific examples of these to benefit the process. In line with the inhibitory learning model of exposure (Craske et al., 2008; Craske et al., 2014), the virtual therapist then helped participants evaluate how the outcome of the exposure exercise compared to their expectations, while providing encouragement. Participants were asked to repeat each exercise until they believed they had made sufficient progress towards understanding their expectations were overestimated and learnt that their anxiety is manageable. Sessions were designed to last approximately 20–30 min, but length of engagement was not measured.
2.5 Measures
2.5.1 Treatment outcome
The treatment outcome was measured by changes in symptom severity of social anxiety and fear of negative evaluation.
Social anxiety symptom severity was assessed using the Social Phobia Scale (SPS) (Mattick and Clarke, 1998). SPS is a 20-item self-report questionnaire using a 5-point Likert scale, designed to assess performance and scrutiny fears. SPS has shown good psychometric properties in previous research (Thompson et al., 2019). Cronbach’s α values for pre- and post-treatment SPS were .97 and .98, respectively.
Fear of being negatively evaluated by others was assessed using the brief version of the Fear of Negative Evaluation scale (FNE-B) (Leary, 1983). FNE-B is a 12-item self-report questionnaire using a 5-point Likert scale and has demonstrated good psychometric properties (Weeks et al., 2005) in previous research. Cronbach’s α values for pre- and post-treatment FNE-B were .94 and .95, respectively.
2.5.2 Outcome expectancy
A 4-item adaptation of the Credibility/Expectancy Questionnaire (Borkovec and Nau, 1972) was used to assess participants’ expectations of a successful outcome from VRET. One item (“How successful do you feel this treatment would be in decreasing a different fear, for example, strong anxiety about taking tests?”) was removed from the original scale as it was not relevant to the current research. The remaining four items were assessed on a 9-point Likert scale, with a Cronbach’s α of 0.89. The CEQ has demonstrated good psychometric properties (Devilly and Borkovec, 2000) in previous research.
2.5.3 Therapeutic alliance
The Virtual Therapist Alliance Scale (VTAS) (Miloff et al., 2020) was used to assess the strength of the therapeutic relationship with the virtual therapist. VTAS is primarily adapted from the Working Alliance Inventory (Horvath and Greenberg, 1989) with additional items to assess empathy and copresence associated with the virtual therapist. It is a 17-item self-report questionnaire assessed using a 5-point Likert scale. VTAS has demonstrated good psychometric properties (Miloff et al., 2020) in previous research. Cronbach’s α values were 0.93, 0.97 and 0.94 for sessions one, two, and three respectively.
2.5.4 Presence
The iGroup Presence Questionnaire (IPQ) (Schubert et al., 2001) was used to assess the sense of presence within the virtual environment. The IPQ is a 14-item self-report questionnaire assessed using a 7-point Likert scale, and comprises three separate subscales which cover spatial presence, involvement, and realness. IPQ has demonstrated good psychometric properties (Schwind et al., 2019) in previous research, albeit employing more sophisticated hardware compared with the current study. Cronbach’s α values were 0.92, 0.90 and 0.86 for sessions one, two, and three respectively.
Social presence was measured using the Networked Minds Measure of Social Presence (NMMSP) (Harms and Biocca, 2004). The NMMSP is a 36-item self-report questionnaire assessed using a 7-point Likert scale, and comprises six subscales which cover co-presence, attentional allocation, perceived message understanding, perceived affective understanding, perceived emotional interdependence, and perceived behavioral interdependence. NMMSP has demonstrated good psychometric properties (Harms and Biocca, 2004) in previous research. Cronbach’s α values were 0.86, 0.91 and 0.96 for sessions one, two, and three respectively.
2.6 Statistical analyses
2.6.1 Data cleaning and imputation
Several measures (session one presence, session one social presence, session two presence, session three therapeutic alliance, pre-/post-treatment social anxiety, pre-/post-treatment fear of negative evaluation) were transformed using Box-Cox transformations after not meeting at least one assumption for parametric analyses (normality, skewedness, kurtosis, homogeneity of variance and extreme values). For presence and social presence, we used the full scales rather than subscales due to the small sample size.
Most measures contained some missing data, and Little’s test returned insignificant results for both the raw (χ2 = 73.07, df = 99, p = 0.98) and transformed datasets (χ2 = 82.56, df = 99, p = 0.88) indicating they were missing at random. Missing values were then imputed using the Expectation-Maximization method in all datasets.
Analyses were run on both the raw and transformed datasets, and their imputed equivalents. We only report findings from the raw imputed dataset as it improved both parametric and model assumptions and minimized bias by not requiring listwise/pairwise deletion. As the amount of missing data in sessions 2 and 3 exceeded the recommended limit for imputation (Jakobsen et al., 2017), only session 1 therapeutic alliance, presence and social presence were used in statistical analyses. Imputation introduced a small number of outliers and influential values into the dataset. These were deemed genuine values and did not indicate a pattern of widespread bias. We therefore decided not to exclude these values as doing so would re-introduce bias due to missing data.
2.6.2 Analytic strategy
Multiple regression models were run in SPSS to assess the influence of outcome expectancy, therapeutic alliance, presence, and social presence on changes in treatment outcomes (social anxiety and fear of negative evaluation). Post-treatment outcome scores were used as the dependent variable. Independent variables were entered into the model in two steps: (1) pre-treatment scores were added to control for baseline levels of the respective treatment outcome variable, and (2) variable of interest was then added in second block. SPSS output included R2 values, and effect sizes (f2) were calculated for each parameter of interest (Selya et al., 2012). Regressing post-treatment scores onto pre-treatment scores is considered the most appropriate method for allowing the assessment of other factors on pre-post change (Mattes and Roheger, 2020). However, this assumes linearity between pre- and post-treatment measures. This assumption was met for correlations between pre- and post-treatment social anxiety, and pre- and post-treatment fear of negative evaluation (see Table 2).
TABLE 2. Correlations between measures.
Upon visual inspection, some variables did not appear linearly related to the treatment outcome change scores. We probed this by creating quadratic terms for every variable of interest, adding them in a third block to their respective hierarchical regression model. Because adding quadratic terms to a model can introduce multicollinearity, we centered all predictors, including quadratic variables. For the models assessing the relationship between outcome expectancy and fear of negative evaluation change, and the relationship between therapeutic alliance and fear of negative evaluation change, this significantly improved model fit. Therefore, the reported findings for these relationships include results from quadratic parameters.
As age and gender have been shown to impact social anxiety (Stein and Stein, 2008; Asher et al., 2017), we added them in further blocks to each hierarchical regression model to assess if they improved model fit. No model reported a significant change in model fit, therefore output reflects the models which do not include these factors.
The PROCESS macro for SPSS (Hayes, 2022) was used to assess the mediating influence of therapeutic alliance on the relationship between outcome expectancy and treatment outcome. This used 5000 bootstrap samples and 95% confidence intervals. Models were run for both social anxiety and fear of negative evaluation, controlling for pre-treatment scores by adding them as covariates.
2.6.3 Sensitivity analyses
Sensitivity analyses were conducted after data was collected and analyzed to determine the minimum treatment effect size required to achieve a sufficient statistical power with our same sample size. Using G*Power, we calculated the required effect size (f2) for each linear and quadratic parameter of interest in every model. This was based on power (80%), sample size, number of predictors being tested, and total number of predictors in model. For both linear and quadratic predictors, the effect size required for achieving 80% power was 0.40.
3 Results
Raw data for this study have been published in an open-access repository (Chard et al., 2022). Descriptive statistics are shown in Table 3, with correlations between each measure presented in Table 2.
TABLE 3. Descriptive statistics for self-report questionnaires.
3.1 Participant attrition
Six participants did not complete the post-treatment assessment. Welch’s t-test indicated no significant difference in pre-treatment social anxiety between participants who did (M = 1.30, SD = 1.00) and did not (M = 0.82, SD = 0.47) complete the post-treatment assessment, t(18.60) = 1.54, p = 0.14. There was also no significant difference in pre-treatment fear of negative evaluation between participants who did (M = 3.54, SD = 0.93) and did not (M = 3.21, SD = 0.88) complete the post-treatment assessment, t(9.52) = 0.78, p = 0.45.
Nine participants did not complete the outcome expectancy and session one therapeutic alliance questionnaires. Five participants did not complete the session one presence questionnaire. Six participants did not complete the session one social presence questionnaire. A series of Welch’s t-tests found no significant differences in pre-treatment social anxiety between participants who did and did not complete these questionnaires, t(range) = 0.95–1.58, p(range) = 0.13–0.36.
3.2 Relationship between study predictors and VRET treatment outcome
3.2.1 Outcome expectancy
Outcome expectancy was positively associated with changes in social anxiety between pre- and post-treatment, B = 0.24, ΔF(1, 19) = 7.58, p = 0.01, ΔR2 = 0.06, f2 = 0.86. This effect size exceeded the required effect size found under the sensitivity analysis (0.40), suggesting the current study is sufficiently powered to detect this effect. Higher outcome expectancy was associated with greater increases to social anxiety, whereas lower levels of outcome expectancy were associated with greater decreases to social anxiety.
No linear relationship was found between outcome expectancy and changes in fear of negative evaluation, B = −0.09, ΔF(1, 19) = 0.40, p = 0.54, ΔR2 = 0.01, f2 = 0.03. However, adding the quadratic parameter for outcome expectancy significantly improved model fit, B = 0.39, ΔF(1, 18) = 10.19, p = 0.01, ΔR2 = 0.14, f2 = 0.58. Figure 3 shows that low and high levels of outcome expectancy were related to an increase in fear of negative evaluation, whereas medium levels of outcome expectancy were associated with a decrease in fear of negative evaluation. The size of the quadratic effect exceeded that required by the sensitivity analysis (0.40), whilst the linear component was very small. Therefore, the current study is sufficiently powered to detect the quadratic effect of outcome expectancy on fear of negative evaluation. These results suggest that the relation between outcome expectancy and changes to fear of negative evaluation varies based on outcome expectancy levels.
FIGURE 3. Line graph representing relationship between outcome expectancy and fear of negative evaluation change scores.
3.2.2 Therapeutic alliance
Alliance with the virtual therapist was positively associated with changes in social anxiety between pre- and post-treatment, B = 0.28, ΔF(1, 19) = 11.66, p = 0.003, ΔR2 = 0.07, f2 = 0.58. This effect size exceeded the required effect size found under the sensitivity analysis (0.40), suggesting sufficient power to detect this effect. Higher levels of alliance were associated with greater increases to social anxiety, whereas lower levels of alliance were associated with greater decreases to social anxiety.
No linear relationship was found between therapeutic alliance and changes in fear of negative evaluation, B = −0.04, ΔF(1, 19) = 0.08, p = 0.78, ΔR2 = 0.002, f2 = 0.01. Adding a quadratic parameter for therapeutic alliance significantly improved model fit, B = 0.29, ΔF(1, 18) = 4.92, p = 0.04, ΔR2 = 0.08, f2 = 0.27. Low and high levels of therapeutic alliance were related to an increase in fear of negative evaluation, whereas medium levels of therapeutic alliance were associated with a decrease in fear of negative evaluation (see Figure 4). Neither the linear nor quadratic effect of therapeutic alliance on fear of negative evaluation reached the necessary effect size required for achieving a sufficient level of power (0.40), however. Thus, the lack of power precludes conclusions about this relationship in our data.
FIGURE 4. Line graph representing relationship between therapeutic alliance and fear of negative evaluation change scores.
3.2.3 Presence
Presence was positively associated with changes in social anxiety between pre- and post-treatment, B = 0.26, ΔF(1, 19) = 6.83, p = 0.02, ΔR2 = 0.05, f2 = 0.36. Higher levels of presence were associated with greater increases to social anxiety, whereas lower levels were associated with greater decreases to social anxiety.
However, presence was also negatively associated with changes in fear of negative evaluation between pre- and post-treatment, B = −0.35, ΔF(1, 19) = 7.56, p = 0.01, ΔR2 = 0.11, f2 = 0.39. Therefore, higher levels of presence were related to greater decreases in fear of negative evaluation, whereas lower levels were related to greater increases in fear of negative evaluation. Nevertheless, the effects of presence on changes in social anxiety and fear of negative evaluation are smaller than the required effect for achieving a sufficient level of power (0.40). Thus, the current study was not sensitive enough to detect these effects.
3.2.4 Social presence
There was no significant relationship between social presence and changes in social anxiety, B = 0.09, ΔF(1, 19) = 0.80, p = 0.38, ΔR2 = 0.01, f2 = 0.05. However, social presence was negatively associated with changes in fear of negative evaluation between pre- and post-treatment, B = −0.45, ΔF(1, 19) = 20.15, p < 0.001, ΔR2 = 0.20, f2 = 1.05. As such, higher levels of social presence were related to greater decreases in fear of negative evaluation, whereas lower levels of social presence were related to greater increases in fear of negative evaluation. Whilst the effect of social presence on social anxiety was small, its effect on fear of negative evaluation was large and exceeded that required by the sensitivity analysis (0.40). Therefore, this effect is sufficient for achieving necessary power.
3.3 Mediating role of therapeutic alliance on the relationship between outcome expectancy and treatment outcome
Results from the mediation analysis are presented in Figure 5. Outcome expectancy was found to be significantly associated with an increase in social anxiety scores between pre- and post-treatment (B = 0.10, t = 2.75, p = 0.01). However, when therapeutic alliance was added to the model, the influence of outcome expectancy was reduced and was no longer significantly related to social anxiety change (B = −0.06, t = −0.61, p = 0.55). Outcome expectancy was a significant predictor of therapeutic alliance (B = 0.23, t = 11.15, p < 0.001), but alliance showed no relationship with changes in social anxiety (B = 0.70, t = 1.79, p = 0.09). The indirect effect of outcome expectancy on social anxiety change via therapeutic alliance was found to be non-significant, B = 0.16, boot SE = 0.11, 95% CI [-0.06, 0.39]. Thus, we did not identify indirect effects of therapeutic alliance on the link between outcome expectancy and treatment outcome.
FIGURE 5. Mediation model displaying the indirect effect of outcome expectancy on treatment outcome via therapeutic alliance. Note: a, coefficient for CEQ effect on VTAS; b, coefficient for VTAS effect on SPS change; c’, coefficient for direct effect of CEQ on SPS change; c, coefficient for total effect of CEQ on SPS change. *p < 0.05, **p < 0.001.
4 Discussion
This study investigated the effect of various user experience and treatment mechanisms on symptom change in a small-scale pilot trial of VRET targeting stuttering-specific social anxiety. Our findings show that high levels of presence, therapeutic alliance and outcome expectancy were associated with increases in social anxiety between pre- and post-treatment, whereas low levels were associated with decreases in social anxiety. No relationship was found between social presence and changes in social anxiety. In addition, high levels of presence and social presence were associated with a decline in fear of negative evaluation, whereas low values were related to an increase in fear of negative evaluation. Both outcome expectancy and therapeutic alliance were quadratically related to changes in fear of negative evaluation. Lastly, therapeutic alliance was found not to mediate the relationship between outcome expectancy and treatment outcome.
Our results show that presence and social presence were the only two variables linearly related to symptom reduction as hypothesized. These factors significantly predicted a decrease in fear of negative evaluation, suggesting they may be related to successful treatment outcome in VRET targeting social anxiety. Nevertheless, as the sensitivity analysis revealed that the effect of presence was not sufficiently powered, caution must be taken when interpreting this finding. The direction of the presence effect is in line with a previous study where presence was associated with symptom reduction in VRET targeting public speaking anxiety (Reeves et al., 2021). However, presence has not been consistently linked with reduction of social anxiety symptoms (Price et al., 2011; Kahlon et al., 2019). The influence of social presence has not previously been examined but this is likely to be more relevant to user experience in social VRET exercises. Previous research suggests social presence is important for eliciting emotional reactions from social cues in VR (Pfaller et al., 2021), and the fear of being negatively evaluated by others is central to the experience of social anxiety (Rapee and Heimberg, 1997). Our findings therefore suggest that social presence is an influential factor for engaging with social cues within VRET and eliciting relevant fear responses which support symptom change.
The negative links between presence and social anxiety are particularly intriguing, as the opposite effect was found for fear of negative evaluation. Other studies have demonstrated no relation between presence and treatment outcome (Price et al., 2011; Kahlon et al., 2019), but this result suggests that heightened presence in VRET exercises may negatively impact social anxiety symptoms. Again, caution must be taken in this interpretation given the results from the sensitivity analysis. Fear of negative evaluation is a core emotional reaction underlying social anxiety (Rapee and Heimberg, 1997), and the two measures are typically strongly positively related (Weeks et al., 2005). However, we found no correlation between social anxiety and fear of negative evaluation change scores (see Table 2), suggesting they were likely affected differently by VRET. This result might be driven by presence, given its mixed associations with treatment outcome. Presence may have helped participants engage in cognitive processing required by inhibitory learning techniques to target feared expectations of being negatively evaluated. However, therapeutic strategies targeting the wider cognitive and behavioral mechanisms underlying social anxiety (Clark and Wells, 1995; Rapee and Heimberg, 1997; Iverach et al., 2017) may have been insufficient or even detrimental for some participants. For example, the virtual therapist may have been ineffective at supporting the reduction of safety behaviors such as avoidance of eye contact. Without an external individual to recognize problematic behaviors and offer reflections to encourage effective change, heightened presence may have provided a context for participants to continue practicing these behaviors. This would contribute to further problematic thoughts and prevention of new non-fear associations with social cues. Presence is therefore suggested to provide an adequate learning context within VR, but the learning depends on the experience and effectiveness of therapeutic strategies employed.
An alternative argument is that there is no relationship between presence and social anxiety changes, but that both are the result of state anxiety during exposure exercises. One previous study found a negative association between presence and social anxiety (Felnhofer et al., 2014). Authors suggested that high levels of state anxiety may have led to less behavioral exploration of the virtual environment, in turn reducing presence. However, state anxiety during exposure is also suggested to facilitate treatment success (Foa and Kozak, 1986). In the current study, several participants reported a lack of state anxiety during exposure exercises, which may have minimized symptom reduction or even contributed to an increase in symptoms. Lower state anxiety may have also minimized behavioral inhibition and increased presence, thus explaining the negative association between presence and social anxiety change. There may be truth in both interpretations of findings, as presence and anxiety have been suggested to be dependent on each other (Peperkorn et al., 2015). Experimental manipulation of presence and anxiety is required to draw more conclusive remarks about their relationship.
Heightened outcome expectancy and therapeutic alliance were also found to predict an increase in social anxiety. This goes against previous research findings, which suggest that therapeutic alliance (Bouchard et al., 2017) and outcome expectancy (Constantino et al., 2011) are both strong predictors of symptom reduction. It is also unusual given the therapeutic relationship is considered crucial for other treatments in stuttering (Lindsay and Langevin, 2017; Sønsterud et al., 2019). Other studies have found no influence of these factors for different treatment formats (Mörtberg, 2014; Jazaieri et al., 2018), but this is the first study to suggest they may hinder VRET success. Novelty of treatment methods may underlie this effect in the current VRET protocol. Several participants reported being impressed and excited about completing VRET, which may itself have encouraged positive rapport with the virtual therapist. This positive approach to treatment is generally desirable, but it may have also hindered some participants’ engagement with treatment strategies. For some, it may have allowed problematic thoughts and behaviors to persist, maintaining or increasing social anxiety. The beliefs and faith placed in technology-assisted treatment may have also overinflated expectations and contributed to less engagement and poorer treatment outcomes. These interpretations raise further questions regarding the ability of VR treatment to recreate certain core in vivo treatment functions. Further research is required to experimentally test novel methods, such as virtual therapists. Future VRET protocols should also consider techniques to temper expectations about treatment outcome that might be conflated with the novelty of using VR.
Outcome expectancy and therapeutic alliance were also significantly related to changes in fear of negative evaluation, but this relationship was not linear. Our finding of a quadratic relationship suggests that high and low values of outcome expectancy and therapeutic alliance were associated with an increase in fear of negative evaluation, whereas medium-sized values were related to a decline. However, additional sensitivity analyses suggested that the effect of therapeutic alliance was not sufficiently powered, therefore these links should therefore be interpreted cautiously. A similar argument to above could be applied, as high levels of outcome expectancy and therapeutic alliance may have reduced treatment engagement, allowed maladaptive processes to persist, and in turn increased fear of negative evaluation. In line with original hypotheses, lower levels of outcome expectancy and therapeutic alliance may have also reduced faith and hope in the therapeutic process, thus reducing engagement and causing fears to persist and increase. Technology-assisted treatments may therefore need to strike a balance in user expectations and engagement. Our findings suggest that treatment outcome may be optimized when therapeutic alliance and outcome expectancy are neither too high nor too low.
No evidence was found within our sample for the mediating effect of therapeutic alliance on the relationship between outcome expectancy and treatment outcome. Therapeutic alliance has been suggested as a mechanism through which outcome expectancy affects symptom reduction in other treatment formats targeting other mental health issues (Sauer-Zavala et al., 2018; Vîslă et al., 2018). Nevertheless, this effect was also not present in another study of VRET targeting social anxiety (Draheim and Anderson, 2019). Our findings suggest that the strength of bond with the virtual therapist did not explain outcome expectancy’s association with social anxiety and fear of negative evaluation change in PWS. This is in line with our previous interpretation of results, which suggests that the eagerness and enthusiasm to participate in a novel treatment may have triggered both outcome expectancy and alliance separately. As such, treatment enthusiasm may underlie the effects of both alliance and outcome expectancy on treatment outcome, rather than therapeutic explaining the influence of outcome expectancy.
4.1 Limitations
Our study is not without its limitations. First, the low sample size is problematic for the conclusions drawn from some of the findings. Recruitment was a particular challenge given PWS represent a small proportion of the general population (Craig et al., 2002), and not all are affected by elevated social anxiety (Craig and Tran, 2014). In addition, United Kingdom COVID-19 restrictions at the time (March 2020—July 2021) posed challenges to recruitment as in-person engagement and participation was not always permitted. The low sample size has resulted in reduction of statistical power for all analyses, thus increasing the chance of type I and II errors. Sensitivity analyses indicated that the significant effects of presence on social anxiety, and presence and therapeutic alliance on fear of negative evaluation, did not meet the necessary effect size required for achieving a sufficient level of power. The required effect size was large for all analyses, demonstrating the challenges posed by using a small sample. The models assessing presence obtained large effect sizes but narrowly missed this threshold. As a result, caution should be taken in interpreting these effects.
Second, therapeutic alliance was measured at the end of each treatment session once all therapist interaction for that week was completed. This approach has been used in several other studies (Woody and Adessky, 2002; Bouchard et al., 2017; Jazaieri et al., 2018). However, some authors have suggested that this might inflate perceived alliance, and cause less stability in this measure, particularly early in treatment (Kivity et al., 2021). Instead, measuring alliance prior to each session is suggested to capture the “typical” alliance across the previous week. In the current study, we decided to measure alliance at the end of the session to maximize responses in session one, circumventing the effects of participant dropout in the later stages of the trial and thus reducing the risk of missing data. Additionally, as participants could complete each treatment session over the course of a week, the alliance measure may have captured this longer-term perception of the therapeutic relationship.
4.2 Strengths
Despite these limitations, our study has several strengths. First, it represents findings from a first pilot trial to assess the impacts of presence, social presence, outcome expectancy and therapeutic alliance on treatment outcome in remotely delivered smartphone VRET for social anxiety. Even though the results indicate that our treatment protocol requires more work, it also showed some positive effects in terms of reducing fear of negative evaluation—a core component of social anxiety. As this treatment format will likely become more prevalent given its suitability to making mental healthcare more accessible, our study represents a first step in empirically validating a VRET protocol for reducing social anxiety in people who stutter. Further research is required to improve the VRET design, and the increased uptake of digital and self-guided methods during the COVID-19 pandemic will hopefully accelerate this.
Second, we used imputation of missing scores to reduce bias introduced by missing data. Participant attrition is almost inevitable in trials of social anxiety treatment, but the use of appropriate methods to deal with missing data and resultant bias was limited in related papers (Bouchard et al., 2017). In one study, missing post-treatment scores were replaced with pre-treatment scores (Hayes et al., 2007), therefore assuming a null effect in participants who dropped out and biasing findings. In this study, we imputed missing values using the Expectation-Maximization method. We also ensured that the pattern of missingness met the relevant assumptions to use this method. This method replaces missing data with estimated values, thereby minimizing the bias they will have on results. As a result, we can be more confident in the conclusions we draw from our findings.
4.3 Conclusion
With rapid development in technological capabilities, VRET is likely to become a popular treatment choice for social anxiety. In a recent report, the World Health Organization recommended scaling up remote delivery of mental health provision using digital technologies, considering the vast treatment gap between individuals with problems and those receiving treatment (World Health Organization, 2022). Nevertheless, increased digital mental health provision requires more in-depth insights into the experience of the technologies being used, not just their efficacy. Research has begun to examine whether VR is an effective tool for treating a variety of anxiety-related problems. However, the role of user experience and treatment mechanisms in affecting symptom improvement in this format is still poorly understood. In this paper, we provide an early indication of how change may be affected in remotely delivered smartphone VRET targeting social anxiety in people who stutter. Our unexpected findings of an increase in social anxiety related to heightened presence, outcome expectancy and therapeutic alliance suggest techniques that should be avoided or adapted in future virtual treatment protocols. Furthermore, they indicate that the positive influence of expectations might not directly translate from in vivo exposure therapy to VRET, and that the two treatment modalities might require different approaches in tackling the same problems. As such, these findings are a useful resource for the future development of VRET targeting social anxiety, especially for people who stutter.
Data availability statement
The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://osf.io/umswv/.
Ethics statement
The studies involving humans were approved by the Science Engineering Technology Research Ethics Committee, Imperial College London. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article (Reference No. 21IC7055).
Author contributions
Conceptualization (IC); Literature review (IC); Intervention design (IC, NZ, and LP); Study design (IC and NZ); Data collection and analysis (IC); Writing original draft (IC); Writing reviews and editing (IC, NZ, and LP); Supervision (NZ and LP). All authors contributed to the article and approved the submitted version.
Funding
The research was funded by a training grant from UK Research and Innovation and Imperial College London (No. EP/R513052/1).
Acknowledgments
A Preprint of this paper is available on PsyArXiv (doi: 10.31234/osf.io/tjdhx).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2023.1159549/full#supplementary-material
Abbreviations
CBT, Cognitive Behavioral Therapy; CEQ, Credibility/Expectancy Questionnaire; FNE-B, brief version of the Fear of Negative Evaluation scale; IPQ, iGroup Presence Questionnaire; NMMSP, Networked Minds Measure of Social Presence; PWS, People Who Stutter; SPS, Social Phobia Scale; VR, Virtual Reality; VRET, Virtual Reality Exposure Therapy; VTAS, Virtual Therapist Alliance Scale.
References
American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders. Washington, DC: American Psychiatric Association.
Anderson, P. L., Price, M., Edwards, S. M., Obasaju, M. A., Schmertz, S. K., Zimand, E., et al. (2013). Virtual reality exposure therapy for social anxiety disorder: A randomized controlled trial. J. Consult. Clin. Psychol. 81 (5), 751–760. doi:10.1037/a0033559
Asher, M., Asnaani, A., and Aderka, I. M. (2017). Gender differences in social anxiety disorder: A review. Clin. Psychol. Rev. 56, 1–12. doi:10.1016/j.cpr.2017.05.004
Biocca, F., Harms, C., and Burgoon, J. K. (2003). Toward a more robust theory and measure of social presence: review and suggested criteria. Presence Teleoperators Virtual Environ. 12 (5), 456–480. doi:10.1162/105474603322761270
Bloodstein, O., and Bernstein Ratner, N. (2008). A handbook on stuttering. Clifton Park, NY: Thomson/Delmar Learning.
Borkovec, T. D., and Nau, S. D. (1972). Credibility of analogue therapy rationales. J. Behav. Ther. Exp. Psychiatry 3 (4), 257–260. doi:10.1016/0005-7916(72)90045-6
Bouchard, S., Dumoulin, S., Robillard, G., Guitard, T., Klinger, E., Forget, H., et al. (2017). Virtual reality compared with in vivo exposure in the treatment of social anxiety disorder: A three-arm randomised controlled trial. Br. J. Psychiatry 210 (4), 276–283. doi:10.1192/bjp.bp.116.184234
Bouchard, S., Robillard, G., Larouche, S., and Loranger, C. (2012). “Description of a treatment manual for in virtuo exposure with specific phobia,” in Virtual reality in psychological, medical and pedagogical applications. Editor C. Eichenberg (Rijeka, Croatia: IntechOpen), 82–108.
Brundage, S. B., Graap, K., Gibbons, K. F., Ferrer, M., and Brooks, J. (2006). Frequency of stuttering during challenging and supportive virtual reality job interviews. J. Fluen. Disord. 31 (4), 325–339. doi:10.1016/j.jfludis.2006.08.003
Chard, I., Van Zalk, N., and Picinali, L. (2022). Virtual reality exposure therapy for reducing social anxiety in stuttering. Frankfurt, Germany: OSF.
Chard, I., Van Zalk, N., and Picinali, L. (2023). Virtual reality exposure therapy for reducing social anxiety in stuttering: A randomized controlled pilot trial. Front. Digital Health 5, 1061323. doi:10.3389/fdgth.2023.1061323
Chard, I., and van Zalk, N. (2022). Virtual reality exposure therapy for treating social anxiety: A scoping review of treatment designs and adaptation to stuttering. Front. Digital Health 4, 842460. doi:10.3389/fdgth.2022.842460
Chesham, R. K., Malouff, J. M., and Schutte, N. S. (2018). Meta-analysis of the efficacy of virtual reality exposure therapy for social anxiety. Behav. Change 35 (3), 152–166. doi:10.1017/bec.2018.15
Clark, D. M., and Wells, A. (1995). “A cognitive model of social phobia,” in Social phobia: Diagnosis, assessment and treatment. Editors R. G. Heimberg, M. R. Liebowitz, D. A. Hope, and F. R. Schneier (New York, NY: Guilford Press).
Constantino, M. J., Arnkoff, D. B., Glass, C. R., Ametrano, R. M., and Smith, J. Z. (2011). Expectations. J. Clin. Psychol. 67 (2), 184–192. doi:10.1002/jclp.20754
Craig, A., Hancock, K., Tran, Y., Craig, M., and Peters, K. (2002). Epidemiology of stuttering in the community across the entire life span. J. Speech, Lang. Hear. Res. 45 (6), 1097–1105. doi:10.1044/1092-4388(2002/088
Craig, A., and Tran, Y. (2014). Trait and social anxiety in adults with chronic stuttering: conclusions following meta-analysis. J. Fluen. Disord. 40, 35–43. doi:10.1016/j.jfludis.2014.01.001
Craske, M. G., Kircanski, K., Zelikowsky, M., Mystkowski, J., Chowdhury, N., and Baker, A. (2008). Optimizing inhibitory learning during exposure therapy. Behav. Res. Ther. 46 (1), 5–27. doi:10.1016/j.brat.2007.10.003
Craske, M. G., Treanor, M., Conway, C. C., Zbozinek, T., and Vervliet, B. (2014). Maximizing exposure therapy: an inhibitory learning approach. Behav. Res. Ther. 58, 10–23. doi:10.1016/j.brat.2014.04.006
Devilly, G. J., and Borkovec, T. D. (2000). Psychometric properties of the credibility/expectancy questionnaire. J. Behav. Ther. Exp. Psychiatry 31 (2), 73–86. doi:10.1016/S0005-7916(00)00012-4
Draheim, A. A., and Anderson, P. L. (2019). Working alliance does not mediate the relation between outcome expectancy and symptom improvement following cognitive behavioural therapy for social anxiety disorder. Cognitive Behav. Ther. 12, e41. doi:10.1017/S1754470X19000266
Ezrati-Vinacour, R., and Levin, I. (2004). The relationship between anxiety and stuttering: A multidimensional approach. J. Fluen. Disord. 29 (2), 135–148. doi:10.1016/j.jfludis.2004.02.003
Felnhofer, A., Kothgassner, O. D., Hetterle, T., Beutl, L., Hlavacs, H., and Kryspin-Exner, I. (2014). Afraid to be there? Evaluating the relation between presence, self-reported anxiety, and heart rate in a virtual public speaking task. Cyberpsychol. Behav. Soc. Netw. 17 (5), 310–316. doi:10.1089/cyber.2013.0472
Foa, E. B., and Kozak, M. J. (1986). Emotional processing of fear: exposure to corrective information. Psychol. Bull. 99 (1), 20–35. doi:10.1037//0033-2909.99.1.20
Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., et al. (2017). Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol. Med. 47 (14), 2393–2400. doi:10.1017/s003329171700040x
Greenberg, R. P., Constantino, M. J., and Bruce, N. (2006). Are patient expectations still relevant for psychotherapy process and outcome? Clin. Psychol. Rev. 26 (6), 657–678. doi:10.1016/j.cpr.2005.03.002
Harms, C., and Biocca, F. (2004). “Internal consistency and reliability of the networked minds measure of social presence,” in Seventh annual international workshop: Presence 2004. Editors M. Alcaniz, and B. Rey (Valencia: Universidad Politecnica de Valencia).
Haug, T., Nordgreen, T., Öst, L. G., Tangen, T., Kvale, G., Hovland, O. J., et al. (2016). Working alliance and competence as predictors of outcome in cognitive behavioral therapy for social anxiety and panic disorder in adults. Behav. Res. Ther. 77, 40–51. doi:10.1016/j.brat.2015.12.004
Hayes, A. F. (2022). Introduction to mediation, moderation, and conditional process analysis. New York, NY: Guilford Press.
Hayes, S. A., Hope, D. A., Van Dyke, M. M., and Heimberg, R. G. (2007). Working alliance for clients with social anxiety disorder: relationship with session helpfulness and within-session habituation. Cogn. Behav. Ther. 36 (1), 34–42. doi:10.1080/16506070600947624
Horigome, T., Kurokawa, S., Sawada, K., Kudo, S., Shiga, K., Mimura, M., et al. (2020). Virtual reality exposure therapy for social anxiety disorder: A systematic review and meta-analysis. Psychol. Med. 50 (15), 2487–2497. doi:10.1017/S0033291720003785
Horvath, A. O., and Greenberg, L. S. (1989). Development and validation of the working alliance inventory. J. Couns. Psychol. 36 (2), 223–233. doi:10.1037/0022-0167.36.2.223
Iverach, L., and Rapee, R. M. (2014). Social anxiety disorder and stuttering: current status and future directions. J. Fluen. Disord. 40, 69–82. doi:10.1016/j.jfludis.2013.08.003
Iverach, L., Rapee, R. M., Wong, Q. J. J., and Lowe, R. (2017). Maintenance of social anxiety in stuttering: A cognitive-behavioral model. Am. J. Speech-Language Pathology 26 (2), 540–556. doi:10.1044/2016_AJSLP-16-0033
Jakobsen, J. C., Gluud, C., Wetterslev, J., and Winkel, P. (2017). When and how should multiple imputation be used for handling missing data in randomised clinical trials: A practical guide with flowcharts. BMC Med. Res. Methodol. 17 (1), 162. doi:10.1186/s12874-017-0442-1
Jazaieri, H., Goldin, P. R., and Gross, J. J. (2018). The role of working alliance in CBT and MBSR for social anxiety disorder. Mindfulness 9 (5), 1381–1389. doi:10.1007/s12671-017-0877-9
Kahlon, S., Lindner, P., and Nordgreen, T. (2019). Virtual reality exposure therapy for adolescents with fear of public speaking: A non-randomized feasibility and pilot study. Child Adolesc. Psychiatry Ment. Health 13 (1), 47. doi:10.1186/s13034-019-0307-y
Kampmann, I. L., Emmelkamp, P. M. G., Hartanto, D., Brinkman, W. P., Zijlstra, B. J. H., and Morina, N. (2016a). Exposure to virtual social interactions in the treatment of social anxiety disorder: A randomized controlled trial. Behav. Res. Ther. 77, 147–156. doi:10.1016/j.brat.2015.12.016
Kampmann, I. L., Emmelkamp, P. M. G., and Morina, N. (2016b). Meta-analysis of technology-assisted interventions for social anxiety disorder. J. Anxiety Disord. 42, 71–84. doi:10.1016/j.janxdis.2016.06.007
Kivity, Y., Strauss, A. Y., Elizur, J., Weiss, M., Cohen, L., and Huppert, J. D. (2021). The alliance mediates outcome in cognitive–behavioral therapy for social anxiety disorder, but not in attention bias modification. Psychotherapy Res. 31 (5), 589–603. doi:10.1080/10503307.2020.1836423
Klinger, E., Bouchard, S., Legeron, P., Roy, S., Lauer, F., Chemin, I., et al. (2005). Virtual reality therapy versus cognitive behavior therapy for social phobia: A preliminary controlled study. Cyberpsychology Behav. 8 (1), 76–88. doi:10.1089/cpb.2005.8.76
Leary, M. R. (1983). A brief version of the fear of negative evaluation scale. Personality Soc. Psychol. Bull. 9 (3), 371–375. doi:10.1177/0146167283093007
Lindsay, A., and Langevin, M. (2017). Psychological counseling as an adjunct to stuttering treatment: clients’ experiences and perceptions. J. Fluen. Disord. 52, 1–12. doi:10.1016/j.jfludis.2017.01.003
Ling, Y., Nefs, H. T., Morina, N., Heynderickx, I., and Brinkman, W. P. (2014). A meta-analysis on the relationship between self-reported presence and anxiety in virtual reality exposure therapy for anxiety disorders. PLoS ONE 9 (5), e96144. doi:10.1371/journal.pone.0096144
Lombard, M., and Ditton, T. (1997). At the heart of it all: the concept of presence. J. Computer-Mediated Commun. 3 (2), 0. doi:10.1111/j.1083-6101.1997.tb00072.x
Luborsky, L. (1976). “Helping alliances in psychotherapy,” in Successful psychotherapy. Editor J. L. Cleghhorn (New York, NY: Brunner/Mazel), 92–116.
Mattes, A., and Roheger, M. (2020). Nothing wrong about change: the adequate choice of the dependent variable and design in prediction of cognitive training success. BMC Med. Res. Methodol. 20 (1), 296. doi:10.1186/s12874-020-01176-8
Mattick, R. P., and Clarke, J. C. (1998). Development and validation of measures of social phobia scrutiny fear and social interaction anxiety11Editor’s note: this article was written before the development of some contemporary measures of social phobia, such as the Social Phobia and Anxiety Inventory (Turner et al., 1989). We have invited this article for publication because of the growing interest in the scales described therein. S.T. Behav. Res. Ther. 36 (4), 455–470. doi:10.1016/S0005-7967(97)10031-6
Miloff, A., Carlbring, P., Hamilton, W., Andersson, G., Reuterskiöld, L., and Lindner, P. (2020). Measuring alliance toward embodied virtual therapists in the era of automated treatments with the virtual therapist alliance scale (VTAS): development and psychometric evaluation. J. Med. Internet Res. 22 (3), e16660. doi:10.2196/16660
Miragall, M., Baños, R. M., Cebolla, A., and Botella, C. (2015). Working alliance inventory applied to virtual and augmented reality (WAI-VAR): psychometrics and therapeutic outcomes. Front. Psychol. 6, 1531. doi:10.3389/fpsyg.2015.01531
Montgomery, P., Grant, S., Mayo-Wilson, E., Macdonald, G., Michie, S., Hopewell, S., et al. (2018). Reporting randomised trials of social and psychological interventions: the CONSORT-SPI 2018 extension. Trials 19 (1), 407. doi:10.1186/s13063-018-2733-1
Morina, N., Brinkman, W. P., Hartanto, D., and Emmelkamp, P. M. G. (2014). Sense of presence and anxiety during virtual social interactions between a human and virtual humans. PeerJ 2, e337. doi:10.7717/peerj.337
Mörtberg, E. (2014). Working alliance in individual and group cognitive therapy for social anxiety disorder. Psychiatry Res. 220 (1), 716–718. doi:10.1016/j.psychres.2014.07.004
Ngai, I., Tully, E. C., and Anderson, P. L. (2015). The course of the working alliance during virtual reality and exposure group therapy for social anxiety disorder. Behav. Cognitive Psychotherapy 43 (2), 167–181. doi:10.1017/S135246581300088X
Peperkorn, H. M., Diemer, J., and Mühlberger, A. (2015). Temporal dynamics in the relation between presence and fear in virtual reality. Comput. Hum. Behav. 48, 542–547. doi:10.1016/j.chb.2015.02.028
Pfaller, M., Kroczek, L. O. H., Lange, B., Fülöp, R., Müller, M., and Mühlberger, A. (2021). Social presence as a moderator of the effect of agent behavior on emotional experience in social interactions in virtual reality. Front. Virtual Real. 2. doi:10.3389/frvir.2021.741138
Price, M., Mehta, N., Tone, E. B., and Anderson, P. L. (2011). Does engagement with exposure yield better outcomes? Components of presence as a predictor of treatment response for virtual reality exposure therapy for social phobia. J. Anxiety Disord. 25 (6), 763–770. doi:10.1016/j.janxdis.2011.03.004
Provoost, S., Lau, H. M., Ruwaard, J., and Riper, H. (2017). Embodied conversational agents in clinical psychology: A scoping review. J. Med. Internet Res. 19 (5), e151. doi:10.2196/jmir.6553
Rapee, R. M., and Heimberg, R. G. (1997). A cognitive-behavioral model of anxiety in social phobia. Behav. Res. Ther. 35 (8), 741–756. doi:10.1016/s0005-7967(97)00022-3
Reeves, R., Elliott, A., Curran, D., Dyer, K., and Hanna, D. (2021). 360° video virtual reality exposure therapy for public speaking anxiety: A randomized controlled trial. J. Anxiety Disord. 83, 102451. doi:10.1016/j.janxdis.2021.102451
Robillard, G., Bouchard, S., Dumoulin, S., Guitard, T., and Klinger, É. (2010). Using virtual humans to alleviate social anxiety: preliminary report from a comparative outcome study. Stud. Health Technol. Inf. 154, 57–60. doi:10.3233/978-1-60750-561-7-57
Sauer-Zavala, S., Boswell, J. F., Bentley, K. H., Thompson-Hollands, J., Farchione, T. J., and Barlow, D. H. (2018). Expectancies, working alliance, and outcome in transdiagnostic and single diagnosis treatment for anxiety disorders: an investigation of mediation. Cognitive Ther. Res. 42 (2), 135–145. doi:10.1007/s10608-017-9855-8
Scheurich, J. A., Beidel, D. C., and Vanryckeghem, M. (2019). Exposure therapy for social anxiety disorder in people who stutter: an exploratory multiple baseline design. J. Fluen. Disord. 59, 21–32. doi:10.1016/j.jfludis.2018.12.001
Schubert, T., Friedmann, F., and Regenbrecht, H. (2001). The experience of presence: factor analytic insights. Presence Teleoperators Virtual Environ. 10 (3), 266–281. doi:10.1162/105474601300343603
Schwind, V., Knierim, P., Haas, N., and Henze, N. (2019). “Using presence questionnaires in virtual reality,” in Proceedings of the 2019 CHI conference on human factors in computing systems (Glasgow, UK: ACM).
Selya, A. S., Rose, J. S., Dierker, L. C., Hedeker, D., and Mermelstein, R. J. (2012). A practical guide to calculating Cohen’s f2, a measure of local effect Size, from PROC MIXED. Front. Psychol. 3, 111. doi:10.3389/fpsyg.2012.00111
Slater, M. (2018). Immersion and the illusion of presence in virtual reality. Br. J. Psychol. 109 (3), 431–433. doi:10.1111/bjop.12305
Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Trans. R. Soc. B Biol. Sci. 364 (1535), 3549–3557. doi:10.1098/rstb.2009.0138
Sønsterud, H., Kirmess, M., Howells, K., Ward, D., Feragen, K. B., and Halvorsen, M. S. (2019). The working alliance in stuttering treatment: A neglected variable? Int. J. Lang. Commun. Disord. 54 (4), 606–619. doi:10.1111/1460-6984.12465
Stein, M. B., and Stein, D. J. (2008). Social anxiety disorder. Lancet 371 (9618), 1115–1125. doi:10.1016/S0140-6736(08)60488-2
Strojny, P. M., Dużmańska-Misiarczyk, N., Lipp, N., and Strojny, A. (2020). Moderators of social facilitation effect in virtual reality: co-presence and realism of virtual agents. Front. Psychol. 11, 1252. doi:10.3389/fpsyg.2020.01252
Thompson, T., Kaminska, M., Marshall, C., and Van Zalk, N. (2019). Evaluation of the social phobia scale and social interaction anxiety scale as assessments of performance and interaction anxiety. Psychiatry Res. 273, 725–731. doi:10.1016/j.psychres.2019.01.103
Vîslă, A., Constantino, M. J., Newkirk, K., Ogrodniczuk, J. S., and Söchting, I. (2018). The relation between outcome expectation, therapeutic alliance, and outcome among depressed patients in group cognitive-behavioral therapy. Psychotherapy Res. 28 (3), 446–456. doi:10.1080/10503307.2016.1218089
Walkom, G. (2016). “Virtual reality exposure therapy: to benefit those who stutter and treat social anxiety,” in 2016 International Conference on Interactive Technologies and Games (ITAG), Nottingham, UK, 26-27 October 2016 (IEEE), 36–41. doi:10.1109/iTAG.2016.13
Wallach, H. S., Safir, M. P., and Bar-Zvi, M. (2009). Virtual reality cognitive behavior therapy for public speaking anxiety: A randomized clinical trial. Behav. Modif. 33 (3), 314–338. doi:10.1177/0145445509331926
Weeks, J., Heimberg, R. G., Fresco, D. M., Hart, T. A., Turk, C. L., Schneier, F. R., et al. (2005). Empirical validation and psychometric evaluation of the Brief Fear of Negative Evaluation Scale in patients with social anxiety disorder. Psychol. Assess. 17 (2), 179–190. doi:10.1037/1040-3590.17.2.179
Wiederhold, B. K., and Wiederhold, M. D. (2005). “The effect of presence on virtual reality treatment,” in Virtual reality therapy for anxiety disorders: Advances in evaluation and treatment (Washington, DC, US: American Psychological Association), 77–86.
Woody, S. R., and Adessky, R. S. (2002). Therapeutic alliance, group cohesion, and homework compliance during cognitive-behavioral group treatment of social phobia. Behav. Ther. 33 (1), 5–27. doi:10.1016/S0005-7894(02)80003-X
World Health Organization (2022). World mental health report: Transforming mental health for all. Geneva, Switzerland: World Health Organization.
Keywords: virtual reality, exposure therapy, social anxiety, stuttering, outcome expectancy, therapeutic alliance, presence
Citation: Chard I, Van Zalk N and Picinali L (2023) Virtual reality exposure therapy for reducing social anxiety associated with stuttering: the role of outcome expectancy, therapeutic alliance, presence and social presence. Front. Virtual Real. 4:1159549. doi: 10.3389/frvir.2023.1159549
Received: 06 February 2023; Accepted: 28 August 2023;
Published: 06 September 2023.
Edited by:
Laura E. Watkins, Emory University, United StatesReviewed by:
Vangelis Lympouridis, University of Southern California, United StatesJames Lewis, Nottingham Trent University, United Kingdom
Copyright © 2023 Chard, Van Zalk and Picinali. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Ian Chard, i.chard18@imperial.ac.uk