A randomized controlled trial to pilot the efficacy of a computer-based intervention with elements of virtual reality and limited therapist assistance for the treatment of post-traumatic stress disorder

Although well-established therapies exist for post-traumatic stress disorder (PTSD), barriers to seek mental health care are high. Technology-based interventions may play a role in improving the reach of efforts to treat, especially when therapist availability is low. The goal of the current randomized controlled trial was to pilot the efficacy of a computer-based trauma intervention with elements of virtual reality (VR; 3MR system) and limited therapist involvement for the treatment of PTSD in a childhood sexual abuse (CSA) and war veteran sample and to compare this to “treatment as usual” (TAU). TAU consisted of evidence-based approaches such as imaginal exposure, EMDR, or narrative exposure therapy. A total of 44 patients with PTSD were included and randomly assigned to 12 sessions of 3MR intervention or TAU (completer n 3MR = 12, TAU = 18). Several measures (PCL-5, BDI-II, OQ-45-2, and the M.I.N.I. 5.0.0.) were administered to measure symptoms of PTSD and depression and scores of overall well-being at pre, post, and a three-month follow-up measurement. Analyses suggest that symptoms of PTSD and depression in the 3MR condition decreased, and overall well-being increased between pre and post measurements. Results did not indicate any clear differences between the treatment conditions over time which suggests that treatment gains of the 3MR intervention seem no less than those of TAU. Finally, both treatment conditions produced similar remission rates of PTSD and depression. Therefore, the 3MR intervention could possibly constitute an appropriate treatment alternative. The small sample size as well as evident drop-out rates in the 3MR condition (45%) do warrant further research. The procedures of this study were approved by the Medical Ethical Research Committee (MERC) of the Erasmus Medical Center in Rotterdam (MEC-NL46279.078.13) and pre-registered via ClinicalTrials.gov (Protocol Record CI1-12-S028-1).

Key practitioner message

• The Multi Modal Memory Restructuring (3MR) system is a computer-based trauma intervention with elements of virtual reality (VR) and limited therapist involvement for PTSD.

• We conducted a randomized controlled trial to pilot the efficacy of the 3MR intervention, compared to “treatment as usual” (TAU).

• Patients were survivors of childhood sexual abuse and war veterans.

• Findings indicate that the 3MR intervention decreased PTSD and depression symptoms between pre, post and follow-up measurements.

• There were no clear differences found between the two treatment conditions which suggests that the treatment gains of the 3MR intervention seem no less than those of TAU. However, the small sample size and the evident drop-out rate warrant further research.

Introduction

Although many people experience traumatic events during their life, only a small minority (approximately 5.6% – 7.8%) develops post-traumatic stress disorder (PTSD) afterwards (1–3). Estimated prevalence rates vary across high and low income countries, as well as between trauma types (2, 4, 5). Physical violence is associated with significant population burden of PTSD (i.e., the associated number of years of PTSD per trauma type on a population level) and predicts subsequent physical violence and intimate partner sexual violence (4). Furthermore, war-related traumatic events are associated with a more chronic course of PTSD symptom recovery relative to exposure to other traumatic events, showing the longest median duration (5 years) of PTSD symptoms (4). In general, the majority of individuals with PTSD fail to recover even after many years and mean symptom duration is significantly longer than previously thought (6, 4). The DSM-5 (Diagnostic and Statistical Manual of Mental Disorders 5th Edition) clusters PTSD symptoms in the categories re-experiencing, avoidance, negative alterations in mood and cognition, and alterations in arousal and reactivity (7). People with PTSD for example re-experience the traumatic event unwantedly by means of memory flashbacks or nightmares, avoid stimuli associated with the traumatic event and experience numbing of general responsiveness and symptoms of increased arousal. PTSD symptoms often chronically impair peoples’ daily life, but nevertheless barriers to seek mental health care seem high. Researchers found in a military sample (n = 6,000) that only 23%–40% of respondents who met criteria for a mental disorder sought professional help (8). Similar, it was reported that only 18.9% of rape victims in a National Women’s Study (n = 4,006) sought formal or informal help for their PTSD or major depressive symptoms (9). Reasons for peoples’ reluctance in seeking help might relate to fear of stigmatization, fear to be seen as weak or to be treated differently (8, 10). In addition, there exist different structural barriers to health care (11). An example is that of waiting lists. Waiting times of several months are commonplace and have resulted in numerous reports warranting change (e.g., 12, 13). Although widely examined and approved therapies for PTSD, such as Eye Movement Desensitization Reprocessing (EMDR) and (trauma-focused) cognitive behavioral therapy [(CBT), 14–16] exist, these current standards of at least 8 to 12 weekly sessions with a therapist lasting for 60 to 90 min can be seen as intensive and therefore costly, in particular in low and middle income countries. In a world that is changing, and where factors as confined access to mental health care due to long travel distances, financial restraints and a limited number of available therapists is current, there remains an ongoing need for interventions that aim to improve the effectiveness, cost-effectiveness, and above all accessibility to treatment for individuals with PTSD.

The current study for the first time describes the efficacy of a novel computer-based trauma intervention with elements of virtual reality (VR) and limited therapist involvement for the treatment of PTSD, offered via the 3MR system. The 3MR system (17) comprises a software application that focuses on the restructuring and relearning of past events. The 3MR system allows patients to visualize past events using personal photos, narrative texts, online geographical maps and patient-created 3D virtual worlds that can be viewed on a computer screen. The 3MR intervention hinges on two fields of research, namely of VR containing patient created 3D virtual worlds and of computer-based interventions enabling patients to follow through therapy sessions in their own environment and without the direct presence of a therapist.

Virtual reality exposure therapy (VRET)

Exposure to the traumatic memories or cues for the traumatic event often plays an important role in reducing symptoms of PTSD (18). Exposure in vivo might often be difficult to establish, mainly due to practical reasons such as high costs (e.g., travelling to other countries). Also, it might be dangerous to return to the original surroundings of the traumatic event (e. g., in case of a warzone). Using VRET to extinct fearful stimuli can overcome these issues since it uses computer-generated environments to simulate feared stimuli (e.g., 19). In these virtual environments, users can be systematically exposed to specific stimuli within a contextually relevant setting, for example a warzone or airplane for soldiers with war-related PTSD (20, 21). Many forms of VR exist, and can be defined as “the use of computer and behavioral interfaces to simulate the behavior of 3D entities that interact in real-time with each other and with a user immersed via sensorimotor channels” (22). Experts (23) mention a wide range of applications that fit this definition, from 3D video games to large immersive VR rooms (the so-called CAVEs) and describe that over the last few years, large expensive VR installations have made room for Head Mounted Displays (HMDs) and other more affordable technologies used in mental health care. Recent studies also describe the development of VR over the last decades and the increase of research interest on the use of VR to improve mental health (24, 25). This has led to applications for the treatment of anxiety disorders (e.g. 19), resulting in several studies describing positive treatment outcomes for several disorders (26), including PTSD (e.g., 27–30, 20). In a systematic review, researchers included ten studies that used VRET to conduct exposure in CBT treatment for PTSD and found that VRET is a potentially promising treatment option (31). The researchers also highlight several limitations to this research field (e.g., only few studies available in the literature, no standardized number of therapy sessions and the non-use of intent-to-treat analysis). Similar, in a recent meta-analysis researchers compared ten clinical trials on the efficacy of VRET for the treatment of PTSD. They found that VRET for PTSD significantly outperformed inactive control conditions and did not differ from active control conditions (32).

Computer-based interventions

Considering the need for accessible treatments, computer-based interventions are known for their ability to reach large groups of people. This type of intervention has several benefits compared to traditional therapy; they are often personalized and tailored to the needs of a diverse group of users, they can reach a large population at relatively low cost and they can be used from a person’s own home (33). Generally, computerized interventions yield comparable effect sizes as traditional psychosocial interventions in the treatment of depression and anxiety (27, 33–36). Differences exist into what extent assistance is offered during these interventions (e.g., no, administrative or therapist assistance; 37). Drop-out rates in computer-based interventions are considered a cause of concern (38). Level of therapist involvement however does seem to influence drop-out (37). Altogether, interventions that require limited therapist involvement (with costs of therapists arguably accounting for the largest proportion when treating patients) and in addition lower expenses such as travel costs, can potentially improve cost-effectiveness and access to mental health care.

The goal of the current randomized controlled trial was to (1) pilot the efficacy of a computer-based trauma intervention with elements of VR (3MR system) and limited therapist assistance for the treatment of PTSD in a CSA and war veteran sample; and (2) to compare this to “treatment as usual” (TAU). We therefore tested whether the 3MR intervention decreased symptoms of PTSD, depression and increased overall well-being at post and follow-up measurements, and if the outcome of the 3MR intervention was comparable to TAU. It was expected that 3MR would perform similarly to TAU.

Method

Procedure

Study enrollment

Patients enrolled for treatment via the specialized mental health care centers of PsyQ (locations Rotterdam-Kralingen, Spijkenisse, and The Hague), Reinier van Arkel (Psychotrauma Centrum Zuid-Nederland), and the ambulatory of the Erasmus University Rotterdam (Department of Clinical Psychology). Potential participants that presumably met in- and exclusion criteria were given a detailed information sheet about the project. There were at least five days between the first and second consultation so that potential participants had the chance to (re-)consider whether they truly wanted to participate in this study. If individuals were interested in participation, they filled in an informed consent¹.

In- and exclusion criteria

Following consent, the in- and exclusion criteria were checked in an extensive interview by telephone that was administered by trained psychologists and psychology master students. Assessment of the in- and exclusion criteria included a semi-structured clinical interview [Mini International Neuropsychiatric Interview Plus - Dutch Version 5.0.0 [M.I.N.I. Plus 5.0.0], (39)] and a self-report questionnaire [Dissociative Experiences Scale [DES], (40)]. Participants were excluded if they met criteria for a current bipolar disorder, current psychotic episode, if they were actively suicidal (defined as “high risk” according to the Mini International Neuropsychiatric Interview Plus - Dutch Version 5.0.0 [M.I.N.I. Plus 5.0.0., (39, 41)], or scored a total score of ≥40 on the Dissociative Experiences Scale [DES, (40)]. Co-morbidity as such was not an exclusion criterion, but PTSD had to be the primary diagnosis according to the M.I.N.I. Plus 5.0.0. (Initially, it was agreed to include patients that met criteria for the diagnosis of PTSD and/or Depression [according to the M.I.N.I. 5.0.0.] as primary diagnosis. However, since recruitment took place at the trauma departments of the participating mental health care centers, only one patient did not meet criteria for the diagnosis of PTSD [only for Depression] at initial screening and inclusion. Therefore, the main focus of the article is on PTSD [e.g., background, introduction, discussion]). Use of medication was no exclusion criterion, provided that the dose was stable for at least two weeks at the beginning of the therapy, remained stable throughout therapy, and was closely monitored.

Measurements

To determine PTSD and depression symptom levels, overall well-being and to check the criteria for meeting the diagnosis of PTSD and/or depression at pre-, post- and 3-month follow-up measurements, primary and secondary outcome measures (PTSD checklist for the DSM 5 with LEC and extended Criterion A* [PCL-5, (42)]; Beck Depression Inventory – Second Edition [BDI-II, (43)]; Outcome Questionnaire – 45 – Second Edition [OQ-45-2, (44)]) were assessed via online self-report questionnaires, as well as the M.I.N.I. Plus 5.0.0. via telephone. (The M.I.N.I. Plus 5.0.0. and the PCL-5 are based on the criteria for PTSD set by the DSM IV and DSM-5 respectively, and thus differ. However, in light of the emergence of the Dutch PCL-5 questionnaire during the onset of this study (2013), this was the soundest option to reflect the diagnostic changes in the field of PTSD at that time, with consideration of existent measures. For an overview of differences between DSM IV and DSM-5 we refer to Friedman (45)]. Assessors were independent but not blinded to treatment condition. At 12-months post treatment, another follow-up measurement was conducted, this assessment is ongoing and is not further described in the present study. Participants were asked to preferably not seek other forms of therapy between the post and the 3-month follow-up measurement unless indicated by the therapist. In the control condition, this restriction was not given due to practical reasons.

Randomization

This study had a randomized controlled design. The control condition was TAU. Randomization was conducted after inclusion. Eligible patients were randomly assigned to two conditions by an independent researcher via a random-numbers table and its allocation sequence was computer-generated. A stratified randomization procedure was employed for clients with CSA or war related PTSD. Patients were assigned to groups by the first author (MM) (i.e., disclosure of assigned therapy condition by opening randomization envelopes set up by the independent researcher), following the assessment of inclusion and exclusion criteria and randomization. Participants were not compensated for their contribution to the study and were free to leave the study at any time and (in the 3MR condition) receive TAU instead.

Participants and flow

The research population consisted of both war veterans and survivors of CSA. Selection of this trauma population had a practical background since these trauma types were most prevalent in the participating mental health care centers. Survivors of CSA could have either single or multiple/recurrent traumatic experiences that had occurred between 0 and 18 years old. Recruited veterans presumably served (a) Dutch military mission(s) in Lebanon, Bosnia-Herzegovina, Iraq, or Afghanistan. Following pre-screening, a total of 83 patients were referred via the participating mental health care centers. Of the 83 individuals that were contacted by the researchers, 48 registered interest in the study. Of these 48 individuals, three did not meet in- and exclusion criteria and one could not join the study due to practical reasons (not able to follow therapy sessions at home). See Figure 1 for a flow of the participants through the trial.

FIGURE 1

Figure 1. Flow diagram of the study.

Participants were aged between 20 and 62 years, with an average age of 39 years. Forty-eight percent was female, 47.7% had previously followed therapy, and 43.2% took prescription drugs during the study period. See Appendix 1 Table 1 for participant descriptives specified per treatment condition (3MR and TAU). Appendix 2 Table 2 shows that independent samples t-tests revealed no significant baseline differences between conditions. A total of 14 patients dropped out of the study. See also Figure 1 and section 4.3 Drop-out analysis. Thirteen individuals could be classified as therapy drop-out, whereas one individual dropped out because of practical reasons. Originally, it was planned to recruit a total of 144 participants to meet sufficient power levels. Patient recruitment turned out to be more difficult than anticipated. After many attempts to facilitate the recruitment we had to stop the inclusion of patients after two years because of practical and financial reasons.

TABLE 1

Table 1. Descriptive statistics of patients in 3MR and TAU conditions.

TABLE 2

Table 2. Independent samples t-test baseline (pre) differences between 3MR and TAU condition.

Materials

Semi-structured clinical interview

To assess primary diagnosis of PTSD and/or depression, the Dutch version of the M.I.N.I. Plus 5.0.0 was administered (M.I.N.I. Plus 5.0.0. (39), M.I.N.I. Plus 5.0.0 NL [i.e., the Dutch version for use in the Netherlands (41)]. The M.I.N.I. Plus 5.0.0. is a structured clinical interview used to assess Axis I Disorders according to DSM IV. All questions are yes/no questions and based on the answers, it is determined whether the patient meets the criteria for a certain disorder (e.g., “In the past month, have you avoided thinking about the event, or have you avoided things that remind you of the event?”). The M.I.N.I. 5.0.0. has excellent interrater reliability (κ > 0.75), very good test-retest reliability (κ > 0.75), as well as validity (39).

Self-report questionnaires

 The Dissociative Experiences Scale [DES (40)] was used to measure dissociative experiences that the participant may be suffering from due to PTSD. The DES consists of 28 self-report items rated on a scale of 0 to 100. Subjects indicate to what extent they experience certain symptoms such as amnesia. An example item is “Some people have the experience of finding themselves in a place and having no idea how they got there. Select a number to show what percentage of the time this happens to you.” The DES has good test-retest reliability (r = 0.84) and split-half reliability (r = 0.71–0.96), as well as good internal consistency and construct validity (ρ = 0.64) (40).

Self-reported symptoms of PTSD were assessed using the PTSD checklist for the DSM 5 with LEC and extended Criterion A [PCL-5 (42, 46)]. The PCL-5 is a 20-item self-report measure assessing the symptoms of PTSD according to the DSM 5. An example item is “In the past month, how much were you bothered by repeated, disturbing, and unwanted memories of the stressful experience?” on a scale of 0 (not at all) to 4 (extremely). The PCL-5 was found to have strong internal consistency (α = 0.94), test-retest reliability (r = 0.82), and convergent (rs = 0.74 to 0.85) and discriminant (rs = 0.31 to 0.60) validity (Blevins et al., 2015).

The Beck Depression Inventory – Second Edition (BDI-II (43); BDI-II-NL [Dutch version (47)] is a 21-item self-report instrument that assesses severity of depression. Each item has four options ranging from 0 (“not present”) to 3 (“present all the time”) within the previous two weeks. Higher scores indicate a more severe depression. An example item is “I don’t feel I am worse than anybody else - I am critical of myself for my weaknesses or mistakes - I blame myself all the time for my faults - I blame myself for everything bad that happens”. The BDI-II has shown good internal consistency reliability [α = 0.93 amongst college students, α = 0.92 amongst outpatients, (43)].

Overall well-being was assessed using the Outcome Questionnaire – 45 – Second Edition [OQ-45–2, (44)]. It measures symptom distress (SD), interpersonal relationships (IR) and difficulties in social roles (SR). The participant indicates how often the statements applied to them in the past week on a scale of 0 (“never”) to 4 (“almost all the time”). An example item is “I have difficulty concentrating”. It has been shown to have good test-retest reliability (r = 0.79), good internal consistency reliability (r = 0.92) as well as good concurrent validity (α = 0.68–0.80) in Dutch clinical populations (48).

Apparatus

The Multi Modal Memory Restructuring (3MR) system is a software application, which focuses on the restructuring and relearning of past events and can be operated by the patient without or with minimal therapist assistance. The 3MR system allows people to visualize past events using personal photos, narrative texts, online geographical maps, and patient-created 3D virtual worlds. Patients view the 3MR application on their computer screen. In this trial there was no use of Head Mounted Displays. The 3MR system consists of a period overview and a diary, which contains the tools “Text”, “Images”, “Website”, “Media”, “Webcam” and “3D world”. Patients install the 3MR system on their own PC or laptop. The 3MR system runs at any PC or laptop that has a 32 or 64 bit version of Windows XP, Windows Vista or Windows 7, Windows Internet Explorer 6 or higher, a processor with a clock speed of at least 1 GHz, 512 MB of intern memory (RAM), 2 GB of free hard disk space and a graphics card which supports OpenGL. The therapy manual that contained 12 therapy sessions was handed-out in hardcopy. Illustrative screenshots of the application are online available at https://doi.org/10.4121/uuid:cacc5b31-047e-4e88-b341-cff18d76de49.

Treatment

3MR Intervention

Patients randomly assigned to the 3MR group received a one-hour face-to-face introduction appointment with a trained therapist to be introduced to the 3MR system and treatment manual. The treatment rationale was explained thoroughly and especially the importance of non-avoiding was discussed and highlighted. Treatment sessions were scheduled two times per week. After this introduction appointment, patients autonomously followed through the 12 sessions of 3MR intervention at home with use of the treatment manual. At session six, the participant was called by the therapist to check progress and to see whether there were any questions or problems. The 3MR system and an accompanying therapy manual were used to guide patients through 12 therapy sessions targeting the traumatic memories of the patients. During sessions, patients completed assignments as described in the therapy manual (abbreviated version shown here) varying from “Search for the location of this event on Google Maps, use Google Street view and answer the following questions; “Where is this?”, “What happened here?”, “What feeling does looking back at this location give you?”, to “Do you have pictures you have on your computer (add these via the tool “Images”) or in hard copy (add these via the tool “Webcam”) that date from that time period?” The upload of a picture was then followed by the questions; “Which people are on this picture?”, “Why is this picture important to illustrate your memory?”. The assignments build up to the construction of a personalized 3D environment, reflecting the actual traumatic memory of the participant with the use of corresponding 3D items. In the 3D world builder, several pre-selectable formats existed (e.g., a “desert” environment in the military version, and a “room” environment in the CSA version). Patients added items to these environments from a library, which contains hundreds of specific items such as vehicles, houses, soldiers and civilians in the military version, and coaches, tables, closets and beds in the CSA version. While looking at this personal created virtual 3D environment, patients answered classical exposure questions from the therapy manual, such as “What do you see looking at this situation?”, “What do you hear?”, “What do you smell?”. After use of the 3D world, patients were instructed to distance themselves from the memory by actively zooming out of the 3D environment before closing it.

During the 3MR intervention the therapist could be contacted by the patient by e-mail or telephone and he or she offered feedback or help when requested. In practice, extra requests for therapist assistance were not often done and therapist assistance remained mostly confined to the mid-term contact moment initiated by the therapist at session six.

Treatment as usual

The control condition consisted of 12 sessions of face to face “treatment as usual” (TAU), which in this trial was a trauma-focused intervention for PTSD (i.e., Imaginal Exposure [IE], Eye Movement Desensitization and Reprocessing [EMDR], or Narrative Exposure Therapy [NET]). These approaches are applied as TAU because they are known to lead to high effect-sizes [e.g., (14)] and are recommended in guidelines from different institutions (e.g., American Psychological Association [APA], and ISTSS (14)). A total of 22 certified psychologists working at the Psychotrauma departments of PsyQ and Reinier van Arkel and the ambulatory of the Erasmus University Rotterdam were involved in TAU and/or 3MR therapy assistance. The post measurement was conducted after 12 3MR or TAU therapy sessions. For ethical reasons ongoing TAU’s were not forced to stop after this point; whether or not patients had followed additional TAU during the period between the post measurement and the follow-up measurement was recorded at the three month follow-up registration.

Data analysis

To include patients with one or more missed measurements and take into account the nested data structure of multiple measurements within a participant, a multilevel model (also known as mixed random effect or hierarchical model) was used to analyze the data. Hereby, pre, post, and 3-month follow-up measurements of the PTSD and depression symptoms are defined at Level 1 and patients at Level 2, with condition (3MR or TAU) as a Level 2 predictor variable [see e.g., (49); for a general overview of multilevel analysis procedures]. Hereby, a Bayesian approach was used to deal with estimation problems often encountered with small sample sizes at the higher level (49). Here, we deviated from our preregistered analysis plan (clinicaltrials.gov/ct2/show/NCT02234076) to accommodate the sample size which was lower than expected. Bayesian estimation takes both background information, the prior distribution, and the information in the data, expressed in terms of the likelihood function of the data given the parameters, into account (49). With both distributions and the use of Bayes theorem to update the prior distribution with the information in the data (50), the posterior distribution is defined for each parameter of interest. In doing so, the so-called Markov chain Monte Carlo (MCMC) method is used to estimate parameter values. The interested reader is referred to an extensive body of literature on Bayesian estimation (see e.g. Van de Schoot et al., 2014) and Bayesian multilevel modeling [see e.g. (49), (51–52)]. All models were created in Stan computational framework (http://mc-stan.org), accessed with the brms package (53) in R (R Core Team, 2020). SPSS 25 was used to generate data files from raw data files and analyze descriptive information on participants baseline differences and drop-out. Multilevel analyses were performed blinded for group allocation.

First, to assess the effects of the 3MR condition over time, a model with random intercepts, an uninformative prior to model these intercepts, and a main effect of time was fitted to the 3MR group for each outcome. Taking a Bayesian perspective, the outcome, e.g. PCL-5 score, is assumed to be drawn from a normal probability distribution with an unknown mean and variation parameter that is considered a random variable and described by a probability distribution (Nalborczyk et al., 2019).

$PCLscor{e}_{ti}\sim \mathrm{Normal}\left({\alpha }_i+\beta tim{e}_t,{\sigma }_e\right)$

${\alpha }_i\sim \mathrm{Normal}\left(\mu ,{\sigma }_{\alpha }\right)$

class="mb15">Here, the mean PCL-5 score per patient, i, at a specific timepoint, t, is determined by a person-specific intercept and the regression coefficient for time, β. The person-specific intercepts, ${\alpha }_i$, are assumed to have a normal distribution with the overall mean PCL-5 score and a variance component ${\sigma }_{\alpha }$ (i.e. the prior distribution). For the PCL-5 scores the uninformative prior distribution had a mean of 38 and standard deviation of 25 and for the BDI-II scores a mean of 20 and standard deviation of 25, based on values reported (54). The prior distribution for the OQ-45-2 scores had a mean of 80 and a standard deviation of 50, following previous findings (48).

Subsequently, a multilevel model was fitted to each outcome including random intercepts, random slopes for the effect of time, a regression coefficient for time, for condition, and one for a cross-level interaction effect between time and condition. This means that the decrease or increase in a patient’s PCL score over time was allowed to vary across patients. As such, an additional prior distribution was specified for the person-specific slopes which were centered around the grand (i.e. mean) slope of time $\beta$. Given the novelty of 3MR, there is no prior information on its effectiveness and its relation to TAU and an uninformative prior was specified with a mean of zero and standard deviation of 100.

$\begin{array}{r}PCLscor{e}_{ti}\sim \\ \mathrm{Normal}({\alpha }_i+{\beta }_{i}tim{e}_t+\beta conditio{n}_i+\beta tim{e}_{t}conditio{n}_i,{\sigma }_e)\end{array}$

${\alpha }_i\sim \mathrm{Normal}\left(\mu ,{\sigma }_{\alpha }\right)$

${\beta }_i\sim \mathrm{Normal}\left(\beta ,{\sigma }_{\beta }\right)$

class="mb15">For the final estimates to be trustworthy, model convergence has to be obtained for each individual parameter (49). Convergence was assessed by evaluating caterpillar charts of the samples, plotting the posterior distribution, and by assessing the sensitivity of the results with respects to varying the number of chains, iterations, and burn-in phase. For those parameters not discussed here, the default prior was used to reach convergence of the models. The default priors in the brms package are chosen to be non or very weakly informative so they do not influence the results much [see (53)].

Once fitted, the parameter estimates were reported with their 95% credibility interval. This interval can be interpreted as having a 95% probability that the population value is within the limits of the interval (50). Furthermore, for the condition coefficient, histograms of the posterior samples of the slopes for condition are plotted marking the highest density interval (HDI). The HDI is a type of credibility interval that indicates the points within the interval which have a higher probability density than points outside the interval and is allowed to have unequal tails (55).

Furthermore, the dichotomous data of the M.I.N.I. Plus 5.0.0. NL were studied using descriptive statistics (diagnosis PTSD and/or current depression YES/NO) on pre, post, and follow-up measurements.

 Considering missing data due to practical hurdles in data collection; of three dropped out patients only pre diagnostic data according to the M.I.N.I. Plus 5.0.0. NL were available (which means no questionnaire data at all). In the completer group, of one participant no pre questionnaire data were available, but post questionnaire data and pre, post and follow-up M.I.N.I. Plus 5.0.0. NL data were available. Also, of one participant no post and follow-up questionnaire data were available, but pre questionnaire data and pre and post M.I.N.I. Plus 5.0.0. NL data were available. All questionnaire data were taken into account with use of the multilevel analysis. All available M.I.N.I. Plus 5.0.0. data were used in the descriptive analysis. In all other analyses the available n is noted when relevant. See also Figure 1 for a flow chart of participants through the study.

Results

Descriptive statistics

Table 3 shows the descriptive statistics of the symptom scales (PCL-5, BDI-II and OQ-45-2) in both treatment conditions (3MR and TAU). Figures 2–4 illustrate the individual patient trajectories over time for the three symptom scales and show there is individual variability in the symptom changes over time in both treatment conditions.

TABLE 3

Table 3. Descriptives per scale per measurement moment and condition.

FIGURE 2

Figure 2. Individual patient trajectories over time for the PCL-5 in both treatment conditions, with the dark line representing the mean scores.

FIGURE 3

Figure 3. Individual patient trajectories over time for the BDI-II in both treatment conditions, with the dark line representing the mean scores.

FIGURE 4

Figure 4. Individual patient trajectories over time for the OQ-45-2 in both treatment conditions, with the dark line representing the mean scores.

M.I.N.I. Plus 5.0.0.

Descriptive statistics of the M.I.N.I. Plus 5.0.0. data show that at post measurement 81.1% of the completers in the 3MR condition (n = 12) no longer met criteria for the diagnosis of PTSD, and 72.2% of the completers in the TAU condition (n = 18). In the 3MR condition, 75% of the completers that met diagnostic criteria for depression at pre measurement no longer met criteria at post measurement. In the TAU condition, the rate was 50%. In both conditions one participant that did not meet diagnostic criteria at pre-measurement, did meet criteria at post-measurement. At the three month follow-up measurement, in the 3MR condition (n = 11) 18.2% of the completers met criteria for PTSD and/or depression. In the TAU condition (n = 16) this was 25% both for PTSD and depression. In the TAU condition, two patients relapsed into diagnosis of PTSD and one into diagnosis of depression. In the 3MR condition two patients relapsed into prior diagnosis of PTSD and one participant met criteria for depression only at the follow-up measurement.

Drop-out analysis

Drop-out analysis revealed that drop-outs (available data n = 11) tended to score somewhat but not significantly different than completers (available data n = 29) on pre measurements of the PCL-5 [dropout M = 49.91, SD = 13.12, completers M = 46.28, SD = 11.49, t(38) = −0.859, p = 0.396], BDI-II-NL [dropout M = 29.19, SD = 6.21, completers M = 26.52, SD = 9.51, t(38) = −0.859, p = 0.396] and OQ-45-2 [dropout M = 94.36, SD = 20.76, completers M = 82.48, SD = 20.68, t(38) = −1.621, p = 0.113]. When comparing treatment conditions, the 3MR condition suffered from a higher drop-out rate (45%) than the TAU condition (18%). This effect was noticeable but not significant χ² (1) = 3.771; p = 0.104). Furthermore, slightly more patients in the CSA group dropped out than in the military group (CSA = 37.5%, military = 25%).

Multilevel analysis

Results of the multilevel analyses are discussed per outcome measure.

PCL-5

A random-intercept model with a main effect of time was fitted to the data of the 3MR condition. The results are shown in Table 4. The regression coefficients and 95% credibility intervals show that PTSD symptoms measured with the PCL-5 decreased over time with an average decrease of 22.04 points between the pre and post measurement and 14.39 between pre and follow-up measurement. Both intervals do not include zero.

TABLE 4

Table 4. Posterior mean, standard error, 95% credible interval for each parameter in model predicting PCL-5 scores.

To compare the effects of the 3MR and TAU condition, a model with cross-level interactions was fitted, for which the parameter estimates are also shown in Table 4. Given the many parameters and few data, reaching convergence proved to be difficult as is evident by the wide 95% credibility intervals. Overall, the 95% credibility intervals of time did not include zero indicating that PTSD symptoms decreased over time on average. This decrease in symptoms becomes slightly more uncertain over time as depicted by the wider 95% credibility interval. Appendix 3 Figure 5 shows plots of the estimates of the posterior samples of the slope for the interaction effects with the highest density intervals (HDIs). These show that the regression estimates are quite uncertain as both zero as well as positive values are included in the intervals. Still, the mode of the distribution and 50% HDI seems to be either smaller than zero or centered around zero, indicating that the treatment gains of the 3MR intervention do at least not seem less when compared to TAU with regards to decreasing PTSD symptoms.

FIGURE 5

Figure 5. Histogram of mode of posterior distribution with 50% and 95% HDI for slopes of condition predicting PCL-5 scores. Note that a negative β coefficient indicates lower PTSD symptoms in the 3MR condition than in the TAU condition.

BDI-II

The parameter estimates for predicting BDI-II scores measuring depression symptoms are shown in Table 5. The model fitted to the 3MR data shows a decrease of depression symptoms over time. On average, BDI-II scores of participants in the 3MR condition decreased by 10.20 points [95%CrI (-15.02, −5.54)] between the pre and post measurement and −9.03 from pre to follow-up [95%CrI (-14.24, −4.21)], again excluding zero from both 95% credibility intervals.

TABLE 5

Table 5. Posterior mean, standard error, 95% credible interval for each parameter in model predicting BDI-II scores.

Further comparison of the 3MR and TAU conditions by means of adding a cross-level interaction, shows a decrease in depression symptoms over time. This decrease becomes slightly more uncertain as time between the measurements increases, but the 95% credibility interval still does not include zero. The posterior samples plotted in Appendix 4 Figure 6 for the slopes of the interaction effects show that the mode of the distributions is negative. This implies that in most samples the decrease in depression symptoms was slightly larger in the 3MR group compared to TAU. Also, the 50% HDI does not include zero for the interaction between condition and the pre to follow-up measurement. Although much uncertainty is still involved, this seems to suggest that the treatment gains of the 3MR intervention with regards to decreasing depression symptoms seem not less than TAU, and sometimes even lean towards a preference for 3MR.

FIGURE 6

Figure 6. Histogram of mode of posterior distribution with 50% and 95% HDI for slopes of condition predicting BDI-II scores. Note that a negative β coefficient indicates lower depression symptoms in the 3MR group than in the TAU condition.

OQ-45-2

Table 6 shows the parameter estimates for predicting overall well-being as measured with the OQ-45-2 scores. A random intercept model with a main effect of time was fitted to the data of the 3MR condition. As indicated by the average decrease of 19.90 points from the pre to post measurement and 15.65 from pre to follow-up, and the corresponding 95% credibility intervals not containing zero, the 3MR condition seems to lower OQ-45-2 scores over time, indicating an increase in overall well-being.

TABLE 6

Table 6. Posterior mean, standard error, 95% credible interval for each parameter in model predicting OQ-45-2 scores.

Comparing both conditions with a model including a cross-level interaction effect shows that overall OQ-45-2 scores decrease over time. With longer time periods however, the 95% credibility interval of the effect of time includes zero, suggesting less certainty that overall well-being changes over time. Similarly, the 95% credibility intervals of the interaction effects, which are on average negative, include zero. Appendix 5 Figure 7 shows plots of the estimates of the posterior samples of coefficients of the interaction effects. As shown, the mode of the distributions is negative and for the difference between the pre and follow-up measurement it is centered around zero. Again, wide HDIs show the uncertainty in the effect of time for 3MR compared to TAU, but treatment gains seem no less than in the TAU condition.

FIGURE 7

Figure 7. Histogram of mode of posterior distribution with 50% and 95% HDI for slopes of interaction effects predicting OQ-45-2 scores. Note that a negative β coefficient indicates lower OQ-45-2 scores in the 3MR group.

Discussion

This paper describes a randomized controlled trial to pilot the efficacy of a computer-based trauma intervention with elements of VR (namely patient created 3D virtual worlds viewed on a computer screen) and limited therapist assistance (3MR system) for the treatment of PTSD in a sample of 44 CSA and war veteran patients. The 3MR intervention was compared to “treatment as usual” (TAU), which consisted of evidence-based trauma focused therapy such as IE, EMDR, or NET. When exploring the effects of the 3MR intervention, results show that symptoms of both PTSD and depression seemed to decrease between pre and post measurements (after 12 sessions of respectively 3MR or TAU). Overall well-being increased over time. When compared to TAU, no clear differences seem to emerge between the two treatment conditions when considering symptoms of PTSD, depression and overall well-being over time. This possibly indicates that the treatment gains of the 3MR intervention are no less than those of the TAU. In addition, results of the semi-structured clinical interviews indicate that in both treatment conditions diagnoses of PTSD and depression declined from pre to post measurements. However, for example the wide credibility intervals in the results indicate that there are still large uncertainties in the effects, and that more data are needed to overcome these.

The results are in line with current literature on the efficacy of interventions for PTSD that use VR, and computer-based interventions. For example, researchers (28) describe several interventions in their meta-analysis on RCTs for the treatment of anxiety disorders (among which PTSD) and conclude that VRET is an effective and equal medium for exposure therapy. When considering the outcomes of computer-based interventions for the treatment of for example PTSD, our findings are in line with previous research which indicates that in general, computerized interventions yield comparable effect sizes as traditional psychosocial interventions in the treatment of depression and anxiety (33–36). The application of the 3MR intervention seems promising, since the intervention was followed through autonomously by patients at home and required only limited therapist assistance. Therefore, it might not only lower expenses through decreased therapists’ costs but could also reduce travel expenses and perceived barriers for PTSD patients in remote areas or for who fear stigmatization. However, this study has several limitations. A major limitation is that it is based on a small number of patients. To allow for more conclusive results on the effects of the 3MR intervention, additional data would be required. Furthermore, although our analysis did not show differences between drop-out rates in the 3MR condition (45%) and the control group (18%), this percentage is high. This was rather expected since computer-based interventions generally suffer from high drop-out rates, especially when therapist involvement is low (38, 37). We expect that this low therapist involvement is also the cause of our relatively high drop-out rates. Future studies could benefit from more specific interventions to improve dropout, such as more frequent (online) contact with a therapist. Yet, high drop-out rates have also been reported in trials applying standard forms of therapy. Researchers for example report a drop-out rate of 41% of patients with CSA randomized to receive CBT, which was significantly higher than the rate of patients in the present-centered therapy and waitlist conditions (56). We further experienced more serious struggles in patient recruitment than anticipated. The sample size of the study is still small, even though patients were recruited during two years. Reasons for experienced patient recruitment difficulties might be sought for in the patients’ and therapists’ hurdles in acceptance of e-Mental Health applications for instance (57, 32). Although patients using the 3MR intervention in this research project were generally positive in evaluating the 3MR intervention (32), further research needs to further examine this issue. Since only the 3MR condition was computer-based, participants were not blind to the treatment condition. Consequently, expectations about the potential efficacy of a computer-based intervention might have had an impact on treatment outcome. Although we did not measure such expectations, our observation suggested that the willingness of participants to participate in one of the two conditions was equally distributed.

Lastly, trials with larger sample sizes are needed and until realized, the results of this study should be interpreted with appropriate caution. Moreover, since individual differences in treatment gain are present in this trial, further insight into the question for which patients this intervention might be specifically beneficial is needed.

In conclusion, our data seem to indicate that the 3MR intervention may be effective in treating PTSD and its application might be especially relevant when therapist availability is low and the intention to enlarge reach of treatment efforts and improve cost-effectiveness are present. However, the small sample size and consequent large uncertainties in the estimations of the comparison between the effects of the 3MR intervention and TAU, and high drop-out rates warrant further research.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Medical Ethics Research Committee (MERC) of the Erasmus Medical Center in Rotterdam (MEC-NL46279.078.13). The patients/participants provided their written informed consent to participate in this study.

Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

MvM, IF, NM, CvdH, WB substantially contributed to the conception, design, and interpretation of data for the work; MvM, WB, IY substantially contributed to the analysis of data for the work; MvM draftied the work; IF, NM, CvdH, WB, MvM, IY, MT, JR, EvE, KvS, MB, revised it critically for important intellectual content; IF, NM, CvdH, WB, MvM, IY, MT, JR, EvE, KvS, MB provide approval for publication of the content. All authors contributed to the article and approved the submitted version.

Funding

This research is funded by NWO as part of the Virtual eCoaching and Storytelling technology for Post-traumatic stress disorder treatment (VESP) project (no. 314-99-104).

Acknowledgments

We like to thank Wendy Schalke, Gloria Gribling, and Jeffrey van der Starre for their efforts at the Psychotrauma departments of Reinier van Arkel Group and PsyQ Parnassia Group. Acknowledgements also to Tara Vis, Ivana van Berkel, Linsey van Berkum, Sarah Hatuluwaja, and Lara Kohlen for their efforts in screening and interviewing our participants. Thanks to Isabel Kampmann from the department of Clinical Psychology and Psychotherapy from the University of Münster for her help with the randomization procedures. This research is funded by NWO as part of the Virtual eCoaching and Storytelling technology for Post-traumatic stress disorder treatment (VESP) project (no. 314-99-104).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnote

¹The informed consent of this study is available from the corresponding author on reasonable request.

References

1. Koenen KC, Ratanatharathorn A, Ng L, McLaughlin KA, Bromet EJ, Stein DJ, et al. Posttraumatic stress disorder in the world mental health surveys. Psychol Med. (2017) 47:2260–74. doi: 10.1017/S0033291717000708

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Atwoli L, Stein DJ, Koenen KC, McLaughlin KA. Epidemiology of posttraumatic stress disorder: prevalence, correlates and consequences. Curr Opin Psychiatry. (2015) 28(4):307–11. doi: 10.1097/yco.0000000000000167

PubMed Abstract | CrossRef Full Text | Google Scholar

3. De Vries GJ, Olff M. The lifetime prevalence of traumatic events and posttraumatic stress disorder in the Netherlands. J Trauma Stress. (2009) 22(4):259–67. doi: 10.1002/jts.20429

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Kessler RC, Aguilar-Gaxiola S, Alonso J, Benjet C, Bromet EJ, Cardos G, et al. Trauma and PTSD in the WHO world mental health surveys. Eur J Psychotraumatol. (2017) 8(sup5):1353383. doi: 10.1080/20008198.2017.1353383

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Benjet C, Bromet E, Karam EG, Kessler RC, McLaughlin KA, Ruscio AM, et al. The epidemiology of traumatic event exposure worldwide: results from the world mental health survey consortium. Psychol Med. (2016) 46(2):327–43. doi: 10.1017/s0033291715001981

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Morina N, Wicherts JM, Lobbrecht J, Priebe S. Remission from post- traumatic stress disorder in adults: a systematic review and meta-analysis of long term outcome studies. Clin Psychol Rev. (2014) 34(3):249–55. doi: 10.1016/j.cpr.2014.03.002

PubMed Abstract | CrossRef Full Text | Google Scholar

7. American Psychiatric Association. Diagnostic and statistical manual of mental disorders [DSM V]. Washington, D.C.: American Psychiatric Association (2013).

8. Hoge CW, Castro CA, Messer SC, McGurk D, Cotting DI, Koffman RL, et al. Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care. N Engl J Med. (2004) 351:13–22. doi: 10.1056/NEJMoa040603

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Lewis S, Resnick HS, Ruggiero KJ, Smith DW, Kilpatrick DG, Best CL, et al. Assault, psychiatric diagnoses, and sociodemographic variables in relation to help-seeking behavior in a national sample of women. J Trauma Stress. (2005) 18(2):97–105. doi: 10.1002/jts.20012

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Regier DA, Narrow WE, Rae DS, Manderscheid RW, Locke BZ, Goodwin FK. The de facto US mental and addictive disorders service system: epidemiologic catchment area prospective 1-year prevalence rates of disorders and services. Arch Gen Psychiatry. (1993) 50(2):85–94. doi: 10.1001/archpsyc.1993.01820140007001

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Andrade LH, Alonso J, Mneimneh Z, Wells JE, Al-Hamzawi A, Borges G, et al. Barriers to mental health treatment: results from the WHO world mental health surveys. Psychol Med. (2014) 44(6):1303–17. doi: 10.1017/S0033291713001943

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Mental Healthcare Foundation. While We Are Waiting. Retrieved from: https://www.mentalhealth.org.uk/publications/while-we-are-waiting (2018).

13. Nederlandse Zorg Autoriteit. Voortgangsrapportage wachttijden in de GGZ. Retrieved from: https://puc.overheid.nl/nza/doc/PUC_210611_22/1/ (2017).

14. Cusack K, Jonas DE, Forneris CA, Wines C, Sonis J, Middleton JC, et al. Psychological treatments for adults with posttraumatic stress disorder: a systematic review and meta-analysis. Clin Psychol Rev. (2016) 43:128–41. doi: 10.1016/j.cpr.2015.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Ford J. Trauma memory processing in posttraumatic stress disorder psychotherapy: a unifying framework. J Trauma Stress. (2018) 31(6):933–42. doi: 10.1002/jts.22344

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Morina N, Koerssen R, Pollet TV. Interventions for children and adolescents with posttraumatic stress disorder: a meta-analysis of comparative outcome studies. Clin Psychol Rev. (2016) 47:41–54. doi: 10.1016/j.cpr.2016.05.006

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Brinkman WP, Vermetten E, Van den Steen M, Neerincx M. Cognitive engineering of a military multi-modal memory restructuring system. J CyberTher Rehabil. (2011) 4:83–99.

Google Scholar

18. Rauch S, Foa E. Emotional processing theory (EPT) and exposure therapy for PTSD. J Contemp Psychother. (2006) 36:61–5. doi: 10.1007/s10879-006-9008-y

CrossRef Full Text | Google Scholar

19. Lucifora C, Grasso GM, Nitsche MA, D’Italia G, Sortino M, Salehinejad MA, et al. Enhanced fear acquisition in individuals with evening chronotype. A virtual reality fear conditioning/extinction study. J Affect Disord. (2022) 311:344–52. doi: 10.1016/j.jad.2022.05.033

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Parsons T, Rizzo A. Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: a meta-analysis. J Behav Ther Exp Psychiatry. (2008) 39:250–61. doi: 10.1016/j.jbtep.2007.07.007

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Gerardi M, Cukor J, Difede J, Rizzo A, Rothbaum BO. Virtual reality exposure therapy for post-traumatic stress disorder and other anxiety disorders. Current Psychiatry Rep. (2010) 12:298–305. doi: 10.1007/s11920-010-0128-4

CrossRef Full Text | Google Scholar

22. Fuchs P, Moreau G, Guitton P. Virtual reality: concepts and technologies. 1st Ed. Leiden: CRC Press (2011).

23. Rizzo AA, Bouchard S. Virtual reality for psychological and neurocognitive interventions. New York: Springer (2019).

24. Wenrui D, Diea H, Shenga X, Xiaoyua L, Jingwena Z, Chen Q. The efficacy of virtual reality exposure therapy for PTSD symptoms: a systematic review and meta-analysis. J Affect Disord. (2019) 257:698–709. doi: 10.1016/j.jad.2019.07.086

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Lake J. The integrative management of PTSD: a review of conventional and CAM approaches used to prevent and treat PTSD with emphasis on military personnel. Advances in Integrative Medicine. (2015) 2:13–23. doi: 10.1016/j.aimed.2014.10.002

CrossRef Full Text | Google Scholar

26. Morina N, Ijntema H, Meyerbröker K, Emmelkamp P. Can virtual reality exposure therapy gains be generalized to real-life? A meta-analysis of studies applying behavioral assessments. Behav Res Ther. (2015) 74:18–24. doi: 10.1016/j.brat.2015.08.010

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Kuhn E, Owen JE. Advances in PTSD treatment delivery: the role of digital technology in PTSD treatment. Curr Treat Options Psychiatry. (2020) 7:88–102. doi: 10.1007/s40501-020-00207-x

CrossRef Full Text | Google Scholar

28. Carl E, Stein AT, Levihn-Coon A, Pogue JR, Rothbaum B, Emmelkamp P, et al. Virtual reality exposure therapy for anxiety and related disorders: a meta- analysis of randomized controlled trials. J Anxiety Disord. (2019) 61:27–36. doi: 10.1016/j.janxdis.2018.08.003

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Reger GM, Koenen-Woods P, Zetocha K, Smolenski DJ, Holloway KM, Rothbaum BO, et al. Randomized controlled trial of prolonged exposure using imaginal exposure vs virtual reality exposure in active duty soldiers with deployment-related posttraumatic stress disorder (PTSD). J Consult Psychol. (2016) 84:946–59. doi: 10.1037/ccp0000134

CrossRef Full Text | Google Scholar

30. Gamito P, Oliveira J, Rosa P, Morais D, Duarte N, Oliveira S, et al. PTSD elderly war veterans: a clinical controlled pilot study. Cyber Psychology, Behavior, and Social Networking. (2010) 13:43–7. doi: 10.1089/cyber.2009.0237

CrossRef Full Text | Google Scholar

31. Gonçalves R, Pedrozo AL, Coutinho ES, Figueira I, Ventura P. Efficacy of virtual reality exposure therapy in the treatment of PTSD: a systematic review. PloS one. (2012) 7(12):e48469. doi: 10.1371/journal.pone.0048469

CrossRef Full Text | Google Scholar

32. Van Meggelen M. A computer-based intervention with elements of virtual reality and limited therapist assistance for the treatment of PTSD: efficacy, acceptance and future implications. Rotterdam: Erasmus University Rotterdam (2019).

33. Amstadter AB, Broman-Fulks J, Zinzow H, Ruggiero KJ, Cercone J. Internet-based interventions for traumatic stress-related mental health problems: a review and suggestion for future research. Clin Psychol Rev. (2009) 29(5):410–20. doi: 10.1016/j.cpr.2009.04.001

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Carlbring P, Westling BE, Ljungstrand P, Ekselius L, Andersson G. Treatment of panic disorder via the internet: a randomized trial of a self-help program. Behav Ther. (2001) 32(4):751–64. doi: 10.1016/S0005-7894(01)80019-8

CrossRef Full Text | Google Scholar

35. Proudfoot J, Goldberg D, Mann A, Everitt B, Marks I, Gray JA. Computerized, interactive, multimedia cognitive-behavioural program for anxiety and depression in general practice. Psychol Med. (2003) 33(2):217–27. doi: 10.1017/S0033291702007225

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Proudfoot J, Ryden C, Everitt B, Shapiro DA, Goldberg D, Mann A, et al. Clinical efficacy of computerized cognitive-behavioural therapy for anxiety and depression in primary care: randomized controlled trial. Br J Psychiatry. (2004) 185:45–54. doi: 10.1192/bjp.185.1.46

CrossRef Full Text | Google Scholar

37. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. (2012) 32:329–42. doi: 10.1016/j.cpr.2012.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med. (2009) 39(05):705–12. doi: 10.1017/S0033291708004224

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Sheehan DV, Lecrubier Y, Sheehan KH, Janavs J, Weiller E, Weiller E, et al. The validity of the Mini international neuropsychiatric interview (MINI) according to the SCID-P and its reliability. Eur Psychiatry. (1997) 12:232–41. doi: 10.1016/S0924-9338(97)83297-X

CrossRef Full Text | Google Scholar

40. Bernstein EM, Putnam FW. Development, reliability, and validity of a dissociation scale. J Nerv Ment Dis. (1986) 174(12):727–35. doi: 10.1097/00005053-198612000-00004

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Van Vliet IM, Leroy H, van Megen HJGM. M.I.N.I. Plus. M.I.N.I. International neuropsychological interview. In: Nederlandse versie 5.0.0. [Dutch version 5.0.0]. Utrecht, Netherlands: UMC (2000).

42. Blevins CA, Weathers FW, Davis MT, Witte TK, Domino JL. The posttraumatic stress disorder checklist for DSM-5 (PCL-5): development and initial psychometric evaluation. J Trauma Stress. (2015) 28:489–98. doi: 10.1002/jts.22059

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Beck AT, Steer RA, Brown GK. Beck depression inventory-second edition manual. San Antonio, TX, United States: The Psychological Corporation (1996).

44. Lambert MJ, Hansen NB, Umphress V, Lunnen K, Okisshi J, Burungame GM, et al. Administration and scoring manual for the outcome questionnaire (OQ- 45.2). Wilmington, DE: American Professional Credentialing Services LLC (1996).

45. Friedman MJ. Finalizing PTSD in DSM-5: getting here from there and where to go next. J Trauma Stress. (2013) 26(5):548–56. doi: 10.1002/jts.21840

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Boeschoten MA, Bakker A, Jongedijk RA, Olff M. PTSD checklist for the DSM-5 – Nederlandstalige versie. In voorbereiding (2014).

47. Van der Does AJW. BDI-II-NL handleiding: de nederlandse versie van de beck depression inventory. 2nd edn Enschede, Ipskamp: Swets (2002).

48. De Jong K, Nugter AM, Polak MG, Wagenborg JEA, Spinhoven P, Heiser WJ. The outcome questionnaire (OQ-45) in a Dutch population: a cross cultural validation. Clinical Psychology and Psychotherapy. (2007) 14:288–301. doi: 10.1002/cpp.529

CrossRef Full Text | Google Scholar

49. Hox JJ, Moerbeek M, Van de Schoot J. Multilevel analysis: techniques and applications. New York, NY: Routledge (2010).

50. Van de Schoot R, Kaplan D, Denissen J, Asendorpf JB, Neyer FJ, Van Aken MAG. A gentle introduction to Bayesian analysis: applications to developmental research. Child Dev. (2014) 85:842–60. doi: 10.1111/cdev.12169

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Sorensen T, Hohenstein S, Vasishth S. Bayesian Linear mixed models using stan: a tutorial for psychologists, linguists, and cognitive scientists. Quantitative Methods for Psychology. (2016) 12:175–200. doi: 10.20982/tqmp.12.3.p175

CrossRef Full Text | Google Scholar

52. Nalborczyk L, Bataiiler C, Lœvenbruck H, Vilian A, Bürkner PC. An introduction to Bayesian multilevel models using brms: a case study of gender effects on vowal variability in standard Indonesian. J Speech Lang Hear Res. (2019) 62:1225–42. doi: 10.1044/2018_JSLHR-S-18-0006

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Bürkner P. BRMS: an R package for Bayesian multilevel models using Stan. J Stat Softw. (2017) 80:1–28. doi: 10.18637/jss

CrossRef Full Text | Google Scholar

54. Wortmann JH, Jordan AH, Weathers FW, Resick PA, Dondanville KA, Hall-Clark B, et al. Psychometric analysis of the PTSD checklist-5 (PCL-5) among treatment-seeking military service members. Psychol Assess. (2016) 28(11):1392–403. doi: 10.1037/pas0000260

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Makowksi D, Ben-Shachar MS, Lüdecke D. Bayestestr: describing effects and their uncertainty, existence and significance within the Bayesian framework. Journal of Open Source Software. (2019) 4:154. doi: 10.21105/joss.01541

CrossRef Full Text | Google Scholar

56. McDonagh A, Friedman M, McHugo G, Ford J, Sengupta A, Mueser K, et al. Randomized trial of cognitive-behavioral therapy for chronic posttraumatic stress disorder in adult female survivors of childhood sexual abuse. J Consult Clin Psychol. (2005) 73:515–24. doi: 10.1037/0022-006X.73.3.515

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Feijt MA, De Kort YAW, Bongers IMB, Ijsstelsteijn WA. Perceived drivers and barriers to the adoption of eMental health by psychologists: the construction of the levels of adoption of eMental health model. J Med Internet Res. (2018) 20(4):e153. doi: 10.2196/jmir.9485

PubMed Abstract | CrossRef Full Text | Google Scholar