Richard Hedström
Märta Wallinius
Kristina Sygel
Chris N. W. Geraets- 1Research and Development Unit, Regional Forensic Psychiatric Clinic, Växjö, Sweden
- 2Evidence-Based Forensic Psychiatry, Department of Clinical Sciences Lund, Psychiatry, Lund University, Lund, Sweden
- 3Department of Psychiatry and Neurochemistry, Center for Ethics, Law and Mental Health, Institute of Neuroscience and Physiology, The Sahlgrenska Academy at University of Gothenburg, Gothenburg, Sweden
- 4Department of Forensic Psychiatry, National Board of Forensic Medicine, Stockholm, Sweden
- 5Department of Psychiatry, University Medical Center Groningen, Groningen, Netherlands
Background: Reliable and valid assessment of paranoia is important in forensic psychiatry for providing adequate care. VR technology may add to current assessment procedures, as it enables observation within realistic (social) situations resembling the complexity of everyday life. VR constitutes a promising tool within forensics, due to the restricted nature of forensic psychiatric hospitals and ethical challenges arising from observing potentially dangerous behaviors in real life.
Objective: To investigate the feasibility of VR assessment for paranoid ideation in forensic psychiatric inpatients qualitatively by assessing the experiences of patients and a clinician, and to explore how the VR measures relate to established clinical measures.
Methods: One clinician (experienced psychiatrist) and 10 forensic psychiatric inpatients with a history or suspicion of paranoid ideation were included. Patients participated in two immersive VR scenarios (bus and supermarket) during which paranoia was assessed by the clinician. Qualitative interviews were performed with patients and the clinician performing the assessment to investigate experiences and feasibility. Further, measures of paranoia, social anxiety, and positive symptoms were obtained.
Results: Nine out of 10 participants with varying levels of paranoid ideation completed the assessment. Manifest inductive content analyses of the interviews revealed general experiences, advantages such as enabling observing participants from a different perspective, and challenges of the VR assessment, such as a lack of objectivity and the laboriousness of the assessment for the clinician. Although more paranoia was experienced during the supermarket scenario, correlates with classical measures were only significant for the bus scenario.
Discussion: The VR assessment was appreciated by most patients and the clinician. Based on our results short, standardized VR assessment scenarios are feasible, however, they do not appear reliable or objective for assessing paranoia. The clinical usefulness is most likely as a collaborative tool and add-on measure to existing methods.
1 Introduction
Paranoid ideation is one of the most common symptoms of a psychotic disorder (Freeman, 2007). Such thoughts can be mild or severe and manifest as persecutory delusions, which are characterized by the belief that harm is occurring, or will occur, and that someone intends to inflict harm (Freeman, 2007). Up to 90% of patients with a psychotic disorder experience paranoid ideation to some degree (Moutoussis et al., 2007). In forensic psychiatry, where patients with a combination of severe mental disorders and criminal conduct are treated, paranoid ideation is prevalent and has been proposed as an underlying, and sometimes even causal, risk factor for violent offenses (Coid et al., 2016; Darrell-Berry et al., 2016).
Reliable and valid assessment of paranoia is important for providing adequate healthcare for patients with psychotic disorders and may in some cases also be relevant to the formulation of violence-preventive strategies for patients. Current assessments in forensic psychiatry mainly consist of clinical interviews, self-reports, and staff observations (Aboraya et al., 2005). A downside of self-reports and interviews is that they rely on memory, insight, and motivation of patients. Furthermore, the secluded environment strongly differs from life outside the clinic, which can affect the reliability and ecological validity of observations. Novel technologies such as virtual reality (VR) could potentially provide new possibilities for psychiatric assessments (Freeman et al., 2017; van Bennekom et al., 2017; Geraets et al., 2022).
Immersive VR enables patients to interact with computer-generated virtual worlds, usually by wearing a head-mounted display. VR simulations have been shown to trigger emotional, psychological, and physical reactions similar to real-life reactions (Martens et al., 2019). To effectively elicit such emotions and responses, VR simulations have to induce a sense of “presence.” For a VR user to experience presence requires experiencing a sense of both “place illusion” and “plausibility illusion”; that is believability of both the virtual environment itself and the unfolding scenario (Slater, 2009; Skarbez et al., 2017; Geraets et al., 2021).
Several systematic reviews have investigated the existing evidence on VR assessment for paranoia (Freeman et al., 2017; van Bennekom et al., 2017; Rus-Calafell et al., 2018). An advantage of VR for the assessment of paranoia lies in its controllability. In a VR environment, we can manipulate, and thus know, whether virtual characters (avatars) show friendly, neutral or hostile behavior. In contrast, when using self-report of daily life situations, it is unknown whether the self-reported paranoia or hostility reflects a persecutory delusion or whether it reflects an actual, imminent social threat. As such, in a VR situation with exclusively neutral cues, a patient’s self-reported paranoid ideation and level of perceived hostility are easier to evaluate. Among the reviewed VR paradigms are scenarios involving riding the underground with several avatars, and exploring public environments such as cafés, a library, and a supermarket, while informing on thoughts of participants about the virtual scenarios directly afterward (Rus-Calafell et al., 2018). The reviews conclude that almost all reviewed VR scenarios could elicit and measure self-reported paranoia in clinical and non-clinical populations to some extent (Freeman et al., 2017; van Bennekom et al., 2017; Rus-Calafell et al., 2018). However, not all studies agree that VR can reliably differentiate between clinical and non-clinical groups. In accordance with this, many of the current studies have been performed as proof-of-concept studies for VR environments or to investigate mechanisms. A lack of knowledge still exists concerning the clinical use and potential for VR-assisted assessment of paranoid ideations.
Virtual reality technology may add value to current forensic psychiatric assessment procedures, as it enables observation within realistic (social) situations resembling the complexity of everyday life, where cognition and behavior can be monitored in real time (Riva, 1997; Freeman et al., 2017; Pan and Hamilton, 2018). Further, VR has the advantage that you can (repeatedly) expose people to (social) situations that are controlled, safe, and can be accessed within the isolated high-security environment of the clinic. VR has been used safely in both patients with paranoia and forensic psychiatric patients (e.g., Pot-Kolder et al., 2018; Klein Tuente et al., 2020). Thus, the use of VR in assessments constitutes a promising tool especially within forensic psychiatry, considering the restricted nature of forensic psychiatric hospitals and the ethical challenges arising from observing potentially dangerous behaviors in real-life settings (Freeman et al., 2017; van Bennekom et al., 2017).
In the current pilot study, we investigated a novel VR-assisted assessment for paranoid ideation in forensic psychiatric inpatients. This was done both qualitatively and quantitatively by (1) evaluating how the VR-assisted assessment was experienced by patients and the clinician, (2) investigating how clinicians’ observations from VR scenarios can add to paranoia assessments, and (3) describing how the VR measures relate to established clinical measures of paranoia and anxiety.
2 Materials and methods
2.1 Design and participants
In total, 10 forensic psychiatric inpatients from a high-security forensic psychiatric clinic in Sweden were included in this single-group, mixed-methods pilot study. Inclusion criteria were: (1) aged 18 or older, (2) currently receiving forensic psychiatric treatment, and (3) a history of, or indications of current, paranoid ideation. Exclusion criteria were: (1) insufficient command of the Swedish language, (2) inability to provide informed consent due to current psychiatric status, (3) presence of an organic brain disease, e.g., dementia or epilepsy, and (4) posing severe security risks (e.g., violence) hindering safe participation. A clinician was recruited, from the consultant psychiatrists at the clinic, to perform the VR-assisted assessments.
Both quantitative and qualitative data were gathered from the patients through interviews, self-reports, observations, and medical record reviews. Qualitative (interview) data was obtained from the clinician conducting the VR-assisted assessment.
2.2 Ethics
This study was approved by the Swedish Ethics Review Authority (2021-06353-01) and was conducted according to the principles of the Declaration of Helsinki. Treating psychiatrists only referred patients to the study if they were assessed as able to provide informed consent and did not pose current and severe security risks. The patients were informed, verbally and in writing, about the study by a research assistant. Specific care was taken to explain that participation or non-participation would not in any way affect the patients’ inpatient care and that they could cease participation at any time without providing a reason. If patients were willing to participate, written, informed consent was obtained.
2.3 Procedures
Patients were referred to the study by their treating psychiatrist, according to inclusion and exclusion criteria. After eligible participants had been informed of the study, those who wanted to participate signed informed consent. Subsequently, background data were collected, and the participants completed the VR scenarios and measurements. The clinician could discontinue a VR scenario when there was a risk of harm (e.g., due to falling). Participation took between 1.5 and 4 h and could be completed in 1 or 2 days (in case it was too intensive to finish all the measures on the same day). Four participants completed all measures within the same day. Five patients completed the clinical trait measures (see the measurement section) the next day, and one patient finished it 2 days later. During the VR scenarios and VR-specific measures, both a clinician and a research assistant were present. After completion of participation, participants received a gift voucher of 99 SEK (approx. $9) for the kiosk at the clinic.
2.4 VR scenarios
In this study, the Social Worlds© VR software created by CleVR (Delft, The Netherlands) was used. Participants used an Oculus Rift S to view the virtual environments, wore headphones, and moved around using Oculus controllers, see Figure 1. The clinician guided the participant through the VR scenarios by using a microphone. First, a 2-min VR practice scenario was performed, and then a supermarket and bus scenario each lasting 5 min. These two virtual environments were chosen as these are neutral environments and relatable to most people, even for those who have been incarcerated or involuntarily confined to a hospital for a long period of time.
Figure 1. The VR setup and the VR environments as seen by the patients in Scenarios 1 and 2. Reproduced with permission from CleVR.
Practice scenario: The participant was immersed in a virtual supermarket and was free to explore the supermarket to become accustomed to the VR experience and equipment, and to report signs of cybersickness. The virtual environment was programmed to include 12 freely moving customer avatars, which meant a moderate level of crowding where participants passed avatars throughout the supermarket, but never were surrounded by them. Avatars would shortly turn their attention to participants only if they came close. No other interaction with avatars was possible and the avatars were set to continuously show a neutral facial expression toward the participant.
Scenario 1: Directly after the practice scenario, the participant was instructed through the headphones to search the virtual supermarket for milk and to inform the clinician once he or she had found it. Next, the participant was instructed to find the cashier and to queue behind the avatars until the scenario ended after 5 min. Due to the random movements of the 12 avatars (acting neutrally) participants faced different numbers of avatars while queuing, and some participants experienced avatars “cutting the line” from behind. During this scenario only ambient sounds were present and avatars, including the cashier, would not respond to any potential comments by participants.
Scenario 2: The participant was seated on a bus which left the bus station and drove through a city neighborhood. On the bus were 11 other avatars, all programmed to act neutrally. The participant could not move around the environment but could look around the bus in 360 degrees as it moved. A female and a male avatar were seated opposite the participant. During the bus ride, the participant overheard the female avatar (enacted by the clinician using the microphone with voice distortion) having a (scripted) heated telephone conversation about her being late (“Mhm. I’ve told you I will be late! Mhm. No!” etc.). The male avatar would occasionally look at the participant, pick up his phone or look across at the female avatar at moments when she spoke louder. After ending the call, the female avatar turned to the participant and asked for directions in a calm and neutral manner. The scenario ended after the participant answered.
2.5 Quantitative measurements
2.5.1 Sociodemographic characteristics
Age, gender, diagnoses, and IQ scores were collected from patients’ medical records. IQ scores were measured with the Wechsler Adult Intelligence Scale Version R, III, or IV. Reliability for the WAIS-R has only been estimated for its subscales, which were deemed moderate to good (ranging from 0.65 to 0.88) (Wechsler, 1996). Full-scale reliability for the later versions were deemed excellent: 0.90 for WAIS-III (Wechsler and Nyman, 2003) and 0.96 for WAIS-IV (Wechsler and Nyman, 2010).
2.5.2 Clinical trait measures
Positive psychotic symptoms were assessed using the Positive and Negative Syndrome Scale interview (PANSS). The Positive symptom subscale is made up of seven items measuring the presence and severity of positive symptoms on a 7-point Likert scale. This subscale has good test-retest reliability of α = 0.81 and interrater-reliability of 0.73 (Kay et al., 1989).
Social anxiety was measured with the Social Interaction Anxiety Scale (SIAS). The SIAS consists of 20 items assessing the tendency to fear and avoid social situations on a scale from 0 (not at all) to 4 (extreme), resulting in a total score ranging between 0-80. Internal and test-retest reliability for the original English version is considered very good, while showing discriminant validity toward non-clinical samples, depression, and other anxiety disorders (Mattick and Clarke, 1998).
Paranoid thoughts were assessed with the Revised Green Paranoid Thoughts Scale (R-GPTS). The R-GPTS has two subscales: Ideas of Reference (8 items) and Ideas of Persecution (10 items). Items are rated on a 5-point Likert scale, ranging from 0 (not at all) to 4 (totally), the total score ranges from 0 to 72. The instrument is considered to have excellent reliability, especially at elevated levels of paranoia (Freeman et al., 2021).
2.5.3 VR measures
State anxiety levels were measured before and directly after each VR scenario on a verbal analog scale (VAS) by rating current anxiety on a scale from 0 (not at all) to 100 (extremely anxious).
VAS perceived hostility was assessed after each VR scenario by asking: “How hostile were the people in the VR environment toward you, on a scale from 0 (not hostile) to 100 (extremely hostile).”
State paranoia in each VR scenario was assessed with the 20-item State Social Paranoia Scale (SSPS). Ten items assess state paranoia, i.e., negative intention about the virtual characters (e.g., “Someone was hostile toward me”) and 10 items describe positive or neutral interpretations of the virtual characters. Items were rated on a 5-point Likert scale, resulting in scores ranging from 10 to 50 on state paranoia. The original English instrument has demonstrated excellent internal reliability and adequate test-retest reliability. Combined with clear divergent and convergent validity it is considered to have good psychometric properties (Freeman et al., 2007).
Presence in VR was assessed after completing both scenarios using the 14-item Igroup Presence Questionnaire (IPQ). Items were scored on a 7-point Likert scale and analyzed according to previously established factors of general presence, spatial presence, involvement, and realness. The instrument has demonstrated good psychometric properties (Schubert et al., 2001).
2.6 Qualitative data
During the scenarios, the participant’s behavior in VR (physical and verbal expressions) was observed by the clinician. Behavior was rated using a structured observation protocol with open questions assessing (1) social physical behavior (e.g., “Does the participant look at avatars, does he avoid avatars, etc.”), (2) emotional expressions (“What emotions does the patient show during the VR session? How?”), (3) verbal expressions (“Does the patient say anything? What?”), and (4) other observations considered relevant for the assessment of symptoms of paranoia, see Supplementary Table 1 for the observation protocol. The clinician’s observations of the participant’s physical and verbal expressions in VR were summarized and manually divided into categories.
The clinician conducted a semi-structured interview with the participant regarding his/her experiences in VR directly after the scenarios, to gain more insights into symptom-related experiences in VR. The interviews were audio recorded and transcribed using the NVivo software.
To assess the acceptability and feasibility of the VR simulation, the researcher conducted semi-structured interviews (see Supplementary Table 2) with both the participant and the clinician on their experiences with the VR assessment. All interviews were audio recorded and transcribed using the NVivo software.
2.7 Data analyses
Descriptive statistics were calculated for all quantitative data by presenting the mean and standard deviation or n, and the median and range. Explorative, non-parametric Spearman’s rho correlation analyses were performed to assess relations between VR state paranoia measures (SSPS and VAS scores) and the clinical trait measures (PANSS, R-GPTS, and SIAS). Significance was accepted at 0.05 due to the explorative design of the study. For visualization of VR state and trait measures spider graphs were made per participants, by transforming the scores of each measure to percentages.
Analysis of the qualitative data was conducted through manifest inductive content analysis using NVivo, in accordance with the process described by Vears and Gillam (2022). The interview scripts were coded by authors RH and MW and then summarized into content categories and subcategories using an iterative process.
3 Results
Between March 2022 and May 2022 23 inpatients from the clinic’s high-security units, and 53 inpatients from the clinic’s medium security units were assessed for inclusion with their treating psychiatrist (see Figure 2). Exclusion occurred mainly due to not having a history of paranoid ideation or current symptoms, a cognitively impaired psychiatric state, lacking Swedish language skills and/or severe risks of violence. The risk of violence was most pronounced for high-security candidates, but was also a factor when excluding several patients in medium security. Out of the original 76 patients, 23 medium security inpatients fulfilled the criteria and were approached for the study. In total, 11 patients signed informed consent. One patient subsequently withdrew consent, resulting in a final sample of 10 participants: a 43% inclusion rate. Table 1 shows the demographic and clinical characteristics of the participants: 9 out of 10 had a current diagnosis of a psychotic disorder. Participants represented diverse psychiatric treatment histories, with a wide range of psychotic experiences and length of outpatient and inpatient care.
Figure 2. Visualization of the selection and recruitment process.
Table 1. Participant characteristics and clinical measures (n = 10).
The sample showed a diverse presentation of current positive psychotic symptoms as measured with the PANSS. Among the 10 participants, the most noteworthy symptoms were suspiciousness/persecution (n = 1 extreme case, n = 2 severe cases) and delusions (n = 2 severe cases, n = 1 moderately severe case). Absent or minimal positive psychotic symptoms were found among three participants. Regarding paranoid ideation as measured in R-GPTS, one participant showed very severe ideas of social references, while four participants showed severe or very severe ideas of persecution, respectively (see Table 1 and Figure 3).
Figure 3. Visualization of trait and VR state measures per participant. The standardized data of participants #1 to #10 is provided for paranoia and anxiety related traits (PANSS, R-GPTS, and SIAS) and state (VAS scores and SPSS) measures. To enable comparison each score was transformed into percentages by dividing the score by the maximum score possible of that measure and multiplying by 100.
3.1 Experiences of VR-assisted assessment by the patients and the clinician
Nine participants completed both VR scenarios. One participant only completed the bus scenario, as the clinician discontinued the supermarket scenario (after ± 3 min) after observing balance issues. Two other participants reported minor symptoms of cybersickness and showed varying degrees of balance issues during the supermarket scenario, but could continue. One participant asked to be seated during the supermarket scenario, due to fear of standing while wearing the VR-glasses. No serious adverse events were reported.
Participants experienced moderate presence in VR on all four subscales of the IPQ (range 0–6): general M = 5.1 (SD = 1.0) spatial presence M = 4.0 (SD = 0.9), involvement M = 3.4, (SD = 1.1), and realness M = 2.9 (SD = 1.1). Table 2 presents the manifest inductive content analyses of the interviews with the patients and the clinician regarding their perceptions of the VR-assisted paranoia assessment. In total, three content categories with specific subcategories were identified: (1) advantages of VR in assessment, (2) experiences with VR assessment, and (3) challenges administering VR scenarios.
Table 2. Manifest inductive content analysis with illustrative quotes from interviews with participants and clinician.
3.2 Clinician observations during VR
Using the structured observation protocol (Supplementary Table 1), the clinician mainly noted participants’ direction of gaze and movement patterns, see Table 3. Interpretations of interactions were made in four cases, and in three cases participants gave verbal information on how they interpreted the avatars’ behaviors or actions and recorded observations of emotions with positive,
Table 3. Summary of clinician’s observations according to type of behavior/reaction.
neutral, or negative valence. Furthermore, several participants narrated their experiences during the session. In two cases, the clinician noted interpretations of the emotional valence of the participants’ communication with the avatars. Furthermore, the clinician noted that two patients did not fully experience the bus scenario as anticipated, which was observed as they did not listen to and/or answer the female avatar as expected; one answered the female character as she was on the phone, and both were unsure whether the female avatar’s final question was directed at them.
3.3 Associations between VR assessments and clinical measures of paranoia and anxiety
Means, standard deviations, medians, and ranges of the VR assessment measures are presented in Table 4. Both VR paranoia measures, the SSPS state paranoia, and the single-item VAS hostility measure, correlated highly for both the supermarket (r = 0.90, p < 0.001) and the bus scenario (r = 0.94, p < 0.0001). On average, slightly more anxiety and paranoia were elicited by the supermarket scenario than the bus scenario. This is also reflected in the visualizations of the standardized scores of the state VR measures and clinical trait measures for each participant in Figure 3, where a strong heterogeneity between the profiles of patients can be noted. Some participants only showed elevated scores on established trait measures (#1, #2, #10) and not on VR measures, while some showed almost no paranoia or anxiety symptoms on any of the measures (#3, #7). Others (#8, #9) demonstrated a more integrated picture with elevated scores on both trait and VR measures. One participant (#5) reported a high level of experienced hostility in the supermarket scenario, but low scores on all other measures.
Table 4. VR measures.
3.3.1 Supermarket scenario
Although the supermarket scenario triggered slightly more paranoid ideas, state paranoia (SPSS) and VAS hostility scores did not correlate with any of the clinical trait measures significantly. Also, for VAS anxiety scores, no clinical trait measures correlated. This indicates that people higher in trait anxiety and paranoia were not more prone to feeling anxious or paranoid during the VR exposures.
3.3.2 Bus scenario
State paranoia (SSPS) in the VR bus scenario correlated strongly with social interaction anxiety (r = 0.64, p = 0.05), ideas of social reference (r = 0.77, p < 0.01), ideas of persecution (r = 0.66, p = 0.04) and the paranoid thoughts total score (r = 0.81, p < 0.01), but not with PANSS delusions or suspiciousness. Similarly, VR VAS hostility during the bus scenario correlated strongly with ideas of social reference (r = 0.69, p = 0.03), and the total score on paranoid thoughts (r = 0.67, p = 0.04), but not with social interaction anxiety or PANSS delusions and suspicious. Thus in contrast to the supermarket, significant associations were found, however, when interpreting it should be noted that people scored rather low on state paranoia and anxiety measures for this scenario.
4 Discussion
The present study is, to our knowledge, the first to examine the feasibility and clinical relevance of VR-assisted assessment of paranoid ideations in a clinical forensic psychiatric setting. The ten participants showed a wide range in severity of psychotic symptoms. Overall, many were positive and curious about the assessment, even though hesitations and fears of the VR technology and its possible effects emerged. From the clinician’s view, VR enabled observations of patients from a third-person perspective, and to initiate conversations on paranoia that otherwise would be difficult to create preconditions for. Several challenges were identified, such as difficulties in the practical use of VR while simultaneously performing clinical assessments and a lack of objectivity due to amongst others, variance in the avatar’s automated behavior.
Self-reports of paranoid ideations in VR were partially related to trait paranoid ideation and social anxiety, but a lack of associations to the clinically assessed PANSS scales, was demonstrated.
4.1 VR-assisted assessment as perceived by patients and clinician
Many participants described VR scenarios as a novel and interesting experience when compared to standard clinical interviews. Several participants who initially worried about the VR technology and its possible impact on their wellbeing, still preferred VR over standard interviews. Interestingly, worries about VR, slight discomfort from equipment or side effects (cybersickness) did not make patients drop out of the experiment. The only discontinued participant was initiated by the clinician, as there was a risk of falling due to balance issues in VR which were clearly noticeable to the clinician but described by the participant as minor cybersickness. Although postural instability has been reported in other VR research (Tian et al., 2022), several studies with forensic and paranoid patients have not reported high rates of cybersickness or balance problems (Rus-Calafell et al., 2018; Klein Tuente et al., 2020). The balance issues could be related to the participants’ unfamiliarity with VR, or the single-session format in which three scenarios were performed in a short period of time. Adaptations such as placing someone in a seated position may solve this issue.
Regarding two of the most central features of VR – experiences of presence and immersion – participants had diverse experiences. Some described experiencing strong presence and immersion, while some described circumstances affecting presence and immersion negatively, e.g., the feeling of wearing the equipment, restrictions in possible actions with the VR environment, and avatars’ “unnatural” expressions and movements. Thus, several participants described experiencing breaks in presence that could be classed as breaks in either place illusion, plausibility illusion, or co-presence (Slater, 2009). Conversely, others reported a continued sense of presence and strong emotions, even when for example bumping into avatars or not being answered when talking to avatars in the supermarket scenario. More investigations of factors affecting presence and immersion are needed.
The clinician appreciated the VR assessment as an alternative means of communication with patients, underlining how VR facilitated observations and conversations with the participants who were most suspicious or hostile to forensic psychiatry and clinical interviews. This is in line with previous studies, indicating that VR interventions potentially increase the degree of personal disclosure in other psychiatric settings (Pan and Hamilton, 2018). We therefore, humbly, suggest that VR may constitute a way to establish constructive two-way communication with patients who, due to hostility, severe paranoia, or previous negative experiences from standard interventions in compulsory (forensic) psychiatric care have been less responsive to previous interventions. Further clinical advantages with VR were described by the clinician as a means for roleplay and observations for patients with limited exposure to social situations, either because of long-term inpatient care or more generalized social difficulties.
The scenarios created for the current study were short and designed to be both neutral and standardized, while letting participants approach avatars and explore the surroundings. This design still contained variations between administrations, and follow-up interviews revealed that the scenarios themselves also carried different meanings for different participants. For example, avatars moving randomly and participants exploring freely in the supermarket scenario meant that participants could bump into avatars and avatars rush past participants, causing different experiences and social situations. Thus, the ability to interact with VR environments, while potentially contributing to maintaining the plausibility illusion of a VR scenario, also creates more variation in the assessment scenario, thereby limiting standardization and objectivity (Pan and Hamilton, 2018). Furthermore, interviews revealed that the scenarios related differently to patients’ specific psychotic symptoms, cognitions, and social learning histories. A wide range of attitudes and reactions were reported to the supposedly neutral scenario environments, ranging from excitement about a virtual environment outside the clinic to VR environments triggering anxieties. These differences must be taken into consideration when evaluating our results since these experiences could be one explanation for why the three most paranoid participants did not report paranoid ideations during VR.
A prominent challenge for the clinician was the limited feasibility of simultaneously conducting clinical assessments while managing the VR technology. We acknowledge that this could be mitigated through automated VR scenarios, or through having an assistant performing the VR scenarios, allowing for the clinician to completely focus on observations of patients’ behaviors. However, when protocols are less strict and not focused on objective assessment, this might also release strain on the clinician, as has been observed in treatment studies where conducting interactive scenarios and providing feedback (thus observing) with similar software was feasible for clinicians (Nijman et al., 2022). Also, providing more thorough training for the clinician (than was done for the current study) with the VR hardware and software seems to be an important feature if such assessments are to be implemented in clinical practice.
4.2 Paranoia in VR and associations with standard assessments
Paranoia in the VR scenarios was reported by some participants and was partly observed by the clinician. The clinician’s observations showed a range of reactions to the scenarios (e.g., curiosity, hesitancy, lack of interest), but no strong signs of paranoid ideations and behavior among participants. For the majority of those demonstrating paranoia during the VR scenarios, the supermarket scenario provoked more paranoia than the bus scenario on both the single-item VAS scale and the state paranoia questionnaire. Inspection of the graphs demonstrating the overall paranoid symptoms presented by each participant showed that those presenting high scores at PANSS and R-GPTS scales, in general, did not experience anxiety or paranoia to a large degree during the VR scenarios. When associations were investigated in correlation analyses, no significant associations between VR-related measures of state paranoia and the clinically assessed PANSS scales were demonstrated, such relations were only found for self-reports of social anxiety and paranoid ideations (in the bus scenario). However, it also should be noted that VR state paranoia measures do not necessarily need to match the interview and questionnaire-based scores, as they concern different timeframes. I.e., the VR measure concerns the past 5 min, whereas the R-GPTS concerns the past month, and the PANSS the past week. In previous experimental studies, participants in VR scenarios have expressed ideas of persecution, social evaluative concerns, risk of physical harm and emphasized social cues in both patients and non-clinical people with high trait paranoia (e.g., being approached too closely, being the subject of the avatars’ attention) (Freeman et al., 2008; Riches et al., 2020).
Given the obvious limitations of a very small sample, our results must be seen as preliminary. However, it provides information for future research that VR-elicited paranoia should be investigated in relation to clinically assessed paranoia, with no assumptions on the inherent overlap. Also, differences between clinical and non-clinical populations should be investigated, as a part of a validation process and investigating the utility as a potential “objective” assessment form, as small and non-significant differences have been reported previously when assessing paranoia during a VR underground train ride (Fornells-Ambrojo et al., 2015; Veling et al., 2016), though the majority of research did find significant differences between such groups (Rus-Calafell et al., 2018).
To be noted, two participants self-reported paranoid ideations in R-GPTS without receiving an elevated score of paranoia in the clinical PANSS interview. This could have been due to current real stressors and conflicts in the forensic psychiatric environment for these specific participants (i.e., actual threats), which was captured as paranoia by the R-GPTS. Fornells-Ambrojo et al. (2015) have previously highlighted the role of environment and adverse life-events in creating and maintaining paranoid ideations, recommending VR assessment to consider participants’ social background and different neighborhoods. For our sample, these aspects were arguably important, and we recommend continued VR research in forensic settings to take the social environments of patients into consideration.
4.3 Therapeutic misconceptions
Expectations reflecting therapeutic misconceptions were evident during recruitment, administration, and follow-up interviews. Therapeutic misconception denotes the misunderstanding and conflation of research goals, protocols, and procedures with clinical treatment effects (Thong et al., 2016). While many participants seemed to understand the study, three out of ten expressed both hopes and fears of VR as an “objective” measure of their mental health, even after thorough information on the study aims and procedures was provided by the research assistant. These participants either disagreed with their current diagnosis and wished for it to be reexamined through this study, or rather worried about showing potential early signs of new psychotic episodes that would be detected through VR. Referring psychiatrists expressed similar expectations about paranoid ideations being discernable through VR scenarios, despite the study repeatedly being presented as a pilot project without the possibility of diagnosing psychotic symptoms, and that no individual findings would be reported back to the referring psychiatrist. These misconceptions of the diagnostic capabilities of VR were comparable to therapeutic misconceptions encountered in psychiatric treatment studies, e.g., potential benefits from the study for the participant’s care, and misconceptions concerning the purpose of the study (Appelbaum et al., 2012). Our study design contained several elements previously found to increase the risk of therapeutic misconception, e.g., researchers having simultaneous clinical assignments and conducting the study in forensic psychiatric settings (Pedersen et al., 2021). The current sample is also characterized by established risk factors for therapeutic misconception in psychosis patients: residential living, poor independence in activities of daily functioning, cognitive deficits, and positive psychotic symptoms (Thong et al., 2016). The expectations and fears that were voiced in our study underline the importance of clear communication adjusted to the participants’ responsiveness, to decrease the risk of therapeutic misconception during research.
4.4 Limitations
The present study has several limitations. Firstly, 57% of the approached patients were not willing to participate, and we have no information on how these patients may have differed from those who chose to participate. Thus, the actual representativeness of the sample is unknown. However, this participation rate is comparable to other studies in forensic psychiatry (Pedersen et al., 2021). Second, participants’ IQ scores were collected through medical records, but measured at different points in time during their illness and cannot be assumed to be reliable measures of current intellectual functioning. Because of recruitment criteria, our sample may have fewer cognitive impairments when compared to patients in similar units and with similar lengths of stay, limiting the generalizability. However, the nature of this study, being an explorative feasibility study, does not entail strict considerations regarding representativeness and generalizability.
Further, the order of the two scenarios was not randomized and therefore we cannot rule out that this might have influenced the assessment and account for differences between assessments from the supermarket and bus scenarios. Also, we do not know whether the (minor) symptoms of cybersickness may have influenced results, a larger sample is needed to investigate this. Finally, the validity of self-report measures on paranoia poses additional limitations in forensic settings, where some individuals may face actual, physical danger in the forensic hospital environment (e.g., due to threats from fellow inpatients).
5 Conclusion
In this study, VR-assisted assessment of paranoid ideations proved overall acceptable to forensic psychiatric patients with different presentations of psychotic symptoms, social experiences, familiarity with VR technology and attitudes to their diagnosis. The VR format was appreciated by a subgroup of patients and the clinician, although the VR assessment (both the clinical model and the practical use of VR technology while conducting clinical assessments) should be revised to enhance practicality. Standardized VR assessment scenarios seem feasible to perform, however, the current research shows that they do not appear reliable as a stand-alone, objective assessment of paranoid ideations in forensic psychiatric patients.
Even though our VR assessment model did not identify clearly defined paranoid symptoms, there seems to be value in introducing VR-assisted assessment to forensic psychiatric practices. Moving forward, VR-assisted assessment could be examined as a collaborative and personalized tool, which could be especially relevant for patients who are hard to engage through standard methods. Accordingly, we recommend future research on personalized VR assessment scenarios for forensic psychiatric patients. Such scenarios could, for instance, examine more therapeutic-oriented goals instead of aiming for objective symptom measurements. Finally, the forensic psychiatric setting added additional challenges concerning expectations of VR technology, patients’ preoccupations with diagnoses in inpatient care, as well as the very real security concerns patients may face, all of which should be examined in future studies.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
This study involving humans was approved by the Swedish Ethical Review Authority Box 2110 750 02 Uppsala. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any identifiable images or data included in this article.
Author contributions
CG and MW designed the study. RH and CG wrote the first draft of the manuscript and carried out the quantitative analysis of the results. RH and MW analyzed the qualitative interview data. KS analyzed observation protocols from the VR scenarios. All authors contributed to manuscript revision and read and approved the submitted version.
Funding
This study was supported by the Regional Forensic Psychiatric Clinic in Växjö, Södra sjukvårdsregionen, Region Kronoberg, and the Swedish Research Council for Health, Working Life and Welfare (2018–01409).
Acknowledgments
A special thank you to the participating patients, Natalia Platonova and Ola Ståhlberg at the National Board of Forensic Medicine, Sweden, and treating psychiatrists and staff at the forensic psychiatric clinic where data collection was performed.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1242243/full#supplementary-material
References
Aboraya, A., France, C., Young, J., Curci, K., and Lepage, J. (2005). The validity of psychiatric diagnosis revisited: The clinician’s guide to improve the validity of psychiatric diagnosis. Psychiatry Edgmont. Pa Townsh. 2, 48–55.
Appelbaum, P. S., Anatchkova, M., Albert, K., Dunn, L. B., and Lidz, C. W. (2012). Therapeutic misconception in research subjects: Development and validation of a measure. Clin. Trials 9, 748–761. doi: 10.1177/1740774512456455
Coid, J. W., Ullrich, S., Bebbington, P., Fazel, S., and Keers, R. (2016). Paranoid ideation and violence: Meta-analysis of individual subject data of 7 population surveys. Schizophr. Bull. 42, 907–915. doi: 10.1093/schbul/sbw006
Darrell-Berry, H., Berry, K., and Bucci, S. (2016). The relationship between paranoia and aggression in psychosis: A systematic review. Schizophr. Res. 172, 169–176. doi: 10.1016/j.schres.2016.02.009
Fornells-Ambrojo, M., Freeman, D., Slater, M., Swapp, D., Antley, A., and Barker, C. (2015). How do people with persecutory delusions evaluate threat in a controlled social environment? A qualitative study using virtual reality. Behav. Cogn. Psychother. 43, 89–107. doi: 10.1017/S1352465813000830
Freeman, D. (2007). Suspicious minds: The psychology of persecutory delusions. Clin. Psychol. Rev. 27, 425–457. doi: 10.1016/j.cpr.2006.10.004
Freeman, D., Loe, B. S., Kingdon, D., Startup, H., Molodynski, A., Rosebrock, L., et al. (2021). The revised Green, et al., Paranoid Thoughts Scale (R-GPTS): Psychometric properties, severity ranges, and clinical cut-offs. Psychol. Med. 51, 244–253. doi: 10.1017/S0033291719003155
Freeman, D., Pugh, K., Antley, A., Slater, M., Bebbington, P., Gittins, M., et al. (2008). Virtual reality study of paranoid thinking in the general population. Br. J. Psychiatry 192, 258–263. doi: 10.1192/bjp.bp.107.044677
Freeman, D., Pugh, K., Green, C., Valmaggia, L., Dunn, G., and Garety, P. (2007). A measure of state persecutory ideation for experimental studies. J. Nerv. Ment. Dis. 195, 781–784. doi: 10.1097/NMD.0b013e318145a0a9
Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., et al. (2017). Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol. Med. 47, 2393–2400. doi: 10.1017/S003329171700040X
Geraets, C. N. W., Stouwe, E. C. D., van der Pot-Kolder, R., and Veling, W. (2021). Advances in immersive virtual reality interventions for mental disorders: A new reality? Curr. Opin. Psychol. 41, 40–45. doi: 10.1016/j.copsyc.2021.02.004
Geraets, C. N. W., Wallinius, M., and Sygel, K. (2022). Use of virtual reality in psychiatric diagnostic assessments: A systematic review. Front. Psychiatry 13:828410. doi: 10.3389/fpsyt.2022.828410
Kay, S. R., Opler, L. A., and Lindenmayer, J.-P. (1989). The positive and negative syndrome scale (PANSS): Rationale and standardisation. Br. J. Psychiatry 155, 59–65.
Klein Tuente, S., Bogaerts, S., Bulten, E., Keulen-de Vos, M., Vos, M., Bokern, H., et al. (2020). Virtual reality aggression prevention therapy (VRAPT) versus waiting list control for forensic psychiatric inpatients: A multicenter randomized controlled trial. J. Clin. Med. 9:2258. doi: 10.3390/jcm9072258
Martens, M. A. G., Antley, A., Freeman, D., Slater, M., Harrison, P. J., and Tunbridge, E. M. (2019). It feels real: physiological responses to a stressful virtual reality environment and its impact on working memory. J. Psychopharmacol. 33, 1264–1273. doi: 10.1177/0269881119860156
Mattick, R. P., and Clarke, J. C. (1998). Development and validation of measures of social phobia scrutiny fear and social interaction anxiety11Editor’s note: This article was written before the development of some contemporary measures of social phobia, such as the Social Phobia and Anxiety Inventory (Turner, et al., 1989). We have invited this article for publication because of the growing interest in the scales described therein. S.T. Behav. Res. Ther. 36, 455–470. doi: 10.1016/S0005-7967(97)10031-6
Moutoussis, M., Williams, J., Dayan, P., and Bentall, R. P. (2007). Persecutory delusions and the conditioned avoidance paradigm: Towards an integration of the psychology and biology of paranoia. Cognit. Neuropsychiatry 12, 495–510. doi: 10.1080/13546800701566686
Nijman, S. A., Pijnenborg, G. H. M., Vermeer, R. R., Zandee, C. E. R., Zandstra, D. C., van der Vorm, D., et al. (2022). Dynamic interactive social cognition training in virtual reality (DiSCoVR) versus virtual reality relaxation (VRelax) for people with a psychotic disorder: A single-blind multicenter randomized controlled trial. Schizophr. Bull. 19:272. doi: 10.1093/schbul/sbac166
Pan, X., and Hamilton, A. F. (2018). Why and how to use virtual reality to study human social interaction: The challenges of exploring a new research landscape. Br. J. Psychol. Lond. Engl. 1953, 395–417. doi: 10.1111/bjop.12290
Pedersen, S. H., Bergman, H., Berlin, J., and Hartvigsson, T. (2021). Perspectives on recruitment and representativeness in forensic psychiatric research. Front. Psychiatry 12:647450. doi: 10.3389/fpsyt.2021.647450
Pot-Kolder, R. M. C. A., Geraets, C. N. W., Veling, W., van Beilen, M., Staring, A. B. P., Gijsman, H. J., et al. (2018). Virtual-reality-based cognitive behavioural therapy versus waiting list control for paranoid ideation and social avoidance in patients with psychotic disorders: a single-blind randomised controlled trial. Lancet Psychiatry 5, 217–226. doi: 10.1016/S2215-0366(18)30053-1
Riches, S., Bird, L., Chan, N., Garety, P., Rus-Calafell, M., and Valmaggia, L. (2020). Subjective experience of paranoid ideation in a virtual reality social environment: A mixed methods cross-sectional study. Clin. Psychol. Psychother. 27, 337–345. doi: 10.1002/cpp.2431
Riva, G. (1997). Virtual reality as assessment tool in psychology. Stud. Health Technol. Inform. 44, 71–79. doi: 10.3233/978-1-60750-888-5-71
Rus-Calafell, M., Garety, P., Sason, E., Craig, T. J. K., and Valmaggia, L. R. (2018). Virtual reality in the assessment and treatment of psychosis: a systematic review of its utility, acceptability and effectiveness. Psychol. Med. 48, 362–391. doi: 10.1017/S0033291717001945
Schubert, T., Friedmann, F., and Regenbrecht, H. (2001). The experience of presence: Factor analytic insights. Presence Teleoperators Virtual Environ. 10, 266–281. doi: 10.1162/105474601300343603
Skarbez, R., Brooks, F. Jr., and Whitton, M. (2017). A Survey of Presence and Related Concepts. ACM Comput Surv 50 96:39. doi: 10.1145/3134301
Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 364, 3549–3557. doi: 10.1098/rstb.2009.0138
Thong, I. S. K., Foo, M. Y., Sum, M. Y., Capps, B., Lee, T. S., Ho, C., et al. (2016). Therapeutic misconception in psychiatry research: A systematic review. Clin. Psychopharmacol. Neurosci. 14, 17–25. doi: 10.9758/cpn.2016.14.1.17
Tian, N., Lopes, P., and Boulic, R. (2022). A review of cybersickness in head-mounted displays: raising attention to individual susceptibility. Virtual Real. 26, 1409–1441. doi: 10.1007/s10055-022-00638-2
van Bennekom, M. J., de Koning, P. P., and Denys, D. (2017). Virtual reality objectifies the diagnosis of psychiatric disorders: A literature review. Front. Psychiatry 8:163. doi: 10.3389/fpsyt.2017.00163
Vears, D. F., and Gillam, L. (2022). Inductive content analysis: a guide for beginning qualitative researchers. Focus on Health Professional Education: A Multi-Professional J. 23, 111–127. doi: 10.11157/fohpe.v23i1.544
Veling, W., Counotte, J., Pot-Kolder, R., Van Os, J., and Van Der Gaag, M. (2016). Childhood trauma, psychosis liability and social stress reactivity: A virtual reality study. Psychol. Med. 46, 3339–3348. doi: 10.1017/S0033291716002208
Wechsler, D. (1996). Manual for the Wechsler Adult Intelligence Scale–Revised Swedish Edition. Stockholm: Psykologiförlaget.
Wechsler, D., and Nyman, H. (2003). WAIS III: Wechsler Adult Intelligence Scale, 3rd Edn. Stockholm: Psykologiförlaget.
Keywords: virtual reality, assessment, diagnostics, paranoia, psychiatry, mental disorders, forensic psychiatry
Citation: Hedström R, Wallinius M, Sygel K and Geraets CNW (2023) Virtual reality-assisted assessment of paranoid ideation in forensic psychiatric inpatients: A mixed-methods pilot study. Front. Psychol. 14:1242243. doi: 10.3389/fpsyg.2023.1242243
Received: 22 June 2023; Accepted: 30 October 2023;
Published: 07 December 2023.
Edited by:
Paolo Taurisano, University of Bari Aldo Moro, ItalyReviewed by:
Valeria Sebri, European Institute of Oncology (IEO), ItalyAnson Kai Chun Chau, The Chinese University of Hong Kong, Hong Kong SAR, China
Klaudia Proniewska, Jagiellonian University Medical College, Poland
Copyright © 2023 Hedström, Wallinius, Sygel and Geraets. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Chris N. W. Geraets, c.n.w.geraets@umcg.nl