Daniel Eckhoff
Christian Sandor 
Gladys L. Y. Cheing
Jan Schnupp
Alvaro Cassinelli- 1School of Creative Media, City University of Hong Kong, Kowloon, Hong Kong SAR, China
- 2University of Paris-Saclay, Gif-sur-Yvette, France
- 3Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
- 4Department of Neuroscience, City University of Hong Kong, Kowloon, Hong Kong SAR, China
Augmented Reality (AR) overlays computer-generated visual, auditory or other sensory information onto the real world. Due to recent technological advancement in the field, it can become increasingly difficult for the user to differentiate between sensory information coming from real and virtual objects, leading to interesting perceptual phenomena. For example, an AR experience in which users can experience their own hands in flames has been shown to elicit heat illusions on the affected hands. In this study, we investigate the potential that AR has for top-down modulation of pain and thermal perception. We assessed thermal pain and detection thresholds on the participant’s right hand while covering it with realistic virtual flames. We compared this experience to a baseline condition with no additional stimuli. We also report on a condition in which the hand is covered by a blue fluid not instantly associated with fire. We found that experiencing a virtual burning hand induces analgesic as well hyperalgesic effects as participants begin to feel heat related pain at lower temperatures and cold related pain at higher temperatures. The experience also impacts significantly on the lowest temperature at which participants starts perceiving warmth. The blue fluid do not affect the thresholds corresponding to the baseline condition. Our research thus confirms previous experiments showing that pain and thermal perception can be manipulated by by AR, while providing quantitative results on the magnitude of this effect.
1 Introduction
Augmented Reality (AR) is an emerging interactive technology that changes the user’s perception of the real world by overlaying virtual sensory stimuli, or through the modification or removal of sensory information of real objects (Azuma et al., 2001). When these virtual sensory stimuli are coherently merged with the real world and experienced in an immersive manner (e.g., through Head-Mounted Displays), they can induce a plausibility illusion (Slater et al., 2009), thereby suspending disbelief and triggering a natural associated response. For example, Weir et al. (2013) and Eckhoff et al. (2020) have reported that an AR experience during which users experience their own hands covered in flames create involuntary heat and olfactory illusions. This raises the question of the extent to which this AR-induced cross-modal illusion can modulate thermal perception.
One way to assess thermal perception is to determine thermal perception thresholds. Thermal perception thresholds correspond to the temperatures at which a person begins to perceive a thermal stimulus as warm or cold, while thermal pain thresholds are the temperatures at which a person begins to perceive heat or cold as painful. These thresholds are usually determined by applying a thermal stimulator to the skin, slowly increasing or decreasing the temperature until the subject reports a thermal sensation, and the start of a painful sensation triggered by heat or cold respectively. Threshold values are influenced by a number of factors, including the method of testing (Defrin et al., 2006), the rate of heating, (Yarnitsky and Ochoa, 1990), the ambient temperature (Strigo and Carli, 2000), the location on the skin (Defrin et al., 2006) or skin temperature (Pertovaara et al., 1996). Threshold can also vary among participants and depend on their gender (Rhud and Meagher, 2001; Averbeck et al., 2017) or medical condition (Curković et al., 1993).
One of the most important cognitive-behavioural techniques to relieve pain is distraction, as suggested by the gate control theory of Melzack and Wall (1965). That is also why Virtual Reality (VR) has been the object of a lot of research in pain perception. In contrast to AR, VR lets the user perceive only virtual sensory information, without information from the real world. AR and VR experiences can induce significant feelings of presence, changing the user’s behavior as if the experience was real (Sanchez-Vives and Slater, 2005; Slater et al., 2009; Gonzalez-Franco and Lanier, 2017). The sense of ‘being there’ - immersed in a virtual environment - is responsible for a significant level of distraction, which is why VR has attracted much more research on pain perception than AR, even in clinical settings Pimentel et al. (2021); Wilson and Scorsone (2021). For example, Hoffman et al. (2000, 2004) placed patients suffering from burn injuries in a virtual snow world and demonstrated analgesic effects through subjective and fMRI measurements. However, it is yet unclear what type and/or which elements in these virtual environments are crucial to elicit these subjective experiences. For example, Mühlberger et al. (2007) had participants move through a virtual winter (snowy) landscape as well as through a predominantly yellow and red autumn landscape, and found similar analgesic effects, hinting at the possibility that the analgesic effect is rather a consequence of distracting the participant away from their pain.
A few articles examine the role of distraction in VR on pain perception by controlling the degree of cognitive load. Surprisingly, Demeter et al. (2018) reported that there was no significant difference in pain reduction between VR experiences with high and low cognitive load. Both seem to reduce pain perception compared to a neutral control. Subsequent studies by Barcatta et al. (2022) found that participants who report a higher level of emotional distress exhibit higher pain thresholds at lower cognitive load. They conclude then that individual differences should be taken into account when designing VR treatments for analgesic purposes.
Cognitive factors or cross-modal interactions can also greatly modulate the perception of pain. Longo et al. (2009) reported that looking at one’s own body part in real pain actually reduces that pain intensity. They delivered potentially painful stimuli to a participant’s hand while the participants were looking at their own hand, a box, or somebody else’s hand. Looking at one’s own had was associated with the lowest reported pain ratings. Similar results have been shown when participants were looking at an embodied hand during a Rubber Hand Illusion (RHI) experiment (Botvinick and Cohen, 1998). In the original RHI experiment, while a subject observes a rubber hand being stroked by a brush, the subject’s real hand, out of view, is also simultaneously stroked. The congruence between the visual and tactile stimuli leads to an illusory sense of ownership over the rubber hand, as well as a perceptual disconnect from the real limb. Subsequent works report analgesic effects on heat stimuli during the RHI (Valenzuela-Moguillansky et al., 2011; Hegedüs et al., 2014). Giummarra et al. (2015) argues that these effects may arise from a reduced awareness and disownership of one’s own body parts, as the subject attends to a healthy and pain-free limb. In their experiment, they could also show the reverse effect, that is, an increase sensitivity to pain or hyperalgesia by presenting a fake hand covered with burn injuries. Likewise, Bauer et al. (2018) found that the mechanical pain threshold was lowered as the embodied rubber hand appears to be threatened by pricking it with a sharp knife.
In the realm of VR, researchers have examined how pain perception can be modulated by the appearance and interaction of embodied body parts, i.e., a virtual representation of a limb collocated with the real limb in egocentric space. Martini et al. (2013) used a VR system that paints a virtual embodied VR arm respectively with red, blue, or green. In their experiment, the reddened skin slightly decreased HPT, while the blue skin increased it. The experimental conditions hints at a different mechanism for pain modulation where distraction plays a secondary role compared to virtual embodiment. In general, the effect of embodiment in VR on pain perception has been the subject of numerous studies (Martini et al., 2015; Martini, 2016; Käthner et al., 2019; Hoffman, 2021).
These results are consistent with the fact that viewing an embodied limb in pain activates pain-related brain areas González-Franco et al. (2014). It has been shown that just looking at painful images can trigger spinal nociceptive responses (Vachon-Presseau et al., 2011), as well as activate somatosensory and motor brain regions and pathways involved in processing the experience of pain (Fan and Han, 2008; Lamm et al., 2011). What is more, the mere thought of pain activates pain-processing brain regions such as the anterior cingulate cortex, medial prefrontal cortex, and anterior insula (Fan and Han, 2008; Lamm et al., 2011).
Gandy et al. (2010) points out a key difference between AR and VR experiences: with AR, the participant can directly observe their own body and its movement in real time, which is not possible in VR. For this reason, AR has been used as a successful therapy to treat phantom pain (Carrino et al., 2014; Ortiz-Catalan et al., 2016; Dunn et al., 2017) and phobias (Juan et al., 2005; Botella et al., 2010; Baus and Bouchard, 2014).
In VR, the experience can either be completely disembodied, or participants can embody limbs of a virtual avatar. However, virtual embodiment is a fragile perceptual experience that is not fully experienced by every subject Dewez et al. (2019). Moreover, it is difficult to integrate VR experiences into everyday life because the real world cannot be perceived simultaneously.
While AR experiences might not be able to distract participants from pain as much as VR can, AR allows participants to perceive augmentations close to, or over their actual body. Experiencing virtual flames on one’s hand in AR has been shown to reliably induce illusory heat sensations in some participants, we decided to investigate the extent to which AR can modulate both thermal pain and pain detection thresholds in healthy volunteers.
In particular, we wanted to address the following research questions: RQ 1: Does displaying virtual flames on the user’s hand lead to a change in Cold Pain Thresholds (CPT) and Heat Pain Thresholds (HPT)? RQ 2: Does displaying virtual flames on the user’s hand lead to a change in Cold Detection Thresholds (CDT) or Warm Detection Thresholds (WDT)?
2 Materials and methods
To investigate the effect of a visual-auditory illusion (representing fire) on cutaneous sensations of heat and cold, we used Quantitative Sensory Tests (QSTs) to measure temperature thresholds for painful and non-painful thermal stimuli.
In a within-subject design study, participants were exposed to three different visual-auditory conditions: realistic fire, blue fluid, and a baseline condition with no AR manipulation of the visual-auditory input. This allowed us to quantitatively examine how thermal sensitivity and pain thresholds were modulated by our AR system.
All conditions involved participants wearing a Video See-Through Head-Mounted Display (HMD). We asked all eligible participants 21) to keep looking at their dominant right hand while we measured the four QST values described above (WDT, CDT, HPT, and CPT). The setup and stimulus conditions are illustrated in Figure 1. In the Fire condition, participants experienced their own hands as burning in red fire (Figure 1C). The experienced involved simulating fire flames espousing the shape and precisely following the movement of the participant’s real hand. The visuals were complemented with the (spatially located) sound of a burning torch. While the main goal of the experiment is to produce quantitative results concerning the impact of AR-induced heat illusions on pain perception, we also took the opportunity to introduce an Blue (Figure 1D) condition that could contribute to a qualitative understanding on the level of realism that is actually necessary for such heat illusions to take place.
FIGURE 1. The experimental setup and the three experimental conditions: the participant sees the graphics covering their hand through the AR headset. (A) Side view of the participant wearing the AR headset, left arm over a computer mouse, right arm resting on a cushion, with the thermal stimulator in contact with the thenar prominence of the right hand. (B) Baseline condition. Participant is looking through the headset on their right hand with no additional auditory and visual stimuli. (C) Fire condition, experiencing their own hand engulfed in a highly realistic simulation of flames. (D) Blue condition, experiencing somewhat flame like visuals covering their hand, but with a “cool” blue color and falling rather than rising dynamics.
Our Blue condition resembles the realistic red fire in several ways, engulfing the hand in an identical way and moving with essentially the same dynamics, but the experience is distorted to render it ambiguous: the ‘flames’ fall downwards and present a bluish color atypical of flames emanating from burning materials rich in carbon. Also, the visuals are accompanied with the sound of a howling wind, creating a disassociation with a hot fire.
Finally, we compared these two conditions to a baseline, neutral condition in which participants observed their right hand through the AR headset without any superimposed visual or auditory stimuli.
2.1 Participants
We recruited twenty-four right-handed healthy participants (range 19–65 years old, average 26 ± 11, 9 identified as female, 14 identified as male) from our university. All participants signed a consent form prior to their participation. The work was conducted in accordance with protocols approved by the Human Subjects Ethics Sub-Committee of the City University of Hong Kong. We offered 100 Hong Kong Dollars as a compensation for their participation. All subjects self-reported as healthy with no hearing impairment. Before the experiment, we examined their color and stereo vision. We used a standard Ishihara color test on a calibrated LCD screen to diagnose red-green color deficits, and a Random Dot stereo test (Stereo Optical Company Inc, Chicago, United States) set to 200 arcsec minimum level of stereopsis. We excluded three of the twenty-four participants due to color-blindness or insufficient stereo vision.
2.2 Augmented reality system
To experience AR, the participants wore a HMD that allowed them to see 3D computer-generated visual stimuli superimposed on their real-world view. We chose to use a Video See-Through HMD. The device uses a pair of frontal cameras mounted on the display to capture the real world that is then electronically blended with computer-generated imagery (Rolland et al., 1995). This allows for the creation of visually coherent virtual graphics (Collins et al., 2017; Itoh et al., 2021). We used the Varjo XR-3 (Varjo, Oslo, Finland) with a pixel density of 3,000 pixels per inch, a refresh rate of 90 Hz, horizontal field of view 115° and vertical 80°. The dual front-facing cameras of the XR-3 have a resolution of 12 MP. We used the Valve Lighthouse tracking system to track the pose of the headset in space. To deliver sound, we used open over-the-ear headphones (Beyerdynamic GmbH and Co. KG, Heilbronn, Germany).
Our interactive AR platform captures the volumetric representation and movement of the participant’s hand in real-time. We achieved that with the Ultraleap hand tracking sensor (Leap Motion, Inc, San Francisco, California, United States) of the Varjo XR-3 together with the Ultraleap SDK. For both conditions, as seen in Figures 1C,D, we developed a real-time fluid simulation based on the Navier–Stokes equations. For the Fire condition, we created a physically correct fire simulation. The Blue condition differs in color as well as the fluid dynamics used in a way that does not resemble usual fire. We developed the whole AR experience inside the Unreal engine (Epic Games, Cary, North Carolina, United States) and used FMOD (Firelight Technologies Pty Ltd., Melbourne, Australia) for playing back realistic spatial sound effects.
2.3 Thermal stimulation
To assess thermal detection or pain thresholds, we followed the Quantitative Sensory Testing (QST) protocol of the German Research Network on Neuropathic Pain (Rolke et al., 2006). Noxious and non-noxious thermal stimuli were delivered using a commercially available thermal stimulator (Medoc TSA-2001; Ramat Yishai, Israel). This device uses a Peltier thermode with a stimulating area of 30 mm2. The thermode was placed on the C7 hand dermatome (palmar of the second metacarpal) of the dominant right hand. Thresholds were determined via a method of limits.
For each sensory measurement, the visuals, sounds, and temperature ramps began at the same time. Cold and warm detection thresholds were measured first (CDT, WDT). Then cold pain and heat pain thresholds were determined (CPT, HPT). This order of measurements are defined by the German Research Network on Neuropathic Pain and are highly reproducible and were not found to influence each other (Rolke et al., 2006; Heldestad et al., 2010).
The thermal stimulator provided ramped stimuli
2.4 Subjective ratings
We additionally assessed their level of motion sickness after the experiment through a Simulator Sickness Questionnaire (SSQ) (Kennedy et al., 1993). The SSQ is a self-reported list of 16 symptoms that participants are asked to rate in severity in a 4-point scale (from “absent” to “strong”). The questionnaire yields a total score and sub-scores on nausea, oculomotor symptoms and disorientation. It is an established method to assess symptoms after simulator use and is widely applied in virtual reality research (Cobb et al., 1999; Davis et al., 1999; Green, 2004; Moss and Muth, 2011).
A final post-study questionnaire assessed the level of presence of the experience delivered by the AR system. The goal is to assess whether they had the impression that the virtual sensory stimuli were part of the real environment. To measure the level of presence and immersion, we adapted the presence questionnaire used in (Regenbrecht and Schubert, 2021). Our presence questionnaire consists of five 7-point Likert scale questions namely Q1: How natural was the appearance of the fire on your right hand? Q2: How much did the visual display quality interfere or distract you from observing and interacting with the visuals covering your right hand? Q3: Did you have the impression of seeing the virtual objects as merely flat images or as three-dimensional objects? Q4: Did you pay attention at all to the difference between real and virtual objects? Q5: Did the virtual objects appear to be on a screen, or did you have the impression that they were located on your hand)?
2.5 Procedures
We conducted the experiment in a sound-proof room in order to keep distractions at a minimum. A single experimenter performed all the tests. Participants sat on a chair in front of a black table as seen in Figure 1A. The temperature of the air-conditioned room was maintained at 23°C. To get acquainted with the procedure, each subject performed a testing session consisting of one HPT and one CDT measurement. We presented three different stimuli in AR to all participants (See Figure 1), with either no additional graphics applied for baseline measurements (Figure 1B), realistic virtual flames covering the right hand (Figure 1C), or dark blue visuals covering the hand (Figure 1D). We asked the participant to keep looking at their hand in each condition. The order of the conditions were randomized to minimize carry-over effects (Yarnitsky and Ochoa, 1990; Greffrath et al., 2007). In each condition, we measured the CDT, CPT, WDT, and HPT. The same order of sensory measurements was used for all participants. We performed the statistical analysis in Python 3.10 using the SciPy 1.6.0 package.
3 Results
3.1 Thermal pain and detection thresholds
Figure 2 depicts the outcomes of the thermal QST (HPT, CPT, WDT, CDT) for all three conditions. Threshold measurements (in °C) were averaged across subjects for each of the three conditions. Because of the small sample size, determining the distribution of pain thresholds (HPT, CPT) was important for choosing an appropriate statistical method. Therefore, we performed a Shapiro-Wilk test, which showed that the CPTs could be considered normally distributed (p > 0.05) in all conditions. We conducted one-way repeated-measures ANOVA (one factor: “Condition” with three levels) on mean CPTs. Post hoc analysis was conducted with Tukey HSD tests. The effect size of all significant results was also calculated according to Cohen’s coefficient (Cohen’s d). Since sphericity is violated for CPT (ϵ = 0.76), Huyn-Feldt corrected results are reported. The one-way repeated measures ANOVA showed an effect of the factor Condition for the CPT (F1.44,28.79 = 7.010, p = 0.007). Post hoc tests have shown that the condition Fire significantly increased the CPT (p = 0.043, d = 0.35, M = 1.84, SD = 2.37) compared to the Baseline condition and compared to the Blue condition (p = 0.002, d = 0.5, M = 2.62, SD = 4.2).
FIGURE 2. Mean thermal pain and detection thresholds for each condition. The whiskers denote the standard error. Connected bars represent significant difference
However, HPTs significantly diverted from the normal distribution (p = 0.043) during the Fire condition and were therefore analyzed using non-parametric Wilcoxon Signed-Rank tests. They revealed that experiencing virtual flames on one’s hand in the condition Fire lead to a significant lower HPT p < 0.001, d = 0.51, M = 1.55, SD = 1.66) compared to the Baseline and to the Blue condition (p = 0.007, d = 0.47, M = 1.43, SD = 2.22). There was no significant difference in the Baseline and Blue condition (p = 0.81, M = 0.12, SD = 1.95).
We found the detection thresholds (WDT, CDT) to be not normally distributed (Shapiro-Wilk test (p < 0.05)). In particular, WDTs were found to be positively skewed, whereas CDTs were found to be negatively skewed. These were therefore pairwise analyzed using a non-parametric Wilcoxon Signed-Rank test. The burning hand significantly decreased the WDT (p < 0.001, d = 0.69, M = −1.26, SD = 1.38) compared to the Baseline. We found no other significant differences.
3.2 Simulator sickness questionnaire
The mean total score of the SSQ of 17.28 ± 3.81 indicated only little to no motion sickness effects induced by our experimental platform. Fatigue and salivation increasing were the most frequently reported symptom (n = 2). General discomfort, eyestrain, difficulty focusing, nausea, fullness of the head and dizziness with eyes open was reported by one participant in each case. All of the symptoms were rated as slight and none as moderate or strong.
3.3 Presence
On a questionnaire measuring presence in AR (7 point Likert scale), adapted from the work by Regenbrecht and Schubert (2021), participants reported experiencing a high degree of presence (5.30 ± 1.40), indicating that our AR system created an effective illusion through virtual graphics that were well registered with the real world.
To investigate whether the presence score and the perceived realism of the experience affected the thresholds, we calculated correlations (Pearson’s r) for all differences in the threshold measures compared with baseline. However, we found no significant correlations.
3.4 Paradoxical heat illusion
Although we did not specifically ask about paradoxical heat sensations, four of the 21 participants reported the sensation of paradoxical heat. They confused a decreasing temperature ramp during a threshold measurement with an increasing one in the Fire condition.
4 Discussion
The major finding of this study is that visual-auditory AR can effectively modulate nociception and thermoception of real thermal stimuli. Exposure to the burning hand in AR leads to (1.) a significantly lower HPT and CPT, and (2.) to a significantly lower WDT. The Blue condition does not change thresholds significantly compared to the baseline.
Our observations confirm that our AR experience can exert both hyperalgesic and analgesic effects. Seeing one’s hand on fire changes the observer’s sensitivity such that less heat is needed to produce sensations of heat pain, but also such that more cold is required to induce sensations of cold pain. In addition, less heat is needed to trigger sensation of warmth. The cross-modal effects thus modulated cutaneous perception in multiple ways. Our finding of hyperalgesic and analgesic changes is in accordance with previous studies done in VR (Martini, 2016). To the best of our knowledge, this is the first study to show a top-down modulation of HPT, CPT and WDT in AR.
There has been a big evidence for the red–warm, blue–cold association. It has been shown that children as young as about 9 years of age associate the color red with warmth and blue with cold (Morgan et al., 1975). These colour-temperature associations can even lead to some thermoregulatory responses. Rugierri and Petruzziello found that body temperature increased significantly when participants looked through red, orange, and yellow glasses (Ruggieri and Petruzziello, 1988). Blue, green or violet did not lead to a significant change in body temperature. Moseley and Arntz have shown that a noxious stimulus, when associated with a red cue, hurts more and is actually perceived as hotter, whereas the opposite is true when associated with a blue cue (Moseley and Arntz, 2007). In a study, more closely related to our’s, by Ho et al. (2014), the WDT was found to be lowered when the color of participants’ real hand was changed to red by a projector. Interestingly, a blue hand increased WDT. We were able to reproduce the results in our Fire condition. However, in our case, the Blue condition did not significantly change any thermal detection or thermal pain threshold, even though the cue was blue. This could indicate that the dynamics of the flames lead to some perceptual ambiguity.
Our work also raises the question whether, or how strongly, the location of the virtual fire affects thermal perception. Would a virtual fire at some distance from the thermal stimulator, or even just the presence of flames in the nearby environment, also lead to a change in HPTs? While Martini et al. (2013) used a VR system that superimposes a red, blue, or green color on a virtual embodied arm to show that the reddened arm significantly decreased HPT and a blue one increased it, they also included a condition involving a virtual stimulus presented outside the limb (a coloured spot), and observed that this did not lead to a change in HPT. Their study therefore shows the importance of a virtual stimulus that is co-located with the real arm (e.g., an “embodied” experience). Likewise, González-Franco et al. (2014) showed that, when an embodied hand in VR is stabbed with a knife, pain-related brain activity can be observed in the EEG. However, when the knife misses the hand and ends up stabbing a table, this pain-related activity is greatly reduced. Given these results by Martini et al. (2013) and González-Franco et al. (2014), we would strongly predict that the changes in pain and detection thresholds we observed would be reduced, and may disappear entirely, if the fire was not displayed at “a safe distance from”, rather than on, the hand. The need to keep the number of stimulus presentations within each experimental session small prevented us from testing this prediction within the scope of this initial study. While quantitative sensory measurements have been shown to have high day-to-day repeatability, habituation may occur between successive measurements due to peripheral receptor fatigue, especially in HPTs Agostinho et al. (2009).
The simple yet effective virtual stimulus employed by Martini et al. (2013) challenges the assumption that a high degree of realism is needed to produce modulatory effects. However, it is interesting to note that our experiment induced changes in pain thresholds that were much larger than those reported in their work. Their HPT decreased by around 0.4°C, while in our study the average decrease was 1.55 °C. We therefore hypothesize that the level of realism and the intensity of the heat suggested by the visual-auditory illusion may be positively correlated with how strongly cutaneous heat sensations are modulated. This seems all the more likely given that it has already been shown that the degree of realism and fidelity of graphics inside a virtual environment positively correlate with the level of the stress response that it can trigger in participants (Slater, 2009; Weiß et al., 2021). A vivid image of flames may be particularly effective, as this is universally recognized as a potential threat (Erlich et al., 2013). Additionally, the experience of realistic, simulated flames has been shown to elicit involuntary heat illusions in participants (Weir et al., 2013; Eckhoff et al., 2020), and we suspect that there could be causal relationships between these involuntary illusions and the strong modulatory effects on HPT, CPT and WDT we describe here, even if the nature and direction of these causal links remain unclear.
The fact that we observed much greater changes in thresholds than by Martini et al. (2013) could also be due to our use of AR rather than VR technology. Indeed, the illusion requires that the participant “embodies” the limb whose image is manipulated, something that is easier to do when the limb in question is the real one, instead of an embodied representation in VR generated by computer graphics. Perani et al. (2001) demonstrated that different neural networks are activated when participants observe either a real hand or a virtual hand in different degrees of realism. In the cases of the virtual hands, they found only limited differences in activation due to the different degrees of realism, compared to the response of viewing the real hand. In addition, Kanayama et al. (2021) compared participants’ oscillatory neural activity using EEG during a rubber hand illusion in VR with a virtual rubber hand and with a real rubber hand without the use of VR. Although both experimental setups were able to evoke the RHI, they found different oscillatory activities in each condition, suggesting that VR embodiment alters basic perceptual mechanisms. Therefore, results from VR experiments with virtual embodied bodies may not be directly comparable with similar experiments performed in AR.
As explained earlier, the Blue condition was intentionally designed not to create a heat illusion by changing the color and dynamics of the visuals and adding a howling wind noise that might suggest cold rather than heat. That condition did not show the same strong modulatory effects on pain and thermal perception, which adds weight to our interpretation of the results that the observed changes in pain modulation do not come from a distraction effect, but rather from the participants’ perception of the red visuals as indicative of the presence of real red fire covering their hand. What we do not yet know is how important the role of the sound was in our experiment. What would have happened, for example, if the sound effects for the two conditions had been switched, so that the hot looking fire visual would have been paired with the sound of a cold wind, or the realistic fire sound had been paired with the blue visuals that did not look like “real” fire? Perhaps some of our participants might have interpreted the blue visuals as fire if they had been accompanied by a realistic crackling fire sound.
Our perception and responses are shaped by innate factors and experiences gained from life (Welch and Warren, 1980; Ernst and Bülthoff, 2004; Shams and Beierholm, 2010). For example, the sound of a crackling fire is easily recognizable and well-understood, even by infants. Erlich et al. (2013) found that infants demonstrated significantly enhanced heart rate deceleration, larger eye-blinks, and more visual orienting when listening to evolutionary fear-relevant sounds - including crackling fire. Here, seeing and hearing the fire could have triggered similar responses, increasing alertness and pain perception. Rhud and Meagher (2001) found that exposure to loud noise resulted in a fear-related hypoalgesia in women but more surprise-related hyperalgesia in men. However, in their study, they used noise sounds, rather than innate fear-related sounds. In light of the above, an unrealistic blue visuals accompanied by the sound of a howling wind represents an ambiguous stimuli and may not trigger an automatic response (learned or otherwise), explaining its negligible effect on heat perception modulation. Interestingly though, the fact that this synthetic experience is unusual suggests that it can be used as a control for distraction in our experiment, but further research is needed in this direction.
As reported earlier, four of our participants spontaneously reported a paradoxical heat illusion during a cold stimulus so strong as to prompt them to interrupt the experiment and question the correct functioning of the thermal stimulator. This only happened in the fire condition, suggesting that the paradoxical effect is correlated to the visualization of the moving flames. Paradoxical heat illusions have been reported in as much as 35% of participants during a cold stimulus threshold experiments with pre-heating of the hand, and only 9.8% without pre-heating (Hämäläinen et al., 1982). A 19% (4 out of 21) occurrence of the paradoxical effect in our experiment may then suggest that the virtual imagery could also induce the illusion of real pre-heating, an interesting question left for further studies.
Given the lack of available effect sizes for modulating effects of this AR experience on HPT, CPT, WDT, and CDT data, we did not perform a priori sample size calculations. In our study, we did not observe a change in CDTs. This could be since perception of cold and warm stimuli involve different neurobiological mechanisms (Patapoutian et al., 2003), presumably responding differently to cross-modal stimulation. A larger experimental sample size may be required to detect more subtle effects on the modulation of the CDT.
5 Conclusion
We believe this study provides valuable insights on the potential of AR to bias thermal perception as well as thermal pain perception, including a quantitative evaluation of analgesic as well hyperalgesic effects. We found that AR-induced heat-illusions create analgesic as well hyperalgesic effects as participants begin to feel heat related pain at lower temperatures and cold related pain at higher temperatures. The experience also impacts significantly on the lowest temperature at which participants starts perceiving warmth. This opens up the fascinating perspective of being able to use carefully crafted AR experiences to modulate and mitigate clinical pain. AR pain treatments would be entirely non-invasive and very safe, and our results give reason to believe that they could also be quite effective. Compared to VR, AR did not yet attract much attention in pain research. So, it is our hope that his article will lead to more studies exploring possible clinical pain interventions.
Data availability statement
The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by the Human Subjects Ethics Sub-Committee of the City University of Hong Kong. The patients/participants provided their written informed consent to participate in this study.
Author contributions
DE conceived the study, implemented the experimental setup and collected and analyzed the data. DE, AC, GC, and JS interpreted the data. DE, AC, CS, and JS designed the experiment. DE wrote the manuscript with the help of all authors. All authors revised the manuscript up to the final version.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Agostinho, C. M. S., Scherens, A., Richter, H., Schaub, C., Rolke, R., Treede, R.-D., et al. (2009). Habituation and short-term repeatability of thermal testing in healthy human subjects and patients with chronic non-neuropathic pain. Eur. J. Pain 13, 779–785. doi:10.1016/j.ejpain.2008.10.002
Averbeck, B., Seitz, L., Kolb, F. P., and Kutz, D. F. (2017). Sex differences in thermal detection and thermal pain threshold and the thermal grill illusion: A psychophysical study in young volunteers. Biol. Sex. Differ. 8, 1–13. doi:10.1186/s13293-017-0147-5
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., and MacIntyre, B. (2001). Recent advances in augmented reality. IEEE Comput. Graph. Appl. 21, 34–47. doi:10.1109/38.963459
Barcatta, K., Holl, E., Battistutta, L., van der Meulen, M., and Rischer, K. M. (2022). When less is more: Investigating factors influencing the distraction effect of virtual reality from pain. Front. Pain Res. 2, 1–11. doi:10.3389/fpain.2021.800258
Bauer, A., Hagenburger, J., Plank, T., Busch, V., and Greenlee, M. W. (2018). Mechanical pain thresholds and the rubber hand illusion. Front. Psychol. 9, 1–8. doi:10.3389/fpsyg.2018.00712
Baus, O., and Bouchard, S. (2014). Moving from virtual reality exposure-based therapy to augmented reality exposure-based therapy: a review. Front. Hum. Neurosci. 8, 1–15. doi:10.3389/fnhum.2014.00112
Botella, C., Bretón-López, J., Quero, S., Baños, R., and García-Palacios, A. (2010). Treating cockroach phobia with augmented reality. Behav. Ther. 41, 401–413. doi:10.1016/j.beth.2009.07.002
Botvinick, M., and Cohen, J. (1998). Rubber hands ‘feel’ touch that eyes see. Nature 391, 756. doi:10.1038/35784
Carrino, F., Rizzotti, D., Gheorghe, C., Kabasu Bakajika, P., Francescotti-Paquier, F., and Mugellini, E. (2014). “Augmented reality treatment for phantom limb pain,” in Virtual, augmented and mixed reality. Applications of virtual and augmented reality. Editors D. Hutchison, T. Kanade, J. Kittler, J. M. Kleinberg, A. Kobsa, F. Matternet al. (Cham: Springer International Publishing), 8526, 248–257. doi:10.1007/978-3-319-07464-1_23
Cobb, S. V. G., Nichols, S., Ramsey, A., and Wilson, J. R. (1999). Virtual reality-induced symptoms and effects (VRISE). Presence. (Camb). 8, 169–186. doi:10.1162/105474699566152
Collins, J., Regenbrecht, H., and Langlotz, T. (2017). Visual coherence in mixed reality: a systematic enquiry. Presence Teleoperators Virtual Environ. 26, 16–41. doi:10.1162/PRES_a_00284
Curković, B., Vitulić, V., Babić-Naglić, D., and Dürrigl, T. (1993). The influence of heat and cold on the pain threshold in rheumatoid arthritis. Z. Rheumatol. 52, 289–291.
Davis, E. T., Scott, K., Pair, J., Hodges, L. F., and Oliverio, J. (1999). Can audio enhance visual perception and performance in a virtual environment? Proc. Hum. Factors Ergon. Soc. Annu. Meet. 43, 1197–1201. doi:10.1177/154193129904302206
Defrin, R., Shachal-Shiffer, M., Hadgadg, M., and Peretz, C. (2006). Quantitative somatosensory testing of warm and heat-pain thresholds: The effect of body region and testing method. Clin. J. Pain 22, 130–136. doi:10.1097/01.ajp.0000154048.68273.d8
Demeter, N., Pud, D., and Josman, N. (2018). Cognitive components predict virtual reality-induced analgesia: Repeated measures in healthy subjects. Front. Robot. AI 4, 1–9. doi:10.3389/frobt.2017.00070
Dewez, D., Fribourg, R., Argelaguet, F., Hoyet, L., Mestre, D., Slater, M., et al. (2019). “Influence of personality traits and body awareness on the sense of embodiment in virtual reality,” in Proceedings - 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2019 (Beijing, China: IEEE), 123–134. doi:10.1109/ISMAR.2019.00-12
Dunn, J., Yeo, E., Moghaddampour, P., Chau, B., and Humbert, S. (2017). Virtual and augmented reality in the treatment of phantom limb pain: A literature review. NeuroRehabilitation 40, 595–601. doi:10.3233/NRE-171447
Eckhoff, D., Cassinelli, A., Liu, T., and Sandor, C. (2020). “Psychophysical effects of experiencing burning hands in augmented reality,” in Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics). Editors P. Bourdot, V. Interrante, R. Kopper, A.-H. Olivier, H. Saito, and G. Zachmann (Cham: Springer International Publishing), 12499. LNCS, 83–95. doi:10.1007/978-3-030-62655-6_5
Erlich, N., Lipp, O. V., and Slaughter, V. (2013). Of hissing snakes and angry voices: Human infants are differentially responsive to evolutionary fear-relevant sounds. Dev. Sci., 894–904. doi:10.1111/desc.12091
Ernst, M. O., and Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends Cogn. Sci. 8, 162–169. doi:10.1016/j.tics.2004.02.002
Fan, Y., and Han, S. (2008). Temporal dynamic of neural mechanisms involved in empathy for pain: an event-related brain potential study. Neuropsychologia 46, 160–173. doi:10.1016/j.neuropsychologia.2007.07.023
Gandy, M., Catrambone, R., MacIntyre, B., Alvarez, C., Eiriksdottir, E., Hilimire, M., et al. (2010). “Experiences with an AR evaluation test bed: Presence, performance, and physiological measurement,” in 9th IEEE International Symposium on Mixed and Augmented Reality 2010: Science and Technology, ISMAR 2010 - Proceedings (Seoul, Korea: IEEE), 127–136. doi:10.1109/ISMAR.2010.5643560
Giummarra, M., Fitzgibbon, B., Georgiou-Karistianis, N., Beukelman, M., Verdejo-Garcia, A., Blumberg, Z., et al. (2015). Affective, sensory and empathic sharing of another’s pain: The Empathy for Pain Scale. Eur. J. Pain 19, 807–816. doi:10.1002/ejp.607
Gonzalez-Franco, M., and Lanier, J. (2017). Model of illusions and virtual reality. Front. Psychol. 8, 1–8. doi:10.3389/fpsyg.2017.01125
González-Franco, M., Peck, T. C., Rodríguez-Fornells, A., and Slater, M. (2014). A threat to a virtual hand elicits motor cortex activation. Exp. Brain Res. 232, 875–887. doi:10.1007/s00221-013-3800-1
Green, B. G. (2004). Temperature perception and nociception. J. Neurobiol. 61, 13–29. doi:10.1002/neu.20081
Greffrath, W., Baumgärtner, U., and Treede, R.-D. (2007). Peripheral and central components of habituation of heat pain perception and evoked potentials in humans. Pain 132, 301–311. doi:10.1016/j.pain.2007.04.026
Hämäläinen, H., Vartiainen, M., Karvanen, L., and Järvilehto, T. (1982). Paradoxical heat sensations during moderate cooling of the skin. Brain Res. 251, 77–81. doi:10.1016/0006-8993(82)91275-6
Hegedüs, G., Darnai, G., Szolcsányi, T., Feldmann, Á., Janszky, J., and Kállai, J. (2014). The rubber hand illusion increases heat pain threshold: Rubber hand illusion and pain perception. Eur. J. Pain 18, 1173–1181. doi:10.1002/j.1532-2149.2014.00466.x
Heldestad, V., Linder, J., Sellersjö, L., and Nordh, E. (2010). Reproducibility and influence of test modality order on thermal perception and thermal pain thresholds in quantitative sensory testing. Clin. Neurophysiol. 121, 1878–1885. doi:10.1016/j.clinph.2010.03.055
Ho, H.-N., Iwai, D., Yoshikawa, Y., Watanabe, J., and Nishida, S. (2014). Combining colour and temperature: A blue object is more likely to be judged as warm than a red object. Sci. Rep. 4, 1–5. doi:10.1038/srep05527
Hoffman, H. G., Doctor, J. N., Patterson, D. R., Carrougher, G. J., and Furness, T. A. (2000). Virtual reality as an adjunctive pain control during burn wound care in adolescent patients. Pain 85, 305–309. doi:10.1016/S0304-3959(99)00275-4
Hoffman, H. G., Richards, T. L., Coda, B., Bills, A. R., Blough, D., Richards, A. L., et al. (2004). Modulation of thermal pain-related brain activity with virtual reality: evidence from fMRI. NeuroReport 15, 1245–1248. doi:10.1097/01.wnr.0000127826.73576.91
Hoffman, H. G. (2021). Interacting with virtual objects via embodied avatar hands reduces pain intensity and diverts attention. Sci. Rep. 11, 10672. doi:10.1038/s41598-021-89526-4
Itoh, Y., Langlotz, T., Sutton, J., and Plopski, A. (2021). Towards indistinguishable augmented reality: A survey on optical see-through head-mounted displays. ACM Comput. Surv. 54, 1–36. doi:10.1145/3453157
Juan, M. C., Alcañiz, M., Monserrat, C., Botella, C., Baños, R. M., and Guerrero, B. (2005). Using augmented reality to treat phobias. IEEE Comput. Graph. Appl. 25, 31–37. doi:10.1109/MCG.2005.143
Kanayama, N., Hara, M., and Kimura, K. (2021). Virtual reality alters cortical oscillations related to visuo-tactile integration during rubber hand illusion. Sci. Rep. 11, 1–13. doi:10.1038/s41598-020-80807-y
Käthner, I., Bader, T., and Pauli, P. (2019). Heat pain modulation with virtual water during a virtual hand illusion. Sci. Rep. 9, 1–12. doi:10.1038/s41598-019-55407-0
Kennedy, R. S., Lane, N. E., Berbaum, K. S., and Lilienthal, M. G. (1993). Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3, 203–220. doi:10.1207/s15327108ijap0303_3
Lamm, C., Decety, J., and Singer, T. (2011). Meta-analytic evidence for common and distinct neural networks associated with directly experienced pain and empathy for pain. NeuroImage 54, 2492–2502. doi:10.1016/j.neuroimage.2010.10.014
Longo, M. R., Betti, V., Aglioti, S. M., and Haggard, P. (2009). Visually induced analgesia: Seeing the body reduces pain. J. Neurosci. 29, 12125–12130. doi:10.1523/JNEUROSCI.3072-09.2009
Martini, M., Perez-Marcos, D., and Sanchez-Vives, M. V. (2013). What color is my arm? Changes in skin color of an embodied virtual arm modulates pain threshold. Front. Hum. Neurosci. 7, 438. doi:10.3389/fnhum.2013.00438
Martini, M., Kilteni, K., Maselli, A., and Sanchez-Vives, M. V. (2015). The body fades away: Investigating the effects of transparency of an embodied virtual body on pain threshold and body ownership. Sci. Rep. 5, 1–5. doi:10.1038/srep13948
Martini, M. (2016). Real, rubber or virtual: The vision of ”one’s own” body as a means for pain modulation. a narrative review. Conscious. Cogn. 43, 143–151. doi:10.1016/j.concog.2016.06.005
Melzack, R., and Wall, P. D. (1965). Pain mechanisms: A new theory: A gate control system modulates sensory input from the skin before it evokes pain perception and response. Science 150, 971–979. doi:10.1126/science.150.3699.971
Morgan, G. A., Goodson, F. E., and Jones, T. (1975). Age differences in the associations between felt temperatures and color choices. Am. J. Psychol. 88, 125. doi:10.2307/1421671
Moseley, L. G., and Arntz, A. (2007). The context of a noxious stimulus affects the pain it evokes. Pain 133, 64–71. doi:10.1016/j.pain.2007.03.002
Moss, J. D., and Muth, E. R. (2011). Characteristics of head-mounted displays and their effects on simulator sickness. Hum. Factors 53, 308–319. doi:10.1177/0018720811405196
Mühlberger, A., Wieser, M. J., Kenntner-Mabiala, R., Pauli, P., and Wiederhold, B. K. (2007). Pain modulation during drives through cold and hot virtual environments. CyberPsychol. Behav. 10, 516–522. doi:10.1089/cpb.2007.9996
Ortiz-Catalan, M., Guðmundsdóttir, R. A., Kristoffersen, M. B., Zepeda-Echavarria, A., Caine-Winterberger, K., Kulbacka-Ortiz, K., et al. (2016). Phantom motor execution facilitated by machine learning and augmented reality as treatment for phantom limb pain: A single group, clinical trial in patients with chronic intractable phantom limb pain. Lancet 388, 2885–2894. doi:10.1016/S0140-6736(16)31598-7
Patapoutian, A., Peier, A. M., Story, G. M., and Viswanath, V. (2003). ThermoTRP channels and beyond: Mechanisms of temperature sensation. Nat. Rev. Neurosci. 4, 529–539. doi:10.1038/nrn1141
Perani, D., Fazio, F., Borghese, N. A., Tettamanti, M., Ferrari, S., Decety, J., et al. (2001). Different brain correlates for watching real and virtual hand actions. NeuroImage 14, 749–758. doi:10.1006/nimg.2001.0872
Pertovaara, A., Kauppila, T., and Hämäläinen, M. M. (1996). Influence of skin temperature on heat pain threshold in humans. Exp. Brain Res. 107, 497–503. doi:10.1007/BF00230429
Pimentel, D., Kalyanaraman, S., Fillingim, R., and Halan, S. (2021). The effects of VR use on pain experienced during a tattoo procedure: A pilot study. Front. Virtual Real. 2, 1–16. doi:10.3389/frvir.2021.643938
Regenbrecht, H., and Schubert, T. (2021). Measuring presence in augmented reality environments: Design and a first test of a questionnaire. arXiv e-prints, 1–7. doi:10.48550/ARXIV.2103.02831
Rhud, J. L., and Meagher, M. W. (2001). Noise stress and human pain thresholds: Divergent effects in men and women. J. Pain 2, 57–64. doi:10.1054/jpai.2000.19947
Rolke, R., Baron, R., Maier, C., Tölle, T. R., Treede, D. R., Beyer, A., et al. (2006). Quantitative sensory testing in the German research network on neuropathic pain (DFNS): Standardized protocol and reference values. Pain 123, 231–243. doi:10.1016/j.pain.2006.01.041
Rolland, J. P., Holloway, R. L., and Fuchs, H. (1995). “Comparison of optical and video see-through, head-mounted displays,” in Photonics for industrial applications. Editor H. Das (Boston, MA), 293–307. doi:10.1117/12.197322
Ruggieri, V., and Petruzziello, M. (1988). Psychophysiology of analysis of connotative decodification: The role of thermic and somesthetic systems in decodification of chromatic stimuli. Percept. Mot. Ski. 66, 435–442. doi:10.2466/pms.1988.66.2.435
Sanchez-Vives, M. V., and Slater, M. (2005). From presence to consciousness through virtual reality. Nat. Rev. Neurosci. 6, 332–339. doi:10.1038/nrn1651
Shams, L., and Beierholm, U. R. (2010). Causal inference in perception. Trends Cogn. Sci. 14, 425–432. doi:10.1016/j.tics.2010.07.001
Slater, M., Khanna, P., Mortensen, J., and Yu, I. (2009). Visual realism enhances realistic response in an immersive virtual environment. IEEE Comput. Graph. Appl. 29, 76–84. doi:10.1109/MCG.2009.55
Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Phil. Trans. R. Soc. B 364, 3549–3557. doi:10.1098/rstb.2009.0138
Strigo, I. A., Carli, F., and Bushnell, M. (2000). Effect of ambient temperature on human pain and temperature perception. Anesthesiology 92, 699–707. doi:10.1097/00000542-200003000-00014
Vachon-Presseau, E., Martel, M. O., Roy, M., Caron, E., Jackson, P. L., and Rainville, P. (2011). The multilevel organization of vicarious pain responses: Effects of pain cues and empathy traits on spinal nociception and acute pain. PAIN® 152, 1525–1531. doi:10.1016/j.pain.2011.02.039
Valenzuela-Moguillansky, C., Bouhassira, D., and O’Regan, J. K. (2011). The role of body awareness in pain. J. Conscious. Stud. 18, 110–142.
Weir, P., Sandor, C., Swoboda, M., Nguyen, T., Eck, U., Reitmayr, G., et al. (2013). “Burnar: Involuntary heat sensations in augmented reality,” in Proceedings - IEEE Virtual Reality (Orlando, Florida: IEEE), 43–46. doi:10.1109/VR.2013.6549357
Weiß, S., Klassen, N., and Heuten, W. (2021). “Effects of image realism on the stress response in virtual reality,” in Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology (Osaka Japan: ACM), 1–10. doi:10.1145/3489849.3489885
Welch, R. B., and Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychol. Bull. 88, 638–667. doi:10.1037/0033-2909.88.3.638
Wilson, K., and Scorsone, G. (2021). The use of virtual reality Technologies to reduce anxiety and improve experience in chemotherapy patients during treatment. Front. Virtual Real. 2, 1–8. doi:10.3389/frvir.2021.695449
Keywords: augmented reality, thermoception, nociception, pain, embodiment, presence
Citation: Eckhoff D, Sandor C, Cheing GLY, Schnupp J and Cassinelli A (2022) Thermal pain and detection threshold modulation in augmented reality. Front. Virtual Real. 3:952637. doi: 10.3389/frvir.2022.952637
Received: 25 May 2022; Accepted: 23 August 2022;
Published: 14 September 2022.
Edited by:
Jens Grubert, Hochschule Coburg, GermanyReviewed by:
Lisa-Marie Vortmann, University of California, San Diego, United StatesVerena Biener, Hochschule Coburg, Germany
Copyright © 2022 Eckhoff, Sandor , Cheing, Schnupp and Cassinelli. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Daniel Eckhoff, ZGFuaWVsLmVja2hvZmZAZ21haWwuY29t