Séamas Weech
Claudia Martin Calderon
Michael Barnett-Cowan- Department of Kinesiology, University of Waterloo, Waterloo, ON, Canada
Sensory dynamics can be re-shaped by environmental interaction, allowing adaptation to altered or unfamiliar conditions that would otherwise provoke challenges for the central nervous system. One such condition occurs in virtual reality, where sensory conflict is thought to induce cybersickness. Although the sensory re-weighting process is likely to underlie adaptation to cybersickness, evidence of a link between sensory re-weighting dynamics and cybersickness is rare. Here, we characterize the relationship between sensory re-weighting in a balance control task and cybersickness. Participants were exposed to visual oscillation while standing in tandem stance. The sway path length of the center of pressure (COP) was measured and averaged for each level of visual oscillation, and a ratio was computed between high and low oscillation magnitudes to reflect the relative contributions of multiple sensory sources of information concerning balance control. Results showed a significant relationship between the magnitude dependency of sway and common sub-scales of cybersickness: disorientation [r(21) = 0.45, p = 0.028] and oculomotor discomfort [r(21) = 0.45, p = 0.033]. We conclude that participants who reported less cybersickness were better-able to down-weight visual information at high magnitude oscillations, thus demonstrating a lower dependency between sway and visual magnitude. The results confirm the utility of balance control as an indicator of cybersickness, and support the role of multisensory re-weighting in determining an individual's tolerance to VR applications.
Introduction
Motion sickness is often experienced in conditions where abnormal relationships exist between sensory cues (Money, 1970; Reason and Brand, 1975; Reason, 1978; Oman, 1990). Habituation to motion sickness due to continued exposure to the novel conditions is thought to reflect sensorimotor learning or re-weighting, whereby the internal models linking prior expectations, motor output, and sensory feedback are updated (Oman, 1990; Oman and Cullen, 2014). Adapting internal models of sensory dynamics usually depends upon environmental interaction, whereby the statistics of a novel environment are repeatedly exposed to the central nervous system (Harris and Wolpert, 1998; Lackner and DiZio, 2005). Findings from neurophysiology suggest a highly-dispersed network of cortical and sub-cortical regions that underlie the re-weighting process (Wolpert et al., 1998; Andersen and Buneo, 2003; Block and Bastian, 2012; Medendorp et al., 2018), and specific neural units that encode and integrate unexpected multisensory cues have been identified in animal models (Brooks and Cullen, 2013; Oman and Cullen, 2014).
There is considerable evidence that sensory re-weighting occurs after exposure to conditions where the normal relationships between sensory cues are disrupted experimentally. Kitazaki and Kimura (2010) reported down-weighting of vestibular cues in postural stabilization when the cues were rendered irrelevant for estimating spatial orientation of the body. Others have shown similar effects over a period of weeks (Dilda et al., 2014). Experimental derivations of sensory weights in spatial orientation tasks reveal that patients with bilateral vestibulopathy weight vision two to three times higher than control participants, consistent with long-term adaptation effects (Alberts et al., 2017).
It has been proposed that individual variability in the speed and extent to which sensory re-weighting occurs may explain a portion of the heterogeneity in cybersickness in virtual environments (e.g., Weech and Troje, 2017; Weech et al., 2018a,b, 2019). Successful habituation to the novel conditions occurs if the internal models of sensory interactions are adapted to account for conflicts between efferent and afferent signals, or between sensory cues across channels. Such changes can occur extremely rapidly, especially when postural stability is threatened by a failure to re-weight cues (Carver et al., 2006; Jeka et al., 2010).
Evidence supports the theory that a change in sensory weights over time is associated with a change in the severity of motion sickness. Seasickness is associated with atypical sensory weights for vision, proprioception, and the vestibular sense, as measured by computerized dynamic posturography (Shahal et al., 1999). Successful habituation to seasickness following extended sea travel was associated with a reduction in vestibular weighting after 6 months that had disappeared after 12 months (Tal et al., 2010). This suggests an initial reduction in vestibular weights due to the conflict between vestibular and visual cues on-board, followed by a restoration of normal vestibular weights once habituation has occurred. On the other hand, Tal et al. (2010) also found that individuals who did not habituate to seasickness by 12 months demonstrated a linear increase in vestibular weights, indicating that a failure to down-weight vestibular cues prevents seasickness adaptation. Extended spaceflight has also been shown to lead to down-weighting of vestibular cues due to a central reinterpretation of spatial orientation cues conveyed by the vestibular system in a zero-gravity environment (Black et al., 1995; Black and Paloski, 1998). Specifically concerning cybersickness, evidence suggests a reduction in cybersickness when the sensory re-weighting process is facilitated by adding noise to a sensory channel. Results of experiments using bone-conducted vibration of the vestibular system (Weech et al., 2018a) and noisy galvanic vestibular stimulation (Weech et al., 2020b) reveal that exposure to a tonic or phasic noise stimulus leads to improved comfort in VR applications. These results are considered to reflect an optimal integration of sensory cues in conditions of uncertainty, according to statistical (Bayesian) principles of cue combination (Ernst and Banks, 2002; Ernst and Bülthoff, 2004; Butler et al., 2010).
Here we asked if the short-term dynamics of sensory re-weighting predict the experience of cybersickness in virtual reality. We characterized sensory re-weighting effects in terms of the extent to which participants' postures were affected by visual information in a balance control task. Given the dynamic nature of posture-related feedback cues, the maintenance of stable posture requires adjustments to sensory feedback cue weights from the visual, proprioceptive, and vestibular senses (Nashner and Berthoz, 1978; Bronstein et al., 1990; Horak and MacPherson, 1996; Peterka, 2002). Non-linear response properties of the balance control system suggest a role for sensory re-weighting in maintaining stability: small magnitude visual oscillations produce oscillatory postural responses with gains of ~1, but increasing visual oscillation magnitude leads to lower sway with gains of <1 (Kiemel et al., 2006).
Using an established paradigm for assessing sensory re-weighting in the control of balance (e.g., Oie et al., 2002, 2005; Allison et al., 2006; Jeka et al., 2008), we measured the extent to which visually-induced sway was modulated at different amplitudes of visual field oscillation. To this end, we derived the ratio between body sway at low- and high-amplitudes of visual field oscillation, and used this measure as an index of sensory re-weighting (Peterka, 2002). Following the balance control task, we collected measures of cybersickness produced by exposure to virtual reality content, and assessed the relationship between sensory re-weighting and cybersickness severity. Given our focus on short-term sensory re-weighting dynamics (as opposed to long-term adaptation), we examined the cybersickness response for a single bout of VR.
Our main outcome measures were twofold. First, we measured the non-linear response properties of the balance control system across visual oscillation magnitudes (small magnitude visual oscillations produce oscillatory postural responses with gains of ~1, but increasing visual oscillation magnitude leads to lower sway with gains of <1; Kiemel et al., 2006). Second, we asked participants to report subjective linear vection strength while they were exposed to radially-expanding optic flow. Vection has been implicated as a strong predictor for cybersickness (Keshavarz et al., 2015; Weech et al., 2018b; but c.f. Webb and Griffin, 2002) and as such, we assessed vection and its association with cybersickness and indices of sensory re-weighting.
Methods
Participants
Twenty-three adults (15 women, age = 21.04 (M) ± 2.74 (SD) yrs, range 18–29) participated in the study. We screened participants for exclusion criteria using a self-report questionnaire that was completed by individual participants prior to the study. Exclusion criteria for the study included any musculoskeletal disorders, balance/vestibular disorders, uncorrected hearing/visual deficits, or previous/ongoing neurological conditions (e.g., stroke). Participants were informed of all procedures and apparatuses and provided written consent. All participants provided informed written consent prior to taking part in the study. Remuneration was provided to each participant ($10 per hr). All procedures were carried out with the approval of the institutional ethics board and in accordance with the Declaration of Helsinki.
Procedure
Balance Control
In the first part of the study, we collected balance control data using a set of two force plates (4060-05; Bertec, Columbus, OH) arranged in a fore-aft layout and separated by ~1 cm. Vertical ground reaction force and moments of force were recorded using a custom-built LabVIEW program (National Instruments, Austin, TX) over a 120 s period for each trial. Force plate data were amplified online using an internal digital pre-amplifier, sampled at a rate of 1,000 Hz, and stored for off-line analysis. The force plates were calibrated before data collection for each participant. Once collected, the force plate data were low-pass filtered (6 Hz, dual-pass 2nd-order Butterworth filter), and the COP parameter (sway path length) was extracted using a custom-made LabVIEW program.
Participants were instructed to stand unshod in a “tandem” stance with one foot on each of the force plates and with the left foot behind the right foot, with the arms rested at their sides in a comfortable posture (Figure 1A). A projection screen (1 × 1.8 m; 1.3 m lens to screen distance) was positioned at a distance of 0.57 m from the participant, and we back-projected images onto the screen using a projector (1920 × 1080 resolution; PROPixx DLP, VPixx Technologies). Each participant wore a pair of goggles (80 × 50 deg visual field) to prevent their ability to see the frame of the projection screen, which would diminish visual-postural coupling and vection.
Figure 1. (A) Depiction of the balance control task. Participants stood in a modified tandem stance with hands by their sides while observing the moving visual stimulus on the screen. (B) Depiction of the VR task. Participants held a gamepad while exposed to the VR application with the head-mounted display.
In each trial, a field of 500 randomly-located dots (blue, 3° visual angle) populated the background (black). In addition, a fixation dot (purple) appeared at the participant's individually measured eye-height. Participants were asked to fixate on this purple fixation dot, and to maintain their posture throughout the task. The field of dots (and the fixation dot) always adhered to a coherent sinusoidal global motion (left-right linear translation, 0.2 Hz frequency), but the motion amplitude differed in each trial. The amplitude of oscillation for each trial was administered in a randomized order according to the method of constant stimuli, selected from four levels (4, 8, 12, and 16 mm) that were repeated three times each, resulting in 12 trials. Each trial lasted for 2 min. A further three trials of 30 s duration were conducted in the same stance, where participants viewed a vection-inducing optic flow stimulus. This consisted of 250 white dots on a black background moving at a constant 2 m/s velocity, where radial expansion, linear perspective, and relative size cues gave rise to the impression of linear translation of the observer in the anterior-posterior axis (as in Weech et al., 2020a). Participants were permitted to take breaks between trials.
Once the first 12 trials were complete, participants were exposed to three trials consisting of radially-expanding optic flow while they maintained the same tandem stance. Participants were told they might experience the sensation of illusory self-motion, “vection,” and were given the example that vection can occur when looking out of a window at a moving vehicle. Each of these trials lasted for 30 s. After each trial, participants were asked to verbally rate their experience of vection (0–10, where 0 indicates feeling no vection and 10 indicates the maximum possible vection). After all trials were completed, participants were asked if they were experiencing any discomfort or sickness symptoms (none reported any symptoms).
Virtual Reality
In the second part of the study, participants played 30 min of a VR application that has previously been identified as highly nauseogenic (e.g., Weech et al., 2018b), consisting of a zero-gravity space-walk simulation (ADR1FT, Three One Zero). The VR environment was presented with a head mounted display (Rift CV1, Oculus VR, Menlo Park, CA; 90 Hz refresh rate, 1080 × 1200 resolution per eye) and the environment was rendered by a high-end graphics card (NVIDIA GTX1070). The headset position was tracked by a combination of inertial (accelerometer/gyroscope) and optical (1 × infrared Oculus camera) sensors that were part of the commercial device package, and this movement was translated into motion of the observer viewpoint in the VR task. The packaged software of the headset was used to calibrate the capture space and the inter-pupillary distance of the headset for each participant. Participants interacted with the VR environment using a handheld gamepad (Xbox One, Microsoft). The instructions were to “explore the environment, and to investigate the interior and exterior of the space station”; these instructions aimed at encouraging dynamic exploration of the environment and exposure to nauseogenic conflicts between visual and inertial cues. A depiction of the setup is shown in Figure 1B.
During exploration in VR, cybersickness levels were collected using a quick verbal report (Fast Motion Sickness scale, FMS: “On a scale from 0 to 20 with 0 being no sickness and 20 being severe sickness, how do you feel?”; Keshavarz and Hecht, 2011). Participants were informed that they could request early termination if their sickness level became intolerable, in accordance with ethical considerations for their safety and well-being. A multi-item self-report questionnaire was completed after VR exposure (Simulator Sickness Questionnaire, SSQ; Kennedy et al., 1993). Both measures have been validated (e.g., Kennedy et al., 1993; Keshavarz and Hecht, 2011; Keshavarz et al., 2015).
While participants explored in VR, we used an electroencephalography (EEG) cap to measure their neural activity. This measurement was for the purposes of another research question and as such the results are not reported here.
Results
First, we computed the ratio between average COP path length at high (16 mm) and low (4 mm) visual oscillations, which we term the “COP ratio.” COP ratios >1 indicate higher sway at higher visual oscillation magnitudes than at low magnitudes; COP ratios <1 indicate lower sway at high visual oscillations magnitudes; and a ratio of 1 indicates equality between sway at low and high oscillation magnitudes. The average COP ratio across participants was 1.04 (SD = 0.11, range = 0.86–1.31).
Our main result was that we observed a significant, positive correlation between SSQ scores and COP ratios for both the disorientation [Pearson's r(21) = 0.45, p = 0.028] and oculomotor discomfort [r(21) = 0.45, p = 0.033] subscales. At the same time, we found no significant relationship between COP ratios and either SSQ total scores [r(21) = 0.39, p = 0.067] or the nausea subscale [r(21) = 0.31, p = 0.15] (Figure 2).
Figure 2. Correlations between COP ratios and SSQ subscales (A–C) and SSQ total score (D). Solid lines indicate linear trends. Significant correlations were observed in (A,B) (ps < 0.05).
We found no other significant relationships between COP ratios and MSSQ total scores [r(21) = 0.13, p = 0.54], mean FMS scores [r(21) = 0.23, p = 0.28], maximum FMS scores [r(21) = 0.32, p = 0.14], or the slope of FMS scores over time during the VR task [r(21) = 0.05, p = 0.83]. Similarly, there was no association between mean COP path length when we analyzed the data across all balance conditions and any of the sickness-related outcome measures [SSQ total scores (r(90) = 0.10, p = 0.35), MSSQ total scores (r(90) = −0.01, p = 0.94), mean FMS scores (r(90) = −0.11, p = 0.28; Figure 3)]. In addition, we observed that participant sex had no effect on either COP ratios [r(21) = 0.20, p = 0.36], COP path lengths (ps ≥ 0.42), FMS scores [r(21) = 0.13, p = 0.57], or SSQ scores [total: r(21) = 0.02, p = 0.94; subscales: ps ≥ 0.32].
Figure 3. (A) Change in FMS scores over time during the VR task (each participant color coded). Lines that terminate before 30 min indicate the participant asked to stop the task early. Correlations for data pooled over balance control conditions are shown with trends (blue lines). (B) SSQ total scores by mean COP length. (C) MSSQ total scores by mean COP length. (D) Mean FMS scores by mean COP length. All correlations are non-significant (ps > 0.05).
We also found no significant relationships in the vection trials between COP path length and MSSQ total scores, SSQ total scores, or SSQ subscales scores (all ps ≥ 0.18). Similarly, we found no significant relationships between verbal ratings of vection strength and MSSQ total scores, SSQ total scores, or SSQ subscales scores (all ps ≥ 0.16).
Discussion
This study aimed to assess the relationship of sensory re-weighting indices with cybersickness, as well as vection and its association with said measures. Our results propose that short-term sensory re-weighting differences are related to susceptibility to cybersickness. This relationship suggests that the non-linear response properties of the balance control system may predict susceptibility to cybersickness. Our results also suggest that measures of postural re-weighting are more sensitive to individual differences in cybersickness than linear measures, specifically COP path length during quiet stance and vection, as well as verbal ratings of vection.
Although there is a rich literature on the use of predictive models for cybersickness (Kim et al., 2005; Dennison et al., 2016; Weech et al., 2018b; Walter et al., 2019), the current results contribute to a previously sparse literature on the link between sensory re-weighting and cybersickness. One previous study examined the relationship between self-reported carsickness and the time course of habituation to galvanic vestibular stimulation (GVS) during balance control (Balter et al., 2004). Carsickness is produced due to large vestibular self-motion cues and limited visual self-motion cues and it was expected that short-term down-weighting of vestibular cues enables habituation to carsickness. No difference in GVS habituation was observed between groups of individuals who were susceptible or impervious to carsickness (Balter et al., 2004). However, the study was limited due to the fact that sickness was only quantified by self-reported measures that rely on accurate recall by participants. Similarly to Balter et al. (2004), we found no significant correlation between sensory re-weighting (COP ratios) and historic recall of motion sickness (MSSQ total scores, which include an item on carsickness). The MSSQ measure is problematic, as it relies on accurate participant recall over a 10 year period. Conversely, in the current study the SSQ was completed after VR exposure, and is often treated as a gold-standard measure of sickness symptomatology (Balk et al., 2013; but c.f. Kim et al., 2018).
Results of the SSQ subscales in relation to COP ratios revealed a significant positive correlation: participants with higher COP ratios reported higher SSQ subscale (oculomotor and disorientation) scores. COP ratios of <1 reflect lower visually induced sway with higher oscillation amplitudes. This reflects the sensory re-weighting occurring during the control of balance and in this case suggests short-term down-weighting of visual cues. The positive correlation with SSQ subscales scores suggests that failure to down-weight visual cues (indicated by higher COP ratios) to maintain postural stability relates to higher cybersickness susceptibility. Therefore, this result can be taken as evidence that sensory re-weighting dynamics play a predictive role in determining cybersickness in virtual reality.
The current results align with previous findings that show effects of “noisy” vestibular stimulation on motion sickness. Stimulation of the vestibular organs via bone-conducted vibration (Weech et al., 2018a) or galvanic vestibular stimulation (Weech et al., 2020b) reduces cybersickness, and these effects are consistent with statistical principles of sensory re-weighting whereby multimodal cues are used according to their reliability, or noise-level (Ernst and Banks, 2002; Ernst and Bülthoff, 2004; Butler et al., 2010). In light of the current results, it would be valuable to identify if individual differences in sensory re-weighting also predict the extent to which a reduction in cybersickness can be achieved using noisy stimulation. Such an investigation would provide valuable insight into how noise affects the perceptual decision-making processes in individuals who demonstrate different baseline levels of sensory re-weighting.
We predicted all SSQ subscales (disorientation, oculomotor, and nausea) and SSQ total scores to be significantly correlated with the sensory re-weighting index. Our results show that both the nausea subscale and SSQ total scores were not significantly related to COP ratios. It may be that factors other than sensory re-weighting are strong contributors to nausea symptoms, although we were unable to identify those factors here. However, it is also possible that the non-significant findings relate to the power of the statistical tests used in the current study. A statistical power analysis based on our data (n = 23, alpha = 0.05) was completed for the correlations between the sensory re-weighting index and the SSQ total scores, and SSQ subscales. The effect size (ES) for the COP ratio correlations with the oculomotor and disorientation SSQ subscales is considered to be medium (r = 0.45) using Cohen's (1992) criteria. The power for these correlations, with a sample size of n = 23 and an alpha of 0.05, is 0.60. The power for the correlations between COP ratios and the SSQ total scores and the SSQ nausea subscale (n = 23, alpha = 0.05) were 0.467 and 0.307, respectively. Despite the decrease in power for these two correlations the ES are still considered to be medium (r = 0.39 [SSQ total] and r = 0.31 [SSQ nausea]) (Cohen, 1992). Thus, it is possible that the non-significant findings are related to the limited amount of statistical power achieved with the current sample size, which suggests that a replication of this study in a larger sample would be a useful way to better understand the discrepancy in correlations across SSQ subscales.
Since sensory re-weighting was measured here via behavioral correlates, and without neurophysiology, alternative explanations for the observed effects are conceivable. For instance, could the current results be equally explained as an effect of visual dominance on cybersickness? Evidence suggests that increased visual dominance over other senses in postural control can modulate the extent to which novel environmental conditions can be accounted for in sensorimotor control (Brady et al., 2012), and that the ability to adapt to unfamiliar conditions can benefit from reducing visual dependency (Bloomberg et al., 2015). However, the current results argue for sensory re-weighting, rather than visual dominance, as the key contributor. The primary outcomes in this study reflect the participant's propensity to switch from a state where vision dominates to one where other cues strongly contribute, indicating a rearranged set of internal weights for sensory cues. Visual dominance, on the other hand, would manifest as strong coupling to the visual stimulus at all magnitudes. If visual dominance were the key factor, we would expect see a correlation between sickness and COP path length for the (e.g.,) 16 mm condition. Given that we do not, it follows that sensory re-weighting is a more likely candidate for the observed effects. At the same time, future efforts should further dissociate these two related factors and their roles in cybersickness. Additionally, it could be considered whether the results simply reflect individual differences in tracking behavior that modulated sickness across participants. Since tracking the fixation dot could be achieved by moving either the eyes or the head-on-body, COP ratios could be partially determined by the tracking strategy adopted. Although head-on-body motion is typically the focus in tasks such as ours (Oie et al., 2002, 2005; Allison et al., 2006; Jeka et al., 2008), the use of eye tracking in future tasks should be employed to identify strategy differences. Finally, given that our focus here was on the re-weighting of cues to resolve sensory conflicts, we did not compute postural stability measures to assess the ecological theory of cybersickness (Riccio and Stoffregen, 1991); it is conceivable that the COP ratios we measured share some overlap with non-linear measures of postural dynamics, and an experiment designed to separately assess those outcomes would be a valuable next step.
While there is currently no direct evidence for the neural locus of the re-weighting effects discussed here, recent evidence has outlined a possible mechanism linking motion sickness to sensory conflict. Primate neurophysiology research by Oman and Cullen (2014) and Cullen (2012) shows that vestibular neurons in the rhesus brainstem exhibit cancelation of vestibular input produced by active head movement (termed “reafference”) while input produced by passive movement (termed “exafference”) is not canceled. Although there has been no direct link established between the activation of these vestibular units and other areas of brainstem that are causally involved in the emetic response, such a link has been hypothesized to exist (Suzuki et al., 2012; Oman and Cullen, 2014). At the same time, other research by Cullen and coworkers has shown evidence for neurons in the cerebellum (rostral fastigial nucleus) that preferentially code exafference (Brooks and Cullen, 2013) and it is the activity of these cerebellar units that is thought to drive adaptation of sensorimotor control strategies due to sensory rearrangements (Oman and Cullen, 2014). While it is unclear whether reafferent-canceling cells are responsive to stimuli from other modalities (e.g., optic flow), Oman and Cullen (2014) report informal evidence of negative results. In the context of these findings, we reason that the individual differences in sensory re-weighting we observed here would manifest in differential blood-oxygen level dependent activity, detectable using neuroimaging; in a future study these prospective differences should be identified and used as input to a prediction/classification algorithm with cybersickness as an output. Similar procedures targeting brainstem activity have been used to sensitively identify the perceptual experience of migraine (Cao et al., 2002), which shares some characteristics with cybersickness (e.g., headache, nausea), thus lending hope to the prospect of classifying cybersickness using brainstem fMRI. While other studies have used imaging techniques to identify areas associated with motion sickness (e.g., medial pre-frontal cortex; pre-genual anterior cingulate cortex; Kim et al., 2011), this is currently an understudied area.
It appears likely that the re-weighting process is highly dispersed across multiple cortical and sub-cortical regions (Andersen and Buneo, 2003; Block and Bastian, 2012; Medendorp et al., 2018). The cerebellum plays a central role in adapting to motor sensory prediction errors (e.g., throwing during prism adaptation) as patients with cerebellar lesions do not demonstrate sensorimotor adaptation (Thach et al., 1992; Earhart et al., 2002). There is also evidence that down-weighting vestibular cues relative to other senses during balance control takes place centrally, perhaps at the level of the cerebellum (Dilda et al., 2014; but note that sensory re-weighting may not require intact cerebellar cortex or cerebellar nuclei: Block and Bastian, 2012). The vestibular nuclei are also implicated in multimodal information processing (Angelaki and Cullen, 2008; Sadeghi et al., 2012; Oman and Cullen, 2014), and neuroimaging data shows that the posterior parietal cortex—to which the vestibular nuclei project—is selectively activated during sensory re-weighting (Clower et al., 1996). Yates et al. (2014) identified nausea and emetic centers that are connected to the vestibular system regions. Some of these overlapping regions include those implicated in multimodal information processing–the vestibular nuclei and cerebellum (fastigial nuclei and uvula-nodulus). These regions need to be further probed to reveal their role as nausea, emesis and sensory processing centers. Neuroimaging techniques and single cells animal recordings offer plausible next steps to further examine these subcortical and cerebellar regions in addition to the use of non-invasive brain stimulation (TMS and tDCS) to probe superficial regions strongly implicated in multisensory processing, such as the posterior parietal cortex (Bremmer et al., 2001).
In summary, we used measures of postural fluctuations (sway path length) at different levels of visual oscillation to compute a COP ratio, indicative of sensory re-weighting dynamics. We then exposed participants to a nauseogenic VR experience and collected their reported cybersickness scores. We observed evidence of a positive correlation between cybersickness SSQ subscales (oculomotor and disorientation) and COP ratios. We conclude that a lower COP ratio, indicative of more successful down-regulation of visual cues during high amplitude oscillations, is associated with lower cybersickness: participants who were unable to down-weigh visual information were more susceptible to cybersickness. The proposed mechanism for this association lies in the greater magnitude and number of sensory conflicts experienced when conflicting sensory cues cannot be effectively organized through the sensory re-weighting process (Dilda et al., 2014; Weech and Troje, 2017). These results support the use of postural stability measures and the role of sensory re-weighting as potential indicators of cybersickness susceptibility and tolerance to virtual reality.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
The studies involving human participants were reviewed and approved by University of Waterloo Research Ethics Board. The patients/participants provided their written informed consent to participate in this study.
Author Contributions
SW: conceptualization, data curation, formal analysis, investigation, methodology, project administration, software, supervision, visualization, writing–original draft preparation, review, and editing. CMC: investigation, project administration, writing–original draft preparation, review, and editing. MB-C: conceptualization, funding acquisition, project administration, resources, supervision, and writing–review and editing. All authors contributed to the article and approved the submitted version.
Funding
This research was supported by grants to MB-C from Oculus Research, the Ontario Research Fund, Canadian Foundation for Innovation's John R. Evans Leaders Fund, and the Natural Sciences and Engineering Research Council of Canada. The sponsors had no influence in the design or execution of the current research.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Alberts, B. B., Selen, L. P., Verhagen, W. I., Pennings, R. J., and Medendorp, W. P. (2017). Bayesian quantification of sensory re-weighting in a familial bilateral vestibular disorder (DFNA9). J. Neurophysiol. 119, 1209–1221. doi: 10.1152/jn.00082.2017
Allison, L. K., Kiemel, T., and Jeka, J. J. (2006). Multisensory re-weighting of vision and touch is intact in healthy and fall-prone older adults. Exp. Brain Res. 175, 342–352. doi: 10.1007/s00221-006-0559-7
Andersen, R. A., and Buneo, C. A. (2003). Sensorimotor integration in posterior parietal cortex. Adv. Neurol. 93, 159–177.
Angelaki, D. E., and Cullen, K. E. (2008). Vestibular system: the many facets of a multimodal sense. Annu. Rev. Neurosci. 31, 125–150. doi: 10.1146/annurev.neuro.31.060407.125555
Balk, S. A., Bertola, M. A., and Inman, V. W. (2013). “Simulator sickness questionnaire: twenty years later,” in Proceedings of the Seventh International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design (Bolton Landing, NY: Public Policy Center, University of Iowa), 257–263. doi: 10.17077/drivingassessment.1498
Balter, S. G., Stokroos, R. J., Van De Laar, M. M., Hendrice, N., and Kingma, H. (2004). Habituation to galvanic vestibular stimulation for analysis of susceptibility to carsickness. Acta Otolaryngol. 124, 690–694. doi: 10.1080/00016480410017242
Black, F. O., and Paloski, W. H. (1998). Computerized dynamic posturography: what have we learned from space? Otolaryngol. Head Neck Surg. 118, 45–51. doi: 10.1016/S0194-5998(98)70009-9
Black, F. O., Paloski, W. H., Doxey-Gasway, D. D., and Reschke, M. F. (1995). Vestibular plasticity following orbital spaceflight: recovery from postflight postural instability. Acta Otolaryngol. 115, 450–454. doi: 10.3109/00016489509125296
Block, H. J., and Bastian, A. J. (2012). Cerebellar involvement in motor but not sensory adaptation. Neuropsychologia 50, 1766–1775. doi: 10.1016/j.neuropsychologia.2012.03.034
Bloomberg, J. J., Peters, B. T., Cohen, H. S., and Mulavara, A. P. (2015). Enhancing astronaut performance using sensorimotor adaptability training. Front. Syst. Neurosci. 9:129. doi: 10.3389/fnsys.2015.00129
Brady, R. A., Peters, B. T., Batson, C. D., Ploutz-Snyder, R., Mulavara, A. P., and Bloomberg, J. J. (2012). Gait adaptability training is affected by visual dependency. Exp. Brain Res. 220, 1–9. doi: 10.1007/s00221-012-3109-5
Bremmer, F., Schlack, A., Shah, N. J., Zafiris, O., Kubischik, M., Hoffmann, K. P., et al. (2001). Polymodal motion processing in posterior parietal and premotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys. Neuron 29, 287–296. doi: 10.1016/S0896-6273(01)00198-2
Bronstein, A. M., Hood, J. D., Gresty, M. A., and Panagi, C. (1990). Visual control of balance in cerebellar and parkinsonian syndromes. Brain 113, 767–779. doi: 10.1093/brain/113.3.767
Brooks, J. X., and Cullen, K. E. (2013). The primate cerebellum selectively encodes unexpected self-motion. Curr. Biol. 23, 947–955. doi: 10.1016/j.cub.2013.04.029
Butler, J. S., Smith, S. T., Campos, J. L., and Bülthoff, H. H. (2010). Bayesian integration of visual and vestibular signals for heading. J. Vis. 10:23. doi: 10.1167/10.11.23
Cao, Y., Aurora, S. K., Nagesh, V., Patel, S. C., and Welch, K. M. A. (2002). Functional MRI-BOLD of brainstem structures during visually triggered migraine. Neurology 59, 72–78. doi: 10.1212/WNL.59.1.72
Carver, S., Kiemel, T., and Jeka, J. J. (2006). Modeling the dynamics of sensory reweighting. Biol. Cybern. 95, 123–134. doi: 10.1007/s00422-006-0069-5
Clower, D. M., Hoffman, J. M., Votaw, J. R., Faber, T. L., Woods, R. P., and Alexander, G. E. (1996). Role of posterior parietal cortex in the recalibration of visually guided reaching. Nature 383:618. doi: 10.1038/383618a0
Cohen, J. (1992). Statistical power analysis. Curr. Dir. Psychol. Sci. 1, 98–101. doi: 10.1111/1467-8721.ep10768783
Cullen, K. E. (2012). The vestibular system: multimodal integration and encoding of self-motion for motor control. Trends Neurosci. 35, 185–196. doi: 10.1016/j.tins.2011.12.001
Dennison, M. S., Wisti, A. Z., and D'Zmura, M. (2016). Use of physiological signals to predict cybersickness. Displays 44, 42–52. doi: 10.1016/j.displa.2016.07.002
Dilda, V., Morris, T. R., Yungher, D. A., MacDougall, H. G., and Moore, S. T. (2014). Central adaptation to repeated galvanic vestibular stimulation: implications for pre-flight astronaut training. PLoS ONE 9:e112131. doi: 10.1371/journal.pone.0112131
Earhart, G. M., Fletcher, W. A., Horak, F. B., Block, E. W., Weber, K. D., Suchowersky, O., et al. (2002). Does the cerebellum play a role in podokinetic adaptation? Exp. Brain Res. 146, 538–542. doi: 10.1007/s00221-002-1238-y
Ernst, M. O., and Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–433. doi: 10.1038/415429a
Ernst, M. O., and Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends Cogn. Sci. 8, 162–169. doi: 10.1016/j.tics.2004.02.002
Harris, C. M., and Wolpert, D. M. (1998). Signal-dependent noise determines motor planning. Nature 394:780. doi: 10.1038/29528
Horak, F. B., and MacPherson, J. M. (1996). “Postural equilibrium and orientation,” in Handbook of Physiology, eds L. B. Rowell and J. T. Sheperd (New York, NY: Oxford University Press), 255–292.
Jeka, J. J., Allison, L. K., and Kiemel, T. (2010). The dynamics of visual reweighting in healthy and fall-prone older adults. J. Mot. Behav. 42, 197–208. doi: 10.1080/00222895.2010.481693
Jeka, J. J., Oie, K. S., and Kiemel, T. (2008). Asymmetric adaptation with functional advantage in human sensorimotor control. Exp. Brain Res. 191, 453–463. doi: 10.1007/s00221-008-1539-x
Kennedy, R. S., Lane, N. E., Berbaum, K. S., and Lilienthal, M. G. (1993). Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3, 203–220. doi: 10.1207/s15327108ijap0303_3
Keshavarz, B., and Hecht, H. (2011). Validating an efficient method to quantify motion sickness. Hum. Factors 53, 415–426. doi: 10.1177/0018720811403736
Keshavarz, B., Riecke, B. E., Hettinger, L. J., and Campos, J. L. (2015). Vection and visually induced motion sickness: how are they related? Front. Psychol. 6:472. doi: 10.3389/fpsyg.2015.00472
Kiemel, T., Oie, K. S., and Jeka, J. J. (2006). Slow dynamics of postural sway are in the feedback loop. J. Neurophysiol. 95, 1410–1418. doi: 10.1152/jn.01144.2004
Kim, H. K., Park, J., Choi, Y., and Choe, M. (2018). Virtual reality sickness questionnaire (VRSQ): motion sickness measurement index in a virtual reality environment. Appl. Ergon 69, 66–73. doi: 10.1016/j.apergo.2017.12.016
Kim, J., Napadow, V., Kuo, B., and Barbieri, R. (2011). “A combined HRV-fMRI approach to assess cortical control of cardiovagal modulation by motion sickness,” in 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Boston, MA: IEEE), 2825–2828. doi: 10.1109/IEMBS.2011.6090781
Kim, Y. Y., Kim, H. J., Kim, E. N., Ko, H. D., and Kim, H. T. (2005). Characteristic changes in the physiological components of cybersickness. Psychophysiology 42, 616–625. doi: 10.1111/j.1469-8986.2005.00349.x
Kitazaki, M., and Kimura, T. (2010). Effects of long-term adaptation to sway-yoked visual motion and galvanic vestibular stimulation on visual and vestibular control of posture. Presence 19, 544–556. doi: 10.1162/pres_a_00017
Lackner, J. R., and DiZio, P. (2005). Vestibular, proprioceptive, and haptic contributions to spatial orientation. Annu. Rev. Psychol. 56, 115–147. doi: 10.1146/annurev.psych.55.090902.142023
Medendorp, W. P., Alberts, B. B., Verhagen, W. I., Koppen, M., and Selen, L. P. (2018). Psychophysical evaluation of sensory re-weighting in bilateral vestibulopathy. Front. Neurol. 9:377. doi: 10.3389/fneur.2018.00377
Nashner, L., and Berthoz, A. (1978). Visual contribution to rapid motor responses during postural control. Brain Res. 150, 403–407. doi: 10.1016/0006-8993(78)90291-3
Oie, K., Carver, S., Kiemel, T., Barela, J., and Jeka, J. (2005). 5.24 The dynamics of sensory re-weighting: a temporal symmetry. Gait Posture 21:S29. doi: 10.1016/S0966-6362(05)80099-2
Oie, K. S., Kiemel, T., and Jeka, J. J. (2002). Multisensory fusion: simultaneous re-weighting of vision and touch for the control of human posture. Cogn. Brain Res. 14, 164–176. doi: 10.1016/S0926-6410(02)00071-X
Oman, C. M. (1990). Motion sickness: a synthesis and evaluation of the sensory conflict theory. Can. J. Physiol. Pharmacol. 68, 294–303. doi: 10.1139/y90-044
Oman, C. M., and Cullen, K. E. (2014). Brainstem processing of vestibular sensory exafference: implications for motion sickness etiology. Exp. Brain Res. 232, 2483–2492. doi: 10.1007/s00221-014-3973-2
Peterka, R. J. (2002). Sensorimotor integration in human postural control. J. Neurophysiol. 88, 1097–1118. doi: 10.1152/jn.2002.88.3.1097
Reason, J. T. (1978). Motion sickness adaptation: a neural mismatch model. J. R. Soc. Med. 71, 819–829. doi: 10.1177/014107687807101109
Riccio, G. E., and Stoffregen, T. A. (1991). An ecological theory of motion sickness and postural instability. Ecol. Psychol. 3, 195–240. doi: 10.1207/s15326969eco0303_2
Sadeghi, S. G., Minor, L. B., and Cullen, K. E. (2012). Neural correlates of sensory substitution in vestibular pathways following complete vestibular loss. J. Neurosci. 32, 14685–14695. doi: 10.1523/JNEUROSCI.2493-12.2012
Shahal, B., Nachum, Z., Spitzer, O., Ben-David, J., Duchman, H., Podoshin, L., et al. (1999). Computerized dynamic posturography and seasickness susceptibility. Laryngoscope 109, 1996–2000. doi: 10.1097/00005537-199912000-00019
Suzuki, T., Sugiyama, Y., and Yates, B. J. (2012). Integrative responses of neurons in parabrachial nuclei to a nauseogenic gastrointestinal stimulus and vestibular stimulation in vertical planes. Am. J. Physiol. Regul. Integr. Comp. Physiol. 302, 965–975. doi: 10.1152/ajpregu.00680.2011
Tal, D., Bar, R., Nachum, Z., Gil, A., and Shupak, A. (2010). Postural dynamics and habituation to seasickness. Neurosci. Lett. 479, 134–137. doi: 10.1016/j.neulet.2010.05.044
Thach, W. T., Goodkin, H. P., and Keating, J. G. (1992). The cerebellum and the adaptive coordination of movement. Annu. Rev. Neurosci. 15, 403–442. doi: 10.1146/annurev.ne.15.030192.002155
Walter, H. J., Li, R., Munafo, J., Curry, C., Peterson, N., and Stoffregen, T. A. (2019). Unstable coupling of body sway with imposed motion precedes visually induced motion sickness. Hum. Mov. Sci. 64, 389–397. doi: 10.1016/j.humov.2019.03.006
Webb, N. A., and Griffin, M. J. (2002). Optokinetic stimuli: motion sickness, visual acuity, and eye movements. Aviat. Space Environ. Med. 73, 351–358.
Weech, S., Kenny, S., and Barnett-Cowan, M. (2019). Presence and cybersickness in virtual reality are negatively related: a review. Front. Psychol. 10:158. doi: 10.3389/fpsyg.2019.00158
Weech, S., Kenny, S., Calderon, C. M., and Barnett-Cowan, M. (2020a). Limits of subjective and objective vection for ultra-high frame rate visual displays. Displays 64:101961. doi: 10.1016/j.displa.2020.101961
Weech, S., Moon, J., and Troje, N. F. (2018a). Influence of bone-conducted vibration on simulator sickness in virtual reality. PLoS ONE 13:e0194137. doi: 10.1371/journal.pone.0194137
Weech, S., and Troje, N. F. (2017). Vection latency is reduced by bone-conducted vibration and noisy galvanic vestibular stimulation. Multisens. Res. 30, 65–90. doi: 10.1163/22134808-00002545
Weech, S., Varghese, J. P., and Barnett-Cowan, M. (2018b). Estimating the sensorimotor components of cybersickness. J. Neurophysiol. 120, 2201–2217. doi: 10.1152/jn.00477.2018
Weech, S., Wall, T., and Barnett-Cowan, M. (2020b). Reduction of cybersickness during and immediately following noisy galvanic vestibular stimulation. Exp. Brain Res. 238, 427–437. doi: 10.1007/s00221-019-05718-5
Wolpert, D. M., Miall, R. C., and Kawato, M. (1998). Internal models in the cerebellum. Trends Cogn. Sci. 2, 338–347. doi: 10.1016/S1364-6613(98)01221-2
Keywords: sensory re-weighting, vection, self-motion perception, motion sickness, virtual reality
Citation: Weech S, Calderon CM and Barnett-Cowan M (2020) Sensory Down-Weighting in Visual-Postural Coupling Is Linked With Lower Cybersickness. Front. Virtual Real. 1:10. doi: 10.3389/frvir.2020.00010
Received: 17 June 2020; Accepted: 27 July 2020;
Published: 02 September 2020.
Edited by:
Charles McMaster Oman, Massachusetts Institute of Technology, United StatesReviewed by:
Heiko Hecht, Johannes Gutenberg University Mainz, GermanyJacob Bloomberg, Johnson Space Center (NASA), United States
Copyright © 2020 Weech, Calderon and Barnett-Cowan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Séamas Weech, seamas.weech@mail.mcgill.ca