Sungu Nam
Kyoung-Mi Jang
Moonyoung Kwon2
Jaeseung Jeong- 1Korea Advanced Institute of Science and Technology, Daejeon, South Korea
- 2Korea Research Institute of Standards and Science, Daejeon, South Korea
Virtual reality (VR) is a rapidly developing technology that simulates the real world. However, for some cybersickness-susceptible people, VR still has an unanswered problem—cybersickness—which becomes the main obstacle for users and content makers. Sensory conflict theory is a widely accepted theory for cybersickness. It proposes that conflict between afferent signals and internal models can cause cybersickness. This study analyzes the brain states that determine cybersickness occurrence and related uncomfortable feelings. Furthermore, we use the electroencephalogram (EEG) microstates and functional connectivity approach based on the sensory conflict theory. The microstate approach is a time–space analysis method that allows signals to be divided into several temporarily stable states, simultaneously allowing for the exploration of short- and long-range signals. These temporal dynamics can show the disturbances in mental processes associated with neurological and psychiatric conditions of cybersickness. Furthermore, the functional connectivity approach gives us in-depth insight and relationships between the sources related to cybersickness. We recruited 40 males (24.1 ± 2.3 years), and they watched a VR video on a curved computer monitor for 10 min to experience cybersickness. We recorded the 5-min resting state EEG (baseline condition) and 10-min EEG while watching the VR video (task condition). Then, we performed a microstate analysis, focusing on two temporal parameters: mean duration and global explained variance (GEV). Finally, we obtained the functional connectivity data using eLoreta and lagged phase synchronization (LPS). We discovered five sets of microstates (A–E), including four widely reported canonical microstates (A–D), during baseline and task conditions. The average duration increased in microstates A and B, which is related to the visual and auditory networks. The GEV and duration decreased in microstate C, whereas those in microstate D increased. Microstate C is related to the default mode network (DMN) and D to the attention network. The temporal dynamics of the microstate parameters are from cybersickness disturbing the sensory, DMN, and attention networks. In the functional connectivity part, the LPS between the left and right parietal operculum (OP) significantly decreased (p < 0.05) compared with the baseline condition. Furthermore, the connectivity between the right OP and V5 significantly decreased (p < 0.05). These results also support the disturbance of the sensory network because a conflict between the visual (V5) and vestibular system (OP) causes cybersickness. Changes in the microstates and functional connectivity support the sensory conflict theory. These results may provide additional information in understanding brain dynamics during cybersickness.
Introduction
Virtual reality (VR) is evolving rapidly to simulate the real world. Because it is a virtual world, it can simulate various things, including unrealistic things. Thus, researchers have rapidly developed and increased its use for multiple purposes, such as movies, video games, sports, and medical tools for psychiatrists and therapists to treat mental illness (Kim, 2005; Porras et al., 2018).
However, VR still has unresolved problems, such as cybersickness, which is the main obstacle for users and content makers. Watching 3D movies or playing 3D games can trigger cybersickness. Users might experience cybersickness while immersed in VR content when they perceive self-movement caused by optical flow while they are immobile. Its symptoms include sweating, yawning, dizziness, spatial disorientation, instability, fatigue, nausea, and vomiting (McCauley and Sharkey, 1992; Mazloumi Gavgani et al., 2018). Several researchers explained cybersickness using sensory conflict, eye movement hypothesis, subjective vertical conflict, postural instability, and other theories (Laviola, 2000; Gallagher and Ferrè, 2018; Lim et al., 2018).
Among these studies, the most widely accepted cybersickness theory is the sensory conflict theory. It proposes that conflict between afferent signals from visual, vestibular, and somatic sensors and signals from internal models of the central nervous system can cause cybersickness (Zhang et al., 2016). In support of the sensory conflict theory, researchers have demonstrated that a strong illusion of apparent self-movement called vection, generated using visual movement in a wide field of view, often precedes cybersickness (Palmisano et al., 2017). Vection can occur when motion signals from visual inputs collide with stationary signals provided by vestibular and/or proprioceptive inputs. An fMRI study showed that during vection, activities of the human motion-sensitive middle temporal (MT+) area desynchronized between the left and right hemispheres (Miyazaki et al., 2015). Furthermore, this MT area interacts with the parieto-insular vestibular cortex, which is the center of vestibular sensation. Another study showed decreased functional connectivity between MT+/V5 and several other striatal and extra-striatal visual processing regions compared to resting states (Hampson et al., 2004). Together, these results indicate a significant relationship between the visual and vestibular systems, supporting the sensory conflict theory—the theory can explain cybersickness neurophysiologically.
Electroencephalogram (EEG) is commonly used to evaluate cybersickness during VR experiences (Kim et al., 2005; Lim et al., 2021). During an EEG recording, specific topography (or microstate) will persist for 50–100 ms before changing to a different topography (Michel and Koenig, 2018). Thus, we can divide the signals into several temporarily stable states to explore short and long ranges simultaneously. Furthermore, these stable states are highly reproducible within and across participants; a clustering algorithm can group microstates into a finite set of classes based on topographical similarity (Khanna et al., 2014). These microstates offer various parameters with physiological relevance to explain variation across behavioral and psychological states and neuropsychiatric disorders (Al Zoubi et al., 2019; Bréchet et al., 2020; Kalburgi et al., 2020). Also, in contrast to conventional EEG power spectrum analysis techniques that seek general understanding of cybersickness, the microstate analysis leverages global large-scale brain networks to examine complex mental activity. Furthermore, compared to the slow dynamics in fMRI signals, microstate analysis of the EEG offers a better temporal resolution that enables fast changes in global network dynamics of the brain in response to cybersickness on a sub-second time scale, which is a significant advantage over other methods (Michel and Koenig, 2018). Therefore, microstates are an appropriate tool for evaluating the disturbances in mental processes associated with neurological and psychiatric conditions of cybersickness.
Recently, researchers have gained physiological and pathological inspiration by exploring the functional connectivity of the brain using high-density (HD) EEG, which has the advantages of being non-invasive, cheap, and portable (Piano et al., 2017). Because microstate analysis is limited to data on the scalp, finding sources for in-depth analysis and observing the relationships between them could lead to more diverse interpretations. In addition, it was more important in our study because it can overcome certain limitations, such as volume conduction problems; we used HD EEG (257 channel) (Barzegaran and Knyazeva, 2017). A recent study (Li et al., 2021) showed that functional connectivity between the frontal and left parietal regions decreased during cybersickness. Furthermore, the frontal–occipital functional connectivity increased when fully immersed in trained VR (Kang et al., 2021). However, the authors stated limitations, such as small sample size, few EEG channels (5-channel), and a single analysis for cybersickness.
Several studies have reported gender-based differences in motion sickness symptoms, stating that women are generally more sensitive to cybersickness than are men (Paillard et al., 2013; Freitag et al., 2016; Rangelova et al., 2020). An early finding postulated that these differences occur because women have a larger field of view than men (Kennedy and Frank, 1985). Additionally, male subjects were less affected by hormones although they had more VR experiences (Clemes and Howarth, 2005). Although some studies argue that gender differences can be reduced by adjusting the inter-pupillary distance (Stanney et al., 2020), others state otherwise (Hsiao et al., 2019; Melo et al., 2019; Curry et al., 2020). Based on these reasons, gender differences can affect the result and we must consider that factor carefully.
Our research group planned a long-term study to investigate the severity of cybersickness in stages caused by human factors, such as gender and age. Thus, we would like to compare the following EEG scenarios: in a resting state, during exposure to simple content using a curved monitor, and during exposure to simple and complex contents while wearing a head-mounted display (HMD). This comparison is to investigate and quantify the brain dynamics caused by the severity of cybersickness. Then, we planned to investigate the characteristics of cybersickness from a wider variety of people of different genders and ages and compare them with those previously studied. First, using simple content consisting of land and sky displayed on a curved monitor, we investigated changes in male participants affected by cybersickness (in their 20s) in EEG.
This study used microstates analysis and source connectivity to understand the large-scale brain networks during cybersickness. These dynamic characteristics are crucial for understanding complex brain cognition and behavior (Liao et al., 2017). Furthermore, they can cause the development of new diagnostic or monitoring tools (Dennison et al., 2016). However, previous studies did not show the microstates of cybersickness, which can provide further insight into exploring cybersickness.
Materials and methods
Participants
We recruited 40 male participants (24.1 ± 2.3 years) for the cybersickness EEG test. Based on a previous study conducted by our research group, we can divide the participants using the MSSQ, which is a self-report questionnaire measuring susceptibility to motion sickness (Lim et al., 2021). The mean score was 21.56, with 20 participants scoring higher than the mean score and others scoring lower. The subjects were right-handed and healthy, and they did not take any medication. All subjects participated in this study after signing a prior agreement. The Institutional Review Board (IRB) of the Korea Advanced Institute of Science and Technology and the Ethical Committee of the Korea Research Institute of Standards and Science (KRISS) approved the experimental procedures in this study (KH2020-145, KRISS-IRB-2021-03). The participants provided informed written consent.
Evaluation index
The simulation sickness questionnaire (SSQ) is a survey for describing simulator sickness by evaluating 16 symptoms on a 4-point scale (0–3). Based on factor analysis, these symptoms were divided into three general categories: oculomotor, disorientation, and nausea (Kennedy et al., 1993). SSQ measures visually induced cybersickness by assessing symptoms of cybersickness. The participants answered the SSQ twice: before watching VR video content (Supplementary Video 1) (named Pre-SSQ) for a baseline condition and after watching VR video content (named Post-SSQ) for the task condition (Figure 1A). The figure compares the two SSQ results.
Figure 1. EEG recordings during cybersickness condition. (A) Experimental scenario. (B) Scenes could be changed to display the roll and yaw motion. The participant will watch the combined video through a curved monitor.
Experimental protocol
Test procedure
We divided the experimental protocol into two conditions: baseline and task. The baseline condition checks the resting state of EEG, while the task condition is designed to cause cybersickness. With the eyes opened, the baseline condition lasted for 5 min without action. Then, the participants watched a VR video for 10 min to induce cybersickness during the task condition (Figure 1A). Finally, the participants completed the SSQ to measure cybersickness before the baseline and after the task conditions.
Apparatus and stimuli
We created a 10-min VR video as 10 min is enough to induce cybersickness (Sharples et al., 2008). The distance between the participants and the curved monitor that presented the VR video was 65 cm. Additionally, the height of all participants’ chairs was adjusted to fit their eyes in the center of the monitor. A screen fence blocked the view behind the curved monitor. The field of view was controlled for all observers to avoid peripheral surroundings that act as stable landmarks (Kim and Kim, 2019). Through curved computer monitors (Samsung, 120 × 34 cm), the scene viewed by the participants constantly moved back and forth at a speed of 20 m/s, and the rotational angular velocity pseudo-randomly changed every minute rather than sequentially (15°/s–20 m/s, 30°/s–20 m/s, 45°/s–20 m/s, 60°/s–20 m/s, 75°/s–20 m/s, 90°/s–20 m/s, 105°/s–20 m/s, 120°/s–20 m/s, 135°/s–20 m/s, and 150°/s–20 m/s). For example, a participant’s field of view moved forward at a speed of 20 m/s and changed the yaw and roll (Figure 1B) at a speed of 30°/s for the first 30 s, and it moved backward at the same movement and angular speed for the next 30 s. This 1-min video was repeated pseudo-randomly 10 times (for 10 min) with a change in angular velocity. We used the Unity program to create the VR scenes in this study. Several parameters, such as rotational speed, linear movement speed, motion axis, background complexity, and scene fidelity, can affect cybersickness. Thus, we tried to keep the condition as simple as possible to minimize the impact of other variables.
Electroencephalogram recordings
The EEG signals were collected in the electric shielding chamber using a 256-channel HydroCel Geodesic Sensor Net (Electrical Geodesics Incorporated, Eugene, OR, United States). We referenced all electrodes to Cz, and impedance remained below 50 kΩ (Tucker et al., 2003). We recorded EEG using a 0.1–100-Hz band pass at a sampling rate of 1000 Hz. We analyzed EEG data using EEGLAB, which is an open-source toolbox operated in MATLAB (MathWorks, Natick, MA, United States). Furthermore, we used an ADJUST (an automatic EEG artifacts detector based on the joint use of spatial and temporal features) plug-in with EEGLAB to remove EEG artifacts. We applied a 1-Hz high-pass filter in the EEG data pre-processing and a 40-Hz low-pass filter. After filtering, we downsampled signals to 125 Hz and re-referenced them using a common average reference. To eliminate ocular (eye blinks, vertical eye movements, and horizontal eye movements) and muscular artifacts, we first applied independent component analysis (ICA) to the data (as implemented in the EEGLAB toolbox) (Vialatte et al., 2008; McMenamin et al., 2010). Then, we used the ADJUST plug-in to remove the ICA components that contained the artifacts based on the default plug-in criteria (Mognon et al., 2011). We selected epochs (4 min) in the middle of both conditions for this study.
Electroencephalogram microstates analysis
The following steps must be taken after data pre-processing: (1) find the most representative maps from EEG data and (2) fit these maps back into the EEG data and calculate temporal parameters and statistics from it.
In this process, the segmentation stage is performed using two spatial cluster analyses based on a modified version of the k-means algorithm to find the optimal topography set for microstate analysis (Pascual-Marqui et al., 1995). First, it was performed at the individual level for each participant, and then it was performed at the global level for all participants. Global field power was calculated from each epoch to find the most dominant map for each participant. We ignored the polarity of the map while using this clustering method. To obtain the optimal number of clusters, we used a meta-criterion that combines seven independent optimization criteria (Pascual-Marqui et al., 1995; Charrad et al., 2014). Further, to perform at the global level, the dominant maps for all participants (baseline, task) were submitted together to a clustering algorithm to find the most representative maps across subjects. The number of global maps was selected based on the same meta-criterion, which was used at the individual level, thus resulting in the five microstate maps in this study (polarity of the map was ignored as in the individual level). The microstate maps were fitted back into the individual EEG data to define the microstates after they were selected. Each time frame of EEG data was labeled with one of five microstate maps, which was identified during global level clustering analysis, by calculating the spatial correlations (winner-takes-all, and only the correlation above 0.5 was selected) (Pascual-Marqui et al., 1995). Furthermore, we calculated two temporal parameters for each participant: GEV and duration. The duration indicates how long it lasts between each microstate class in milliseconds. The GEV indicates how well the microstate topographies explain the original EEG data. We used the free Cartool (Brunet et al., 2011) software 3.91, programmed by Denis Brunet Cartool, to perform the microstate analysis. The software also includes a meta-criterion calculation.
Electroencephalogram source connectivity
First, we used eLORETA with the LORETA KEY software for source localization (Pascual-Marqui, 2007a,b; Pascual-Marqui et al., 2011). We performed the eLORETA computations in a realistic head model using the MNI152 (Montreal Neurological Institute 152) template (Mazziotta et al., 2001; Fuchs et al., 2002) with the 3D solution space restricted to 6,000 voxels of 5-mm3 resolution, which comprises cortical gray matter determined using the probabilistic Talairach atlas (Lancaster et al., 2000). Next, we applied coordinates of electrode positions in the MNI152 scalp to a digitized MRI version of the Talairach atlas (McConnel Brain Imaging Centre, Montreal Neurological Institute, McGill University). We used the resulting coordinates to compute the eLORETA transformation matrix. Furthermore, we defined the region of interest (ROI). We extracted 10 ROIs (Table 1) from the visual and vestibular system parts, which interact when experiencing motion sickness (zu Eulenburg et al., 2012; Miyazaki et al., 2021). We used the single nearest voxel option as the spatial resolution of the eLORETA was low, and the single centroid voxel was the best representative of the ROI. For the functional connectivity analysis for each pair of ROIs, we used a lagged phase synchronization (LPS) to ignore the instantaneous component of total connectivity and considered only the lagged component (Pascual-Marqui, 2007a,b; Jahng et al., 2017).
Table 1. Region of interest (ROI).
Statistical analysis
We analyzed SSQ with descriptive statistics using SPSS 21 software. We performed a paired t-test to investigate the condition differences in temporal parameters from each microstate. Furthermore, we performed a paired t-test to determine the difference in LPS between the two conditions. The variance smoothing parameter and randomization number were 0 and 5,000, respectively. We created a file containing the maximal thresholds, and the thresholds at probability values of p < 0.01, p < 0.05, and p < 0.01 (Friston et al., 1991). Finally, we used those values to show the significant changes in functional connectivity.
Results
The result of the average SSQ score before (baseline) and after (task) watching VR content showed significant differences (t = −6.85, p < 0.001) (Figure 2). Additionally, the participants felt cybersickness. There were no significant associations between the parameters of the EEG and the degree of cybersickness based on SSQ sub-categories (total, nausea, oculomotor, and disorientation etc.). There were no significant associations between the functional connectivity and the GEV or duration of microstates either.
Figure 2. SSQ score of the participant before and after watching the VR video content. All the participants felt cybersickness (p < 0.001).
Microstate segmentation of 40 participants recorded using 256-channel EEG showed five microstate topographies (Figure 3), which were consistent across conditions. These five microstate classes across participants explained 78.76 and 80.93% of the global variance in the resting and task conditions, respectively. Additionally, five microstate maps showed a high spatial correlation of the topographies ranging from 0.94 to 0.98. The four microstates (A–D) obtained in the two conditions were similar to the four class models maps consistently identified in the literature. Microstates A and B have topographic map fields with diagonal axis orientations, C has anterior–posterior orientations, and D has fronto-central extreme locations (Michel and Koenig, 2018). Microstate E also had a topography similar to that of several studies (Custo et al., 2017; Bréchet et al., 2020).
Figure 3. EEG microstate topographies for both conditions (base, task).
We fitted back five main maps from the baseline and task conditions to each participant’s original EEG using special correlation analysis and winner-takes-all labeling. This fitting method allowed us to uninterruptedly calculate the duration of each microstate and the global variance explained by each map (GEV). Researchers commonly use these parameters to show the spatiotemporal properties of EEG microstates. Microstate duration and GEV were different for each condition. For the resting vs. task condition, microstate C decreased (t = −6.156, p < 0.001) and D increased (t = −8.53, p < 0.001), which was statistically significant (Figure 4A). There were no statistically significant values for the other microstates. Furthermore, we compared the average duration of the microstates under the two conditions (Figure 4B). The other microstate showed significant change except for microstate E. Microstates A, B, and D (t = −7.086, p < 0.001; t = −8.118, p < 0.001; and t = 6.482, p < 0.001, respectively) showed increased values, while microstate C showed decreased values (t = −5.230, p < 0.001). Microstate E showed no significant difference for either GEV (t = −1.067, p = 0.293) and duration (t = −1.051, p = 0.300).
Figure 4. Microstate global explained variance (GEV) and average duration was different for each condition. (A) For resting vs. task condition, microstate C decreased (t = −6.156, *p < 0.001) and D increased (t = −8.53, *p < 0.001) in GEV, which was statistically significant. (B) The average duration of the microstates under the two conditions. The results were significant in all microstates except E.
We observed statistical significance in the LPS when comparing the baseline condition with the task condition; the threshold for significance was t = 2.187 (corresponding to p < 0.05). The functional connectivity between the left parietal operculum (OP1) and right parietal operculum (OP2) decreased. In addition, connectivity between OP2 and the right medial temporal area (V5) decreased. There were no significant associations between the functional connectivity and the GEV or duration of microstates (all absolute r-values < 0.45).
Discussion
This study analyzes the neural mechanisms of cybersickness. To achieve this, we checked microstates and functional connectivity under cybersickness. Our main findings are as follows: (1) there are significantly increased and decreased values of microstate parameters under cybersickness and (2) cybersickness decreases functional connectivity between the visual and vestibular systems.
We selected five maps for microstate analysis. A, B, C, and D are common microstates among these maps, and several studies also referred to E (Custo et al., 2017; Michel and Koenig, 2018; Bréchet et al., 2020). Thus, we investigated microstates A, B, C, and D, which showed significant changes in GEV and duration.
Before discussing the parameter change, it is crucial to understand the sources of each microstate. The source of a microstate is vital in interpreting parameter changes and explaining cybersickness. An fMRI study discovered sources of resting states in microstates (Custo et al., 2017). At first, microstate A showed strong activity in the left, middle, and superior temporal lobes, including Brodmann areas 41 (BA41, primary auditory cortex) and 22 (BA22, Wernicke), and the left insular cortex. Microstate B showed activity in the left and right occipital cortices (cuneus), including Brodmann areas 17 and 18 (primary visual cortex). The posterior cingulate, right insula, and the temporal cortex were also sources of microstate B. Then, microstate C showed strong activation in the precuneus and the posterior cingulate cortex. A second weaker activated area was the left angular gyrus. Lastly, microstate D showed strong activation in the right inferior parietal lobe (BA40) and the right, middle, and superior frontal gyri. The right insula (BA13) was also activated (Bréchet et al., 2019). These aforementioned and other fMRI studies show that microstate A is related to the auditory network, microstate B to the visual network, microstate C to the saliency network, and microstate D to the attention network (Britz et al., 2010).
Microstates A and B, which are related to auditory and visual networks, showed an increase in average duration in our study. Similarly, according to the microstate analysis result of relapsing-remitting multiple sclerosis using HD EEG, patients showed an increased average duration of microstates A and B compared with healthy controls (Gschwind et al., 2016). It was possible to predict the duration of the disease and the annual relapse rate using the two changes of the microstates and apply a stepwise multiple linear regression model. This finding suggests that multiple sclerosis affects the sensory network (visual and auditory), which is consistent with the fMRI studies (Britz et al., 2010). We can also interpret that the increased duration in microstates A and B in our study occurred for a similar reason. Therefore, we can assume that disturbance of the sensory network caused by cybersickness increased A and B durations.
We discovered a decreased average duration and GEV during cybersickness. In addition, microstate C decreased in the visualization condition (Milz et al., 2016) and high cognitive processes, particularly math problems (Bréchet et al., 2019). We also found an association between microstate C and the DMN; additional fMRI studies supported these findings (Damoiseaux et al., 2008; Lei et al., 2014). According to these findings, microstate C is a task-negative network with decreasing activity during the performance of cognitive works. These findings are also related to worse cognitive status and agree with the neural mechanism issue of cybersickness (Zeng et al., 2015). Thus, a decrease in GEV and duration could result in stress from viewing VR video and disturbance of the DMN caused by motion sickness, as confirmed in the studies described above.
However, microstate D showed a relatively increased duration and GEV than microstate C in our study. Furthermore, we observed increased duration and occurrence when participants engaged in a subtraction task (Seitzman et al., 2017). This result suggests an association between microstate D and the dorsal attention network. However, according to a behavioral manipulation study, it increased during rest rather than performing a goal-directed task (Milz et al., 2016). This discovery suggests an association between microstate D and the reflexive aspects of attention, focus switching, and reorientation. It is also associated with wakefulness and reality testing, which increases during meditation (Faber et al., 2014). However, it decreases during schizophrenia (Lehmann et al., 2005) and under hypnosis (Katayama et al., 2007). Furthermore, watching a VR video that moves continuously and rotates makes participants repeat reorientation and focus switching despite cybersickness. They become more aroused simultaneously. Thus, D also increased in our study.
In the microstate analysis, the sensory network was affected when the duration of microstates A and B increased. Therefore, we wanted to check if it appeared in functional connectivity. Thus, the connectivity between V5 and OP decreased, and the left and right OPs decreased (Figure 5). By viewing moving concentric circles, functional connectivity between MT/V5 and other visual processing brain regions decreased compared with resting states (Hampson et al., 2004). However, several studies discovered that the connectivity between the left and right MT decreases when feeling cybersickness (Miyazaki et al., 2015), and the left and right vestibular also decreases (Li et al., 2021). Additionally, a study confirmed a reduced visual–vestibular interaction in monkeys using visual motion stimulation (Chen et al., 2011), and nausea reduced connectivity between the right and left V1 (Toschi et al., 2017). The tendency to feel motion sickness decreased with a lowered gap between the left and right using a simulator and tDCS (Arshad et al., 2015; Ahn et al., 2020). Therefore, our study showed that cybersickness reduced the connectivity of the visual–vestibular network as confirmed earlier by the microstate analysis, which is consistent with the results of previous studies.
Figure 5. Functional connectivity result using lagged phase synchronization when comparing the baseline condition with the task condition (t = 2.187, p < 0.05). The functional connectivity between the left parietal operculum (OP1) and right parietal operculum (OP2) decreased, and connectivity between OP2 and the right medial temporal area (V5) decreased.
Conclusion
In this study, we analyzed the microstate and functional connectivity during cybersickness. We discovered changed microstate parameters under the cybersickness conditions and decreased functional connectivity between visual and vestibular networks. These results provide understanding and information about brain dynamics, which change during cybersickness. To the best of our knowledge, this is the first attempt to analyze the brain states of cybersickness in terms of large-scale brain networks on a sub-second time scale (microstates). We clearly showed that the brain states of cybersickness exhibited specific spatiotemporal patterns of the EEG and could be expressed as a superposition of EEG microstates. This indicates that cybersickness potentially affects global brain dynamics in sub-second time scale. We suggest that these distinct temporal dynamics of EEG microstates can be a potential biomarker for detection of cybersickness in various fields including fighter pilots and flight attendants. However, only men participated because of the limitations of this study. We need further research for the localization and discovery of detailed neural mechanisms; however, this study is a promising start in improving our understanding of cybersickness.
Limitation of the study
This study used simple VR video instead of complex content, such as vertical objects (trees, buildings, etc.). However, using more complex video content would cause differences in the microstate values and network measures. Furthermore, we did not use HMD in this study, which provides more immersive VR content. Future research might identify differences between this study and more immersed stereoscopic content using HMD. One of the significant limitations in the current study is that we only included male participants in the experiment. Thus, the findings obtained from the current study are only relevant for males. We should perform the same protocols of experiments and analyses using both male and female participants in our further investigation to confirm the results. We suggest that the gender-balanced study would clearly show how gender affects the spatiotemporal microstate patterns of brain dynamics in EEGs induced by cybersickness. Furthermore, EEG recordings were made during the baseline and while watching the VR video. Other studies that followed a similar protocol as ours have suggested that decreased alpha power is a sign of cybersickness (Naqvi et al., 2015; Liu et al., 2017). However, several studies have reported that alpha power increases after watching a stimulus (Chuang et al., 2016; Wawrzyk et al., 2018). Thus, EEG results may vary depending on whether it is presenting a visual stimulus or whether it is measured after the visual stimulation disappears. In this study, we measured participants while they were watching VR videos. In the follow-up study, we must check the EEG recording of the resting period after watching the VR video. Additionally, fMRI studies might discover the exact source of cybersickness. Thus, we could have more detailed and drastic results due to more severe cybersickness and enriched information.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by the Institutional Review Board (IRB) of KAIST and the Ethical Committee of “Korea Research Institute of Standards and Science (KRISS).” The patients/participants provided their written informed consent to participate in this study.
Author contributions
SN contributed to the design, conceptualization, interpretation of the data, and writing of the original draft. K-MJ and MK provided data acquisition and expertise on EEG pre-processing. HL and JJ provided expertise on EEG microstates and functional connectivity. All authors contributed to the interpretation of the results, manuscript revision, and read and approved the submitted version.
Funding
This work was supported by the Ministry of Trade, Industry, and Energy (MOTIE), Korea, under the “Regional Industry-based Organization Support Program” (grant number: P0001940) supervised by the Korea Institute for Advancement of Technology (KIAT).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnhum.2022.857768/full#supplementary-material
References
Ahn, M.-H., Park, J. H., Jeon, H., Lee, H.-J., Kim, H.-J., and Hong, S. K. (2020). "Temporal dynamics of visually induced motion perception and neural evidence of alterations in the motion perception process in an immersive virtual reality environment. Front. Neurosci. 14:600839. doi: 10.3389/fnins.2020.600839
Al Zoubi, O., Mayeli, A., Tsuchiyagaito, A., Misaki, M., Zotev, V., Refai, H., et al. (2019). EEG microstates temporal dynamics differentiate individuals with mood and anxiety disorders from healthy subjects. Front. Hum. Neurosci. 13:56. doi: 10.3389/fnhum.2019.00056
Arshad, Q., Cerchiai, N., Goga, U., Nigmatullina, Y., Roberts, R. E., Casani, A. P., et al. (2015). Electrocortical therapy for motion sickness. Neurology 85, 1257–1259.
Barzegaran, E., and Knyazeva, M. G. (2017). Functional connectivity analysis in EEG source space: The choice of method. PLoS One 12:e0181105. doi: 10.1371/journal.pone.0181105
Bréchet, L., Brunet, D., Birot, G., Gruetter, R., Michel, C. M., and Jorge, J. (2019). Capturing the spatiotemporal dynamics of self-generated, task-initiated thoughts with EEG and fMRI. Neuroimage 194, 82–92. doi: 10.1016/j.neuroimage.2019.03.029
Bréchet, L., Brunet, D., Perogamvros, L., Tononi, G., and Michel, C. M. (2020). EEG microstates of dreams. Sci. Rep. 10:17069.
Britz, J., Van De Ville, D., and Michel, C. M. (2010). BOLD correlates of EEG topography reveal rapid resting-state network dynamics. Neuroimage 52, 1162–1170. doi: 10.1016/j.neuroimage.2010.02.052
Brunet, D., Murray, M. M., and Michel, C. M. (2011). Spatiotemporal analysis of multichannel EEG: CARTOOL. Intell. Neurosci. 2011:813870. doi: 10.1155/2011/813870
Charrad, M., Ghazzali, N., Boiteau, V., and Niknafs, A. (2014). NbClust: An R package for determining the relevant number of clusters in a data set. J. Stat. Softw. 61, 1–36.
Chen, A., DeAngelis, G. C., and Angelaki, D. E. (2011). Convergence of vestibular and visual self-motion signals in an area of the posterior sylvian fissure. J. Neurosci. 31, 11617–11627. doi: 10.1523/JNEUROSCI.1266-11.2011
Chuang, S.-W., Chuang, C.-H., Yu, Y.-H., King, J.-T., and Lin, C.-T. (2016). EEG alpha and gamma modulators mediate motion sickness-related spectral responses. Int. J. Neural Syst. 26:1650007. doi: 10.1142/S0129065716500076
Clemes, S. A., and Howarth, P. A. (2005). The menstrual cycle and susceptibility to virtual simulation sickness. J. Biol. Rhythms 20, 71–82. doi: 10.1177/0748730404272567
Curry, C., Li, R., Peterson, N., and Stoffregen, T. A. (2020). Cybersickness in virtual reality head-mounted displays: Examining the influence of sex differences and vehicle control. Int. J. Hum. Comput. Int. 36, 1161–1167.
Custo, A., Van De Ville, D., Wells, W. M., Tomescu, M. I., Brunet, D., and Michel, C. M. (2017). Electroencephalographic resting-state networks: Source localization of microstates. Brain Connect. 7, 671–682. doi: 10.1089/brain.2016.0476
Damoiseaux, J. S., Beckmann, C., Arigita, E. S., Barkhof, F., Scheltens, P., Stam, C., et al. (2008). Reduced resting-state brain activity in the “default network” in normal aging. Cereb. cortex 18, 1856–1864.
Dennison, M. S., Wisti, A. Z., and D’Zmura, M. (2016). Use of physiological signals to predict cybersickness. Displays 44, 42–52. doi: 10.1089/cyber.2020.0613
Faber, P., Lehmann, D., Milz, P., Travis, F., and Parim, N. (2014). EEG microstates differ between transcending and mind wandering. ZNZ Symp. 18:131.
Freitag, S., Weyers, B., and Kuhlen, T. W. (2016). Examining rotation gain in CAVE-like virtual environments. IEEE Trans. Vis. Comput. Graph. 22, 1462–1471. doi: 10.1109/TVCG.2016.2518298
Friston, K. J., Frith, C., Liddle, P., and Frackowiak, R. (1991). Comparing functional (PET) images: The assessment of significant change. J. Cereb. Blood Flow Metab. 11, 690–699.
Fuchs, M., Kastner, J., Wagner, M., Hawes, S., and Ebersole, J. S. (2002). A standardized boundary element method volume conductor model. Clin. Neurophysiol. 113, 702–712.
Gallagher, M., and Ferrè, E. R. (2018). Cybersickness: A multisensory integration perspective. Multisensory Res. 31, 645–674.
Gschwind, M., Hardmeier, M., Van De Ville, D., Tomescu, M. I., Penner, K. I, Naegelin, Y., et al. (2016). Fluctuations of spontaneous EEG topographies predict disease state in relapsing-remitting multiple sclerosis. NeuroImage 12, 466–477. doi: 10.1016/j.nicl.2016.08.008
Hampson, M. I, Olson, R., Leung, H.-C., Skudlarski, P., and Gore, J. C. (2004). Changes in functional connectivity of human MT/V5 with visual motion input. Neuroreport 15, 1315–1319. doi: 10.1097/01.wnr.0000129997.95055.15
Hsiao, C.-Y., Liu, Y.-A., and Wang, M.-J. (2019). Evaluation of Suitable Rest Times for the Use of Optical Head-Mounted Displays. International Conference on Applied Human Factors and Ergonomics. New York: Springer.
Jahng, J., Kralik, J. D., Hwang, D.-U., and Jeong, J. (2017). Neural dynamics of two players when using nonverbal cues to gauge intentions to cooperate during the Prisoner’s Dilemma Game. NeuroImage 157, 263–274. doi: 10.1016/j.neuroimage.2017.06.024
Kalburgi, S. N., Whitten, A. P., Key, A. P., and Bodfish, J. W. (2020). Children with autism produce a unique pattern of EEG microstates during an eyes closed resting-state condition. Front. Hum. Neurosci. 14:288. doi: 10.3389/fnhum.2020.00288
Kang, J. M., Kim, N., Lee, S. Y., Woo, S. K., Park, G., Yeon, B. K., et al. (2021). Effect of Cognitive Training in Fully Immersive Virtual Reality on Visuospatial Function and Frontal-Occipital Functional Connectivity in Predementia: Randomized Controlled Trial. J. Med. Internet Res. 23:e24526. doi: 10.2196/24526
Katayama, H., Gianotti, L. R., Isotani, T., Faber, P. L., Sasada, K., Kinoshita, T., et al. (2007). Classes of multichannel EEG microstates in light and deep hypnotic conditions. Brain topography 20, 7–14. doi: 10.1007/s10548-007-0024-3
Kennedy, R. S., and Frank, L. H. (1985). A Review of Motion Sickness with Special Reference to Simulator Sickness. Orlando FL: Naval Traning equipment center.
Kennedy, R. S., Lane, N. E., Berbaum, K. S., and Lilienthal, M. G. (1993). Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3, 203–220. doi: 10.1186/s13054-016-1208-6
Khanna, A., Pascual-Leone, A., and Farzan, F. (2014). Reliability of resting-state microstate features in electroencephalography. PLoS One 9:e114163. doi: 10.1371/journal.pone.0114163
Kim, G. J. (2005). A SWOT analysis of the field of virtual reality rehabilitation and therapy. Presence 14, 119–146.
Kim, N.-G., and Kim, B.-S. (2019). The effect of retinal eccentricity on visually induced motion sickness and postural control. Appl. Sci. 9:1919.
Kim, Y. Y., Kim, H. J., Kim, E. N., Ko, H. D., and Kim, H. T. (2005). Characteristic changes in the physiological components of cybersickness. Psychophysiology 42, 616–625. doi: 10.1111/j.1469-8986.2005.00349.x
Lancaster, J. L., Woldorff, M. G., Parsons, L. M., Liotti, M., Freitas, C. S., Rainey, L., et al. (2000). Automated Talairach atlas labels for functional brain mapping. Hum. Brain Map. 10, 120–131.
Laviola, J. J. Jr. (2000). A discussion of cybersickness in virtual environments. ACM Sigchi Bull. 32, 47–56.
Lehmann, D., Faber, P. L., Galderisi, S., Herrmann, W. M., Kinoshita, T., Koukkou, M., et al. (2005). EEG microstate duration and syntax in acute, medication-naive, first-episode schizophrenia: A multi-center study. Psychiatry Res. 138, 141–156. doi: 10.1016/j.pscychresns.2004.05.007
Lei, X., Wang, Y., Yuan, H., and Mantini, D. (2014). Neuronal oscillations and functional interactions between resting state networks: Effects of alcohol intoxication. Hum. Brain Map. 35, 3517–3528. doi: 10.1002/hbm.22418
Li, G., O. Onuoha, M. McGill, S. Brewster, C. P. Chen, and F. Pollick, (2021). Decreased Brain Functional Connectivity in VR Users During Cybersickness. 3rd Neuroergonomics Conference 2021 (NEC 2021). Glasgow: Workshop Item
Liao, X., Vasilakos, A. V., and He, Y. (2017). Small-world human brain networks: Perspectives and challenges. Neurosci. Biobehav. Rev. 77, 286–300.
Lim, H. K., Ji, K., Woo, Y. S., Han, D.-u., Lee, D.-H., Nam, S. G., et al. (2021). Test-retest reliability of the virtual reality sickness evaluation using electroencephalography (EEG). Neurosci. Lett. 743:135589. doi: 10.1016/j.neulet.2020.135589
Lim, Y.-H., Kim, J.-S., Lee, H.-W., and Kim, S.-H. (2018). Postural instability induced by visual motion stimuli in patients with vestibular migraine. Front. Neurol. 9:433. doi: 10.3389/fneur.2018.00433
Liu, R., Peli, E., and Hwang, A. D. (2017). Measuring visually induced motion sickness using wearable devices. Electron. Imaging 2017, 218–223. doi: 10.1080/00140139.2019.1582805
Mazloumi Gavgani, A., Walker, F. R., Hodgson, D. M., and Nalivaiko, E. (2018). A comparative study of cybersickness during exposure to virtual reality and “classic” motion sickness: Are they different? J. Appl. Physiol. 125, 1670–1680. doi: 10.1152/japplphysiol.00338.2018
Mazziotta, J., Toga, A., Evans, A., Fox, P., Lancaster, J., Zilles, K., et al. (2001). A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). Philos. Trans. R. Soc. Lond. B Biol. Sci. 356, 1293–1322. doi: 10.1098/rstb.2001.0915
McCauley, M. E., and Sharkey, T. J. (1992). Cybersickness: Perception of self-motion in virtual environments. Presence 1, 311–318.
McMenamin, B. W., Shackman, A. J., Maxwell, J. S., Bachhuber, D. R., Koppenhaver, A. M., Greischar, L. L., et al. (2010). Validation of ICA-based myogenic artifact correction for scalp and source-localized EEG. Neuroimage 49, 2416–2432. doi: 10.1016/j.neuroimage.2009.10.010
Melo, M. C., Raposo, J. V., Coelho, A., Narciso, D. G., and Bessa, M. (2019). Immersive 360° video user experience: Impact of different variables in the sense of presence and cybersickness. Univers. Access Inf. Soc. 18, 77–87.
Michel, C. M., and Koenig, T. (2018). EEG microstates as a tool for studying the temporal dynamics of whole-brain neuronal networks: A review. Neuroimage 180, 577–593. doi: 10.1016/j.neuroimage.2017.11.062
Milz, P., Faber, P. L., Lehmann, D., Koenig, T., Kochi, K., and Pascual-Marqui, R. D. (2016). The functional significance of EEG microstates—Associations with modalities of thinking. Neuroimage 125, 643–656. doi: 10.1016/j.neuroimage.2015.08.023
Miyazaki, J., Yamamoto, H., Ichimura, Y., Yamashiro, H., Murase, T., Yamamoto, T., et al. (2015). Inter-hemispheric desynchronization of the human MT+ during visually induced motion sickness. Exp. Brain Res. 233, 2421–2431. doi: 10.1007/s00221-015-4312-y
Miyazaki, J., Yamamoto, H., Ichimura, Y., Yamashiro, H., Murase, T., Yamamoto, T., et al. (2021). Resting-state functional connectivity predicts recovery from visually induced motion sickness. Exp. Brain Res. 239, 903–921. doi: 10.1007/s00221-020-06002-7
Mognon, A., Jovicich, J., Bruzzone, L., and Buiatti, M. (2011). ADJUST: An automatic EEG artifact detector based on the joint use of spatial and temporal features. Psychophysiology 48, 229–240. doi: 10.1111/j.1469-8986.2010.01061.x
Naqvi, S. A. A., Badruddin, N., Jatoi, M. A., Malik, A. S., Hazabbah, W., and Abdullah, B. (2015). EEG based time and frequency dynamics analysis of visually induced motion sickness (VIMS). Australas. Phys. Eng. Sci. Med. 38, 721–729. doi: 10.1007/s13246-015-0379-9
Paillard, A., Quarck, G., Paolino, F., Denise, P., Paolino, M., Golding, J. F., et al. (2013). Motion sickness susceptibility in healthy subjects and vestibular patients: Effects of gender, age and trait-anxiety. J. Vestib. Res. 23, 203–209. doi: 10.3233/VES-130501
Palmisano, S., Mursic, R., and Kim, J. (2017). Vection and cybersickness generated by head-and-display motion in the Oculus Rift. Displays 46, 1–8.
Pascual-Marqui, R. D. (2007a). Discrete, 3D distributed, linear imaging methods of electric neuronal activity. Part 1: Exact, zero error localization. arXiv [Preprint]. doi: 10.48550/arXiv.0710.3341
Pascual-Marqui, R. D. (2007b). "Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: Frequency decomposition. arXiv [Preprint]. doi: 10.48550/arXiv.0711.1455
Pascual-Marqui, R. D., Lehmann, D., Koukkou, M., Kochi, K., Anderer, P., Saletu, B., et al. (2011). Assessing interactions in the brain with exact low-resolution electromagnetic tomography. Philos. Trans. A Math. Phys. Eng. Sci. 369, 3768–3784. doi: 10.1098/rsta.2011.0081
Pascual-Marqui, R. D., Michel, C. M., and Lehmann, D. (1995). Segmentation of brain electrical activity into microstates: Model estimation and validation. IEEE Trans. Biomed. Eng. 42, 658–665.
Piano, C., Imperatori, C., Losurdo, A., Bentivoglio, A. R., Cortelli, P., and Della Marca, G. (2017). Sleep-related modifications of EEG connectivity in the sensory-motor networks in Huntington Disease: An eLORETA study and review of the literature. Clin. Neurophysiol. 128, 1354–1363. doi: 10.1016/j.clinph.2016.11.019
Porras, D. C., Siemonsma, P., Inzelberg, R., Zeilig, G., and Plotnik, M. (2018). Advantages of virtual reality in the rehabilitation of balance and gait: Systematic review. Neurology 90, 1017–1025.
Rangelova, S., Rehm, K., Diefenbach, S., Motus, D., and André, E. (2020). “Gender differences in simulation sickness in static vs. moving platform VR automated driving simulation,” in International Conference on Human-Computer Interaction, ed H. Krömker (Cham: Springer), 146–165.
Seitzman, B. A., Abell, M., Bartley, S. C., Erickson, M. A., Bolbecker, A. R., and Hetrick, W. P. (2017). Cognitive manipulation of brain electric microstates. Neuroimage 146, 533–543.
Sharples, S., Cobb, S., Moody, A., and Wilson, J. R. (2008). Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays 29, 58–69.
Stanney, K., Fidopiastis, C., and Foster, L. (2020). Virtual reality is sexist: But it does not have to be. Front. Robot. AI. 7:4. doi: 10.3389/frobt.2020.00004
Toschi, N., Kim, J., Sclocco, R., Duggento, A., Barbieri, R., Kuo, B., et al. (2017). Motion sickness increases functional connectivity between visual motion and nausea-associated brain regions. Auton. Neurosci. 202, 108–113. doi: 10.1016/j.autneu.2016.10.003
Tucker, J. S., Orlando, M., and Ellickson, P. L. (2003). Patterns and correlates of binge drinking trajectories from early adolescence to young adulthood. Health Psychol. 22:79.
Vialatte, F.-B., Solé-Casals, J., Maurice, M., Latchoumane, C., Hudson, N., Wimalaratna, S., et al. (2008). Improving the Quality of EEG Data in Patients with Alzheimer’s Disease using ICA. International Conference on Neural Information Processing. New York: Springer.
Wawrzyk, M., Wesołowska, K., Plechawska-Wójcik, M., and Szymczyk, T. (2018). Analysis of Brain Activity Changes Evoked by Virtual Reality Stimuli Based on EEG Spectral Analysis. A Preliminary Study. International Conference on Information Systems Architecture and Technology. New York: Springer.
Zeng, K., Wang, Y., Ouyang, G., Bian, Z., Wang, L., and Li, X. (2015). Complex network analysis of resting state EEG in amnestic mild cognitive impairment patients with type 2 diabetes. Front. Comput. Neurosci. 9:133. doi: 10.3389/fncom.2015.00133
Zhang, L. L., Wang, J. Q., Qi, R. R., Pan, L. L., Li, M., and Cai, Y. L. (2016). Motion sickness: Current knowledge and recent advance. CNS Neurosci. Ther. 22, 15–24.
Keywords: cybersickness, EEG microstate, topographical analysis, electroencephalography, functional connectivity
Citation: Nam S, Jang K-M, Kwon M, Lim HK and Jeong J (2022) Electroencephalogram microstates and functional connectivity of cybersickness. Front. Hum. Neurosci. 16:857768. doi: 10.3389/fnhum.2022.857768
Received: 19 January 2022; Accepted: 29 July 2022;
Published: 22 August 2022.
Edited by:
Vasil Kolev, Bulgarian Academy of Sciences (BAS), BulgariaReviewed by:
Jane Zhen Liang, Shenzhen University, ChinaJustin Maximilian Mittelstädt, German Aerospace Center (DLR), Germany
Copyright © 2022 Nam, Jang, Kwon, Lim and Jeong. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Hyun Kyoon Lim, aGxpbUBrcmlzcy5yZS5rcg==; Jaeseung Jeong, anNqZW9uZ0BrYWlzdC5hYy5rcg==