Spatial Neglect Midline Diagnostics From Virtual Reality and Eye Tracking in a Free-Viewing Environment

Purpose: Virtual reality (VR) and eye tracking may provide detailed insights into spatial cognition. We hypothesized that virtual reality and eye tracking may be used to assess sub-types of spatial neglect in stroke patients not readily available from conventional assessments.

Method: Eighteen stroke patients with spatial neglect and 16 age and gender matched healthy subjects wearing VR headsets were asked to look around freely in a symmetric 3D museum scene with three pictures. Asymmetry of performance was analyzed to reveal group-level differences and possible neglect sub-types on an individual level.

Results: Four out of six VR and eye tracking measures revealed significant differences between patients and controls in this free-viewing task. Gaze-asymmetry between-pictures (including fixation time and count) and head orientation were most sensitive to spatial neglect behavior on a group level analysis. Gaze-asymmetry and head orientation each identified 10 out of 18 (56%), compared to 12 out of 18 (67%) for the best conventional test. Two neglect patients without deviant performance on conventional measures were captured by the VR and eyetracking measures. On the individual level, five stroke patients revealed deviant gaze-asymmetry within-pictures and six patients revealed deviant eye orientation in either direction that were not captured by the group-level analysis.

Conclusion: This study is a first step in using VR in combination with eye tracking measures as individual differential neglect subtype diagnostics. This may pave the way for more sensitive and elaborate sub-type diagnostics of spatial neglect that may respond differently to various treatment approaches.

1. Introduction

Globally, there is an annual incidence of about 16.9 million first-ever strokes and 33 million stroke survivors (Feigin et al., 2014). Stroke is a leading cause of cognitive impairments as approximately one third of stroke survivors live with life-long disability (Singh et al., 2018). Spatial neglect represents a common impairment following stroke affecting at least 30% of stroke survivors (Hammerbeck et al., 2019). However, SN often goes under-diagnosed and consequently under-treated (Bowen et al., 1999; Edwards et al., 2006; Chen et al., 2013). Spatial neglect (SN) constitutes a heterogeneous syndrome with several different, dissociable symptoms or subtypes (Buxbaum et al., 2004; Kerkhoff and Schenk, 2012; Rode et al., 2017). Conventional tests often simply assess one aspect of these underlying deficits. No single conventional neglect test can reliably diagnose all patients, i.e., one patient may pass the first four tests and fail the fifth, another may fail the first and pass the rest. This often relates to different subtypes of neglect e.g., motor and sensory neglect or ego- and allocentric (body and object centered) neglect. Egocentric neglect manifests itself as inattention to stimuli presented in the contralesional hemispace of different body midlines (trunk, head and eyes) and allocentric neglect as inattention to the contralesional half part of objects regardless of their egocentric placement. Many of these subtypes differ in diagnostic measures and prognostic consequences. Ego- and allocentric neglect seem to have different recovery rates (Demeyere and Gillebert, 2019) and different neglect midlines may require different treatment approaches, e.g., oculomotor neglect.

Virtual Reality (VR) combined with eye tracking may be a useful technology to detect different aspects of SN in complex 3D environments. The ability to control and monitor all motor and sensory input and output in high spatial resolution and temporal millisecond precision level may be ideal for assessing subtle impairments in spatial attention. VR has for long been a target for recording diagnostics of cognitive impairments in cognitive neuroscience, although pen-and-paper tests still dominate clinical neuropsychology. Digital tests can provide novel measures to quantify neglect that are too cumbersome to compute from pen and paper tests in clinical contexts.

A literature review by Negut et al. (2016) confirmed virtual reality to be a sensitive neuropsychological assessment tool in detecting cognitive impairment for clinical practice. They identified task performance indicators based on: 1) time measured, 2) number of errors in performing a task, 3) quantitzation of head or body movement. Only one study used head movement, the rest used task-based parameters. A number of studies has explored VR measures to assess neglect, through task based measures, either for training cognitive functions or making assessments (Nolin et al., 2019). For example, Broeren et al. (2007) used cancellation tasks to derive the pattern of search and Yasuda et al. (2020) used object detection tasks to assess near- and far SN. To assess neglect, eye tracking can be used in conjunction with head-mounted displays to scan eye movement patterns (Baheux et al., 2004, 2006). Kim et al. (2004) combined eye tracking with VR task to create a diagnostics tool for SN. Twelve patients and 40 controls were diagnosed based on deviation angle (between mandated and actual gaze position), no-attention time, scanning time, number of cues, failure rate of mission, and ratio of right/left scan. Their deviation angle correlated with line bisection test results. Other studies have used eye tracking in non-VR environments to assess neglect, for example Cazzoli et al. (2016) who measured x-axis gaze position while participants with neglect and visual field defect (VFD) viewed a projected virtual traffic scene. The neglect patients with VFD showed a significant rightward deviation in x-axis gaze position compared to those with no symptoms. But their study did not include neglect patient without visual field defects and all participants used chin rests removing the possibility of head movements. Ptak et al. (2009) assessed neglect with free-viewing of photographs, but used a chin-rest and found a group-level ipsilesional shift in the fixation distribution for the patient group. Results from eye tracking have been promising but not without contradictions. For example, Primativo et al. (2015) found no differences in the number and durations of fixations between SN and non-SN patients in a free viewing task of an albeit asymmetric picture. Studies use different measures derived from gaze measurements to identify neglect, such as re-fixations, mean amplitude and saccade landing position (Paladini et al., 2019) and horizontal fixation frequency (Ptak et al., 2009).

Contrary to setups in previous studies, this study provides both head-mounted immersive VR and eye-tracking that allows for unconstraint head and eye movement. We wanted to investigate to what extent SN can be assessed based on basic continuous measurements of head and eye movement while patients are spontaneously and freely looking around in a simple immersive VR environment and whether these measures can detect individual motor biases across different body midlines.

We hypothesized that virtual reality and eye tracking may be used to assess sub-types of spatial neglect in stroke patients not readily available from conventional assessments. Specifically, the aim was to investigate whether differences in attentional biases across different frames of references (egocentric midlines of body, head, eyes, and allocentric midlines of objects) can be assessed by VR and eye tracking on a group level (patients and controls) and individual level (subtypes diagnostics).

2. Methods

All participants were recruited at Hammel Neurorehabilitation Centre (HNC) and University Research Clinic. Patients with right hemisphere brain injury and behavioral symptoms of SN (with KF-NAP scores larger than zero) were included. KF-NAP was used as a baseline measure to identify neglect patients, due to its high sensitivity to neglect symptoms direct relation to everyday activities and changes in the severity of neglect symptoms during recovery from stroke patients (Chen et al., 2015). Patients with previous brain injury or neurodegenerative diseases as well as bedridden and blind patients were excluded. Healthy age and gender matched controls were recruited from the staff at HNC.

2.1. Conventional Measures

Conventional SN tests were applied for comparison to virtual reality measurements. The Line bisection test from the Behavioral Inattention Test (Wilson et al., 1987) requires the patients to mark the center of each of three (8 inches 20.3 mm) horizontal lines that are printed on a sheet of white paper. In the Apples Cancellation Test (Bickerton et al., 2011) patients are instructed to cancel out targets depicting outlines of apples, but only complete apples without gaps, on a sheet of white paper. The test yields individual scores for both non-lateralized visual attention (accuracy), egocentric and allocentric neglect. The gray scales gradient test requires patients to judge which of two left-right mirror-reversed gray scale gradients appears darker. For each of the 12 pairs of stimuli, one of the grayscales is shaded from white on the left to black on right, and the other is shaded in the opposite direction. The task is highly sensitive to unilateral hemispheric brain damage, and can uncover attentional biases in patients without SN symptoms on conventional cancellation or line bisection tests (Mattingley et al., 2004). The chimeric faces test consists of 12 pairs of chimeric faces generated from portraits of 10 different people smiling and 10 portraits of the same people with a neutral expression arranged vertically (Mattingley et al., 1993; Sarri et al., 2006). Each pair contains two chimeras of the same person, one neutral in the left half and smiling in the right half, and the other vice versa, with the vertical position counterbalanced. Patients are instructed to chose the happier of the two thus revealing left (20/0) or right (0/20) attentional biases. The KF-NAP was developed on the basis of the Catherine Bergego Scale (Azouvi, 1996)—the most widely used behavioral assessment instrument for SN (Azouvi, 2017). KF-NAP is a manual method for systematic observation of SN during everyday activities, including 10 categories: gaze orientation, limb awareness, auditory attention, dressing, grooming, personal belongings, navigation, collisions, having a meal, and cleaning after meals (Chen et al., 2012, 2015). Each category is scored from 0 to 3, with higher scores indicating more severe neglect. The sum score ranging from 0 to 30 indicates the severity of SN with predefined cut-off scores of the severity (0 = none, 1–10 = mild, 11–20 moderate, 21–30 severe). KF-NAP has been shown to be very sensitive to neglect symptoms directly related to everyday activities and to changes in the severity of neglect symptoms as stroke patients are recovering (Chen et al., 2015).

2.2. Virtual Reality and Eye Tracking Procedure and Apparatus

Participants were instructed about wearing a virtual reality headset and the free viewing task: “In a moment you will be placed in a museum. You get a few minutes to look around. You do not have to describe what you see.” We then equipped the participants with the virtual reality headset (Figure 1, Left) and started an eye tracking calibration procedure (staring at dots shown in the head-mounted display). After successful Pupil Labs 2D calibration, they were presented with a simple museum environment (Figure 1, Right). The environment consisted of three sets of three pictures shown on three walls. Head movement was needed to see the entirety of the left and right pictures. The picture sets were either three faces of well known persons, three similar seasonal colored landscapes, or three non-figurative paintings. The picture sets were selected to reflect symmetry of salience, were placed equally far from the middle and counterbalanced. The participants viewed each picture set for 60 s, totalling 180 s of free viewing time. While participants looked around, we logged the VR headset's position and orientation and their gaze. We used an HTC Vive VR headset with Pupil Labs eye tracking equipment, which tracks with 1 degree accuracy (Pupil Labs, 2021). The virtual environment was developed in Unity, which recorded the sensor data from the VR headset and raw raycasted gaze data using Pupil Labs' API without jitter postprocessing. The visual angle of each picture subtended 32° in width and 40° in height. The pictures were spaced 16° apart. The HTC Vive provided 110° field of view, although this depended on the fit (Lynn et al., 2020). Data from VR and Eye trackers were recorded at a 30 Hz sampling rate and analyzed with the saccades package in R.

FIGURE 1

Figure 1. (Left) Experimental setup showing note taker, technician and the participant who wore a head-mounted display whilst sitting in a wheel chair. (Middle) Participants wore an HTC Vive with Pupil Labs eye tracking cameras mounted inside. (Right) The virtual museum contained the participant and three images to the left (−48°), right (+48°), and middle (0°).

2.3. Virtual Reality and Eye Tracking Measure Description, Preparation, and Analysis

After their calculation, all measures were normalized to range from −1 (leftward) to 1 (rightward). For all measures except the fixations, we subtracted the percentage of time spent on the left side from the percentage of time spent on the right side (see Table 1). For these measures we defined impairments through cut-off criteria based on the difference between the percentages of time spent in the left and the right hemispace in the control group (e.g., including the 5th percentile or none of the controls). These support clinical diagnostics e.g., whether patients had allo-, egocentric neglect, or neglect related to head or eye midline deviations.

TABLE 1

Table 1. Virtual reality and eye tracking measurements.

Eye tracking data was filtered to include only data points from looking at the three pictures. Gaze asymmetry measured the position of the participants' gaze projected onto these pictures. Between picture gaze asymmetry left out eye tracking data from the middle picture and subtracted the amount of time spent looking at the left-most picture from the time looking at the right-most picture. These temporal aggregates were solely based on the number of eye tracking samples located on each respective picture. Their fixation counts and the totalled duration of the fixations were separate measures. Within-picture gaze asymmetry divided each picture into a left and right section and compared the participants' time spent gazing on each side, to measure allocentric neglect.

Head- and eye orientation were measured irrespective of gaze in virtual reality in order to assess motor neglect related to different body midlines. We did not correct for head position when using head orientation as the patients were seated in wheel chairs and potentially not sitting fully upright. Head Orientation L/R subtracted the percentage of time (in seconds) participants spent with the head rotated to the right from the percentage of time spent on the left. For example, if a participant's head was oriented to the right side twice as long (66%) as to the left side (33%), this measure reported an imbalance of −33%. Eye Orientation L/R was based on how much time the person was looking to the left of their visual field center line (where the nose is pointing) in comparison to looking right of it aggregated over all three pictures independent of head rotation. Fixation durations and counts are common aggregate measures of gaze data but can behave differently. One could spend equal amounts of gaze time left and right, yet still have fewer fixations and longer durations of fixations to one side. Subtracting the percentage of fixations that happened on the left from those on the right providing the measures fixation count L/R. The measure fixation duration L/R relied on the difference of the sums of fixation durations from the left and the right divided by the sum of all fixation durations.

2.4. Statistical Approach

The conventional tests produced continuous raw scores for the Apples Cancellation and the Line bisection tests, and asymmetry scores for the gray scale gradients and the Chimeric faces. They were analyzed according to their normative cut-off scores from the literature and binary neglect diagnostics were calculated. For the VR and eye tracking measures we relied on the data from the control group. For each measure we derived cut-offs (cu) by adding the mean m_c of the control group to 1.645 times its standard deviation sd_c (95th percentile): cu = m_c±1.645*sd_c. Mann-Whitney tests checked for between group differences unless indicated otherwise. An alpha value <5% (p < 0.05) was considered statistically significant. Two-sided testing with a 5% alpha value (α = 0.05) and 80% power (1−β = 0.80) along with enrolling 18 patients and 16 healthy controls allowed us to detect large effect sizes (Cohen's d > 1.0).

2.5. Participant Characteristics

Twenty three stroke patients and 16 age and gender matched controls from a hospital staff population were recruited. However, five patients were excluded for different reasons: one had a premorbid surgical fixation of the neck, that prevented natural movement of the head; one was easily fatigued and the VR procedure was aborted; one got emotional labile during testing with the conventional test and VR testing was not performed; one patient could not be satisfyingly calibrated for the eye tracking; and one patient had left-sided brain damage and right-sided neglect. Participants were matched to have equal age and gender in the patient (range 51–74, M = 61.4, SD = 6.6, 9F/9M) and control group (range 52–69, M = 60.0, SD = 4.8, 8F/8M), with no significant difference for age according to a t-test (p = 0.491). The patient demographic characteristics are presented in Table 2. Written informed consent was obtained from the participants.

TABLE 2

Table 2. Patient demographics and brain injury characteristics.

3. Results

3.1. Group-Level Results

Table 3 describes the means and p-values from Mann-Whitney tests on both conventional and VR measures used in the study. All conventional measures were significantly different (p < 0.05) between the patient and control groups. Most virtual reality measures differed significantly, too, except gaze asymmetry within picture and eye orientation left/right. The correlations between virtual reality measures are described in Table 4. Figure 2 depicts the density plots of the three pictures split by patient and control groups.

TABLE 3

Table 3. Group-level reports for patient and control groups (conventional and VR tests).

TABLE 4

Table 4. Correlations between VR measures, calculated using a pearson correlation coefficient.

FIGURE 2

Figure 2. Group level spatial distribution of gaze samples on the (solid line) and median gaze (dotted line) of patients and controls. The x-axis spans the horizontal world space of the virtual museum scene from −2.0 to 2.0. The y-axis plots the corresponding gaze sample counts in thousands (k).

3.2. Individual Results

Tables 5, 6 provides an overview of how well the conventional and virtual reality tests identified individual attentional biases in the patient and control groups. Figure 3 show individual gaze distributions.

TABLE 5

Table 5. Patient group demographics and results from conventional and VR measures.

TABLE 6

Table 6. Control group demographics and results from conventional and VR measures.

FIGURE 3

Figure 3. Individual spatial distributions of gaze samples on the three pictures with true picture midline (solid lines) and median gaze (dotted lines) of patients and controls. The x-axis reports the horizontal axis of the virtual museum scene. The y-axis counts the number of gaze samples in thousands (k).

3.3. Gaze Asymmetry

Ten patients (2, 7, 8, 17, 24, 26, 33, 34, 35, 37) and one control (6) had abnormal right-ward bias related to the viewing time of the left and right most pictures, whereas two patients (28, 36) and one control (12) had small left-ward biases contrary to expectations (Figure 4). Right-ward biases could be interpreted as SN behaviors centered at the body midline (egocentric neglect).

FIGURE 4

Figure 4. Gaze Asymmetry Between-pictures. (Left) Individual normalized time difference between looking at left/right picture in patient (black, top) and control (gray, bottom) with normative threshold cut-offs (red lines). (Right) Patient (black, top) and control (gray, bottom) histograms of horizontal gaze sample positions limited to the left and right painting with the true (solid) midline and the (dotted) group median of horizontal gaze positions.

Two patients (28, 33) and none of the controls had right-ward bias related to the viewing time within the left and right half part of the pictures, whereas three patients (26, 34, 37) and one control (18) had left-ward bias (see Figure 5). Even though patients more commonly had deviant within-picture bias than controls, the biases were both right-ward and left-ward and two patients (37, 26) had fairly large left-ward biases contrary to expectation. This measure could be interpreted as SN behaviors related to the object midline (allocentric neglect), even though the behavior of some patients was contrary to expectation.

FIGURE 5

Figure 5. Gaze Asymmetry Within-picture. (Left) Individual time spent looking right and left within all pictures in patient (black, top) and control (gray, bottom) with the empirical cut-offs (red lines). (Right) Spatial distribution of gaze samples within all pictures with true (solid) midline and group median (dotted).

3.4. Head- and Eye-Orientation

In terms of head orientation, 11 of the patients (2, 7, 8, 17, 24, 26, 33, 34, 35, 36, 37) and none of the controls showed an abnormal right-ward bias, whereas one patient (28) and one control (12) had a small abnormal left-ward bias (see Figure 6). Right-ward bias of head orientation can be interpreted as motor neglect or lack of intention to initiate head movements toward the left.

FIGURE 6

Figure 6. Head orientation left/right. (Left) Individual time spent rotating head left and right in patient (black, top) and control (gray, bottom) with normative threshold cut-offs (red lines). (Right) Spatial distribution of patient and control head orientation with true (solid) midline and group median (dotted).

Four patients (8, 33, 37, 39) and two controls (3, 18) had an abnormal right-ward eye position bias, i.e., the eyes spend more time looking to the right than the left regardless of head movement whereas two patients (26, 32) and none of the controls had left-ward eye movement biases (see Figure 7). Right-ward eye movement bias could be interpreted as oculomotor neglect or a lack of intention to move ones eyes to the left.

FIGURE 7

Figure 7. Eye orientation left/right. (Left) Individual time spent rotating eye balls left and right in patient (black, top) and control (gray, bottom) with normative threshold cut-offs (red lines). (Right) Spatial distribution of patient and control eye ball orientation with true (solid) midline and group median (dotted).

3.5. Fixation-Duration and Fixation-Count

Seven patients (7, 8, 17, 33, 34, 35, 37) and one control (3) had abnormal right-ward fixation time bias, i.e., they spent more time on each fixation in the right VR hemispace compared to the left, whereas no patients and no controls exhibited left-ward biases (see Figure 8).

FIGURE 8

Figure 8. Fixation left/right. (Left) Individual time spent fixating eyes in the left and right for patient (black, top) and control (gray, bottom) with normative threshold cut-offs (red lines). For example, –1 indicates that participants spent 100% of the time fixating on the left. (Right) Time-weighted spatial distribution of patient and control fixations, according to world midline, with time-weighted medians (dotted).

Likewise, eight patients (7, 8, 17, 24, 33, 34, 35, and 37) and none of the controls had abnormal right-ward fixation counts, interpreted from the number of saccades, whereas no patients and no controls had left-ward bias (see Figure 9).

FIGURE 9

Figure 9. Fixation count left/right. (Left) Individual fixation count right/left for patient (black, top) and control (gray, bottom) with normative threshold cut-offs (red lines). (Right) Spatial distribution of patient and control fixation counts, according to the world (solid) midline with (dotted) median.

4. Discussion

This study aimed at investigating whether attentional biases across different egocentric midlines of body, head, eyes, and allocentric midlines of objects could be assessed by VR and eye tracking on a group and individual level in patients with SN.

On a group level, gaze asymmetry between-pictures (egocentric neglect) was highly sensitive but gaze asymmetry within-pictures (allocentric neglect) was not. Likewise, head orientation (caputomotor neglect) was highly sensitive, though eye orientation (oculomotor neglect) was not. Finally, fixation time and fixation count were both highly sensitive to right-ward neglect behavior.

On an individual level, even though gaze asymmetry within-pictures (allocentric neglect) was not a significant measure of neglect behavior on a group level, five patients as opposed to one control did in fact revealed deviant behavior in either direction. Likewise did six patients as opposed to two controls revealed deviant eye orientation behavior in either direction.

Overall, gaze-asymmetry between-pictures and head orientation each identified 10 out of 18 (56%) of the stroke patients, compared to 12 out of 18 (67%) for the best of the conventional tests. Interestingly, three patients (34, 38, and 39) were not picked up by any of the conventional neglect test at all, apart from their low to moderate KF-NAP scores used as the inclusion criteria. However, two of these exhibited deviant rightward bias in one or more of the six VR and eye tracking measures. Conversely, four patients (4, 16, 21, 38) were not picked up by any of the VR and eye tracking measures, but were picked up by different conventional tests. This underlines the heterogeneity of spatial neglect and provides evidence for the complementarity of a VR and eye-tracking based free viewing task for neglect diagnostics. We observed a small rightward gaze bias in the control group's means during 180 s of viewing time, contrary to the leftward bias commonly found in the first 1.5 s by Foulsham et al. (2018). Our measures did not consider the initial gaze behavior.

The free-viewing task in our study resembled findings in group-level midline deviations from previous task-based studies, such as the left/right ratio measured by Kim et al. (2004, 2010) and the head orientation deviation found by Ogourtsova et al. (2018). The very high correlation between head orientation and gaze asymmetry between pictures matches findings from Sidenmark and Gellersen (2019), which showed that focusing on a target further than 15° away typically involves head orientation. We found a median undershooting to the right of the leftmost painting in patients (see Figure 2), similar to what Ogourtsova et al. (2018) observed when their participants approached a left-side (−15°) target in a locomotive task. Since allocentric neglect normally is observed across both sides, we decided not to sub-analyse the within-picture asymmetry on the left side further. Our group based results on fixations mirrored non-VR based studies that found fewer fixations for SN patients on the left side both in free viewing (Fellrath and Ptak, 2015; Ohmatsu et al., 2019) and visual search tasks (Cazzoli et al., 2011). Similarly our gaze asymmetry results and fixations matched the rightward median gaze in SN patients in both free viewing and visual search tasks (Machner et al., 2012, 2018). But in contrast to the study by Primativo et al. (2015) ours did find significant group level differences for fixation durations and counts.

The current free-viewing VR task was not sensitive enough to detect neglect in all patients but there are other behavioral measures that can be derived from the data collected in our setup. For example, Sidenmark and Gellersen (2019) showed that people use different combinations of torso, head, and eye rotations to acquire targets, which might differ for SN patients. Scan paths from eye tracking provide another such avenue that could leverage the spatio-temporal data from viewing each picture (e.g., re-fixations, mean amplitude, saccade landing position, Paladini et al., 2019), and initial fixation location (Foulsham et al., 2018). Our free viewing created no best outcomes but could still be analyzed according to measures similar to Dalmaijer et al. (2015) quality of search or best R.

Our approach poses some limitations to the results. The simple museum environment and the free-viewing task created low attentional demands. This may have allowed patients to try spending equal amounts of time on each picture if they expected being tested. Tracking of the headset and eye movements may have addressed motor neglect specifically but missing sensory neglect subtypes. The eye tracking data was constrained by a 30 Hz sampling rate and a fairly narrow field of view offered by the current generation head-mounted displays that provided eye tracking. Our results are limited by the accuracy of the gaze tracker both by spatial accuracy and the temporal resolution of the eye tracking data including the inherent jitter. To account for these limitations our measures did not require high spatial accuracy, except for the within-picture measure, which may account for why no significant group difference was found. While neglect patients might perform worse than controls during eye tracking calibration given the spatial arrangement of focal points across the visual field all patients included in our study successfully passed calibration. However, our results indicated opportunities for detailed measurement of behavior over time, rather than "single-instance" tests with potentially less cognitive strain and we saw opportunities for sub-diagnosis on neglect symptoms. Some of the VR and eye tracking measures incorrectly picked up four controls (3, 6, 12, 18). In total 5/96 (5.2%) observations were right-ward bias false positives and 2/96 (2.1%) were left-ward bias false positive, which fairly closely resembles the expected false positive rates from a cut-off criteria set at the 5th percentile. False positives could be reduced by setting a stricter cut-off criteria e.g., at the first percentile. Conventional neglect tests are often confounded by ceiling effects that yield high specificity (true positive rates), while they suffer from low sensitivity (true negative rates) leading to patients passing these tests while still experiencing neglect related problems in more complex everyday activities. In this study, we wanted to compare the sensitivity of VR and eye tracking measures to conventional measure for patients with clinically evident SN measured by the KF-NAP. However, including a group of stroke patients without SN may have revealed subtle gaze biases caused by inter-hemispheric imbalance in this group, too. This may very well be due to the lack of specificity (false negatives) of conventional neglect test, thus VR and eye tracking may in fact increase specificity of SN assessment. This should be investigated in future studies.

Neglect symptoms may be provoked by multitasking, simultaneous stimuli, mental fatigue, stress, or emotional states (Blini et al., 2016). These are intentionally avoided in conventional assessments usually administered in a well-controlled examination room, but could be purposefully exploited in a controlled VR environment. In addition to ceiling effects, many conventional neglect tests lack ecological validity, i.e., providing no direct link between the task tested (e.g., cancellation or line bisection tests) and activities of daily living. This may also be improved in VR and eye tracking based assessment.

This study underlined the heterogeneity of symptoms of SN and represents a first step in using virtual reality in combination with eye tracking measures for individual differential subtype diagnostics from a free viewing context. Virtual reality and eye tracking hold a potential for individual subtype diagnostics that could inform clinical treatment choices and hence treatment efficacy. Improving sensitivity, specificity and ecological validity by use of VR and eye-tracking measures may provide more accurate diagnostics and prognostics for patients with spatial neglect.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Scientific Ethics Committees for the Central Denmark Region. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

HK, JJ, and LE conceptualized the study and designed the methodology. Analysis of conventional measurements was done by LE and HK. BH and HK were responsible for data verification, cleanup and analysis of VR and eye tracking measures, and visualization of all data. BH created all graphical material and wrote the background with supervision from HK. LE collected and interpreted all clinical measurements. BH, LE, and HK jointly wrote the discussion. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors are indebted to the help of Bianca Clavio Christensen (development and testing of VR and eye tracking), Jonas Lindeløv (initial data cleaning and analysis), Kristian Westergaard Jensen (data collection), Rasmus Hougaard Pedersen (data collection), Alexander Kaiser (data collection), Kiefer Coze (VR implementation), and Théo Toupin (VR implementation).

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2021.742445/full#supplementary-material

References

Azouvi, P. (1996). Functional consequences and awareness of unilateral neglect: study of an evaluation scale. Neuropsychol. Rehabil. 6, 133–150. doi: 10.1080/713755501

CrossRef Full Text | Google Scholar

Azouvi, P. (2017). The ecological assessment of unilateral neglect. Ann. Phys. Rehabil. Med. 60, 186–190. doi: 10.1016/j.rehab.2015.12.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Baheux, K., Yoshizawa, M., Seki, K., and Handa, Y. (2006). Virtual reality pencil and paper tests for neglect: a protocol. Cyberpsychol. Behav. 9, 192–195. doi: 10.1089/cpb.2006.9.192

PubMed Abstract | CrossRef Full Text | Google Scholar

Baheux, K., Yoshizawa, M., Tanaka, A., Seki, K., and Handa, Y. (2004). Diagnosis and rehabilitation of patients with hemispatial neglect using virtual reality technology. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2, 4908–4911. doi: 10.1109/IEMBS.2004.1404357

PubMed Abstract | CrossRef Full Text | Google Scholar

Bickerton, W. L., Samson, D., Williamson, J., and Humphreys, G. W. (2011). Separating forms of neglect using the apples test: validation and functional prediction in chronic and acute stroke. Neuropsychology 25, 567–580. doi: 10.1037/a0023501

PubMed Abstract | CrossRef Full Text | Google Scholar

Blini, E., Romeo, Z., Spironelli, C., Pitteri, M., Meneghello, F., Bonato, M., et al. (2016). Multi-tasking uncovers right spatial neglect and extinction in chronic left-hemisphere stroke patients. Neuropsychologia 92, 147–157. doi: 10.1016/j.neuropsychologia.2016.02.028

PubMed Abstract | CrossRef Full Text | Google Scholar

Bowen, A., McKenna, K., and Tallis, R. C. (1999). Reasons for variability in the reported rate of occurrence of unilateral spatial neglect after stroke. Stroke 30, 1196–1202. doi: 10.1161/01.STR.30.6.1196

PubMed Abstract | CrossRef Full Text | Google Scholar

Broeren, J., Samuelsson, H., Stibrant-Sunnerhagen, K., Blomstrand, C., and Rydmark, M. (2007). Neglect assessment as an application of virtual reality. Acta Neurol. Scand. 116, 157–163. doi: 10.1111/j.1600-0404.2007.00821.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Buxbaum, L. J., Ferraro, M. K., Veramonti, T., Farne, A., Whyte, J., Ladavas, E., et al. (2004). Hemispatial neglect: Subtypes, neuroanatomy, and disability. Neurology 62, 749–756. doi: 10.1212/01.WNL.0000113730.73031.F4

PubMed Abstract | CrossRef Full Text | Google Scholar

Cazzoli, D., Hopfner, S., Preisig, B., Zito, G., Vanbellingen, T., Jäger, M., et al. (2016). The influence of naturalistic, directionally non-specific motion on the spatial deployment of visual attention in right-hemispheric stroke. Neuropsychologia 92, 181–189. doi: 10.1016/j.neuropsychologia.2016.04.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Cazzoli, D., Nyffeler, T., Hess, C. W., and Müri, R. M. (2011). Vertical bias in neglect: a question of time? Neuropsychologia 49, 2369–2374. doi: 10.1016/j.neuropsychologia.2011.04.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, P., Chen, C. C., Hreha, K., Goedert, K. M., and Barrett, A. M. (2015). Kessler foundation neglect assessment process uniquely measures spatial neglect during activities of daily living. Arch. Phys. Med. Rehabil. 96, 869.e1–876.e1. doi: 10.1016/j.apmr.2014.10.023

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, P., Hreha, K., Fortis, P., Goedert, K. M., and Barrett, A. M. (2012). Functional assessment of spatial neglect: a review of the catherine bergego scale and an introduction of the kessler foundation neglect assessment process. Top. Stroke Rehabil. 19, 423–435. doi: 10.1310/tsr1905-423

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, P., McKenna, C., Kutlik, A. M., and Frisina, P. G. (2013). Interdisciplinary communication in inpatient rehabilitation facility: evidence of under-documentation of spatial neglect after stroke. Disabil. Rehabil. 35, 1033–1038. doi: 10.3109/09638288.2012.717585

PubMed Abstract | CrossRef Full Text | Google Scholar

Dalmaijer, E. S., Van der Stigchel, S., Nijboer, T. C. W., Cornelissen, T. H. W., and Husain, M. (2015). Cancellationtools: all-in-one software for administration and analysis of cancellation tasks. Behav. Res. Methods 47, 1065–1075. doi: 10.3758/s13428-014-0522-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Demeyere, N., and Gillebert, C. R. (2019). Ego- and allocentric visuospatial neglect: dissociations, prevalence, and laterality in acute stroke. Neuropsychology 33, 490–498. doi: 10.1037/neu0000527

PubMed Abstract | CrossRef Full Text | Google Scholar

Edwards, D. F., Hahn, M. G., Baum, C. M., Perlmutter, M. S., Sheedy, C., and Dromerick, A. W. (2006). Screening patients with stroke for rehabilitation needs: validation of the post-stroke rehabilitation guidelines. Neurorehabil. Neural Repair. 20, 42–48. doi: 10.1177/1545968305283038

PubMed Abstract | CrossRef Full Text | Google Scholar

Feigin, V. L., Forouzanfar, M. H., Krishnamurthi, R., Mensah, G. A., Connor, M., Bennett, D. A., et al. (2014). Global and regional burden of stroke during 1990-2010: findings from the global burden of disease study 2010. Lancet 383, 245–254. doi: 10.1016/S0140-6736(13)61953-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Fellrath, J., and Ptak, R. (2015). The role of visual saliency for the allocation of attention: Evidence from spatial neglect and hemianopia. Neuropsychologia 73, 70–81. doi: 10.1016/j.neuropsychologia.2015.05.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Foulsham, T., Frost, E., and Sage, L. (2018). Stable individual differences predict eye movements to the left, but not handedness or line bisection. Vision Res. 144, 38–46. doi: 10.1016/j.visres.2018.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Hammerbeck, U., Gittins, M., Vail, A., Paley, L., Tyson, S. F., and Bowen, A. (2019). Spatial neglect in stroke: identification, disease process and association with outcome during inpatient rehabilitation. Brain Sci. 9:374. doi: 10.3390/brainsci9120374

PubMed Abstract | CrossRef Full Text | Google Scholar

Kerkhoff, G., and Schenk, T. (2012). Rehabilitation of neglect: an update. Neuropsychologia 50, 1072–1079. doi: 10.1016/j.neuropsychologia.2012.01.024

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, D. Y., Ku, J., Chang, W. H., Park, T. H., Lim, J. Y., Han, K., et al. (2010). Assessment of post-stroke extrapersonal neglect using a three-dimensional immersive virtual street crossing program. Acta Neurol Scand. 121, 171–177. doi: 10.1111/j.1600-0404.2009.01194.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, K., Kim, J., Ku, J., Kim, D. Y., Chang, W. H., Shin, D. I., et al. (2004). A virtual reality assessment and training system for unilateral neglect. Cyberpsychol. Behav. 7, 742–749. doi: 10.1089/cpb.2004.7.742

PubMed Abstract | CrossRef Full Text | Google Scholar

Lynn, M. H., Luo, G., Tomasi, M., Pundlik, S. E., and Houston, K. (2020). Measuring virtual reality headset resolution and field of view: Implications for vision care applications. Optom. Vis. Sci. 97, 573–582. doi: 10.1097/OPX.0000000000001541

PubMed Abstract | CrossRef Full Text | Google Scholar

Machner, B., Dorr, M., Sprenger, A., von der Gablentz, J., Heide, W., Barth, E., et al. (2012). Impact of dynamic bottom-up features and top-down control on the visual exploration of moving real-world scenes in hemispatial neglect. Neuropsychologia 50, 2415–2425. doi: 10.1016/j.neuropsychologia.2012.06.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Machner, B., Könemund, I., von der Gablentz, J., Bays, P. M., and Sprenger, A. (2018). The ipsilesional attention bias in right-hemisphere stroke patients as revealed by a realistic visual search task: neuroanatomical correlates and functional relevance. Neuropsychology 32, 850–865. doi: 10.1037/neu0000493

PubMed Abstract | CrossRef Full Text | Google Scholar

Mattingley, J. B., Berberovic, N., Corben, L., Slavin, M. J., Nicholls, M. E. R., and Bradshaw, J. L. (2004). The greyscales task: a perceptual measure of attentional bias following unilateral hemispheric damage. Neuropsychologia 42, 387–394. doi: 10.1016/j.neuropsychologia.2003.07.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Mattingley, J. B., Bradshaw, J. L., Phillips, J. G., and Bradshaw, J. A. (1993). Reversed perceptual asymmetry for faces in left unilateral neglect. Brain Cogn. 23, 145–165. doi: 10.1006/brcg.1993.1052

PubMed Abstract | CrossRef Full Text | Google Scholar

Negut, A., Matu, S.-A., Sava, F. A., and David, D. (2016). Virtual reality measures in neuropsychological assessment: a meta-analytic review. Clin. Neuropsychol. 30, 165–184. doi: 10.1080/13854046.2016.1144793

PubMed Abstract | CrossRef Full Text | Google Scholar

Nolin, P., Besnard, J., Allain, P., and Banville, F. (2019). “Assessment and rehabilitation using virtual reality after stroke: a literature review,” in Virtual Reality Technologies for Health and Clinical Applications, Chapter 15, Vol. 2, eds A. S. Rizzo and S. Bouchard (New York, NY: Springer), 307–326.

Google Scholar

Ogourtsova, T., Archambault, P. S., and Lamontagne, A. (2018). Post-stroke visual neglect affects goal-directed locomotion in different perceptuo-cognitive conditions and on a wide visual spectrum. Restor. Neurol Neurosci. 36, 313–331. doi: 10.3233/RNN-170766

PubMed Abstract | CrossRef Full Text | Google Scholar

Ohmatsu, S., Takamura, Y., Fujii, S., Tanaka, K., Morioka, S., and Kawashima, N. (2019). Visual search pattern during free viewing of horizontally flipped images in patients with unilateral spatial neglect. Cortex 113, 83–95. doi: 10.1016/j.cortex.2018.11.029

PubMed Abstract | CrossRef Full Text | Google Scholar

Paladini, R. E., Wyss, P., Kaufmann, B. C., Urwyler, P., Nef, T., Cazzoli, D., et al. (2019). Re-fixation and perseveration patterns in neglect patients during free visual exploration. Eur. J. Neurosci. 49, 1244–1253. doi: 10.1111/ejn.14309

PubMed Abstract | CrossRef Full Text | Google Scholar

Primativo, S., Arduino, L. S., Daini, R., De Luca, M., Toneatto, C., and Martelli, M. (2015). Impaired oculo-motor behaviour affects both reading and scene perception in neglect patients. Neuropsychologia 70, 90–106. doi: 10.1016/j.neuropsychologia.2015.02.020

PubMed Abstract | CrossRef Full Text | Google Scholar

Ptak, R., Golay, L., Müri, R. M., and Schnider, A. (2009). Looking left with left neglect: The role of spatial attention when active vision selects local image features for fixation. Cortex 45, 1156–1166. doi: 10.1016/j.cortex.2008.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Pupil Labs (2021). VR and AR - Eye Tracking Add-ons Technical Specifications. Available online at: https://pupil-labs.com/products/vr-ar/tech-specs/

Rode, G., Pagliari, C., Huchon, L., Rossetti, Y., and Pisella, L. (2017). Semiology of neglect: an update. Ann. Phys. Rehabil. Med. 60, 177–185. doi: 10.1016/j.rehab.2016.03.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Sarri, M., Kalra, L., Greenwood, R., and Driver, J. (2006). Prism adaptation changes perceptual awareness for chimeric visual objects but not for chimeric faces in spatial neglect after right-hemisphere stroke. Neurocase 12, 127–135. doi: 10.1080/13554790600598774

PubMed Abstract | CrossRef Full Text | Google Scholar

Sidenmark, L., and Gellersen, H. (2019). Eye, head and torso coordination during gaze shifts in virtual reality. ACM Trans. Comput. Hum. Interact. 27, 1–40. doi: 10.1145/3361218

CrossRef Full Text | Google Scholar

Singh, R. J., Chen, S., Ganesh, A., and Hill, M. D. (2018). Long-term neurological, vascular, and mortality outcomes after stroke. Int. J. Stroke 13, 787–796. doi: 10.1177/1747493018798526

PubMed Abstract | CrossRef Full Text | Google Scholar

Wilson, B. A., Cockburn, J., and Halligan, P. W. (1987). Behavioural Inattention Test: BIT. Amsterdam: Harcourt Assessment.

Yasuda, K., Kato, R., Sabu, R., Kawaguchi, S., and Iwata, H. (2020). Development and proof of concept of an immersive virtual reality system to evaluate near and far space neglect in individuals after stroke: a brief report. NeuroRehabilitation 46, 595–601. doi: 10.3233/NRE-203014

PubMed Abstract | CrossRef Full Text | Google Scholar