Anti-fatigue Performance in SSVEP-Based Visual Acuity Assessment: A Comparison of Six Stimulus Paradigms

Purpose: The occurrence of mental fatigue when users stare at stimuli is a critical problem in the implementation of steady-state visual evoked potential (SSVEP)-based visual acuity assessment, which may weaken the SSVEP amplitude and signal-to-noise ratio (SNR) and subsequently affect the results of visual acuity assessment. This study aimed to explore the anti-fatigue performance of six stimulus paradigms (reverse vertical sinusoidal gratings, reverse horizontal sinusoidal gratings, reverse vertical square-wave gratings, brief-onset vertical sinusoidal gratings, reversal checkerboards, and oscillating expansion–contraction concentric rings) in SSVEP acuity assessment.

Methods: Based on four indices of α + θ index, pupil diameter, National Aeronautics and Space Administration Task Load Index (NASA-TLX), and amplitude and SNR of SSVEPs, this study quantitatively evaluated mental fatigue in six SSVEP visual attention runs corresponding to six paradigms with 12 subjects.

Results: These indices of mental fatigue showed a good agreement. The results showed that the paradigm of motion expansion–contraction concentric rings had a superior anti-fatigue efficacy than the other five paradigms of conventional onset mode or pattern reversal mode during prolonged SSVEP experiment. The paradigm of brief-onset mode showed the lowest anti-fatigue efficacy, and the other paradigms of pattern reversal SSVEP paradigms showed a similar anti-fatigue efficacy, which was between motion expansion–contraction mode and onset mode.

Conclusion: This study recommended the paradigm of oscillating expansion–contraction concentric rings as the stimulation paradigm in SSVEP visual acuity because of its superior anti-fatigue efficacy.

Introduction

Recently, there have been some research findings across a range of applications in vision science based on steady-state visual evoked potential (SSVEP) (Norcia et al., 2015; Odom et al., 2016; Zheng et al., 2019b). As an essential part of any ophthalmological or optometric examination, visual acuity is the most commonly measured visual function (Fahad et al., 2008). Within 40 years, the SSVEP technique has been used for measuring visual acuity in some studies, demonstrating that SSVEP provides an objective and quantitative method in visual acuity assessment, especially for infants or individuals with intellectual disabilities, hysteria, or malingering (Tyler et al., 1979; Norcia and Tyler, 1985a, b; Hemptinne et al., 2018).

There are some parameters, such as electrode placement, temporal frequency, stimulus area, and sweep duration, related to SSVEP visual acuity assessment, and some studies have given their recommended parameter settings (Yadav et al., 2009; Almoqbel et al., 2011; Hemptinne et al., 2018). As for stimulus paradigms used in SSVEP visual acuity assessment, previous studies have compared some performance, such as sensitive electrodes, harmonic components of SSVEP response, correlation, and agreement between objective SSVEP and subjective psychophysical visual acuity, of six paradigms (reverse vertical sinusoidal gratings, reverse horizontal sinusoidal gratings, reverse vertical square-wave gratings, brief-onset vertical sinusoidal gratings, reversal checkerboards, and oscillating expansion–contraction concentric rings) (Tobimatsu et al., 1993; Chen et al., 2019; Zheng et al., 2019a, 2020; Hamilton et al., 2020).

However, although SSVEP can be an objective method to assess visual acuity, mental fatigue caused by uncomfortable light twinkling and contrast changes of prolonged visual stimulus can decrease arousal level and attention, worsening the SSVEP signal quality and consequently degrading the practical performance (Lee et al., 2010; Zhu et al., 2010; Cao et al., 2014; Chen et al., 2015). Previous studies have indicated that the amplitude and the signal-to-noise ratio (SNR) are related to the mental fatigue of the subjects, with decreasing amplitude and SNR corresponding to the developing fatigue (Wu et al., 2010), which can affect the precision and the accuracy of SSVEP visual acuity results since the threshold determination criterion of SSVEP visual acuity is related to the amplitude and the SNR of electroencephalography (EEG) response (Fahad et al., 2008; Yadav et al., 2009).

To evaluate the mental fatigue of prolonged SSVEP task, previous studies have proved that EEGs in the α band (8–13 Hz) and the θ band (4–7 Hz) can be adapted to assess mental fatigue (Klimesch, 1999; Cao et al., 2014; Kathner et al., 2014; Xie et al., 2016). The θ activity is related to drowsiness, while the α waves appear during relaxed conditions, at decreased attention levels and in a drowsy but wakeful state (Klimesch, 1999). Increased fatigue level is often related to the global increase of EEG power in the α and the θ bands (Klimesch, 1999; Xie et al., 2016). Moreover, pupil diameter can also be an index to evaluate mental fatigue, and the increase of mental fatigue coincides with a decrease in pupil diameter (Hopstaken et al., 2015b; Koo et al., 2018). Besides that, the National Aeronautics and Space Administration Task Load Index (NASA-TLX) is also used as a subjective and quantitative estimation of mental fatigue (Hart and Staveland, 1988; Sampei et al., 2016).

On this basis, in this study, four indices, i.e., the EEG spectral powers of α + θ, SSVEP properties of amplitude and SNR, pupil diameters recorded by the eye tracker, and subjective NASA-TLX, were measured in six SSVEP visual attention runs corresponding to six previously mentioned types of paradigms to compare their anti-fatigue performance (Cao et al., 2014; Xie et al., 2016). We hypothesized that the reversal vertical sinusoidal gratings, reverse horizontal sinusoidal gratings, reverse vertical square-wave gratings, and reversal checkerboards would show a similar anti-fatigue performance since the stimulus mode and the pattern were similar. When staring at the brief-onset vertical sinusoidal gratings, the subjects would become more fatigued because of the constantly changing brightness of the onset and offset mode. As for the oscillating expansion–contraction concentric rings, since the overall brightness was uniform when evoking steady-state motion visual evoked potential (SSMVEP) (Xie et al., 2016; Zheng et al., 2019a), its anti-fatigue property would be better than that of other stimulus paradigms.

Materials and Methods

Subjects

Twelve subjects (two females), aged between 21 and 25 years old and with normal or corrected normal visual acuity, participated in this experiment. They had no history of eye disease. All the subjects gave informed written consent following a protocol approved by the institutional review board of Xi’an Jiaotong University, conforming to the Declaration of Helsinki.

EEG Recordings

In this study, EEG signals were recorded from six occipital electrodes (PO3, PO4, POz, O1, O2, and Oz) according to the 10–20 system with a ground electrode, Fpz, placed on the forehead and a reference electrode, A1, placed on the left earlobe (Listed, 2006). The EEG signals were collected by a g.USBamp acquisition and processing system and an active electrode system g.GAMMAbox (g.tec, Schiedlberg, Austria) at a sampling rate of 1,200 Hz. Besides that, an online band-pass filter from 2 to 100 Hz was imposed to remove artifacts, and an offline notch filter between 48 and 52 Hz was applied to eliminate the power line interference.

Stimulus Designs

As shown in Figure 1, six stimulus paradigms (A: reverse vertical sinusoidal gratings, B: reverse horizontal sinusoidal gratings, C: reverse vertical square-wave gratings, D: brief-onset vertical sinusoidal gratings, E: reversal checkerboards, and F: oscillating expansion–contraction concentric rings) were introduced as six separate experimental runs (Zheng et al., 2020). As for each run, one stimulator was presented to the subjects at the center of a 24.5-in. LCD monitor (PG258Q, ASUS, Taipei, China) with a resolution of 1,920 × 1,080 pixels and a refresh rate of 240 Hz.

FIGURE 1

Figure 1. Examples of six stimulus paradigms (Zheng et al., 2020). (A) Reverse vertical sinusoidal gratings. (B) Reverse horizontal sinusoidal gratings. (C) Reverse vertical square-wave gratings. (D) Brief-onset vertical sinusoidal gratings. (E) Reversal checkerboards. (F) Oscillating expansion–contraction concentric rings. The yellow ring indicates the shape shifting of the same zone during different processes in the paradigm of concentric rings.

The subjects were asked to sit 60 cm away from the monitor with the center at eye level. The visual angle of the stimulator was 4° with a diameter of 148 pixels, in accordance with the recommended visual angle parameter of previous studies (Almoqbel et al., 2011; Ng et al., 2012). The reversal or oscillating details of the six paradigms were the same as in our previous studies, with a contrast of 99.7%, and the duty cycle of paradigm D remained at 0.3 (Zheng et al., 2019a, 2020). According to previous studies (Almoqbel et al., 2011), the spatial frequency of three cycles per degree (cpd) corresponding to 1.0 logMAR optotype and temporal frequency of 7.5 Hz was assigned to all six stimulus paradigms. In the whole experiment, a spatially homogeneous white background with luminance of 208 cd/m² was displayed in pauses and around the stimulators. The stimulus paradigms were controlled by MATLAB (MathWorks, Natick, United States) with the Psychophysics Toolbox (Brainard, 1997).

Experimental Procedure

For each subject, six runs A, B, C, D, E, and F corresponding to six stimulus paradigms A, B, C, D, E, and F were carried out, respectively. An eye tracker (Tobii X2-30, Stockholm, Sweden) was used to monitor the subjects’ eye movements and record their pupil diameter at a sample rate of 30 Hz. Each run consisted of 23 trials with three pre-experimental trials and 20 experimental trials. Each trial lasted 5 s, with an interval of 0.5 s between two trials. During the first three pre-experimental trials, to measure the baseline mental fatigue level from baseline pupil diameter and α + θ band, the subjects stared at a black screen with only a red fixation cross at the position of the center stimulator, so there was no interference from the pupillary light reflexes of the eye to the environmental lighting (Hopstaken et al., 2015a). As for the other 20 trials, the stimulator was presented and the subjects were instructed to binocularly maintain attention on the center target stimulus throughout the experiment. The order of the six runs was random, and there was enough rest time for the subjects between two runs as long as the subjects wished. Additionally, a red fixation cross was presented at the center of the paradigms to aid fixation (Almoqbel et al., 2011). The whole experiment of each subject usually lasted for about 30–45 min, depending on the inter-run rest time governed by the subjects.

NASA-TLX

NASA-TLX, originally developed as a paper-and-pencil questionnaire by NASA Ames Research Center’s Sandra Hart in the 1980s, has become a gold standard for measuring subjective workload across a wide range of applications (Hart and Staveland, 1988; Hart, 2006). Here we used NASA-TLX to assess mental fatigue subjectively as a psychological measurement. We assumed that workload deduced by the NASA-TLX represented the mental fatigue of the subjects (Sampei et al., 2016).

Firstly, the six defined sources of workload – mental demand (MD), physical demand (PD), temporal demand (TD), performance (PE), effort (EF), and frustration (FR) – were explained to the subjects.

The instruction was in Chinese as it is their native language. Then, after the subjects completed each run, they were asked to evaluate the six factors on a 0–100 scale. Next, after all the runs were completed, the subjects were asked to complete a pairwise comparison method of the six defined sources. The weights a, b, c, d, e, and f were assigned to each of the six workload sources from the pairwise comparison results, with weight integers ranging from 0 to 5, and their combinations were C(6, 2) = 15 (Sampei et al., 2016). Finally, the individual NASA-TLX of each run for each subject was derived from a weighted average of the ratings of these six factors:

$$\begin{array}{c}\text{NASA}-\text{TLX}\hfill \\ =\frac{a*\text{MD}+b*\text{PD}+c*\text{TD}+d*\text{PE}+e*\text{EF}+f*\text{FR}}{15}.\hfill \end{array}(1)$$

Signal Processing of the EEG Data

Canonical Correlation Analysis

Canonical correlation analysis (CCA) is a non-parametric multivariable method used to reflect the overall linear correlation between two groups of variables, and it is also used in the analysis of SSVEPs (Lin et al., 2007; Bin et al., 2009; Yan et al., 2018). In our study, it was also used to describe the correlations between the multi-channel SSVEP signals X and reference signal Y_i. X is the six-electrode channel signal in each trial. The reference signal Y_i composed of sine and cosine pairs is constructed at the reference frequency f_i (i = 1, 2, …, N):

$$Y_i=\left(\begin{array}{c}\hfill \sin \left(2\mathrm{\pi }{f}_{i}t\right)\hfill \\ \hfill \cos (2\mathrm{\pi }{f}_{i}t)\hfill \end{array}\right),t=\frac{1}{F_{\text{s}}},\mathrm{\dots },\frac{S}{F_{\text{s}}}.(2)$$

where F_s is the sampling rate, and S is the sample point. Here the reference frequency f_i is set to 1.0, 1.1, …, 35.0 Hz (i.e., N = 341).

The linear transformations of X and Y_i are x = w_x^TX and y_i = w_yi^TY_i, respectively, and the maximum correlation coefficient value ρ_i between X and Y_i can be calculated by the CCA method as:

$${\mathrm{\rho }}_i=\underset{w_x,w_y}{\max }\frac{E[w_x^{T}X{Y}_i^T{w}_{yi}]}{\sqrt{E\left[w_x^{T}X{X}^T{w}_x\right]E[w_{yi}^T{Y}_i{Y}_i^T{w}_{yi}]}}.(3)$$

where E denotes the symbol of the expected value in statistics, and the superscript T indicates the transposed matrix. The maximum correlation coefficient value ρ_i, which represents the maximum correlation between X and Y_i, can be considered as the response to the stimulus paradigm of SSVEPs at the reference frequency f_i (f₁, f₂…, f_N). Therefore, all the ρ_i and their corresponding frequency f_i can be plotted as a CCA spectrum. The ρ_i at the stimulus frequency of 7.5 Hz was regarded as the SSVEP amplitude.

Signal-to-Noise Ratio

Signal-to-noise ratio (SNR) refers to the ratio of signal to noise in a device or system. In our study, the SNR was defined as the ratio of the square of the CCA coefficient at the stimulus frequency of 7.5 Hz to the mean value of the square of the n adjacent points on the CCA spectrum:

$$\text{SNR}=\frac{z{(f)}^2}{\frac{1}{n}*{\sum }_{k=1}^{\frac{n}{2}}\left[z{\left(f+c*k\right)}^2+z{\left(f-c*k\right)}^2\right]}.(4)$$

where n is set to 10, and f is 7.5 Hz. z(f) is the CCA coefficient of the stimulus frequency f on the CCA spectrum. Then, c is the scale value of abscissa on the CCA spectrum, which is set to 0.1.

EEG Spectral Powers of α + θ Band

Common average reference (CAR) fusion is a commonly used EEG spatial filtering method performed by subtracting the mean of all electrode signals from the selected electrode signals to enhance the SNR of the selected electrode signals (Friman et al., 2007; Yan et al., 2019). In this study, we chose Oz electrode in spectral analysis, so the time domain EEG signal V_i to be analyzed can be expressed as:

$$V_i=V_{Oz}-\frac{1}{6}\sum _{j=1}^6{V}_j,(5)$$

where V_j is the EEG signal from six electrode channels (PO3, PO4, POz, O1, O2, and Oz).

As for the processing of the EEG signals, firstly, a band-pass filter of 3–45 Hz was carried out to remove low-frequency drift and high-frequency interference. Then, CAR fusion was used for spatial filtering in each trial. Next, the Welch power spectrum density (PSD) in bins of 0.1 Hz was used for spectral analysis to obtain the EEG spectral powers of the α band of 8–13 Hz and the θ band of 4–7 Hz. Finally, the sum value of the PSD amplitude in the frequency band on the Welch power spectrum was defined as the EEG band power indices of the frequency bands of α and θ (Cao et al., 2014). Hence, the EEG combined index (α + θ) in each trial was obtained.

As the stimulation time and the experimental trial increased, the subject could get more fatigued. Hence, the mean values and SD of each index in the 1–5, 6–10, 11–15, and 16–20 experimental trials of each run were used to represent the corresponding four fatigue levels (i.e., level 1, level 2, level 3, and level 4), respectively (Xie et al., 2016). Fatigue level 4 represented the most fatigued state, while level 1 represented the least fatigued state.

Statistical Analysis

Statistical analyses were carried out using SPSS 22.0 (IBM, Armonk, United States). One-way or two-way repeated-measures analysis of variance (ANOVA) with a significance of P < 0.05 was employed to evaluate the significance of changes in the four indices of α + θ index, SSVEP amplitude and SNR, pupil diameter index, and NASA-TLX index of six paradigms at two fatigue levels, i.e., fatigue level 1 and level 4. The post-hoc analysis with Bonferroni correction method for multiple comparisons was also used when necessary. Besides that, we used equal signs and inequality signs to visualize the anti-fatigue performance among the six paradigms based on each index.

Results

Pre-experimental Trials

As for each subject, we assumed that the initial mental fatigue was the same at the beginning of each run since there was enough rest time between two runs and the order of the presentation of the six runs was random. Here, to verify this assumption, we estimated the difference of initial baseline mental fatigue among six runs corresponding to six paradigms. As the first three pre-experimental trials of each run were presented with a black background, the mean fatigue level of the first three pre-experimental trials can be regarded as the initial mental fatigue for each run. One-way repeated-measures ANOVA was used to analyze the difference in pupil diameter and α + θ band of the first three pre-experimental trials for each paradigm. As shown in Figure 2, there was no significant difference both in the pupil diameter and the α + θ band of the first three pre-experimental trials for each paradigm [F_(5,55) = 0.687, P = 0.635 for pupil diameter; Greenhouse–Geisser correction: F_{(3.238,35.623)} = 0.774, P = 0.525 for the α + θ band], demonstrating that our assumption that the initial mental fatigue was the same at the beginning of each run was credible.

FIGURE 2

Figure 2. Comparison of the mean values and SD of the pupil diameter and α + θ band index of the first three pre-experimental trials for each paradigm over 12 subjects. Statistics were assessed by one-way repeated-measures ANOVA. (A) Pupil diameter index. (B) α + θ band index.

Comparison of NASA-TLX

For the convenience of data analysis, the NASA-TLX of psychological measurement of mental fatigue was normalized per subject by his/her maximal value of six workload sources. The mean values and SD of normalized NASA-TLX for six stimulation paradigms over 12 subjects are shown in Figure 3. One-way repeated-measures ANOVA revealed that there was a significant difference in NASA-TLX among six paradigms [F_(5,55) = 0.074, P = 0.044]. As the corresponding Bonferroni post-hoc analysis shows in Table 1, there was no difference in the mean values of NASA-TLX among paradigms A, B, C, and E, demonstrating that these pattern reversal paradigms had a similar stimulus intensity for the human eyes. Paradigm D had the highest mean value of NASA-TLX than the other paradigms (P < 0.05, respectively), demonstrating that visual stimulation of onset mode had a high stimulus intensity, which may be due to the repetitive attentional demands of continuous flicker and contrast change (Xie et al., 2017). Except for paradigms C and E with a slightly but non-significantly higher NASA-TLX than paradigm F, there was a significant difference in the mean values of NASA-TLX between paradigms F and A, B, and D (P < 0.05, respectively), showing that paradigm F had the lowest value of NASA-TLX, in favor of that motion expansion–contraction SSMVEP stimulation which exhibited a superior anti-fatigue efficacy over the conventional flickering or pattern reversal SSVEP stimulation (Xie et al., 2016). Hence, the anti-fatigue performance of six paradigms based on NASA-TLX was as follows: F > E = A = B = C > D.

TABLE 1

Table 1. Bonferroni post-hoc analysis of National Aeronautics and Space Administration Task Load Index among six stimulus paradigms.

FIGURE 3

Figure 3. Comparison of the mean values and SD of normalized National Aeronautics and Space Administration Task Load Index for six stimulus paradigms over 12 subjects. Statistics were assessed by one-way repeated-measures ANOVA.

Comparison of SSVEP Amplitude and SNR

To compare the changes in amplitude and SNR, the mean values and SD of the SSVEP amplitude and SNR summed over the stimulus frequency of 7.5 Hz in the 1–5 and 16–20 trials of each run were grouped to represent fatigue levels 1 and 4, respectively, as shown in Figure 4. Two-way repeated-measures ANOVA revealed that the interaction of two factors of “stimulus paradigm” and “fatigue level” yielded significance in SSVEP amplitude [F_(5,55) = 2.955, P = 0.020] and SNR [F_(5,55) = 2.695, P = 0.030]. Subsequently, one-way repeated-measures ANOVA found a significant difference in SSVEP amplitude among six paradigms at fatigue level 1 [Greenhouse–Geisser F_{(2.510,27.609)} = 14.116, P < 0.001] and fatigue level 4 [Greenhouse–Geisser F_{(2.637,29.006)} = 11.019, P < 0.001]. As the corresponding Bonferroni post-hoc analysis of SSVEP amplitude among six paradigms at fatigue level 1 and level 4 shows in Tables 2, 3, the SSVEP amplitude induced by different stimulus paradigms was different. Paradigms E and D induced the highest amplitude, and paradigms A, B, and C induced the lowest amplitude, with paradigm F in between.

TABLE 2

Table 2. Bonferroni post-hoc analysis of steady-state visual evoked potential amplitude among six paradigms at fatigue level 1.

TABLE 3

Table 3. Bonferroni post-hoc analysis of steady-state visual evoked potential amplitude among six paradigms at fatigue level 4.

FIGURE 4

Figure 4. Comparison of the mean values and SD of steady-state visual evoked potential amplitude and signal-to-noise ratio (SNR) between fatigue level 1 and fatigue level 4 for each paradigm over 12 subjects. (A) Amplitude. (B) SNR. Statistics were assessed by one-way repeated-measures ANOVA. ^∗∗P < 0.01; ^∗P < 0.05.

One-way repeated-measures ANOVA was also used to analyze the difference in SSVEP amplitude for paradigms A, B, C, D, E, and F between fatigue level 1 and level 4 and found a significant decrease in paradigms B, C, and D [F_(1,11) = 12.201, P = 0.005 for paradigm B; F_(1,11) = 5.047, P = 0.046 for paradigm C; F_(1,11) = 13.749, P = 0.003 for paradigm D]. The same trend of decrease was also found in paradigms A and E without statistical significance [F_(1,11) = 2.550, P = 0.139 for paradigm A; F_(1,11) = 4.353, P = 0.061 for paradigm E]. However, there was no obvious change in amplitude for paradigm F [F_(1,11) = 0.083, P = 0.779].

A similar significant difference of SNR results can also be found among six paradigms at fatigue level 1 [Greenhouse–Geisser F_{(2.427,26.696)} = 11.949, P < 0.001] and fatigue level 4 [Greenhouse–Geisser F_{(2.688,29.569)} = 10.594, P < 0.001] by one-way repeated-measures ANOVA. The corresponding Bonferroni post-hoc analysis of SSVEP SNR among six paradigms at fatigue level 1 and level 4 is shown in Tables 4, 5, revealing that paradigms E and D had the highest SNR and paradigms A, B, and C had the lowest SNR, with paradigm F in between.

TABLE 4

Table 4. Bonferroni post-hoc analysis of steady-state visual evoked potential signal-to-noise ratio among six paradigms at fatigue level 1.

TABLE 5

Table 5. Bonferroni post-hoc analysis of steady-state visual evoked potential signal-to-noise ratio among six paradigms at fatigue level 4.

One-way repeated-measures ANOVA found that SNR had a similar decrease for paradigms A, B, and D between fatigue level 1 and level 4 [F_(1,11) = 6.175, P = 0.030 for paradigm A; F_(1,11) = 12.471, P = 0.005 for paradigm B; F_(1,11) = 8.584, P = 0.014 for paradigm D]. The same but non-significant trend of decrease was also found in paradigm C [F_(1,11) = 1.103, P = 0.316] and paradigm E [F_(1,11) = 1.341, P = 0.271]. However, there was no obvious change in SNR for paradigm F [F_(1,11) < 0.001, P = 0.999].

This implied that the factor of the types of stimulus paradigm had a significant influence on the SSVEP response during prolonged usage. Both SSVEP amplitude and SNR between fatigue level 1 and fatigue level 4 had a downtrend for paradigms A, B, C, D, and E, but paradigm F did not present a significant change in SSVEP amplitude and SNR between fatigue level 1 and level 4. These results were also in line with previous studies such that paradigm F of motion SSMVEP stimulation exhibited a superior anti-fatigue efficacy than conventional flickering or pattern reversal SSVEP stimulation during prolonged SSVEP visual acuity assessment (Xie et al., 2016). Hence, the anti-fatigue performance of six paradigms based on SSVEP amplitude and SNR was as follows: F > E > A = C = B ≥ D.

Comparison of Pupil Diameter Index

For the convenience of data analysis, the pupil diameter corresponding to each paradigm was normalized for each subject by his/her respective baseline pupil diameter of the first three pre-experimental trials of each paradigm. Figure 5A presents the normalized pupil diameter index between fatigue level 1 and fatigue level 4 for six stimulus paradigms over 12 subjects. Two-way repeated-measures ANOVA revealed that the interaction of two factors of “stimulus paradigm” and “fatigue level” yielded significance in normalized pupil diameter index [F_(5,55) = 2.727, P = 0.029]. Subsequently, one-way repeated-measures ANOVA found no significant difference in pupil diameter index among six paradigms at fatigue level 1 [F_(5,55) = 1.796, P = 0.129] and fatigue level 4 [F_(5,55) = 0.170, P = 0.973].

FIGURE 5

Figure 5. Comparison of the mean values and SD of normalized pupil diameter index and normalized α + θ index between fatigue level 1 and fatigue level 4 for six stimulus paradigms over 12 subjects. (A) Normalized pupil diameter index. Statistics were assessed by repeated-measures ANOVA. ^∗∗P < 0.01; ^∗P < 0.05. (B) Normalized α + θ index.

One-way repeated-measures ANOVA was also used to analyze the difference in pupil diameter index for all six paradigms between fatigue level 1 and level 4 and found a significant decrease in paradigms A, B, C, D, and E [F_(1,11) = 18.291, P = 0.001 for paradigm A; F_(1,11) = 15.803, P = 0.002 for paradigm B; F_(1,11) = 15.226, P = 0.002 for paradigm C; F_(1,11) = 8.134, P = 0.016 for paradigm D; F_(1,11) = 13.177, P = 0.004 for paradigm E). However, there was no obvious change in pupil diameter index for paradigm F [F_(1,11) = 0.091, P = 0.769]. This also revealed that paradigm F had better anti-fatigue efficacy than the other five paradigms. Hence, the anti-fatigue performance of the six paradigms based on the pupil diameter index was as follows: F > E = A = B = C = D.

Comparison of α + θ Index

The α + θ band corresponding to each paradigm was normalized for each subject by his/her respective baseline α + θ band of the first three pre-experimental trials of each paradigm. Figure 5B presents the normalized α + θ index between fatigue level 1 and fatigue level 4 for six stimulus paradigms over 12 subjects. Two-way repeated-measures ANOVA revealed that the interaction of two factors of “stimulus paradigm” and “fatigue level” was non-significant in normalized α + θ index [F_(5,55) = 1.930, P = 0.104]. The factor of “stimulus paradigm” had a significant effect on α + θ index [Greenhouse–Geisser F_{(2.466,27.128)} = 13.166, P < 0.001], and the corresponding Bonferroni post-hoc analysis of α + θ index among six paradigms is shown in Table 6. The factor of “fatigue level” also had a significant effect on α + θ index [F_(1,11) = 9.028, P = 0.012]. The α + θ index and its change of paradigms A, B, and C were similar, revealing that paradigms A, B, and C had a similar anti-fatigue performance. Similarly, paradigms D and E also had a close anti-fatigue performance. Although the α + θ index of paradigm F was slightly higher than those of other paradigms at fatigue level 1, there was little change in the α + θ index between fatigue level 1 and level 4, demonstrating that paradigm F had better anti-fatigue efficacy than the other five paradigms. Hence, the anti-fatigue performance of six paradigms based on the α + θ index was as follows: F > A = B = C = E ≥ D.

TABLE 6

Table 6. Bonferroni post-hoc analysis of the α + θ index among six paradigms.

Discussion

In this study, we used four indices of α + θ index, pupil diameter index, NASA-TLX, and SSVEP amplitude and SNR to compare the mental fatigue and anti-fatigue performance of six paradigms in SSVEP visual acuity assessment. First, as for α + θ index, the θ waves tend to appear during meditative, drowsy, hypnotic, or sleeping states, and the increase in θ waves is related to performance decrements on task (Klimesch, 1999; Xie et al., 2016). The α waves appear during wakeful relaxation with closed eyes at decreased attention levels and in a drowsy but wakeful state, and the increase in α waves associated with fatigue is related to the increased mental effort to maintain vigilance level (Cao et al., 2014; Kathner et al., 2014). More specifically, the decreased attention and arousal level caused by mental fatigue are associated with the global increase in the θ and the α activities. Hence, the α + θ index shows a significant increase associated with the increasing fatigue level and also related to the mental alertness level (Eoh et al., 2005; Cao et al., 2014). Second, as for the pupil diameter, it is also a well-documented psycho-physiological proxy of effort, load on memory, and arousal (Peysakhovich et al., 2015), and the decrease in pupil diameter is related to deep breathing, mental work, and sleep. Hence, the increase of mental fatigue coincides with the decrease in pupil diameter (Hopstaken et al., 2015b). Third, NASA-TLX is a gold standard for measuring subjective mental fatigue across a wide range of applications (Hart and Staveland, 1988). Finally, previous studies have proved that the SSVEP amplitude and SNR can be significantly affected by the increasing fatigue level, and the amplitude and SNR of the elicited SSVEP are easily affected by mental states, fatigue, and degree of attention level (Cao et al., 2014; Xie et al., 2016).

This study focused on the mental fatigue effects caused by the long-time SSVEP stimulus of six stimulus paradigms. The results of all the indices of α + θ index, pupil diameter index, NASA-TLX, and SSVEP amplitude and SNR showed that paradigm F of motion expansion–contraction had a superior anti-fatigue efficacy than the other five paradigms of conventional onset mode or pattern reversal SSVEP stimulation during prolonged SSVEP experiment. The paradigm D of brief-onset mode showed the lowest anti-fatigue efficacy, and the other paradigms A, B, C, and E of pattern reversal SSVEP stimulation paradigms showed a similar anti-fatigue efficacy, which was between paradigms D and F. These indices of mental fatigue showed a good agreement. The results showed the anti-fatigue performance calculated averagely through all the four indices of mental fatigue estimation of six paradigms as follows: F > E ≥ A = B = C > D. Besides that, the decrease of SSVEP amplitude and SNR caused by mental fatigue during prolonged EEG experiment especially in paradigms A, B, C, D, and E may consequently deteriorate the SSVEP visual acuity assessment since the threshold determination criterion of SSVEP acuity is related to the amplitude and SNR of SSVEP response (Zheng et al., 2019a). Hence, we recommended paradigm F of oscillating expansion–contraction concentric rings as the stimulation paradigm in SSVEP visual acuity.

The reason for paradigm F to have the highest anti-fatigue property may be because of its uniform brightness and position changes rather than luminance alternations when presented to the subject, which overcame the problem of visual fatigue caused by uncomfortable light twinkling and contrast changes in the pattern reversal and brief-onset mode (Xie et al., 2012, 2016; Han et al., 2018; Yan et al., 2018). According to the theory of visual pathways, the visual system is divided into two major pathways of the parvocellular pathway and the magnocellular pathway (Pokorny and Smith, 1997). The magnocellular pathway contains the detection of dynamic motion and depth, whereas the parvocellular pathway contains the detection of spatial contrast and color information, with a slower propagation than the magnocellular pathway. Previous studies have proposed that attention uses the faster and more dominant signals of the magnocellular pathway to give priority to stimuli and simultaneously enhance the activity of the parvocellular pathway (Pokorny and Smith, 1997; Di Russo and Spinelli, 1999; Yeshurun and Sabo, 2012). If attentional networks are more reliant on parvocellular pathways, extra reaction time and demand are required for attention (Li et al., 2007; Laycock et al., 2008). Hence, the attention demand may be alleviated in motion expansion–contraction mode in paradigm F, while the contrast change in paradigm D of brief-onset mode may be a bit intense, resulting in the increase of attention demand.

There were also some limitations in this study that should be weighted. First, although the temporal frequency of 7.5 Hz was often used in SSVEP visual acuity (Almoqbel et al., 2011; Hamilton et al., 2020), the only one temporal frequency did not fully consider all relevant research since mental fatigue was limited by stimulus frequency to some extent (Won et al., 2016). Second, the number of trials and the time spent on one run were not necessarily the same as those of the SSVEP acuity test in clinical experiments, which may also have a certain influence on the results. Third, we used binocular rather than monocular viewing in our study, and the two may be not completely equivalent. Fourth, we used a consistent spatial frequency rather than a set of sweep spatial frequencies similar to the SSVEP visual acuity experiment, which may also lead to some difference in results from the actual experiment. Finally, in this study, compared to pupil diameter index and α + θ index, there were no corresponding baseline of SSVEP amplitude and SNR in the first three pre-experimental trials of each paradigm because of no visual stimulus during the three trials, so SSVEP amplitude and SNR were not normalized by respective baseline.

Conclusion

To conclude, this study has explored the anti-fatigue performance of six stimulus paradigms (reverse vertical sinusoidal gratings, reverse horizontal sinusoidal gratings, reverse vertical square-wave gratings, brief-onset vertical sinusoidal gratings, reversal checkerboards, and oscillating expansion–contraction concentric rings) used in SSVEP visual acuity assessment. Four indices of α + θ index, pupil diameter index, NASA-TLX, and SSVEP amplitude and SNR were proposed to estimate mental fatigue quantitatively. These indices of mental fatigue showed a good agreement. The results showed that the paradigm of motion expansion–contraction had a superior anti-fatigue efficacy than the other five paradigms of conventional onset mode or pattern reversal mode during prolonged SSVEP experiment. The paradigm of brief-onset mode showed the lowest anti-fatigue efficacy, and the other paradigms of pattern reversal mode showed a similar anti-fatigue efficacy, which was between motion expansion–contraction mode and onset mode.

Except for brief-onset vertical sinusoidal gratings, the four commonly used stimulus paradigms (i.e., reverse vertical sinusoidal gratings, reverse horizontal sinusoidal gratings, reverse vertical square-wave gratings, and reversal checkerboards) in SSVEP acuity assessment had a relatively good anti-fatigue property, indicating that mental fatigue could not affect the SSVEP acuity estimation too much when using the four stimulus paradigms. As for the paradigm of oscillating expansion–contraction concentric rings, it had the highest anti-fatigue property, and we recommended the oscillating expansion–contraction concentric rings as the stimulus paradigm in SSVEP acuity assessment.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation, to any qualified researcher.

Ethics Statement

The studies involving human participants were reviewed and approved by the institutional review board of the Xi’an Jiaotong University. The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

XZ contributed to the study design, data acquisition, analysis, interpretation, manuscript writing, and revision. GX contributed to the study design and the approval of the final version for publication. YZ contributed to the statistical analysis and manuscript drafting. RL and KZ contributed to the data analysis and interpretation. YD contributed to the manuscript writing and revision. JX provided the experimental equipment and approved the final version for publication. SZ conceptualized the study. All authors contributed to the article and approved the submitted version.

Funding

This research was supported by grants from the Special Guidance Funds for the Construction of World-Class Universities (Disciplines) and the Characteristic Development in Central Universities (PY3A071), the National Key Research and Development Program of China (2017YFC1308500), and the National Natural Science Foundation of China (NSFC-51775415).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Almoqbel, F. M., Yadav, N. K., Leat, S. J., Head, L. M., and Irving, E. L. (2011). Effects of sweep VEP parameters on visual acuity and contrast thresholds in children and adults. Graefes. Arch. Clin. Exp. Ophthalmol. 249, 613–623. doi: 10.1007/s00417-010-1469-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Bin, G., Gao, X., Yan, Z., Hong, B., and Gao, S. (2009). An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. J. Neural Eng. 6:046002. doi: 10.1088/1741-2560/6/4/046002

CrossRef Full Text | Google Scholar

Brainard, D. H. (1997). The psychophysics toolbox. Spat. Vis. 10, 433–436. doi: 10.1163/156856897X00357

CrossRef Full Text | Google Scholar

Cao, T., Wan, F., Wong, C. M., da Cruz, J. N., and Hu, Y. (2014). Objective evaluation of fatigue by EEG spectral analysis in steady-state visual evoked potential-based brain-computer interfaces. Biomed. Eng. Online 13:28. doi: 10.1186/1475-925X-13-28

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, X., Wang, Y., Nakanishi, M., Gao, X., Jung, T. P., and Gao, S. (2015). High-speed spelling with a noninvasive brain-computer interface. Proc. Natl. Acad. Sci. U.S.A. 112, E6058–E6067. doi: 10.1073/pnas.1508080112

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, X., Wang, Y., Zhang, S., Xu, S., and Gao, X. (2019). Effects of stimulation frequency and stimulation waveform on steady-state visual evoked potentials using a computer monitor. J. Neural Eng. 16:066007. doi: 10.1088/1741-2552/ab2b7d

PubMed Abstract | CrossRef Full Text | Google Scholar

Di Russo, F., and Spinelli, D. (1999). Spatial attention has different effects on the magno- and parvocellular pathways. Neuroreport 10, 2755–2762. doi: 10.1097/00001756-199909090-00011

PubMed Abstract | CrossRef Full Text | Google Scholar

Eoh, H. J., Chung, M. K., and Kim, S. H. (2005). Electroencephalographic study of drowsiness in simulated driving with sleep deprivation. Int. J. Ind. Ergonom. 35, 307–320. doi: 10.1016/j.ergon.2004.09.006

CrossRef Full Text | Google Scholar

Fahad, A., Leat, S. J., and Elizabeth, I. (2008). The technique, validity and clinical use of the sweep VEP. Ophthal. Physl. Opt. 28, 393–403. doi: 10.1111/j.1475-1313.2008.00591.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Friman, O., Volosyak, I., and Graser, A. (2007). Multiple channel detection of steady-state visual evoked potentials for brain-computer interfaces. IEEE Trans. Biomed. Eng. 54, 742–750. doi: 10.1109/TBME.2006.889160

PubMed Abstract | CrossRef Full Text | Google Scholar

Hamilton, R., Bach, M., Heinrich, S. P., Hoffmann, M. B., Odom, J. V., McCulloch, D. L., et al. (2020). VEP estimation of visual acuity: a systematic review. Doc. Ophthalmol. doi: 10.1007/s10633-020-09770-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Han, C., Xu, G., Xie, J., Chen, C., and Zhang, S. (2018). Highly interactive brain-computer interface based on flicker-free steady-state motion visual evoked potential. Sci. Rep. 8:5835. doi: 10.1038/s41598-018-24008-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Hart, S. G. (2006). Nasa-task load index (NASA-TLX); 20 years later. Proc. Hum. Fact. Ergon. Soc. Annu. Meet. 50, 904–908. doi: 10.1177/154193120605000909

CrossRef Full Text | Google Scholar

Hart, S. G., and Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Adv. Psychol. 52, 139–183. doi: 10.1016/s0166-4115(08)62386-9

CrossRef Full Text | Google Scholar

Hemptinne, C., Liu-Shuang, J., Yuksel, D., and Rossion, B. (2018). Rapid objective assessment of contrast sensitivity and visual acuity with sweep visual evoked potentials and an extended electrode array. Invest. Ophthalmol. Vis. Sci. 59, 1144–1157. doi: 10.1167/iovs.17-23248

PubMed Abstract | CrossRef Full Text | Google Scholar

Hopstaken, J. F., van der Linden, D., Bakker, A. B., and Kompier, M. A. (2015a). A multifaceted investigation of the link between mental fatigue and task disengagement. Psychophysiology 52, 305–315. doi: 10.1111/psyp.12339

PubMed Abstract | CrossRef Full Text | Google Scholar

Hopstaken, J. F., van der Linden, D., Bakker, A. B., and Kompier, M. A. (2015b). The window of my eyes: task disengagement and mental fatigue covary with pupil dynamics. Biol. Psychol. 110, 100–106. doi: 10.1016/j.biopsycho.2015.06.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Kathner, I., Wriessnegger, S. C., Muller-Putz, G. R., Kubler, A., and Halder, S. (2014). Effects of mental workload and fatigue on the P300, alpha and theta band power during operation of an ERP (P300) brain-computer interface. Biol. Psychol. 102, 118–129. doi: 10.1016/j.biopsycho.2014.07.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Klimesch, W. (1999). EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res. Brain Res. Rev. 29, 169–195. doi: 10.1016/s0165-0173(98)00056-3

CrossRef Full Text | Google Scholar

Koo, B. Y., Jang, M. H., Kim, Y. C., and Mah, K. C. (2018). Changes in the subjective fatigue and pupil diameters induced by watching LED TVs. Optik 164, 701–710. doi: 10.1016/j.ijleo.2018.03.077

CrossRef Full Text | Google Scholar

Laycock, R., Crewther, D. P., and Crewther, S. G. (2008). The advantage in being magnocellular: a few more remarks on attention and the magnocellular system. Neurosci. Biobehav. Rev. 32, 1409–1415. doi: 10.1016/j.neubiorev.2008.04.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, P. L., Sie, J. J., Liu, Y. J., Wu, C. H., Lee, M. H., Shu, C. H., et al. (2010). An SSVEP-actuated brain computer interface using phase-tagged flickering sequences: a cursor system. Ann. Biomed. Eng. 38, 2383–2397. doi: 10.1007/s10439-010-9964-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, J. C., Sampson, G. P., and Vidyasagar, T. R. (2007). Interactions between luminance and colour channels in visual search and their relationship to parallel neural channels in vision. Exp Brain Res. 176, 510–518. doi: 10.1007/s00221-006-0773-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Lin, Z., Zhang, C., Wu, W., and Gao, X. (2007). Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. IEEE Trans. Biomed. Eng. 54, 1172–1176. doi: 10.1109/tbme.2006.889197

PubMed Abstract | CrossRef Full Text | Google Scholar

Listed, N. (2006). Guideline 5: guidelines for standard electrode position nomenclature. Am. J. Eeg. Technol. 46, 222–225. doi: 10.1080/1086508x.2006.11079580

CrossRef Full Text | Google Scholar

Ng, K. B., Bradley, A. P., and Cunnington, R. (2012). Stimulus specificity of a steady-state visual-evoked potential-based brain-computer interface. J. Neural Eng. 9:036008. doi: 10.1088/1741-2560/9/3/036008

CrossRef Full Text | Google Scholar

Norcia, A. M., Appelbaum, L. G., Ales, J. M., Cottereau, B. R., and Rossion, B. (2015). The steady-state visual evoked potential in vision research: a review. J. Vis. 15:4. doi: 10.1167/15.6.4

CrossRef Full Text | Google Scholar

Norcia, A. M., and Tyler, C. W. (1985a). Infant VEP acuity measurements: analysis of individual differences and measurement error. Electroencephalogr. Clin. Neurophysiol. 61, 359–369. doi: 10.1016/0013-4694(85)91026-0

CrossRef Full Text | Google Scholar

Norcia, A. M., and Tyler, C. W. (1985b). Spatial frequency sweep VEP: visual acuity during the first year of life. Vis. Res. 25, 1399–1408. doi: 10.1016/0042-6989(85)90217-2

CrossRef Full Text | Google Scholar

Odom, J. V., Bach, M., Brigell, M., Holder, G. E., McCulloch, D. L., Mizota, A., et al. (2016). ISCEV standard for clinical visual evoked potentials: (2016 update). Doc. Ophthalmol. 133, 1–9. doi: 10.1007/s10633-016-9553-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Peysakhovich, V., Dehais, F., and Causse, M. (2015). Pupil diameter as a measure of cognitive load during auditory-visual interference in a simple piloting task. Proc. Manufact. 3, 5199–5205. doi: 10.1016/j.promfg.2015.07.583

CrossRef Full Text | Google Scholar

Pokorny, J., and Smith, V. C. (1997). Psychophysical signatures associated with magnocellular and parvocellular pathway contrast gain. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 14, 2477–2486. doi: 10.1364/josaa.14.002477

PubMed Abstract | CrossRef Full Text | Google Scholar

Sampei, K., Ogawa, M., Torres, C. C. C., Sato, M., and Miki, N. (2016). Mental fatigue monitoring using a wearable transparent eye detection system. Micromachines 7:20. doi: 10.3390/mi7020020

PubMed Abstract | CrossRef Full Text | Google Scholar

Tobimatsu, S., Kurita-Tashima, S., Nakayama-Hiromatsu, M., and Kato, M. (1993). Effect of spatial frequency on transient and steady-state VEPs: stimulation with checkerboard, square-wave grating and sinusoidal grating patterns. J. Neurol Sci. 118, 17–24. doi: 10.1016/0022-510x(93)90239-u

CrossRef Full Text | Google Scholar

Tyler, C. W., Apkarian, P., Levi, D. M., and Nakayama, K. (1979). Rapid assessment of visual function: an electronic sweep technique for the pattern visual evoked potential. Invest. Ophthalmol. Vis. Sci. 18, 703–713.

Google Scholar

Won, D. O., Hwang, H. J., Dahne, S., Muller, K. R., and Lee, S. W. (2016). Effect of higher frequency on the classification of steady-state visual evoked potentials. J. Neural Eng. 13:016014. doi: 10.1088/1741-2560/13/1/016014

CrossRef Full Text | Google Scholar

Wu, Z., Yao, D., Tang, Y., Huang, Y., and Su, S. (2010). Amplitude modulation of steady-state visual evoked potentials by event-related potentials in a working memory task. J. Biol. Phys. 36, 261–271. doi: 10.1007/s10867-009-9181-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Xie, J., Xu, G., Luo, A., Li, M., Zhang, S., Han, C., et al. (2017). The role of visual noise in influencing mental load and fatigue in a steady-state motion visual evoked potential-based brain-computer interface. Sensors 17:1873. doi: 10.3390/s17081873

PubMed Abstract | CrossRef Full Text | Google Scholar

Xie, J., Xu, G., Wang, J., Li, M., Han, C., and Jia, Y. (2016). Effects of mental load and fatigue on steady-state evoked potential based brain computer interface tasks: a comparison of periodic flickering and motion-reversal based visual attention. PLoS One 11:e0163426. doi: 10.1371/journal.pone.0163426

PubMed Abstract | CrossRef Full Text | Google Scholar

Xie, J., Xu, G., Wang, J., Zhang, F., and Zhang, Y. (2012). Steady-state motion visual evoked potentials produced by oscillating Newton’s rings: implications for brain-computer interfaces. PLoS One 7:e39707. doi: 10.1371/journal.pone.0039707

PubMed Abstract | CrossRef Full Text | Google Scholar

Yadav, N. K., Almoqbel, F., Head, L., Irving, E. L., and Leat, S. J. (2009). Threshold determination in sweep VEP and the effects of criterion. Doc. Ophthalmol. 119, 109–121. doi: 10.1007/s10633-009-9177-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Yan, W., Xu, G., Chen, L., and Zheng, X. (2019). Steady-state motion visual evoked potential (SSMVEP) enhancement method based on time-frequency image fusion. Comput. Intell. Neurosci. 2019:9439407. doi: 10.1155/2019/9439407

PubMed Abstract | CrossRef Full Text | Google Scholar

Yan, W., Xu, G., Xie, J., Li, M., and Dan, Z. (2018). Four novel motion paradigms based on steady-state motion visual evoked potential. IEEE Trans. Biomed. Eng. 65, 1696–1704. doi: 10.1109/TBME.2017.2762690

PubMed Abstract | CrossRef Full Text | Google Scholar

Yeshurun, Y., and Sabo, G. (2012). Differential effects of transient attention on inferred parvocellular and magnocellular processing. Vis. Res. 74, 21–29. doi: 10.1016/j.visres.2012.06.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Zheng, X., Xu, G., Wang, Y., Han, C., Du, C., Yan, W., et al. (2019a). Objective and quantitative assessment of visual acuity and contrast sensitivity based on steady-state motion visual evoked potentials using concentric-ring paradigm. Doc. Ophthalmol. 139, 123–136. doi: 10.1007/s10633-019-09702-w

PubMed Abstract | CrossRef Full Text | Google Scholar

Zheng, X., Xu, G., Zhi, Y., Wang, Y., Han, C., Wang, B., et al. (2019b). Objective and quantitative assessment of interocular suppression in strabismic amblyopia based on steady-state motion visual evoked potentials. Vis. Res. 164, 44–52. doi: 10.1016/j.visres.2019.07.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Zheng, X., Xu, G., Wu, Y., Wang, Y., Du, C., Wu, Y., et al. (2020). Comparison of the performance of six stimulus paradigms in visual acuity assessment based on steady-state visual evoked potentials. Doc. Ophthalmol. doi: 10.1007/s10633-020-09768-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhu, D., Bieger, J., Garcia Molina, G., and Aarts, R. M. (2010). A survey of stimulation methods used in SSVEP-based BCIs. Comput. Intell. Neurosci. 2010:702357. doi: 10.1155/2010/702357

PubMed Abstract | CrossRef Full Text | Google Scholar