- 1Department of Psychiatry and Psychotherapy, Medical Center – University of Freiburg, Freiburg, Germany
- 2Faculty of Medicine, University of Freiburg, Freiburg, Germany
- 3Institute for Frontier Areas of Psychology and Mental Health, Freiburg, Germany
- 4Faculty of Biology, University of Freiburg, Freiburg, Germany
- 5Department of Psychosomatic Medicine and Psychotherapy, Medical Center – University of Freiburg, Freiburg, Germany
- 6INSERM U1114, Cognitive Neuropsychology and Pathophysiology of Schizophrenia, Strasbourg, France
- 7Bernstein Center Freiburg, University of Freiburg, Freiburg, Germany
Introduction: During observation of the ambiguous Necker cube, our perception suddenly reverses between two about equally possible 3D interpretations. During passive observation, perceptual reversals seem to be sudden and spontaneous. A number of theoretical approaches postulate destabilization of neural representations as a pre-condition for reversals of ambiguous figures. In the current study, we focused on possible Electroencephalogram (EEG) correlates of perceptual destabilization, that may allow prediction of an upcoming perceptual reversal.
Methods: We presented ambiguous Necker cube stimuli in an onset-paradigm and investigated the neural processes underlying endogenous reversals as compared to perceptual stability across two consecutive stimulus presentations. In a separate experimental condition, disambiguated cube variants were alternated randomly, to exogenously induce perceptual reversals. We compared the EEG immediately before and during endogenous Necker cube reversals with corresponding time windows during exogenously induced perceptual reversals of disambiguated cube variants.
Results: For the ambiguous Necker cube stimuli, we found the earliest differences in the EEG between reversal trials and stability trials already 1 s before a reversal occurred, at bilateral parietal electrodes. The traces remained similar until approximately 1100 ms before a perceived reversal, became maximally different at around 890 ms (p = 7.59 × 10–6, Cohen’s d = 1.35) and remained different until shortly before offset of the stimulus preceding the reversal. No such patterns were found in the case of disambiguated cube variants.
Discussion: The identified EEG effects may reflect destabilized states of neural representations, related to destabilized perceptual states preceding a perceptual reversal. They further indicate that spontaneous Necker cube reversals are most probably not as spontaneous as generally thought. Rather, the destabilization may occur over a longer time scale, at least 1 s before a reversal event, despite the reversal event as such being perceived as spontaneous by the viewer.
1. Introduction
During observation of an ambiguous figure, like the famous Necker cube (Necker, 1832), our perception becomes unstable, and alternates between two, or even more, possible interpretations despite unchanged visual input. Such perceptual reversals can occur spontaneously or suddenly, i.e., without the observer’s expectation or intention to perceive the sensory input in a different way. In contrast, observers can also volitionally control their percept to some degree and both increase and decrease their reversal rate (the number of perceptual reversals over time), however, they cannot prevent the reversals entirely (e.g., Strüber and Stadler, 1999; van Ee et al., 2005; Kornmeier et al., 2009, 2019). Multistable perception can also be induced via binocular rivalry, i.e., if the two eyes simultaneously receive conflicting information. Similar to the situation with classical ambiguous figures, perception alternates between the two percepts corresponding to the two eyes’ input (e.g., Blake, 2001; O’Shea et al., 2013). Multistable perception phenomena have also been reported in other modalities, like audition (e.g., Pressnitzer and Hupe, 2006; Einhäuser et al., 2020) and touch (e.g., Carter et al., 2008; Conrad et al., 2012; Liaci et al., 2016; Hense et al., 2019; Darki and Rankin, 2020).
The phenomenon of multistable perception has fascinated researchers from different disciplines for more than 200 years (Brascamp et al., 2018). Multistable perception is vividly discussed in the contexts of perception and consciousness (e.g., Crick and Koch, 1998; Blake and Logothetis, 2002; Long and Toppino, 2004; Atmanspacher et al., 2008; Giles et al., 2016; Brascamp et al., 2018; Devia et al., 2022), particularly because a separation of sensory processing (which can be kept constant over time) and perceptual processing (which alternates repeatedly) appears to be possible (e.g., Crick and Koch, 1998; Tong et al., 1998; Leopold and Logothetis, 1999; Intaite et al., 2010; Blake et al., 2014; Bartels, 2021). However, despite a large number of experimental studies, the neural mechanisms underlying spontaneous perceptual reversals are so far poorly understood. Problems are both theoretical and practical in nature. Very often, it is unclear what the processes are that (1) necessarily precede a spontaneous perceptual reversal (e.g., the weakening or destabilization of a neural representation) and causally lead to it, and (2) those of the reversal process as such, i.e., the change in dominance (and access to consciousness) of one neural representation by the alternative one in a certain time window, and finally (3) the subsequent processes, related to becoming aware of the reversal event and communicating it with the environment within an experimental paradigm (Brascamp et al., 2018; Devia et al., 2022). The Electroencephalogram (EEG) provides the necessary temporal resolution in the range of milliseconds to differ between these processing steps. However, the precise temporal reference of the reversal event is necessary for the differentiation between the above-described processing steps but at the same time difficult to access due to its endogenous nature.
A number of EEG- and fMRI-studies used participants’ key presses as a time reference for the reversal event (e.g., Basar-Eroglu et al., 1996; Lumer et al., 1998; Sterzer and Kleinschmidt, 2007; Knapen et al., 2011). This method, however, comes with the problem of intra-individual reaction time variability in the range of ± 100 ms (Strüber and Herrmann, 2002; Kornmeier and Bach, 2004, 2012), which is too large to exploit the high temporal resolution of the EEG. An alternative approach used ambiguous motion stimuli with specific stimulus features or presentation modes that allow to narrow down a possible time window of the reversal event (e.g., Pastukhov and Braun, 2013; Pastukhov, 2016; Weilnhammer et al., 2017, 2021). This elegant stimulus design, however, is restricted to a certain class of motion stimuli. De Jong et al. (2020) recently developed a clever method using within-participant intracranial electrophysiological measures to temporally resolve reversals of binocular rivalry stimuli.
Other studies referred to the change in pupil diameter during perceptual reversals (Einhäuser et al., 2008; Hupe et al., 2009; Kloosterman et al., 2015; Brascamp et al., 2021) or used the horizontal eye fixation positions in a free observation paradigm (Polgári et al., 2020). These innovative approaches allowed the implementation of “no report paradigms” and helped to separate task-related and perception-related processing steps (e.g., Brascamp et al., 2018, 2021; Devia et al., 2022). However, the specific role of the pupil response in the reversal process is discussed controversially (Duman et al., 2022). It is particularly unclear, whether the pupil response or a certain change in horizontal eye position precedes or follows the reversal event and how much the latency between the reversal event and the pupil response/eye position change varies across repetitions.
In the present study, we used the so-called “Onset Paradigm” to estimate the timing of the reversal event. The basic idea goes back to the seminal work of Orbach et al. (1963, 1966). They presented the Necker cube discontinuously and found a huge variability of the reversal rate (number of reversals per minute) as a function of the inter-stimulus interval (ISI), i.e., the duration of a blank screen between successive Necker cube presentations. The reversal rate continuously increased from continuous presentations to ISIs of about 400 ms and decreased again for longer ISIs, up to a complete freezing of the percept for ISIs of about 2 s (Leopold et al., 2002; Maier et al., 2003; Kornmeier et al., 2007; Pastukhov and Braun, 2008; Pearson and Brascamp, 2008; Zaretskaya et al., 2010). O’Donnell et al. (1988) were the first to present Necker cubes discontinuously while measuring EEG. They found a P300 event-related potential (ERP) component correlating with a perceptual reversal. Kornmeier and Bach (2004) adopted this idea and developed the “Onset Paradigm”: Participants compared their percepts of subsequently presented Necker cubes and indicated in separate experimental conditions either perceptual reversals (reversal condition) or perceptual stability (stability conditions) from the previously observed stimulus to the current stimulus by a key press in the subsequent ISI. Such Necker cube reversals were labeled “endogenous” because the Necker cube as such is equally compatible with both 3D interpretations and the perceptual reversal cannot be related to certain stimulus features. The idea of this paradigm was, that the discontinuous presentation mode synchronizes the reversal event with stimulus onset, allowing to use the latter as a time reference for the former. Kornmeier et al. calculated ERPs and subtracted stability trials from reversal trials in order to eliminate the ERP signatures related to stimulus onset, decision making, and motor preparation.
In a separate experiment with the same conditions, Kornmeier et al. replaced the ambiguous Necker cube with disambiguated cube versions, corresponding to the two perceptual interpretations of the ambiguous Necker cube stimulus. They labeled perceptual reversals evoked by the disambiguated stimuli as exogenous reversals. Their Onset Paradigm increased the temporal resolution of the reversal event strongly and revealed two highly similar chains of ERP components for endogenous perceptual reversals (Experiment 1) and exogenously induced reversals (Experiment 2) with two exceptions. The ERP chain related to the endogenous reversals started with a small occipital positivity (“Reversal Positivity”) at 160 ms after stimulus onset. This Reversal Positivity was not observed with exogenously induced reversals. It may indicate an early visual “detection” of a maximally unstable perceptual state in the case of an endogenous reversal, as though the early perceptual processing units are not able to handle the sensory information in the first feed-forward loop of sensory information flow (e.g., Kornmeier et al., 2011). The exogenously induced reversal (realized by the computer program) is void of this problem, which may explain the absence of the Reversal Positivity in this case. Furthermore, the subsequent three ERP components related to endogenous reversals were delayed by about 40–60 ms compared to those from the exogenously induced reversals. A comparable processing delay was observed in the reaction time data in a separate experiment and may reflect the necessary time to solve the sensory ambiguity problem.
Since its introduction, the Onset Paradigm has been used by several labs around the world and the findings, reported above, have mainly been confirmed (Kornmeier and Bach, 2004, 2006, 2012; Kornmeier et al., 2007, 2011, 2019; Pitts et al., 2007, 2008, 2010; Atmanspacher et al., 2008; Britz et al., 2010; Intaite et al., 2010, 2013; Ehm et al., 2011; Sandberg et al., 2014; Yokota et al., 2014; Abdallah and Brooks, 2020). For reviews see Pitts and Britz (2011) and Kornmeier and Bach (2012).
For the interpretation of their ERP findings, Kornmeier et al. postulated two separate processes (see Figure 1 for a schematic graphical representation): After a perceptual interpretation has been established, its neural representation starts to slowly destabilize until a state of maximal neural instability has been reached. Assuming that the brain has evolved to keep unstable states as short as possible, this instability becomes resolved by a fast restabilization process of 40—60 ms, as indicated by the ERP results, leading to the reversed perceptual interpretation (Kornmeier and Bach, 2006).
Figure 1. Schematic demonstration of the reversal dynamics. The participant observes an ambiguous Necker lattice discontinuously over a certain time interval and perceives it as facing in one orientation. The perceptual representation is postulated to destabilize over time (blue arrow) until the percept becomes so unstable (indicated by the light blue frame) that a perceptual reversal occurs. After the reversal, the perceptual representation is again temporally stable (dark blue frame). Bottom row: Necker lattice stimuli as ambiguous sensory input. Top row: perceptual interpretation of the observer. The instances R indicate perceptual reversals.
These previous EEG studies indicated that this restabilization is mainly executed in lower visual areas. However, it is so far unclear, which brain areas are affected during the postulated longer-lasting destabilization process and at which time points between two perceptual reversals we can find EEG correlates of a destabilized perceptual state. Hence, the aim of the present study is to identify EEG signatures that reflect states of gradual destabilization preceding a reversal event. Such signatures would allow us to better understand the processes underlying perceptual/neural destabilization. We focused on ERP differences between stable and unstable perceptual states in a time window before a reversal.
2. Materials and methods
2.1. Participants
EEG data from 21 participants (mean age = 25.0 years, standard deviation (SD) = 3.27 years; 11 female) was collected and analyzed. All participants had normal or corrected-to-normal visual acuity as measured with the Freiburg Visual Acuity and Contrast Test (FrACT, Bach, 2018) and gave their informed written consent. The study was approved by the ethics committee of the University of Freiburg, Germany.
2.2. Stimuli
As visual stimuli we used the ambiguous Necker lattice (a combination of nine Necker cubes, Necker, 1832) as first described in Kornmeier et al. (2001) and Kornmeier and Bach (2004) (Figure 2) and two disambiguated lattice variants corresponding to the two perceptual interpretations of the ambiguous lattice, including depth cues, like shading, central projection, and aerial perspective (Figure 2). The lattices were white (40 cd/m2) with a dark background (0.01 cd/m2). This was the luminance for both lattice variants, the disambiguated variant’s luminance was calculated by averaging the luminance across the four corners. The Necker lattices had a size of 7.5° × 7.5° degrees of visual angle. A fixation cross was presented in the center of the lattices (Joos et al., 2020).
Figure 2. The experiment consisted of two experimental conditions, the Ambiguity Condition (top row) and Disambiguation Condition (bottom row). In both conditions, the lattice stimuli were presented for 1000 ms with ISIs of 400 ms in between. Participants had to compare successive stimuli and indicate via button press whether they perceived the current lattice orientation as reversed compared to the previous lattice (reversal trials, R) or as unchanged (stability trials, S). Participants were allowed to respond from stimulus onset until the end of the subsequent ISI. We focused our analysis on the stimulus time window before (Si) and after (Si+1) the reversed percept, including the ISI in between. Diagram adapted from Kornmeier and Bach (2012) and Joos et al. (2020).
2.3. Procedure
In two experimental conditions, either ambiguous or disambiguated variants of the Necker lattice were presented discontinuously (1,000 ms presentation time) with dark screen ISIs (400 ms duration and same color as the background of the lattices) between stimulus presentations. In a 1-back task, participants compared the perceived 3D orientation of the currently presented lattice stimulus (Si+1) with the previously presented stimulus (Si) and indicated perceptual reversals and perceptual stability (same percept across the two stimulus presentations) by different keys. We kept the stimulus presentation time short enough (1,000 ms) to prevent additional reversals during stimulus observation. In the case of ambiguous lattice stimuli, perceptual reversals across stimuli were evoked endogenously by the participants’ perceptual system. In the case of the disambiguated lattice variants, the reversal was evoked by the computer program with a predefined reversal rate of 30%, reflecting average reversal rates of a Necker cube stimulus (Kornmeier et al., 2009). The two experimental conditions were subdivided in experimental blocks of approximately 7 min duration, containing typically some 300 trials.
Before the experiment, participants completed a training session where they were presented with disambiguated Necker lattices. Identical to the experiment, participants needed to indicate with a button press if the percept stayed the same or changed from Si to Si+1. The training duration was approximately 5 min. If participants got less than 90% of the trials correct, the training was repeated.
2.4. EEG recording
The EEG was recorded with 32 active silver/silver chloride electrodes using the extended 10–20 system for the electrode positions on the scalp (American Clinical Neurophysiology Society, 2006), using the BrainVision ActiCHamp amplifier. The data was digitized at a sampling rate of 1000 Hz and online bandpass filtered at 0.01–120 Hz. Impedance was kept below 10 kΩ for all electrodes.
2.5. EEG pre-processing
All pre-processing was done using the MNE-Python package (version 0.23.0) (Gramfort et al., 2013) in Python 3.9.6. The raw data was offline band-pass filtered from 0.01 to 25 Hz and re-referenced to the average of the mastoid electrodes (TP9, TP10). The vertical electrooculogram (vEOG) electrodes were used to determine eye blink artifacts. Due to technical problems, one participant did not have a vEOG electrode, and therefore the Fp1 electrode was used for artifact detection. The vEOG electrode (or Fp1 electrode) was bandpass filtered from 1 to 10 Hz.
To detect eye blink artifacts an Independent Component Analysis (ICA) was conducted. A correlation was calculated between the vEOG electrode and the Independent Components (IC) to detect which components were most correlated to eye movements. If a component surpassed a predefined correlation threshold of r = 0.8, this component was eliminated from the data.
For the remaining artifacts an artifact rejection threshold of ±100 μV (peak-to-peak amplitude) was defined. For the Ambiguous Condition, this resulted in an average (across participants) of 24% of trials being dropped, with a standard deviation of 21%. For the Disambiguation Condition 22% of trials were dropped, with a standard deviation of 20%. Data was baseline corrected, with the average amplitude in a time window between 60 ms before and 40 ms after stimulus onset of Si+1. Trials were labeled as either reversal or stable depending on the participants’ responses to stimulus Si+1. Responses were regarded as valid if given in a time from 150 ms after Si+1 onset to the end of the ISI following Si+1. If a response button was pressed more than once during this response time interval, the latest response was used. In cases of pressing wrong buttons or no responses the trial was ignored.
2.6. Data analysis
For the data analysis we collected reversal trials Sr,i+1, where the perceptual reversal took place from stimulus Sr,i to stimulus Sr,i+1. We compared these reversal trials with stability trials Ss,i+1, where perception stayed unchanged from Ss,i to Ss,i+1. Our specific focus was whether the EEG from the trials Sr,i and the subsequent ISI preceding reversal trials Sr,i+1 differed from the stability trials Ss,i and the subsequent ISI.
2.6.1. Reversal rates
We calculated the number of reversals per minute (reversal rate) by counting the number of reversals per experimental block and dividing the sum by the block duration. One block lasted for 7 min on average. This reversal rate was then averaged within participants across experimental blocks of the same condition. The focus of the present analysis was the identification of EEG signatures related to perceptual instability before a reversal compared to a time window of perceptual stability. For this purpose, it was necessary to have a minimum number of reversal trials per participant. We therefore excluded participants with less than 5 reversals per minute, resulting in a dataset that included 15 participants from the original 21. Table 1 shows the average reversal rates per minute for every participant.
2.6.2. Global field power (GFP)
For each participant, we calculated event-related potentials by separately averaging Sr,i and Sr,i+1 across reversal trials and Ss,i and Ss,i+1 across stability trials. We then used the ERPs to calculate the Global Field Power, “GFP,” which is the spatial standard deviation at a single time point (t) (Lehmann and Skrandies, 1980).
where v(t)j is the voltage at electrode j at time point t, is the mean voltage of all electrodes at time point t, and n is the total number of electrodes. The GFP is calculated for every time point, resulting in a time series of GFP values.
We were interested in EEG correlates of upcoming perceptual reversals. Given, that it is so far unclear at which time point or at which EEG electrode exactly we should expect such correlates, we used the GFP as a measure that integrates across electrodes and thus reduces the dimensionality of the data. Results from the GFP then allowed us to define a temporal region of interest (ROI) for the next analysis steps.
2.6.3. Data analysis based on machine learning
To investigate how much the EEG data differed between stable and reversal trials, an Artificial Neural Network (ANN) was trained. For this we used the Deep4Net ANN via the package Braindecode, version 0.5.1 (Schirrmeister et al., 2017), which was specifically designed for the analysis of EEG data. The data the network received was each participant’s single trials and all the electrodes. The ANN was trained and evaluated individually for every participant in the following way:
(a) From each participant’s trials, 75% were systematically selected for the training and testing of the ANN and the remaining 25% were selected for the evaluation of the ANN. For the training procedure the order of trials was randomized and the reversal trials resampled to match the number of stable trials (the evaluation trials were not randomized or resampled). The data were also all normalized, resulting in all values in a single trial to range between 0 and 1.
As a result of the evaluation, for each trial from the evaluation set we either obtained a 1 or 0 as a “prediction label” from the ANN classification, indicating either the identification of a stable or an upcoming reversal trial. Additionally, for each trial we obtained a binary manual response from the participant indicating either a perceived reversal or perceived stability, which we call “true label.” We then calculated accuracy values as the number of evaluation trials where the predicted label matched the true label, divided by the total number of trials in the evaluation set.
(b) This procedure was executed four times to have each quarter once for the evaluation, resulting in four separate accuracy values in percent discriminability. For each participant, we calculated the median of the resulting four accuracy values. The median accuracy reflects the differentiation between stable and upcoming reversal trials, ideally, indicating how good the destabilization of a perceptual state is reflected in the EEG data. Additionally, and more realistically, the accuracy also reflects how good the ANN was at extracting the relevant information.
One basic idea of this study was, that in the case of endogenous spontaneous perceptual reversals, during observation of an unchanged ambiguous cube stimulus, the EEG contrast described above may reflect a state of perceptual destabilization immediately before the reversal. In contrast, in the case of exogenously induced reversals between the disambiguated cube variants, no such destabilization state should be observed. This should be reflected in a comparison between the ambiguous and disambiguated stimulus conditions.
We applied a bootstrap method to compare distributions of accuracy values for the two conditions in the following way:
(c) We put all the test results from all four test sets together, randomly selected 70% of these data and calculated an accuracy value from these data. This was done separately for each participant and experimental condition (ambiguous and disambiguated stimuli) 1,000 times in a sampling-with-replacement manner.
(d) For each participant we created two accuracy value distributions for the two conditions and compared them with each other by applying a Kolmogorov-Smirnov test.
The analysis steps (a)—(d) were applied for two different temporal ROIs, which were (1) a time window between 300 and 700 ms after onset of the stimulus immediately before an indicated perceptual reversal (Si) and (2) the subsequent ISI. The choice of the Si time window was based on the results from the previous Global Field Power analysis. Within the time range where the p-values comparing the reversal and stable GFP in Si were below 0.05, a 400 ms time window was chosen to match the time window length of the ISI. The beginning of the time range was determined by finding the first time point where the p-values were less than 0.05 after onset of Si and physiologically plausible [150 ms after stimulus onset (Joos et al., 2020)]. This was 304 ms, and therefore, it was rounded down to 300 ms.
2.6.4. Estimating spatial regions of interest
The EEG is a rough measure with a relatively low signal-to-noise ratio. Whether a brain signal of interest can be measured on the scalp, depends on a number of factors, like individual brain anatomy, conductivity and thickness of bones and meninges, etc. (e.g., Bach, 1998; Nunez and Srinivasan, 2006). As a consequence, such a signal may be clearly identified in some participants, but be apparently absent in others, even if the neural processes of interest may be highly similar across participants. For this reason, we performed the above-described analysis on the level of individual participants and selected indicative participants (as defined below) to investigate, which of the 32 EEG electrodes provided the most classification information. We identified 8 indicative participants (see section “3. Results”).
The ANN then was trained and evaluated 8 more times on the indicative participants and with each training and evaluation a subgroup of electrodes (frontal, central, parietal, occipital, left hemisphere, right hemisphere, parietal left hemispheric, and parietal right hemispheric electrodes) was removed. If the accuracy was lower when removing some electrodes compared to others, this suggested that the missing electrodes contained useful information for the ANN to classify the EEG of Si as a reversal or stable trial.
To quantify the results of these different ANN runs, a ratio between the accuracy values from the Ambiguity Condition and Disambiguation Condition was calculated for each participant. The Disambiguation Condition in this case served as a control as the EEG from the trial immediately before the exogenously induced reversal (Si) should have no information about the subsequent reversal (Si+1). As a consequence, no indication of perceptual reversals should be observed in the data from this condition.
2.6.5. Source analysis
Standard EEG electrode positions were assumed for all q = 32 electrodes according to the 10–20 system since no individual EEG electrode positions from the individual participants were available from the dataset. We used the “fsavagerage” (Fischl et al., 1999) template T1 image as provided by the Freesurfer image analysis suite1. The forward model was computed using the boundary element method (BEM, Fuchs et al., 2002) as provided by MNE-Python (Gramfort et al., 2013). Each shell (brain, skull, and scalp tissue) was composed of 5120 vertices. Conductivity was set to 0.3 S/m for brain and scalp tissue, and 0.06 S/m for the skull.
The source space was created using p = 1284 dipoles with icosahedral spacing, i.e., dipoles were placed along the cortical folding by iterative subdivision of a icosahedron (cf.2, Gramfort et al., 2013). In order to reduce computational complexity and based on the reasonable physiological assumption, a fixed orientation of dipoles orthogonal to the surface of the cortical sheet was assumed.
Inverse solutions were calculated for the ERPs spanning 300–700 ms after the onset of Si. This was the same range used for the classification of the ANN as described above. In order to mitigate the problem of missing individual forward models, we adapted a group-inversion scheme similar to the one described by Friston et al. (2015).
As a first step, we concatenated the reversal- and stability-ERPs of all participants. Next, we identified a global set of active sources by calculating the flexible multi-signal classification (FLEX-MUSIC) inverse solution on the concatenated ERPs (Hecker et al., 2023). FLEX-MUSIC is a recently developed improvement of the well-established recursive MUSIC approach to solve EEG inverse problems, that can accurately estimate not only the location but also the spatial extent of neural sources. Finally, weighted minimum norm estimates [wMNE, (Pascual-Marqui, 1999)] were calculated on the global set of active sources for each participant and condition.
3. Results
3.1. Global field power already shows differences in conditions approximately 1,100 ms before the reversal
The GFP of reversal- and stable-ERPs, together with the GFP difference traces (reversal GFP minus stability GFP) are shown in Figure 3. From this Figure, it is clear that in the Disambiguation Condition (red traces, left), the activity in Sr,i is similar to that of Ss,i. As a consequence, the difference GFP trace in the Si time window is close to zero. The GFP difference trace in the Si+1 time window, however, shows a highly significant deviation from the zero line, indicating a clear GFP difference between a reversed and a stable percept (maximal significance at 427 ms after onset if Si+1, p = 1.6 × 10–8, Cohen’s d = 2.19).
Figure 3. Global Field Power (GFP) and p-values of the Disambiguation (red) and Ambiguous (blue) Conditions. The two top panels show the GFP averaged across participants. The dashed, darker traces depict the stable condition, whereas the lighter, continuous trace depicts the reversal condition. The middle two panels depict difference GFP traces of the top row (reversal minus stability) resulting in the difference traces. The shaded area is ± standard error of the mean. The bottom panels show the p-values logarithmically scaled. The orange-red horizontal lines depict alpha values of 0.05, 0.01, and 0.001, respectively. The filled (red and blue) areas indicate statistically significant time periods. The gray areas indicate interstimulus interval time ranges. The first range shown is the ISI preceding Si. The next 1,000 ms represent the onset period of Si and the 1000 ms after the interstimulus interval show Si+1.
The Ambiguity Condition (Figure 3, blue traces, right) presents a different pattern. The difference GFP trace in Si shows a significant deviation from zero, with a similar shape to the trace from Si+1. The most significant effect in the Si time window is at 514 ms after stimulus onset (p = 7.59 × 10–6, Cohen’s d = 1.35). The most significant effect in the Si+1 time window is at 382 ms after stimulus onset (p = 7.3 × 10–6, Cohen’s d = 1.06).
3.2. ANN uses parietal information to classify reversal and stable trials already approximately 1,100 ms before an upcoming reversal
First, the ANN was trained and evaluated on the most statistically significant 400 ms of the stimulus presentation window Si based on the GFP results (i.e., 300–700 ms after stimulus onset, cf. Figure 3). This resulted in a mean accuracy of 59.22% for the Ambiguity Condition and 52.29% for the Disambiguation Condition (with 50% being chance level). Cohen’s d was 1.21 comparing the Ambiguous and Disambiguated Accuracies for Si. Figure 4 indicates that for all but two participants the ANN results showed a higher accuracy in the Ambiguous Condition compared to the Disambiguation Condition.
Figure 4. Median accuracy of individual participants (colored circles) and the grand mean ± standard deviation (brown squares). The x-axis shows the accuracy in the Ambiguous Condition and the y-axis the accuracy in the Disambiguation Condition. The left panel shows the accuracy calculated during the Si stimulus time window and the right panel shows the accuracy during the ISI time window (reversal and stable). The number inserted in the top left shows the mean accuracy ratio (ambiguous divided by disambiguated) accuracies (with the standard deviation). The continuous gray horizontal and vertical lines indicate chance level at 50% accuracy.
Next, the ANN was trained and evaluated on the 400 ms ISI, resulting in a mean accuracy across all participants of 59.26% for Ambiguous and 53% for Disambiguated Conditions (Cohen’s d = 1.21). Again, all but the same two participants as mentioned previously had a higher accuracy in the Ambiguous Condition during the ISI. The average accuracy ratio between ambiguous and disambiguated is slightly larger in Si (1.14) compared to during the ISI (1.12). The standard deviation of the accuracy ratios is also larger in Si (0.13) compared to during the ISI (0.12). Moreover, the accuracies show more variance in the Ambiguity Condition than in the Disambiguation Condition.
The ambiguous and disambiguation accuracy distributions were compared, in both the 300–700 ms in Si and the 400 ms of the ISI, using the Kolmogorov-Smirnov test. The Kolmogorov–Smirnov test was used to quantify the difference between the accuracy distributions of the Ambiguous and Disambiguated condition for each participant. Figure 5 displays distributions based on bootstrapping from three representative example participants together with the related test results. The three examples indicate the strong differences in the discriminatory power of the EEG between participants. The Kolmogorov-Smirnov test results from all participants is shown in Figure 6. One interesting aspect in this graph is the linear relation between the ISI and Si data, indicating that if a participant’s EEG showed a strong discriminatory power in the Si time window, it was also relatively strong in the subsequent ISI. The correlation between the statistic values of these two time windows was ρ = 0.8 (p = 0.0009). Furthermore, 60 % of participants showed larger discriminatory power in the Si time window compared to the ISI time window. Finally, the graph identifies participants 2, 3, 4, 9, 12, 13, 14, and 15 as indicative participants. Participant 13 has more discriminatory EEG power in the ISI compared to Si, while for the others there is more discriminatory power in Si compared to the ISI. The indicative participants were determined by choosing all participants that had a KS statistic score larger than or equal to 0.8 in either the Si time window or the ISI time window.
Figure 5. Examples of accuracy distributions resulting from the bootstrap method and the corresponding Kolmogorov–Smirnov (KS) test statistic for participants 4, 11, and 5. The red distributions represent the Disambiguation Condition accuracies, the blue distributions represent the accuracies in the Ambiguous Condition. The KS test statistics ranges between 0 and 1. The closer the values are to 1, the further apart the distributions are.
Figure 6. Kolmogorov–Smirnov (KS) test statistic values indicating the separability of the distribution of ambiguous and disambiguated accuracies in Si (300–700 ms) and the ISI. The KS test statistic values ranges from 0 to 1. The closer the values are to 1, the further apart the distributions are. The x-axis presents the KS statistic of Si Time Window. The y-axis presents the statistic of the ISI Time Window. The colors correspond to the individual participants. Most participants with high/low separability in the Si Time Window also showed high/low values in the ISI Time Window, with overall better separability in the Si Time Window than in the ISI Time Window (i.e., more data points below the diagonal). Moreover, largest separability can be observed in participants 2, 3, 4, 9, 12, 13, 14, and 15 (indicative participants).
In the following spatial analysis steps, we focused on these 8 indicative participants. We repeated the ANN calculations with different subsets of electrodes in order to identify electrode subsets that were necessary to obtain high discrimination performance.
Figure 7 shows the results of this analysis. Without the removal of any electrodes, our high-performance participants had an average accuracy ratio of 1.19 (SD = 0.13). When the parietal electrodes were removed, the median accuracy was slightly decreased (accuracy ratio = 1.18; SD = 0.12). Remarkably, for all other variants of electrode removal, the accuracy increased. Moreover, removing the left parietal electrodes increases the accuracy slightly more than removing the right parietal electrodes (accuracy ratio = 1.25; SD = 0.095 versus accuracy ratio = 1.22; SD = 0.098). Additionally, removing all electrodes from the left hemisphere increased the accuracy slightly more than removing the right hemisphere electrodes (accuracy ratio = 1.30; SD = 0.12 vs. accuracy ratio = 1.29; SD = 0.16). Overall, removing the left-hemisphere electrodes provides the highest accuracy ratio among all variants.
Figure 7. Accuracy ratio of Si (300–700 ms after stimulus onset) of the 8 indicative participants with different electrode subsets removed with each run. The values show the mean accuracy ratio of all indicative participants. The different colors depict the different participants and the numbers in the top left of each panel represents the mean accuracy ratio (Ambiguity divided by Disambiguation; with the standard deviation). The larger the distances (to the right and to the top) of the individual icons from the horizontal and vertical gray lines are, the higher is the discriminatory power of the respective participant’s EEG data.
3.3. Source localization
Source localization was performed in a two-stage procedure. First, the global set of active sources was identified with the EEG source analysis method FLEX-MUSIC, then the neural activity at these regions was estimated on the level of individual participants.
Notably, a relatively confined set of active regions was found (cf. Figure 8), which is remarkable given the presumably high inter-individual variance of the EEG data. The results of the source localization suggest that the parahippocampal place area (PPA) most significantly differs between reversal and stable trials (p = 3.96 × 10–05; cf. Figure 8 and Table 2). A summary of the brain areas and the corresponding t- and p-values is shown in Table 2. Other regions that significantly differ are the fusiform gyrus, the lingual gyrus, the entorhinal cortex, the posterior cingulate cortex, and the precuneus. Most activity seems to be located in the right hemisphere.
Figure 8. Results of the source localization with thresholded t-values for the Ambiguity Condition. Reversal ERPs compared to stable ERPs. The size of the cluster does not represent the relevance of the cluster. Activity seems to be concentrated mainly in the right hemisphere, specifically the parahippocampal place area (PPA). There also seems to be some activity in the left hemisphere in a similar area. Additionally, the posterior cingulate cortex (PCC) also seems to be active.
Table 2. The location of all significant voxels that are in named brain areas in order of most significant to least significant.
4. Discussion
The primary aim of the present study was to identify EEG correlates of a destabilized perceptual brain state preceding a spontaneous perceptual reversal of an ambiguous Necker lattice. Given the absence of spatial and temporal regions of interest from the EEG literature to focus this question on, we started with a Global Field Power (GFP) analysis. We compared the GFP from time periods during perceptual stability with those about 1 s before a perceptual reversal and found large GFP effects with ambiguous lattice stimuli. As expected, no such effects were observed with the disambiguated control lattice stimuli.
The concept of perceptual destabilization is not well established in the literature. It is, thus, unclear, how common the underlying neural processes are in time and brain areas across participants. We, therefore, focused the analysis steps subsequent to the GFP analysis on the level of individual participants using ANNs. We determined the accuracy of the individually trained ANNs to discriminate between stable and destabilized brain states. This analysis identified eight (out of 15) participants with high EEG discrimination performance already about 1 s before an upcoming reversal. By repeated re-analysis of the data from these eight participants with the systematic removal of subsets of electrodes, we were able to identify the parietal cortex as a critical brain area for the prediction of an upcoming reversal. A subsequent source localization analysis revealed the parahippocampal place area (PPA) as the most statistically significant brain area for reversal prediction. Previous studies identified a network of frontal and parietal regions being active and relevant during spontaneous perceptual reversals (Brascamp et al., 2018; Watanabe, 2021). Interestingly, neither our ANN analysis nor the source analysis indicated the relevance of this network for destabilization. The source localization needs to be taken with caution, due to the missing individual forward models.
4.1. Potential limitations
4.1.1. Limited number of trials and participants
In the present study, we measured 21 participants. Six had to be excluded due to too low reversal rates (see section “2. Materials and methods”), resulting in a dataset of 15 participants. This is not a very large sample to make strong claims about common mechanisms in the general population. On the other hand, the majority of our analyses was realized on the level of individual participants. From our 15 analyzed participants, we identified eight indicative participants. We cannot say whether the effects we identified in 53% of our participants can also be identified in 53% of the population. However, we regard our results as a good starting point for follow-up studies with a specific focus on perceptual representations and their stability criteria. Further below, we will also suggest possible next steps to increase the discrimination accuracies.
4.1.2. Are perceptual reversals during discontinuous stimulus presentation a good model for the continuous case?
The introduction section contains an entire paragraph on the question about how to measure spontaneous endogenous perceptual reversals of an ambiguous stimulus with high temporal resolution. In the present study, we used the Onset-Paradigm as a possible way to address this principal problem. One can now ask whether a changed percept from one stimulus presentation to the next is really comparable with a spontaneous reversal during a continuous observation of the ambiguous stimulus?
We think “yes,” as we have already discussed this question in detail in our review paper in 2012 (Kornmeier and Bach, 2012). In short, continuous observation is never really continuous, because of around 15 eye blinks per minute on average (Sforza et al., 2008), with a duration of around 200 ms (Caffier et al., 2003) and the ensuing saccadic suppression (Binda and Morrone, 2018). Moreover, a number of behavioral studies showed a continuous transition of reversal rates from continuous stimulus presentation to short reoccurring interruptions up to 400 ms duration (Orbach et al., 1963, 1966; Kornmeier et al., 2007). Clearly, however, interrupted stimulus presentation is not the same as continuous presentation. Based on the present results, it may thus be an interesting next step to train ANNs with EEG data from discontinuous presentation and apply the trained ANN to EEG data from continuous observation.
4.1.3. Can the findings from the Necker lattice be generalized across other types of ambiguous stimuli?
We interpret our findings as neural destabilization correlates of perceptual representations. We have already discussed that neither the concept of perceptual representations and their neural correlates nor the concept of destabilization are well established. This becomes obvious, if we compare perceptual reversals of ambiguous figures with perceptual reversals of binocular rivalry stimuli, as recently discussed by Bachmann and Aru (2023). In the case of ambiguous figures, the stimulus features, e.g., the edges of the Necker cube, stay in consciousness but are interpreted differently. As a consequence, only a part of the perceptual representation may become destabilized in the context of a reversal. In the case of binocular rivalry, only the input from one eye enters consciousness, whereas the input from the other eye stays unconscious (if we ignore the intermediate patchwork periods between two distinct percepts). In this case, the whole “perceptual representation” may become destabilized. It may thus be interesting to replicate the current experiment with binocular rivalry stimuli, to see how much of our results can be replicated.
4.2. Why do we only see clear discrimination accuracy in 53% of the participants?
The scalp EEG is a relatively rough measure. Whether a neural signal reaches the scalp and can be detected by the scalp EEG electrodes depends, amongst other factors, on the individual folding of the brain. The activity of neighboring brain sources of activity can be superimposed and weaken each other. Furthermore, neural signals related to a certain processing step are typically superimposed on background activity, not necessarily related to this processing, but affecting the signal-to-noise ratio of the signal in question (Arieli et al., 1996; Bach, 1998). These and other factors contribute to a large inter-individual variability of brain activity, even if participants observe the same stimulus or execute the same task (e.g., Kornmeier et al., 2014). An impressive demonstration of the inter-individual variability during observation of multistable stimuli has recently been presented by Wexler (2018). A huge number of reported EEG effects, including work from our lab, resulted from group statistics and were difficult to observe in a large number of individual participants (e.g., Ehm et al., 2011). Hence, it is not surprising that in the present data some participants showed weak or even no effects. The low signal-to-noise ratio of single-trial EEG may also explain that even for the eight indicative participants the discrimination accuracy remained below 80%. Finally, another potential limitation could be the low number of trials the ANN had available for training. Due to potential fatigue effects in our participants, typically resulting in rising numbers of body movement artifacts, wrong key presses etc., it was difficult to increase the number of trials to be enough for training the ANN.
Beyond these rather technical reasons, another interesting more functional factor may play a role. Polgári et al. (2020) recently published an interesting study about eye-movements during observation of the Necker cube. In the present study, participants were instructed to fixate on a fixation cross in the center of the cube stimuli in order to minimize eye-movement artifacts in the EEG. In Polgári et al.’s (2020) study participants were allowed to freely move their eyes. The authors analyzed fixations and clustered them based on the horizontal fixation location as an either right or left fixation cluster with respect to the vertical midline of the visual field in which the Necker cube was presented centrally. They found that in most cases, in which participants reported a reversal, the eyes also moved from one cluster to the other. However, they recorded about twice as many changes of the horizontal eye position between clusters than reversals, which means that not every cluster change was accompanied by a clear and conscious perceptual reversal. Polgári et al.’s (2020) explanation of this observation is in contrast to the basic assumption underlying the present study. We a priori assumed that participants’ percepts become stable immediately after a reversal and only slowly destabilized toward the next perceptual reversal. Comparing the EEG from the middle of this proposed temporal window of perceptual stability with the EEG close to a reversal should therefore reveal potential EEG correlates of perceptual stability state differences (stable versus destabilized). Polgári et al. (2020) in contrast, postulated that the perceptual brain state can also destabilize in between two button presses, but sometimes restabilizes back to the perceptual interpretation where it started. They further speculate that such unconscious destabilizations may come with a partial or total departure from one of the clustered horizontal eye positions, which would explain the larger number of cluster changes than button presses. Interestingly, indications of unconscious destabilizations during observation of ambiguous figures have also been reported by other groups (e.g., Pastukhov and Braun, 2007; Pastukhov and Klanke, 2016).
As described above, our discrimination accuracy measure is based on a postulated EEG contrast between perceptually stable and unstable brain states. If Polgári et al.’s (2020) postulate of unconscious destabilizations is correct, it may be possible that the time windows we selected and labeled as perceptually stable brain states between two button presses may contain periods of unconscious destabilizations. This may lead to a larger variability and a smaller signal-to-noise ratio of our measure. Accepting Polgári et al.’s (2020) interpretation, horizontal eye-movement data can be used in a future replication of the present study to better determine perceptually stable brain states.
4.3. What can the current results contribute to the understanding of perceptual representations?
When our participants look at a black screen and suddenly a white lattice stimulus occurs, a clear and distinct pattern of EEG activity can be identified within the first 600—800 ms after stimulus onset. This pattern typically has a temporal profile which can be observed, e.g., in the ERP with stimulus onset as the reference time point. In the first 300 ms we see so-called exogenous signatures with typically sharp and relatively large deviations from baseline, followed by broader components, like the P300-family (e.g., Figure 2C in Kornmeier and Bach, 2006). These signatures in the ERP (lower-frequency range) are typically accompanied by modulations in higher frequencies (Ehm et al., 2011). These signatures are typically discussed as processing steps along a hierarchy of perceptual processing on the way to create a stable perceptual interpretation (VanRullen and Thorpe, 2002; Dehaene and Changeux, 2011; Kornmeier and Bach, 2012; Sergent et al., 2017). Interestingly, after the percept has been established and is stable, the measurable signals seem to fade out, raising the fundamental question of how stable perceptual interpretations are represented in the brain over time (e.g., Krüger, 2022). Of course, given the quality of the EEG, we cannot infer that the absence of a measurable signal means no processing in the brain. At this point, ANNs become relevant, because these methods seem to be more sensitive to subtle signals.
Given that no precise spatio-temporal regions of interest were available for our experimental question and taking into account, that the signatures we were looking for, could be highly variable in space and time across observers and at the same time very small, we decided to focus on within-participant statistics and apply ANN methods for our analysis after a group GFP analysis. In the GFP analysis of the Ambiguity Condition we found surprisingly large effects in a time window already about 1 s before a reversed percept was established. Two observations are of particular interest here:
1. In our analysis, we focused on two time windows of interest, the Si time window, where the last lattice stimulus before the perceptual reversal was presented, and the subsequent ISI. In the GFP analysis we found a relatively long time period (about 400 ms) of significance with huge effect sizes up to 1.35. We also found significant effects in the subsequent ISI, but with much shorter time periods and smaller effect sizes.
2. Interestingly, while the GFP analysis indicates that most of the destabilization information seems to necessitate the presence of a stimulus (Si time window), the subsequent ANN analysis draws a more sophisticated picture. In contrast to the GFP results, Figure 6 indicates that our indicative participants showed comparable discriminatory power in both the Si time window and in the subsequent ISI. This supports the observation that the ANN methods are more sensitive. Moreover, it indicates that the destabilization state of the perceptual system can also be read out from EEG data in the absence of a visual stimulus.
During the Si presentation time window, the period of high significance started at about 300 ms after stimulus onset. Interestingly, 300 ms are discussed as an estimate of time, necessary for a perceptual interpretation to become conscious, after the early visual processing steps have finished (Atmanspacher et al., 2008; Dehaene and Changeux, 2011; Kornmeier and Bach, 2012; Atmanspacher and Filk, 2013; Kornmeier et al., 2017a). If we interpret our observed effects as correlates of perceptual destabilization, then the GFP analysis results indicate, that the early visual processing units are less relevant. This hypothesis is further supported by the observation that eliminating the occipital electrodes did not substantially reduce the discriminatory power (cf. Figure 7).
Our current results do not allow strong conclusion about the “nature of a perceptual representation,” nor on the definition of precise spatial regions of interest for subsequent studies. However, the huge effect sizes in precisely defined time windows make this study a perfect starting point for subsequent studies about stable and unstable perceptual brain states.
5. Conclusion
We (Ehm et al., 2011) and others (Britz et al., 2009, 2010; Nakatani and van Leeuwen, 2013) already reported about EEG effects shortly before a reversed percept of an ambiguous Necker lattice is established. Our present results indicate that in some participants such anticipatory activity can be identified already at least 1 s before the reversal. Technical reasons (not enough single trials and number of reversals), unfortunately, did not allow us to go further back in time. It is, thus, possible that EEG indicators of destabilization and, thereby, of an upcoming perceptual reversal are present earlier. It would be highly interesting to see whether this is really the case. Moreover, in the present study, this anticipatory activity was only found in a subgroup of our participants. It is currently unclear, whether similar processing is also present in the other participants but is not measurable with EEG. Follow-up studies with more trials and thus a better signal-to-noise ratio may be able to answer this question.
In his Principles of Psychology, the great psychologist William James wrote about stable mental representations and transient, unstable states between them (James, 1890). He emphasized that the stable states are the “substantive parts” that enter consciousness, while the “transitive parts” are typically very fast and stay unconscious. The basic idea of this has further been elaborated in terms of categorical versus a categorical perceptual/mental states (Atmanspacher, 1992; Feil and Atmanspacher, 2010; Atmanspacher and Fach, 2019).
The present findings indicate that the perceptual system is already in an unstable state about 1 s before a perceptual reversal. The eye-tracking data from Polgári et al. (2020) further indicate that transient periods of perceptual instability may occur between two consciously perceived reversal events. All of this indicates that what James described as the transient state may only be the tip of the iceberg, i.e., the point of maximal instability (Kornmeier and Bach, 2012) and that perceptual instability is a gradual and longer lasting phenomenon rather than being binary and short.
We demonstrated that perceptual instability can be measured with the EEG. Moreover, the effects are large enough to be visible in a considerable number of individual participants. Taking into account the findings from Polgári et al. (2020) it may even be possible to increase the sensitivity of our measures by eliminating time windows of apparent stability, in which eye tracking data, however, would indicate destabilization. This might make some of the non-indicative participants become indicative participants.
We interpret the present EEG effects as a measure of perceptual instability, i.e., as reflecting the difference in neural activity between stable and destabilized perceptual states. An interesting question for subsequent studies, is to see whether this measure reflects perceptual (in)stability on a continuous scale, i.e., whether the size of the effect reflects the amount of instability of the system. Furthermore, a number of studies show deviating perceptual reversal dynamics, when participants with psychiatric disorders observe ambiguous stimuli (e.g., Notredame et al., 2014; Schmack et al., 2015; Robertson et al., 2016; Kornmeier et al., 2017b). These deviating perceptual dynamics in patients may indicate an underlying imbalance between stable and unstable brain states. Deviating patterns of EEG variability and other stability measures of brain activity further confirm these observations (Koshiyama et al., 2018; Foerster et al., 2021; Marques-Carneiro et al., 2021; Hecker et al., 2022; Huang et al., 2022). In summary, the present paradigm together with the identified physiological effects are a good starting point for further research about stability features of brain activity comparing patients with healthy controls. This will be one of the next steps on our agenda.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors upon request.
Ethics statement
The studies involving human participants were reviewed and approved by the University of Freiburg, Germany. The patients/participants provided their written informed consent to participate in this study.
Author contributions
EJ, LH, and JK contributed to the conception and design of the study. EJ and LH contributed to the data acquisition. MW, LH, JK, and AA contributed to the conception of the analysis. MW and LH contributed to the analysis of the data and wrote all analysis code. JK and MW wrote the first draft of the manuscript. LT and JK contributed to the acquisition of the funding. All authors contributed to manuscript revision, read, and approved the submitted version.
Funding
This work was funded by Neurex, the Deutsch-Französische Hochschule (DFH), European Campus (Eucor) seeding money, and the Institute for Frontier Areas of Psychology and Mental Health (IGPP). We acknowledge support by the Open Access Publication Fund of the University of Freiburg.
Acknowledgments
We thank Julia Schipp for the data acquisition.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Footnotes
References
Abdallah, D., and Brooks, J. L. (2020). Response dependence of reversal-related ERP components in perception of ambiguous figures. Psychophysiology 57:e13685. doi: 10.1111/psyp.13685
American Clinical Neurophysiology Society (2006). Guideline 5: Guidelines for standard electrode position nomenclature. J. Clin. Neurophysiol. 23, 107–110.
Arieli, A., Sterkin, A., Grinvald, A., and Aertsen, A. (1996). Dynamics of ongoing activity: Explanation of the large variability in evoked cortical responses. Science 273, 1868–1871. doi: 10.1126/science.273.5283.1868
Atmanspacher, H. (1992). Categoreal and acategoreal representation of knowledge. Cogn. Sys. 3, 259–288.
Atmanspacher, H., Bach, M., Filk, T., Kornmeier, J., and Römer, H. (2008). Cognitive time scales in a necker-zeno model for bistable perception. Open Cybernet. Syst. J. 2, 234–251.
Atmanspacher, H., and Fach, W. (2019). Exceptional experiences of stable and unstable mental states, understood from a dual-aspect point of view. Philosophies 4:7. doi: 10.3390/philosophies4010007
Atmanspacher, H., and Filk, T. (2013). The necker-zeno model for bistable perception. Top. Cogn. Sci. 5, 800–817. doi: 10.1111/tops.12044
Bach, M. (1998). “Electroencephalogram (EEG),” in Functional imaging, eds G. K. von Schulthess and J. Hennig (Philadelphia: Lippincott-Raven), 391–408.
Bach, M. (2018). Homepage of the Freiburg visual acuity & contrast test (‘FrACT’). Available online at: http://www.michaelbach.de/fract.html (accessed May 9, 2023).
Bachmann, T., and Aru, J. (2023). Conscious interpretation: A distinct aspect for the neural markers of the contents of consciousness. Conscious. Cogn. 108:103471. doi: 10.1016/j.concog.2023.103471
Bartels, A. (2021). Consciousness: What is the role of prefrontal cortex?. Curr. Biol. 31, R853–R856. doi: 10.1016/j.cub.2021.05.012
Basar-Eroglu, C., Struber, D., Kruse, P., Basar, E., and Stadler, M. (1996). Frontal gamma-band enhancement during multistable visual perception. Int. J. Psychophysiol. 24, 113–125.
Binda, P., and Morrone, M. C. (2018). Vision During Saccadic Eye Movements. Annu. Rev. Vis. Sci. 4, 193–213. doi: 10.1146/annurev-vision-091517-034317
Blake, R. (2001). A Primer on Binocular Rivalry, Including Current Controversies. Brain Mind 2, 5–38.
Blake, R., Brascamp, J., and Heeger, D. J. (2014). Can binocular rivalry reveal neural correlates of consciousness?. Philos. Trans. R. Soc. B Biol. Sci. 369:20130211. doi: 10.1098/rstb.2013.0211
Blake, R., and Logothetis, N. K. (2002). Visual competition. Nat. Rev. Neurosci. 3, 13–21. doi: 10.1038/nrn701
Brascamp, J., Sterzer, P., Blake, R., and Knapen, T. (2018). Multistable Perception and the Role of the Frontoparietal Cortex in Perceptual Inference. Annu. Rev. Psychol. 69, 77–103. doi: 10.1146/annurev-psych-010417-085944
Brascamp, J. W., de Hollander, G., Wertheimer, M. D., DePew, A. N., and Knapen, T. (2021). Separable pupillary signatures of perception and action during perceptual multistability. eLife 10:e66161. doi: 10.7554/eLife.66161
Britz, J., Landis, T., and Michel, C. M. (2009). Right parietal brain activity precedes perceptual alternation of bistable stimuli. Cereb. Cortex 19, 55–65.
Britz, J., Pitts, M. A., and Michel, C. M. (2010). Right parietal brain activity precedes perceptual alternation during binocular rivalry. Hum. Brain Mapp. 32, 1432–1442. doi: 10.1002/hbm.21117
Caffier, P. P., Erdmann, U., and Ullsperger, P. (2003). Experimental evaluation of eye-blink parameters as a drowsiness measure. Eur. J. Appl. Physiol. 89, 319–325. doi: 10.1007/s00421-003-0807-5
Carter, O., Konkle, T., Wang, Q., Hayward, V., and Moore, C. (2008). Tactile Rivalry Demonstrated with an Ambiguous Apparent-Motion Quartet. Curr. Biol. 18, 1050–1054. doi: 10.1016/j.cub.2008.06.027
Conrad, V., Vitello, M. P., and Noppeney, U. (2012). Interactions between apparent motion rivalry in vision and touch. Psychol. Sci. 23, 940–948. doi: 10.1177/0956797612438735
Darki, F., and Rankin, J. (2020). Perceptual rivalry with vibrotactile stimuli. bioRxiv [Preprint]. doi: 10.1101/2020.09.17.301358
De Jong, M. C., Vansteensel, M. J., van Ee, R., Leijten, F. S. S., Ramsey, N. F., Dijkerman, H. C., et al. (2020). Intracranial recordings reveal unique shape and timing of responses in human visual cortex during illusory visual events. Curr. Biol. 30, 3089–3100.e4. doi: 10.1016/j.cub.2020.05.082
Dehaene, S., and Changeux, J. P. (2011). Experimental and theoretical approaches to conscious processing. Neuron 70, 200–227. doi: 10.1016/j.neuron.2011.03.018
Devia, C., Concha-Miranda, M., and Rodríguez, E. (2022). Bi-stable perception: Self-coordinating brain regions to make-up the mind. Front. Neurosci. 15:805690. doi: 10.3389/fnins.2021.805690
Duman, I., Ehmann, I. S., Gonsalves, A. R., Gültekin, Z., Van den Berckt, J., and van Leeuwen, C. (2022). The No-report paradigm: A revolution in consciousness research? Front. Hum. Neurosc. 16:861517. doi: 10.3389/fnhum.2022.861517
Ehm, W., Bach, M., and Kornmeier, J. (2011). Ambiguous figures and binding: EEG frequency modulations during multistable perception. Psychophysiology 48, 547–558. doi: 10.1111/j.1469-8986.2010.01087.x
Einhäuser, W., da Silva, L. F. O., and Bendixen, A. (2020). Intraindividual Consistency Between Auditory and Visual Multistability. Perception 49, 119–138. doi: 10.1177/0301006619896282
Einhäuser, W., Stout, J., Koch, C., and Carter, O. (2008). Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry. Proc. Natl. Acad. Sci. U. S. A. 105, 1704–1709. doi: 10.1073/pnas.0707727105
Feil, D., and Atmanspacher, H. (2010). Acategorial states in a representational theory of mental processes. J. Consciousness Stud. 17, 72–101.
Fischl, B., Sereno, M. I., Tootell, R. B. H., and Dale, A. M. (1999). High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum. Brain Mapp. 8, 272–284. doi: 10.1002/(SICI)1097-019319998:4<272::AID-HBM10<3.0.CO;2-4
Foerster, F. R., Weibel, S., Poncelet, P., Dufour, A., Delevoye-Turrell, Y. N., Capobianco, A., et al. (2021). Volatility of subliminal haptic feedback alters the feeling of control in schizophrenia. J. Abnorm. Psychol. 130, 775–784. doi: 10.1037/abn0000703
Friston, K., Zeidman, P., and Litvak, V. (2015). Empirical bayes for DCM: A group inversion scheme. Front. Syst. Neurosci. 9:164. doi: 10.3389/fnsys.2015.00164
Fuchs, M., Kastner, J., Wagner, M., Hawes, S., and Ebersole, J. S. (2002). A standardized boundary element method volume conductor model. Clin. Neurophysiol. 113, 702–712. doi: 10.1016/S1388-2457(02)00030-5
Giles, N., Lau, H., and Odegaard, B. (2016). What Type of Awareness Does Binocular Rivalry Assess? Trends Cogn. Sci. 20, 719–720. doi: 10.1016/j.tics.2016.08.010
Gramfort, A., Luessi, M., Larson, E., Engemann, D. A., Strohmeier, D., Brodbeck, C., et al. (2013). MEG and EEG data analysis with MNE-Python. Front. Neurosci. 7:267. doi: 10.3389/fnins.2013.00267
Hecker, L., Tebartz, van Elst, L., and Kornmeier, J. (2023). Source Localization Using Recursively Applied and Projected MUSIC with Flexible Extent Estimation. bioRxiv [Preprint]. doi: 10.1101/2023.01.20.524892
Hecker, L., Wilson, M., Tebartz, van Elst, L., and Kornmeier, J. (2022). Altered EEG variability on different time scales in participants with autism spectrum disorder: An exploratory study. Sci. Rep. 12:13068. doi: 10.1038/s41598-022-17304-x
Hense, M., Badde, S., and Röder, B. (2019). Tactile motion biases visual motion perception in binocular rivalry. Attent. Percept. Psychophys. 81, 1715–1724. doi: 10.3758/s13414-019-01692-w
Huang, Q., Pereira, A. C., Velthuis, H., Wong, N. M. L., Ellis, C. L., Ponteduro, F. M., et al. (2022). GABA B receptor modulation of visual sensory processing in adults with and without autism spectrum disorder. Sci. Transl. Med. 14:eabg7859. doi: 10.1126/scitranslmed.abg7859
Hupe, J. M., Lamirel, C., and Lorenceau, J. (2009). Pupil dynamics during bistable motion perception. J. Vis. 9:10. doi: 10.1167/9.7.10
Intaite, M., Koivisto, M., and Revonsuo, A. (2013). Perceptual reversals of Necker stimuli during intermittent presentation with limited attentional resources. Psychophysiology 50, 82–96. doi: 10.1111/j.1469-8986.2012.01486.x
Intaite, M., Koivisto, M., Ruksenas, O., and Revonsuo, A. (2010). Reversal negativity and bistable stimuli: Attention, awareness, or something else? Brain Cogn. 74, 24–34. doi: 10.1016/j.bandc.2010.06.002
Joos, E., Giersch, A., Hecker, L., Schipp, J., Heinrich, S. P., Tebartz, et al. (2020). Large EEG amplitude effects are highly similar across Necker cube, smiley, and abstract stimuli. PLoS One 15:e0232928. doi: 10.1371/journal.pone.0232928
Kloosterman, N. A., Meindertsma, T., van Loon, A. M., Lamme, V. A. F., Bonneh, Y. S., and Donner, T. H. (2015). Pupil size tracks perceptual content and surprise. Eur. J. Neurosci. 41, 1068–1078. doi: 10.1111/ejn.12859
Knapen, T., Brascamp, J., Pearson, J., van Ee, R., and Blake, R. (2011). The Role of Frontal and Parietal Brain Areas in Bistable Perception. J. Neurosci. 31, 10293–10301. doi: 10.1523/JNEUROSCI.1727-11.2011
Kornmeier, J., and Bach, M. (2004). Early neural activity in Necker-cube reversal: Evidence for low-level processing of a gestalt phenomenon. Psychophysiology 41, 1–8. doi: 10.1016/j.visres.2004.10.006
Kornmeier, J., and Bach, M. (2006). Bistable perception—Along the processing chain from ambiguous visual input to a stable percept. Int. J. Psychophysiol. 62, 345–349.
Kornmeier, J., and Bach, M. (2012). Ambiguous Figures – what happens in the brain when perception changes but not the stimulus. Front. Hum. Neurosci. 6:51. doi: 10.3389/fnhum.2012.00051
Kornmeier, J., and Bach, M. (2014). EEG correlates of perceptual reversals in Boring’s ambiguous old/young woman stimulus. Perception 43, 950–962. doi: 10.1068/p7741
Kornmeier, J., Ehm, W., Bigalke, H., and Bach, M. (2007). Discontinuous presentation of ambiguous figures: How interstimulus-interval durations affect reversal dynamics and ERPs. Psychophysiology 44, 552–560.
Kornmeier, J., Friedel, E., Hecker, L., Schmidt, S., and Wittmann, M. (2019). What happens in the brain of meditators when perception changes but not the stimulus? PLoS One 14:e0223843. doi: 10.1371/journal.pone.0223843
Kornmeier, J., Friedel, E., Wittmann, M., and Atmanspacher, H. (2017a). EEG correlates of cognitive time scales in the Necker-Zeno model for bistable perception. Conscious. Cogn. 53, 136–150. doi: 10.1016/j.concog.2017.04.011
Kornmeier, J., Hein, C. M., and Bach, M. (2009). Multistable perception: When bottom-up and top-down coincide. Brain Cogn. 69, 138–147. doi: 10.1016/j.bandc.2008.06.005
Kornmeier, J., Heinrich, S. P., Atmanspacher, H., and Bach, M. (2001). The reversing “Necker Wall” – a new paradigm with reversal entrainment reveals an early EEG correlate. Invest. Ophthalmol. Vis. Sci. 42:409.
Kornmeier, J., Pfaffle, M., and Bach, M. (2011). Necker cube: Stimulus-related (low-level) and percept-related (high-level) EEG signatures early in occipital cortex. J. Vis. 11:12. doi: 10.1167/11.9.12
Kornmeier, J., Wörner, R., Riedel, A., Bach, M., and Tebartz van Elst, L. (2014). A different view on the checkerboard? Alterations in early and late visually evoked EEG potentials in asperger observers. PLoS One 9:e90993. doi: 10.1371/journal.pone.0090993
Kornmeier, J., Wörner, R., Riedel, A., and Tebartz van Elst, L. (2017b). A different view on the Necker cube—Differences in multistable perception dynamics between Asperger and non-Asperger observers. PLoS One 12:e0189197. doi: 10.1371/journal.pone.0189197
Koshiyama, D., Kirihara, K., Tada, M., Nagai, T., Fujioka, M., Ichikawa, E., et al. (2018). Electrophysiological evidence for abnormal glutamate-GABA association following psychosis onset. Transl. Psychiatry 8:211. doi: 10.1038/s41398-018-0261-0
Krüger, J. (2022). Inattentive Perception, Time, and the Incomprehensibility of Consciousness. Front. Psychol. 12:804652. doi: 10.3389/fpsyg.2021.804652
Lehmann, D., and Skrandies, W. (1980). Reference-free identification of components of checkerboard-evoked multichannel potential fields. Electroencephalogr. Clin. Neurophysiol. 48, 609–621. doi: 10.1016/0013-4694(80)90419-8
Leopold, D. A., and Logothetis, N. K. (1999). Multistable phenomena: Changing views in perception. Trends Cogn. Sci. 3, 254–264. doi: 10.1016/S1364-6613(99)01332-7
Leopold, D. A., Wilke, M., Maier, A., and Logothetis, N. K. (2002). Stable perception of visually ambiguous patterns. Nat. Neurosci. 5, 605–609.
Liaci, E., Bach, M., Tebartz van Elst, L., Heinrich, S. P., and Kornmeier, J. (2016). Ambiguity in Tactile Apparent Motion Perception. PLoS One 11:e0152736. doi: 10.1371/journal.pone.0152736
Long, G. M., and Toppino, T. C. (2004). Enduring interest in perceptual ambiguity: Alternating views of reversible figures. Psychol. Bull. 130, 748–768. doi: 10.1037/0033-2909.130.5.748
Lumer, E. D., Friston, K. J., and Rees, G. (1998). Neural correlates of perceptual rivalry in the human brain. Science 280, 1930–1934.
Maier, A., Wilke, M., Logothetis, N. K., and Leopold, D. A. (2003). Perception of temporally interleaved ambiguous patterns. Curr. Biol. 13, 1076–1085.
Marques-Carneiro, J. E., Krieg, J., Duval, C. Z., Schwitzer, T., and Giersch, A. (2021). Paradoxical sensitivity to sub-threshold asynchronies in schizophrenia: A behavioral and EEG Approach. Schizophr. Bull. Open 2:sgab011. doi: 10.1093/schizbullopen/sgab011
Nakatani, H., and van Leeuwen, C. (2013). Antecedent occipital alpha band activity predicts the impact of oculomotor events in perceptual switching. Front. Syst. Neurosci. 7:19. doi: 10.3389/fnsys.2013.00019
Necker, L. A. (1832). Observations on some remarkable optical phaenomena seen in Switzerland; and on an optical phaenomenon which occurs on viewing a figure of a crystal or geometrical solid. Lond. Edinb. Dublin Philos. Mag. J. Sci. 1, 329–337. doi: 10.1080/14786443208647909
Notredame, C.-E., Pins, D., Deneve, S., and Jardri, R. (2014). What visual illusions teach us about schizophrenia. Front. Integr. Neurosci. 8:63. doi: 10.3389/fnint.2014.00063
Nunez, P. L., and Srinivasan, R. (2006). Electric Fields of the Brain. Oxford: Oxford University Press, doi: 10.1093/acprof:oso/9780195050387.001.0001
O’Donnell, B. F., Hendler, T., and Squires, N. K. (1988). Visual evoked potentials to illusory reversals of the Necker cube. Psychophysiology 25, 137–143.
Orbach, J., Ehrlich, D., and Heath, H. (1963). Reversibility of the Necker cube: I. An examination of the concept of “satiation of orientation.”. Percept. Mot. Skills 17, 439–458.
Orbach, J., Zucker, E., and Olson, R. (1966). Reversibility of the Necker cube: VII: Reversal rate as a function of figure-on and figure-off durations. Percept. Mot. Skills 22, 615–618.
O’Shea, R. P., Kornmeier, J., and Roeber, U. (2013). Predicting visual consciousness electrophysiologically from intermittent binocular rivalry. PLoS One 8:e76134. doi: 10.1371/journal.pone.0076134
Pascual-Marqui, R. D. (1999). Review of Methods for Solving the EEG Inverse Problem. Int. J. Bioelectromagnet. 1, 75–86.
Pastukhov, A. (2016). Perception and the strongest sensory memory trace of multi-stable displays both form shortly after the stimulus onset. Attent. Percept. Psychophys. 78, 674–684. doi: 10.3758/s13414-015-1004-4
Pastukhov, A., and Braun, J. (2007). Perceptual reversals need no prompting by attention. J. Vis. 7, 1–17.
Pastukhov, A., and Braun, J. (2008). A short-term memory of multi-stable perception. J. Vis. 8, 7.1–14. doi: 10.1167/8.13.7
Pastukhov, A., and Braun, J. (2013). Structure-from-motion: Dissociating perception, neural persistence, and sensory memory of illusory depth and illusory rotation. Attent. Percept. Psychophys. 75, 322–340. doi: 10.3758/s13414-012-0390-0
Pastukhov, A., and Klanke, J.-N. (2016). Exogenously triggered perceptual switches in multistable structure-from-motion occur in the absence of visual awareness. J. Vis. 16:14. doi: 10.1167/16.3.14
Pearson, J., and Brascamp, J. (2008). Sensory memory for ambiguous vision. Trends Cogn. Sci. 12, 334–341. doi: 10.1016/j.tics.2008.05.006
Pitts, M. A., and Britz, J. (2011). Insights from intermittent binocular rivalry and EEG. Front. Hum. Neurosci. 5:107. doi: 10.3389/fnhum.2011.00107
Pitts, M. A., Gavin, W. J., and Nerger, J. L. (2008). Early top-down influences on bistable perception revealed by event-related potentials. Brain Cogn. 67, 11–24.
Pitts, M. A., Martínez, A., and Hillyard, S. A. (2010). When and where is binocular rivalry resolved in the visual cortex? J. Vis. 10, 25–25. doi: 10.1167/10.14.25
Pitts, M. A., Nerger, J. L., and Davis, T. J. R. (2007). Electrophysiological correlates of perceptual reversals for three different types of multistable images. J. Vis. 7:6.
Polgári, P., Causin, J.-B., Weiner, L., Bertschy, G., and Giersch, A. (2020). Novel method to measure temporal windows based on eye movements during viewing of the Necker cube. PLoS One 15:e0227506. doi: 10.1371/journal.pone.0227506
Pressnitzer, D., and Hupe, J. M. (2006). Temporal dynamics of auditory and visual bistability reveal common principles of perceptual organization. Curr. Biol. 16, 1351–1357. doi: 10.1016/j.cub.2006.05.054
Robertson, C. E., Ratai, E.-M., and Kanwisher, N. (2016). Reduced GABAergic Action in the Autistic Brain. Curr. Biol. 26, 80–85. doi: 10.1016/j.cub.2015.11.019
Sandberg, K., Barnes, G. R., Bahrami, B., Kanai, R., Overgaard, M., and Rees, G. (2014). Distinct MEG correlates of conscious experience, perceptual reversals and stabilization during binocular rivalry. Neuroimage 100, 161–175. doi: 10.1016/j.neuroimage.2014.06.023
Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J., Glasstetter, M., Eggensperger, K., Tangermann, M., et al. (2017). Deep learning with convolutional neural networks for EEG decoding and visualization: Convolutional neural networks in EEG analysis. Hum. Brain Mapp. 38, 5391–5420. doi: 10.1002/hbm.23730
Schmack, K., Schnack, A., Priller, J., and Sterzer, P. (2015). Perceptual instability in schizophrenia: Probing predictive coding accounts of delusions with ambiguous stimuli. Schizophr. Res. 2, 72–77. doi: 10.1016/j.scog.2015.03.005
Sergent, C., Faugeras, F., Rohaut, B., Perrin, F., Valente, M., Tallon-Baudry, C., et al. (2017). Multidimensional cognitive evaluation of patients with disorders of consciousness using EEG: A proof of concept study. Neuroimage 13, 455–469. doi: 10.1016/j.nicl.2016.12.004
Sforza, C., Rango, M., Galante, D., Bresolin, N., and Ferrario, V. F. (2008). Spontaneous blinking in healthy persons: An optoelectronic study of eyelid motion. Ophthalm. Physiol. Optics 28, 345–353. doi: 10.1111/j.1475-1313.2008.00577.x
Sterzer, P., and Kleinschmidt, A. (2007). A neural basis for inference in perceptual ambiguity. Proc. Natl. Acad. Sci. U. S. A. 104, 323–328.
Strüber, D., and Herrmann, C. S. (2002). MEG alpha activity decrease reflects destabilization of multistable percepts. Cogn. Brain Res. 14, 370–382.
Strüber, D., and Stadler, M. (1999). Differences in top-down influences on the reversal rate of different categories of reversible figures. Perception 28, 1185–1196.
Tong, F., Nakayama, K., Vaughan, J. T., and Kanwisher, N. (1998). Binocular rivalry and visual awareness in human extrastriate cortex. Neuron 21, 753–759.
van Ee, R., van Dam, L. C., and Brouwer, G. J. (2005). Voluntary control and the dynamics of perceptual bi-stability. Vis. Res. 45, 41–55.
VanRullen, R., and Thorpe, S. J. (2002). Surfing a spike wave down the ventral stream. Vis. Res. 42, 2593–2615.
Watanabe, T. (2021). Causal roles of prefrontal cortex during spontaneous perceptual switching are determined by brain state dynamics. eLife 10:e69079. doi: 10.7554/eLife.69079
Weilnhammer, V., Fritsch, M., Chikermane, M., Eckert, A.-L., Kanthak, K., Stuke, H., et al. (2021). An active role of inferior frontal cortex in conscious experience. Curr. Biol. 31, 2868–2880.e8. doi: 10.1016/j.cub.2021.04.043
Weilnhammer, V., Stuke, H., Hesselmann, G., Sterzer, P., and Schmack, K. (2017). A predictive coding account of bistable perception—A model-based fMRI study. PLoS Comput. Biol. 13:e1005536. doi: 10.1371/journal.pcbi.1005536
Wexler, M. (2018). Multidimensional internal dynamics underlying the perception of motion. J. Vis. 18:7. doi: 10.1167/18.5.7
Yokota, Y., Minami, T., Naruse, Y., and Nakauchi, S. (2014). Neural processes in pseudo perceptual rivalry: An ERP and time–frequency approach. Neuroscience 271, 35–44. doi: 10.1016/j.neuroscience.2014.04.015
Keywords: ambiguous figures, perceptual reversals, bistable perception, EEG, Necker cube
Citation: Wilson M, Hecker L, Joos E, Aertsen A, Tebartz van Elst L and Kornmeier J (2023) Spontaneous Necker-cube reversals may not be that spontaneous. Front. Hum. Neurosci. 17:1179081. doi: 10.3389/fnhum.2023.1179081
Received: 03 March 2023; Accepted: 28 April 2023;
Published: 30 May 2023.
Edited by:
Adam James Reeves, Northeastern University, United StatesReviewed by:
Talis Bachmann, University of Tartu, EstoniaDanko Nikolic, Frankfurt Institute for Advanced Studies, Germany
Amedeo D’Angiulli, Carleton University, Canada
Copyright © 2023 Wilson, Hecker, Joos, Aertsen, Tebartz van Elst and Kornmeier. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jürgen Kornmeier, juergen.kornmeier@uni-freiburg.de