Transfer of predictive signals across saccades

Predicting visual information facilitates efficient processing of visual signals. Higher visual areas can support the processing of incoming visual information by generating predictive models that are fed back to lower visual areas. Functional brain imaging has previously shown that predictions interact with visual input already at the level of the primary visual cortex (V1; Harrison et al., 2007; Alink et al., 2010). Given that fixation changes up to four times a second in natural viewing conditions, cortical predictions are effective in V1 only if they are fed back in time for the processing of the next stimulus and at the corresponding new retinotopic position. Here, we tested whether spatio-temporal predictions are updated before, during, or shortly after an inter-hemifield saccade is executed, and thus, whether the predictive signal is transferred swiftly across hemifields. Using an apparent motion illusion, we induced an internal motion model that is known to produce a spatio-temporal prediction signal along the apparent motion trace in V1 (Muckli et al., 2005; Alink et al., 2010). We presented participants with both visually predictable and unpredictable targets on the apparent motion trace. During the task, participants saccaded across the illusion whilst detecting the target. As found previously, predictable stimuli were detected more frequently than unpredictable stimuli. Furthermore, we found that the detection advantage of predictable targets is detectable as early as 50–100 ms after saccade offset. This result demonstrates the rapid nature of the transfer of a spatio-temporally precise predictive signal across hemifields, in a paradigm previously shown to modulate V1.

Introduction

Comparing incoming sensory stimulation with previously generated predictions is an efficient strategy for processing the wealth of visual information. Predicted stimuli can be processed more efficiently and unpredicted surprising stimuli are allocated more processing resources. The brain constantly constructs predictive models of the world which are updated in anticipation of planned movements. With respect to vision, both Descartes and later von Helmholtz made an important discovery about the visual system: when external pressure is used to displace the eyeball, the visual scene moves. However, when we saccade our eyes, the visual world remains stable (Descartes, 1642–1648; von Helmholtz, 1962). This was the first evidence that internal models do not anticipate the mechanically induced change of the visual stimulus but are updated in anticipation of voluntary eye movements. An internal copy of the motor command, called efference copy, is used to update these internal predictions (Sperry, 1950). In hierarchical models of cortical processing, it is conceptualized that higher cortical areas incorporate planned motor signals and provide spatio-temporal predictions for lower level visual areas. Lower visual areas can use these top-down predictive signals to anticipate expected change and process visual information more rapidly and efficiently (Merriam and Colby, 2005; Bar, 2007; Gilbert and Sigman, 2007; Harrison et al., 2007; Kveraga et al., 2007; Friston, 2009; Alink et al., 2010). For example, predictions developed from previous experience can allow an individual to correctly represent the entire shape of an object when it is partially occluded (van Lier et al., 1994; Sugita, 1999; Erlhagen, 2003; Johnson and Olshausen, 2005).

Many models have suggested that predictions are generated in higher cortical areas. Mumford (1992) proposed that flexible templates are formed in higher cortical areas and sent down to lower cortical areas where they explain away the bottom-up input signal. In such a predictive model, only the non-explained, surprising incoming signal is fed forward whereas all other signals explainable by spatio-temporal context are filtered out at the earliest possible cortical processing stage. Rao and Ballard (1999) modeled a hierarchical predictive coding architecture in which higher levels of the model predict responses of the next lower level using feedback. Feedforward connections from lower to higher cortical areas communicate any errors between the predicted response and the actual response. When new models are learned new synaptic connections need to be formed reflecting learned associations (den Ouden et al., 2009). Several models have been proposed demonstrating the importance of the bidirectional influence between higher and lower cortical areas for perception and recognition (Bar et al., 2006; Lamme, 2006; Meyer, 2012). A more formal account of predictive coding has been developed by Friston (2005, 2009, 2010).

For predictive coding to facilitate visual processing it is important that the predictive signal transfers across hemifields rapidly, ensuring that it continues to aid visual recognition across visual fields. This study aims to demonstrate whether a predictive signal is transferred across hemifields and, more precisely, how quickly after saccade completion we can detect prediction effects previously related to V1 processing (Alink et al., 2010).

Previous evidence indicated that the transfer of information across saccades is rapid and accelerates visual perception by about 40 ms (Hunt and Cavanagh, 2009). When subjects saccaded toward a ticking clock and reported the time displayed on the clock, subjects’ response was 39 ms earlier than the actual time. Hunt and Cavanagh (2009) attributed this effect to anticipatory sensory enhancement in the target area in which the eyes fall after saccade. Peterburs et al. (2011) found three ERP components which were related to saccadic updating. The antecedent potential building from 80 to 40 ms prior to saccade was thought to be associated with the planning of the impending saccade, consistent with previous findings in monkey lateral intraparietal area (LIP) and frontal eye field (FEF; Duhamel et al., 1992; Umeno and Goldberg, 2001). The next component in the time course related to spatial updating was a negative ERP 30–70 ms post-saccade onset. Finally the last component to occur in relation to the saccade arrived 200–500 ms after the saccade onset. On closer inspection of this late updating, Peterburs et al. (2011) found evidence in the ERP traces to suggest that positive ERP activity in the time window between 100 and 150 ms after onset was related to interhemispheric transfer. Bellebaum and Daum (2006) also found an early post-saccadic component at 50 ms after offset which was thought to be imperative for saccadic updating. Allowing for approximately 80 ms for saccade duration (Baloh et al., 1975; see also Results below), Peterburs et al. (2011) evidence suggests predictive coding transfer should occur within 20–70 ms after saccade offset if it is indeed relevant for efficient processing.

In this experiment we used an apparent motion paradigm which has previously been proven useful to demonstrate the effect of a predictive mechanism (Schwiedrzik et al., 2007; Alink et al., 2010; see also Hidaka et al., 2011). Visual illusions reflect the fact that the brain draws inferences from the visual input and that prior beliefs (or predictions) are used to construct the percept (Goebel et al., 1998; Brown and Friston, 2012). Apparent motion is an illusion of motion induced by two stationary stimuli that blink on and off alternately. It gives rise to an illusory object moving between the inducing stimuli along the shortest path, but avoiding obstacles (Kolers, 1963; Attneave and Block, 1974; Shepard and Zare, 1983; Goebel et al., 1998; Muckli et al., 2002, 2005; Liu et al., 2004; Larsen et al., 2006). Long distance apparent motion is a particularly suitable paradigm as higher visual areas have larger receptive fields which enable them to process the spatio-temporal dynamics of the illusion, thus creating a prediction with regard to where the illusory motion token is at a certain time (Alink et al., 2010).

In the experimental paradigm used here, targets were presented on the apparent motion trace either in-time or out-of-time with the illusory motion token. Targets were similar in visual features to those stimuli inducing apparent motion. In-time targets fitted the predicted time and place of the illusory motion token better than those presented out-of-time. Moreover, we have shown that participants are significantly more accurate in detecting the more predictable in-time targets than the unpredictable out-of-time targets (Schwiedrzik et al., 2007). However, it should be noted that both in-time and out-of-time targets are masked by illusory motion and are detected less frequently than control stimuli that are not embedded in apparent motion (Schwiedrzik et al., 2007). Our previous brain imaging results with the same paradigm showed that the effect of prediction interacts with incoming information at the level of V1 (Alink et al., 2010). Alink et al. found that unpredictable, out-of-time targets caused a higher activation in V1 than predictable, in-time targets even though out-of-time targets were detected less frequently. In line with predictive coding frameworks (Mumford, 1992; Rao and Ballard, 1999; Friston, 2005, 2009, 2010), the decreased BOLD signal in response to predictable targets was interpreted as consistent with the notion that predicted information is processed more efficiently and thus causes less neural activation. The increased BOLD signal in response to unpredictable targets was thought to be a result of prediction errors. Alink et al. experiments were performed under conditions of central fixation and it is unclear whether predictability effects would also occur when cortical predictions need to be transferred across an eye movement. Since V1 has a precise retinotopic structure, feedback must interact with incoming information at a high spatial and temporal precision. It is unclear whether a predictive signal can quickly transfer to new retinal coordinates or even across visual hemifields. The objective of this study was to examine whether visual predictions transfer across hemifields and to measure the critical time window for such trans-saccadic predictions.

Here, we combined our previous apparent motion paradigm with inter-hemifield saccades to investigate the transfer of the predictive signal to the other hemifield. In contrast to a related study by Szinte and Cavanagh (2011), we added in-time and out-of-time targets on the apparent motion trace to investigate effects of visual predictions. By presenting targets along the apparent motion trace immediately after a saccade we were able to determine how long it takes for the predictive signal to transfer to the new retinal position.

Materials and Methods

Subjects

Thirty subjects were recruited via the online departmental subject pool, 27 were included in the final analysis (see Task and Procedures for reasons of exclusion; mean age 25, range 19–38 years, 19 females). Fourteen subjects performed version A of the experiment and 13 subjects version B. Of those, three subjects took part in both versions. All subjects had normal or corrected-to-normal eye sight, no history of brain damage, and signed informed consent.

Stimuli

Two white rectangles (2.1° each, 12.3° vertically apart) were flashed alternately to induce apparent motion (see Figure 1). Each apparent motion stimulus was displayed for five frames (67 ms) followed by an inter-stimulus interval of another five frames, resulting in a frequency of 3.75 Hz. Targets were the same shape and color, but slightly smaller (1.7°) than the inducing stimuli, to ensure they fall on the apparent motion trace and to account for cortical magnification. Targets were presented on the apparent motion trace at either an upper or lower position (2.5° from the midline) in either the second or fourth frame of the ISI. The targets were presented for 13.3 ms (1 refresh rate) either in-time with a linearly moving illusory token or out-of-time (i.e., at the same time but at the wrong target position, Figure 1B). Targets occurred equally often at the upper and lower target position and during both upward and downward apparent motion. Each trial consisted of 10 cycles of apparent motion. Apparent motion stimulation was continuous and the onsets and offsets of trials were not noticeable. The apparent motion stimulus was placed at the center of the screen with two fixation crosses (0.62° each, one green, one red) at either side (7° eccentricity). Fixation crosses changed color every 2.66 s (10 cycles of apparent motion), always at the beginning of cycle 6 of each trial. In version A of the experiment, targets were displayed in the cycle immediately before and immediately after the color change of the fixation cross (cycles 5 and 6) and also in between the color change (cycle 1). In version B of the experiment, targets were displayed in cycles 7, 8, 9, and 10, i.e., 2–4 cycles after the color change of the fixation cross (see explanation below). Apparent motion stimulation was interrupted with a natural scene display once a minute, enabling subjects to rest their eyes for 20 s and preventing apparent motion breakdown due to adaptation (Anstis and Giaschi, 1985). Stimuli were created using Presentation (Neurobehavioural Systems, Inc., Albany, USA) and presented on a 16 inch Sony Trinitron CRT Monitor (resolution: 1024 by 768, refresh rate: 75 Hz). The setup was similar to Szinte and Cavanagh (2011), however we used a larger vertical distance of the apparent motion stimulus and a much slower saccading rhythm.

FIGURE 1

Figure 1. (A) Schematic depiction of the stimulus display (not in scale). The two apparent motion stimuli flashed in alternation (at 3.75 Hz). During the ISI, a target was flashed at either an upper or lower position on the apparent motion trace. The target was of the same shape and luminance as the apparent motion stimuli though slightly smaller. Subjects maintained their eyes at the red fixation cross and saccaded across the illusion when the fixation cross changed color (every 2.66 s). (B) Time-space diagram of the stimulus display. Predictable targets were flashed in-time with a linearly moving illusory token whereas unpredictable targets were flashed out-of-time with an illusory token, i.e., at the same time as the corresponding predictable target but at the wrong place. Targets were presented either at an early or late delay, and either during upward or downward apparent motion.

Task and Procedure

Each subject was seated in a dark room at a distance of 70 cm from the computer monitor using a chin rest and a forehead support. Eye movements (EyeLink, SR Research, ON, Canada) were recorded throughout.

Pre-test

Prior to the main experiment, a 10 min pre-test was conducted to familiarize subjects with the task, determine their optimal stimulus contrast, and their baseline performance without saccades. Here, the same apparent motion stimulus was presented in the right visual field (7° eccentricity) with a single white fixation cross at the center. The subjects’ task was to keep their eyes at the central fixation cross and detect the targets on the apparent motion trace. The background gray values were varied block-wise in five steps [Michelson contrasts derived from luminance measurements with a photometer (Minolta): 0.80; 0.69; 0.56; 0.43; 0.29] to determine subjects’ individual stimulus contrast for highest detectability of in-time targets compared to out-of-time targets. This optimal contrast value was then used throughout the main saccading experiment. On average, a mean Michelson contrast of .052 (SEM.027) was employed. Mean detection rates across the five contrast values are plotted in Figure 2. Replicating previous findings (Schwiedrzik et al., 2007), in-time targets were detected better than out-of time targets [repeated-measures ANOVA: F(1, 23) = 42.09, p < 0.001]. Overall, detection rates increased with decreasing contrast [F(4, 92) = 3.21, p = 0.016; no interaction]. However, contrast blocks were not counter-balanced, so this could reflect a training effect instead of an effect of contrast.

FIGURE 2

Figure 2. Results of the pre-test. Here, subjects performed the task without saccading, with a fixation cross at the center and the apparent motion display at an eccentricity of 7° to the right. Contrast between background and stimuli was varied in five steps to determine the optimal stimulus contrast for each subject individually. Replicating previous results (Schwiedrzik et al., 2007), mean detection rates were higher for in-time targets than for out-of-time targets (p < 0.001). Error bars indicate 1 SEM.

Main experiment

In the main experiment, participants were instructed to fixate their eyes on the red fixation cross and saccade over the central illusion when the cross changed color, whilst never to rest their eyes directly at the illusion. At the same time, participants detected targets on the apparent motion trace and responded via a button press. Three subjects with frequent saccades to the center of the screen were excluded from the analysis. The experiment was broken up into four runs of 10 min and lasted in total about 1.5 h including pre-test, practice trials, and breaks.

Experimental Design

Two versions of the experiment were run. In version A we anticipated that subjects would rhythmically saccade from left to right without much saccade latency after cue, similarly to Szinte and Cavanagh (2011). Thus, we presented the targets mainly in the cycle before and after the fixation cross color change. However, after initial data analysis we realized that the timing of the cross color change was comparably slow and that subjects in fact showed a significant saccade latency (about 300 ms, see Results below). Therefore, we took this delay in saccading into account in version B of the experiment and presented the targets between 346 and 1303 ms after the saccade cue. We pooled the data from both versions of the experiment in the final data analysis to achieve maximum data coverage across all time windows.

Across all time windows, a total of 1300 trials were presented. To increase statistical power across all time windows, 40% of trials contained in-time targets, 40% contained out-of-time targets, and 20% contained no target. Note that our critical measure was not overall detection rate, but the difference between in-time and out-of-time target detection. Target presence, target timing (in-time or out-of-time), target position along the apparent motion trace, and target presentation time window were randomized and counter-balanced.

Analysis

Only trials with large horizontal saccades occurring within 500 ms after saccade cue were included. From all trials containing a target, detection rates were derived as the proportion of trials where a button press occurred between 150 and 1200 ms after target onset. Trials were sorted with respect to target time distance from individual saccade offset (for each trial and each subject). Note that as we were interested in the re-occurrence of a predictive effect after saccade, saccade offset was our critical point of reference rather than saccade onset as used in several other studies (e.g., Peterburs et al., 2011).

Detection rates for in-time and out-of-time targets were binned into 50 and 100 ms time windows (or bigger, see Figure 3A) and averaged, first within subjects, then across subjects. Data were only included in a bin if more than three trials per subject and more than three subjects contributed to that bin (outlier reduction). Relative differences in detection rates were computed on a single subject level as detection rate [in-time] – detection rate [out-of-time]/(detection rate [in-time] + detection rate [out-of-time]).

FIGURE 3

Figure 3. Results of the main experiment. (A) Diagram of the time windows of interest. Detection rates were analyzed for target occurrence relative to subjects’ individual saccade offset. (B) Mean detection rates for predictable in-time and unpredictable out-of-time targets for the data averaged across the four large time windows of interest. Error bars indicate 1 SEM. (C) Mean detection rates averaged across bins of 100 ms from 600 ms before and 200 ms after saccade offset (zoom). Note that bins always included data centered around the time point labeled on the x-axis. For example, the data point at 50 contains detection rates for targets occurring from 0 (saccade offset) to 100 ms after saccade offset. (D) Detection rates averaged across bins of 50 ms (further zoom) from 150 ms before to 200 ms after saccade offset. Dashed lines indicate data ±1 SEM, stars indicate p < 0.05.

Note that our experimental design implied that we could not compute d^′. Subjects only responded to the presence of a target but not its absence due to the fact that apparent motion stimulation was on-going and the onset and offset of trials were not noticeable. That is, while we could compute hits, misses, and false alarms, correct rejections are not captured with this experimental design.

Results

Mean latency between saccade cue and saccade onset was 307.9 ms (SEM 7.3), mean saccade duration was 89.3 ms (SEM 6.7).

Detection rates for in-time and out-of-time targets, pooled within large time windows according to mean saccade latency and mean saccade duration, are plotted in Figure 3B. As expected, in-time targets were detected more accurately than out-of-time targets [repeated-measures ANOVA; F(1, 26) = 110.26, p < 0.001]. Detection rates decreased after saccade cue and during saccades, leading to a main effect of time window [F(3, 78) = 25.13, p < 0.001]. This effect interacted with target timing [F(3, 78) = 2.77, p = 0.047]. Post hoc comparisons (paired-sample t-tests) for individual time windows revealed a significant detectability difference between in-time and out-of-time targets before the saccade cue and after saccade offset (p < 0.05, Bonferroni-corrected). At the uncorrected level, the detectability difference was also significant in the time window between saccade cue and saccade onset (p = 0.027), and marginally significant during saccade (p = 0.060). Note that the number of trials, and thus statistical power varied across time windows due to their variable length. The average percentage of trials contributing to the individual time windows were as follows: 68% (before and between saccades), 15% (after saccade cue), 5% (during saccade), and 12% (0–200 ms after saccade offset).

Detection rates were binned into 100 ms (Figure 3C) and 50 ms time windows (Figure 3D). Note that bins always included data centered around a specific time point. For example, in the data binned by 100 ms, the data point at 50 pooled over targets occurring from 0 (saccade offset) to 100 ms. Paired-sample t-tests (uncorrected) revealed that the detection advantage of in-time targets disappeared within 100–200 ms after saccade cue (Figure 3C) and reappeared as early as 50–100 ms after saccade offset (Figure 3D).

In Figure 4, the relative difference between in-time and out-of-time target detection rate is plotted for single subjects, for the data binned by 100 ms (Figure 4A) and for the data binned by 50 ms (Figure 4B). Note that some data points overlap and that the number of data entries varies across bins due to differences in individual saccade latencies, differential target presentation in relation to saccade offset, and outlier reduction (see Analysis above). The plots show a positive difference in detection rates (i.e., better detection for in-time than for out-of-time targets) in the majority of subjects in those time window in which we found a significant effect (cf. Figures 3C,D).

FIGURE 4

Figure 4. Relative difference in detection rates between predictable in-time and unpredictable out-of-time targets for single subjects. Data were binned by 100 ms (A) and by 50 ms (B) across the same time windows as in Figures 3C,D. Data points above the midline at 0 depict a positive difference, i.e., in-time targets were better detected than out-of-time targets in that particular subject. Vice versa for negative differences. Note that data points for several subjects may overlap and that the number of data entries varied across individual bins (see Results). As in Figures 3C,D, bins always included data centered around the time point labeled on the x-axis.

A control analysis showed that the detectability difference between in-time and out-of-time targets did not change over the four runs of the experiment [repeated-measures ANOVA; F(3, 27) = 2.916, p > 0.05]. Therefore, targets became neither more nor less predictable within the experimental session, precluding a potential confound of training as raised e.g., by de-Wit et al. (2010).

Discussion

Subjects were required to detect targets presented along the apparent motion trace whilst saccading across the illusion. As a main effect, targets that appeared in-time with the motion illusion were detected more frequently than those appearing out-of-time, replicating previous results (Schwiedrzik et al., 2007; Hidaka et al., 2011). The increased detection rate of in-time targets is an indication that the visual system generates an illusion-related prediction along the apparent motion path. Predicted in-time targets are processed more efficiently, detected better, and cause less fMRI brain activity in V1 (Alink et al., 2010). Previous results indicate that predictions of moving tokens are generated with the contribution of hMT/V5+ and are fed back to retinotopic visual areas (Sterzer et al., 2006; Wibral et al., 2009). Simulations of area V1 show that combined cortical feedback and lateral interaction can lead to precise spatial predictions (Erlhagen, 2003).

The main aim of our experiment was to determine the length of time taken by the predictive signal on the apparent motion trace to transfer across saccades and to re-occur at the new retinotopic position. This effect should occur swiftly (i.e., between 20 and 70 ms) after saccade offset to facilitate visual processing (Bellebaum and Daum, 2006; Peterburs et al., 2011), given that the next saccade is often initiated already after 250 ms in natural viewing conditions. Our results show that the predictive detection advantage of in-time targets is present as early as 50–100 ms after saccade offset. The transfer of the predictive signal occurs timely for visual processing in the next fixation period. This finding suggests that a spatio-temporally precise internal model is transferred across saccades and updated within 50–100 ms. This fast time window relates to the earliest time window in which stable vision is possible after saccades due to saccadic suppression. It is also too early to allow for an entirely new rebuilt apparent motion illusion in the new hemifield and subsequent post-diction to take place. For rebuilt and post-diction, at least half a cycle of apparent motion (133 ms) would need to be presented in the new hemifield (see discussion below).

Interestingly, the same time window of 50 ms and above was found to be critical for the release of saccadic suppression (Deubel et al., 1996). Subjects are unable to detect relatively large displacements of saccadic target stimuli if they occur during saccade or up to 50 ms after saccade offset. Our data is in accordance with this finding: Projecting the predicted target position to a new post-saccadic retinotopic position takes about 50–100 ms. Before this time period, spatio-temporal target displacements will go largely unnoticed because the precise location is not yet transferred. Indeed, Deubel et al. (1996) found that when the target stimulus remains off (“blanked”) for 50 ms or longer after saccade offset, subjects recover the ability to detect target displacements. Furthermore, the earliest ERP time component related to trans-saccadic updating and integrating of visual information starts at 50 ms after saccade offset (Bellebaum and Daum, 2006). The authors relate the parietal ERP component starting at 50 ms to the updating process that matches the efference copy of the motor command to the stimulus location. It is plausible to assume that this reflects the process that transfers the prediction to the new retinotopic position at which it will facilitate processing of in-time targets.

Our data also show that the overall detection of targets is reduced during saccade and until 50 ms after saccade offset. In theory, we cannot exclude the possibility of an in-time prediction effect during these time windows, but the low detection rates do not allow for sufficient statistical power (see Results).

It seems that the visual prediction system has learned its delay times and found ways to compensate the lost 50 ms by correcting its forward prediction. Hunt and Cavanagh (2009) showed that subjects who follow the arms of a fast moving clock with peripheral vision will predate the fixation of the clock by 40–60 ms – a process that might be thought of as a temporal filling-in process to avoid discontinuities introduced by each saccadic eye-movement and its saccadic suppression. Other motion illusions are related to this temporal filling-in: movement into the blind spot is extrapolated in its expected coordinates even when no retinal signal is received (Maus and Nijhawan, 2008). A common demonstration of forward adjustment of predictions is the flash lag illusion (Nijhawan, 2008). It seems that we act on predictions corrected forward in-time unless there is a strong signal overwriting this prediction. Weak error signals as our out-of time stimulus are likely to remain unnoticed like a small signal in a noisy pattern. Strong unexpected transients, however, allow for an immediate update (Maus et al., 2010).

To perceive apparent motion during saccadic eye-movements, the visual system has to keep track of the spatiotopic position of the moving illusion and correct for eye-movement induced shifts at retinotopic positions. Szinte and Cavanagh (2011) measured the precision with which spatiotopic coordinates of the apparent motion illusion are updated while saccadic eye-movements are performed. If the remapping compensation is perfect, vertical apparent motion should appear precisely vertical even if a horizontal saccade is performed across the illusion. However, the findings of Szinte and Cavanagh (2011) suggest differently: the trans-saccadic remapping of the apparent motion end points leads to an overcompensation of the eye-movement amplitude by 5%, and the illusion appears tilted by up to 9°. Interestingly, the compensation was tested at nine different positions and it was found to vary between positions individually, suggesting that the compensation does not follow an overall global correction but depends on locally acquired experience.

Our experiment does not inform us about the spatial precision with which a signal is transferred (apart from the fact that the transfer is precise enough for the in-time/out-of-time difference to take effect). Also, it should be noted that the horizontal saccadic rhythm was much slower in our paradigm compared to Szinte and Cavanagh (2011) and that the illusion did not appear tilted, suggesting that no overcompensation occurred.

The decrease in mean detection rate seen in Figure 3C prior and during saccade could be explained by trans-saccadic suppression and peri-saccadic mislocalization. During trans-saccadic suppression there is a general reduction in visual sensitivity which can occur even prior to saccade onset (Vallines and Greenlee, 2006). Peri-mislocalization could also account for a decrease in target detection as objects which are flashed close to saccade onset are largely mislocalized on the retina from their actual physical position (Ostendorf et al., 2007). This mislocalization may occur due to spatio-temporal mismatch between the saccade and extraretinal eye position information (Ross et al., 2001). Both these models of vision breakdown over saccades could predict a decrease in detection rate of both in-time and out-of-time targets within the illusion.

Szinte and Cavanagh (2011) findings as well as evidence by Rolfs et al. (2011) suggest that there is a close interplay between the remapped visual information and attention. Our observed prediction effect could be explained by smoothly moving visuo-spatial attention, similar to what Shioiri et al. (2002) demonstrated behaviorally. That is, subjects’ attention may have been trained on the dynamics of the illusory motion as they were instructed to detect targets along the apparent motion trace. As attention is transferred across saccades as much as visual information (Rolfs et al., 2011), this may lead to a better detection rate of in-time targets as they appear in the focus of attention. Our results are consistent with dynamical concepts of a fast moving attentional searchlight: such a moving searchlight predicts the location where a stimulus is expected – which is closely related to a moving token or a motion prediction.

However, our results cannot be explained with conventional notions of a static visuo-spatial attention searchlight, as it cannot account for in-time/out-of-time differences. Even when visuo-spatial attention is focused on a center task, the apparent motion illusion in the periphery remains strong (Kohler et al., 2008) and brain activity along the apparent motion trace is increased (Muckli et al., 2005). Gilbert and Sigman (2007) highlight the wealth of top-down influences and note that “the notion of attention itself may be inadequate as a descriptor of the full range of top-down influences that are exerted.”

We propose that the predictive signal is transferred from one hemifield to the next. An alternative would be to assume that the signal could be rebuilt anew or that the presence of an in-time target was inferred by post-diction. Our data show that rebuilding of a detectability advantage of in-time targets must occur until 50–100 ms after saccade offset. For post-diction to be effective in the new hemifield, both the upper and lower apparent motion stimuli must have been presented and perceived for the in-time/out-of-time detectability difference to take effect. Given that half an apparent motion cycle lasted 133 ms, it is unlikely that an entire rebuilt of the predictive signal could have occurred within 50–100 ms after saccade offset.

It is worth mentioning that our results are not in contrast to Yantis and Nakama (1998). Yantis and Nakama (1998) showed that target discrimination degrades if targets are presented on the apparent motion trace, but they did not investigate in-time versus out-of-time differences of target stimuli on the apparent motion path. In line with Yantis and Nakama (1998), also our apparent motion illusion induces motion masking and overall reduces the detectability of both in-time and out-of-time stimuli (Schwiedrzik et al., 2007). When apparent motion is not induced, both types of stimuli are detected equally well. In the presence of the illusion, in-time stimuli are less masked by apparent motion than out-of-time-stimuli. Moreover, Yantis showed that high precision object discrimination is reduced on the apparent motion trace, whereas our paradigm just required the detection of a simple flash without the need of high spatial frequency analysis. High precision object discrimination may be incompatible with the apparent motion illusion as is exemplified by interference of inconsistent stimulus features on the apparent motion path with motion masking: for example, orthogonally oriented Gabor patches along the apparent motion trace slow down the perceived speed of the motion illusion (Georges et al., 2002).

Motion induced blindness provides another example in which static stimuli not fitting to the motion percept are overwritten by a top-down motion prediction even though the non-perceived stimulus induced a stronger V1 signal (Schölvinck and Rees, 2010). One of the most convincing demonstrations of predictive coding overwriting the physical stimulus is given by Hidaka et al. (2009). Three blinking bars triggered a strong apparent motion prime that was followed by a test stimulus of two blinking bars that could either consistently continue the apparent motion direction or that blinked in opposite sequence. In both cases, subjects see consistent apparent motion, indicating that motion prediction overwrites the non-fitting opponent motion. Both the out-of-time stimulus of our study and the apparent motion stimulation in the opponent direction of Hidaka et al.’s (2009) study are less detectable as they are overwritten by top-down predictions.

Conclusion

Our findings are an additional piece of evidence for the theory of a predictive mechanism in the visual system. Predictive signals transfer rapidly across hemifields. At around 50–100 ms after saccade offset, the apparent motion illusion, including its predicted path, is remapped to the corresponding retinotopic position in the other hemifield. The time interval corresponds well to other forms of inter-hemifield update. Future brain imaging experiments could then test whether the earlier observed V1 effect does also cross over to the contralateral hemisphere in the same critical time window.

Consistent with previous research it seems that predictive codes help to maintain information across saccades. Our results suggest that the visual brain does not passively wait to be stimulated but rather constantly forms predictions to allow for consistency across saccades and over space and time.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was funded by BBSRC grant BB/G005044/1.

References

Alink, A., Schwiedrzik, C. M., Kohler, A., Singer, W., and Muckli, L. (2010). Stimulus predictability reduces responses in primary visual cortex. J. Neurosci. 30, 2960–2966.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Anstis, S., and Giaschi, D. (1985). Adaptation to apparent motion. Vision Res. 25, 1051–1062.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Attneave, F., and Block, G. (1974). Absence of masking in the path of apparent movement. Percept. Psychophys. 16, 205–207.

CrossRef Full Text

Baloh, R. W., Sills, A. W., Kumley, W. E., and Honrubia, V. (1975). Quantitative measurement of saccade amplitude, duration, and velocity. Neurology 25, 1065–1065.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bar, M. (2007). The proactive brain: using analogies and associations to generate predictions. Trends Cogn. Sci. (Regul. Ed.) 11, 280–289.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bar, M., Kassam, K. S., Ghuman, A. S., Boshyan, J., Schmid, A. M., Schmidt, A. M., Dale, A. M., Hämäläinen, M. S., Marinkovic, K., Schacter, D. L., Rosen, B. R., and Halgren, E. (2006). Top-down facilitation of visual recognition. Proc. Natl. Acad. Sci. U.S.A. 103, 449–454.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bellebaum, C., and Daum, I. (2006). Time course of cross-hemispheric spatial updating in the human parietal cortex. Behav. Brain Res. 169, 150–161.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Brown, H., and Friston, K. J. (2012). Free-energy and illusions: the corn sweet effect. Front. Psychol. 3:43. doi:10.3389/fpsyg.2012.00043

CrossRef Full Text

den Ouden, H. E. M., Friston, K. J., Daw, N. D., McIntosh, A. R., and Stephan, K. E. (2009). A dual role for prediction error in associative learning. Cereb. Cortex 19, 1175–1185.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Descartes, R. (1642–1648). Über den Menschen. Beschreibung des Menschlichen Körpers. First published as ‘De homine’, trans. K. E. Rothschuh 1969. Heidelberg: Lambert Schneider.

Deubel, H., Schneider, W. X., and Bridgeman, B. (1996). Postsaccadic target blanking prevents saccadic suppression of image displacement. Vision Res. 36, 985–996.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

de-Wit, L., MacHilsen, B., and Putzeys, T. (2010). Predictive coding and the neural response to predictable stimuli. J. Neurosci. 30, 8702–8703.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Duhamel, J. R., Colby, C. L., and Goldberg, M. E. (1992). The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255, 90–92.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Erlhagen, W. (2003). Internal models for visual perception. Biol. Cybern. 88, 409–417.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Friston, K. (2005). A theory of cortical responses. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360, 815–836.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Friston, K. (2009). The free-energy principle: a rough guide to the brain? Trends Cogn. Sci. 13, 293–301.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Friston, K. (2010). The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11, 127–138.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Georges, S., Seriès, P., Frégnac, Y., and Lorenceau, J. (2002). Orientation dependent modulation of apparent speed: psychophysical evidence. Vision Res. 42, 2757–2772.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Gilbert, C. D., and Sigman, M. (2007). Brain states: top-down influences in sensory processing. Neuron 54, 677–696.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Goebel, R., Khorram-Sefat, D., Muckli, L., Hacker, H., and Singer, W. (1998). The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery. Eur. J. Neurosci. 10, 1563–1573.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Harrison, L. M., Stephan, K. E., Rees, G., and Friston, K. J. (2007). Extra-classical receptive field effects measured in striate cortex with fMRI. Neuroimage 34, 1199–1208.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hidaka, S., Nagai, M., and Gyoba, J. (2009). Spatiotemporally coherent motion direction perception occurs even for spatiotemporal reversal of motion sequence. J. Vis. 9, 6.1–12.

CrossRef Full Text

Hidaka, S., Nagai, M., Sekuler, A. B., Bennett, P. J., and Gyoba, J. (2011). Inhibition of target detection in apparent motion trajectory. J. Vis. 11, 1–12.

CrossRef Full Text

Hunt, A. R., and Cavanagh, P. (2009). Looking ahead: the perceived direction of gaze shifts before the eyes move. J. Vis. 9, 1–7.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Johnson, J. S., and Olshausen, B. A. (2005). The recognition of partially visible natural objects in the presence and absence of their occluders. Vision Res. 45, 3262–3276.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kohler, A., Haddad, L., Singer, W., and Muckli, L. (2008). Deciding what to see: the role of intention and attention in the perception of apparent motion. Vision Res. 48, 1096–1106.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kolers, P. A. (1963). Some differences between real and apparent visual movement. Vision Res. 3, 191–206.

CrossRef Full Text

Kveraga, K., Ghuman, A. S., and Bar, M. (2007). Top-down predictions in the cognitive brain. Brain Cogn. 65, 145–168.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lamme, V. A. F. (2006). Towards a true neural stance on consciousness. Trends Cogn. Sci. (Regul. Ed.) 10, 494–501.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Larsen, A., Madsen, K. H., Lund, T. E., and Bundesen, C. (2006). Images of illusory motion in primary visual cortex. J. Cogn. Neurosci. 18, 1174–1180.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Liu, T., Slotnick, S. D., and Yantis, S. (2004). Human MT+ mediates perceptual filling-in during apparent motion. Neuroimage 21, 1772–1780.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Maus, G. W., and Nijhawan, R. (2008). Motion extrapolation into the blind spot. Psychol. Sci. 19, 1087–1091.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Maus, G. W., Weigelt, S., Nijhawan, R., and Muckli, L. (2010). Does area v3a predict positions of moving objects? Front. Psychol. 1:186. doi:10.3389/fpsyg.2010.00186

CrossRef Full Text

Merriam, E. P., and Colby, C. L. (2005). Active vision in parietal and extrastriate cortex. Neuroscientist 11, 484–493.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Meyer, K. (2012). Another remembered present. Science 335, 415–416.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Muckli, L., Kohler, A., Kriegeskorte, N., and Singer, W. (2005). Primary visual cortex activity along the apparent-motion trace reflects illusory perception. PLoS Biol. 3, e265. doi:10.1371/journal.pbio.0030265

CrossRef Full Text

Muckli, L., Kriegeskorte, N., Lanfermann, H., Zanella, F. E., Singer, W., and Goebel, R. (2002). Apparent motion: event-related functional magnetic resonance imaging of perceptual switches and states. J. Neurosci. 22, RC219.

Pubmed Abstract | Pubmed Full Text

Mumford, D. (1992). On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol. Cybern. 66, 241–251.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Nijhawan, R. (2008). Visual prediction: psychophysics and neurophysiology of compensation for time delays. Behav. Brain Sci. 31, 179–198.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ostendorf, F., Fischer, C., Finke, C., and Ploner, C. J. (2007). Perisaccadic compression correlates with saccadic peak velocity: differential association of eye movement dynamics with perceptual mislocalization patterns. J. Neurosci. 27, 7559–7563.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Peterburs, J., Gajda, K., Hoffmann, K.-P., Daum, I., and Bellebaum, C. (2011). Electrophysiological correlates of inter- and intrahemispheric saccade-related updating of visual space. Behav. Brain Res. 216, 496–504.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Rao, R. P. N., and Ballard, D. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79–87.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Rolfs, M., Jonikaitis, D., Deubel, H., and Cavanagh, P. (2011). Predictive remapping of attention across eye movements. Nat. Neurosci. 14, 252–256.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ross, J., Morrone, M. C., Goldberg, M. E., and Burr, D. C. (2001). Changes in visual perception at the time of saccades. Trends Neurosci. 24, 113–121.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schölvinck, M. L., and Rees, G. (2010). Neural correlates of motion-induced blindness in the human brain. J. Cogn. Neurosci. 22, 1235–1243.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schwiedrzik, C. M., Alink, A., Kohler, A., Singer, W., and Muckli, L. (2007). A spatio-temporal interaction on the apparent motion trace. Vision Res. 47, 3424–3433.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shepard, R., and Zare, S. (1983). Path-guided apparent motion. Science 220, 632–634.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Shioiri, S., Yamamoto, K., Kageyama, Y., and Yaguchi, H. (2002). Smooth shifts of visual attention. Vision Res. 42, 2811–2816.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sperry, R. W. (1950). Neural basis of the spontaneous optokinetic response produced by visual inversion. J. Comp. Physiol. Psychol. 43, 482–489.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sterzer, P., Haynes, J.-D., and Rees, G. (2006). Primary visual cortex activation on the path of apparent motion is mediated by feedback from hMT+/V5. Neuroimage 32, 1308–1316.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sugita, Y. (1999). Grouping of image fragments in primary visual cortex. Nature 401, 269–272.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Szinte, M., and Cavanagh, P. (2011). Spatiotopic apparent motion reveals local variations in space constancy. J. Vis. 11, 1–20.

CrossRef Full Text

Umeno, M. M., and Goldberg, M. E. (2001). Spatial processing in the monkey frontal eye field. II. Memory responses. J. Neurophysiol. 86, 2344–2352.

Pubmed Abstract | Pubmed Full Text

Vallines, I., and Greenlee, M. W. (2006). Saccadic suppression of retinotopically localized blood oxygen level-dependent responses in human primary visual area V1. J. Neurosci. 26, 5965–5969.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

van Lier, R., van der Helm, P., and Leeuwenberg, E. (1994). Integrating global and local aspects of visual occlusion. Perception 23, 883–903.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

von Helmholtz, H. (1962). Treatise on Physiological Optics, Vol. 3. New York: Dover Publications.

Wibral, M., Bledowski, C., Kohler, A., Singer, W., and Muckli, L. (2009). The timing of feedback to early visual cortex in the perception of long-range apparent motion. Cereb. Cortex 19, 1567–1582.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Yantis, S., and Nakama, T. (1998). Visual interactions in the path of apparent motion. Nat. Neurosci. 1, 508–512.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text