The Influence of Auditory Information on Visual Size Adaptation

Size perception can be influenced by several visual cues, such as spatial (e.g., depth or vergence) and temporal contextual cues (e.g., adaptation to steady visual stimulation). Nevertheless, perception is generally multisensory and other sensory modalities, such as auditory, can contribute to the functional estimation of the size of objects. In this study, we investigate whether auditory stimuli at different sound pitches can influence visual size perception after visual adaptation. To this aim, we used an adaptation paradigm (Pooresmaeili et al., 2013) in three experimental conditions: visual-only, visual-sound at 100 Hz and visual-sound at 9,000 Hz. We asked participants to judge the size of a test stimulus in a size discrimination task. First, we obtained a baseline for all conditions. In the visual-sound conditions, the auditory stimulus was concurrent to the test stimulus. Secondly, we repeated the task by presenting an adapter (twice as big as the reference stimulus) before the test stimulus. We replicated the size aftereffect in the visual-only condition: the test stimulus was perceived smaller than its physical size. The new finding is that we found the auditory stimuli have an effect on the perceived size of the test stimulus after visual adaptation: low frequency sound decreased the effect of visual adaptation, making the stimulus perceived bigger compared to the visual-only condition, and contrarily, the high frequency sound had the opposite effect, making the test size perceived even smaller.

Introduction

Humans rely on visual information to create a representation of the environment. Perception of size is an important feature to build this representation, but, like other features, it can be subjected to an erroneous perception due to several multi-sensory information. Size perception depends on several factors: spatial context (Leibowitz et al., 1969; Aglioti et al., 1995; Haffenden et al., 2001; Konkle and Oliva, 2012), vergence (Bradshaw et al., 1996; Arnold et al., 2008) and visual adaptation (Blakemore and Sutton, 1969; Pooresmaeili et al., 2013), leading to a misperception of physical size. In this work, we will investigate the influence of adaptation on size perception.

Adaptation is a behavioral technique used to investigate the flexibility of the neural responses to a specific sensory stimulation (for a review see Thompson and Burr, 2009). For example, a prolonged presentation of a stimulus (adapter) with constant size can affect the size of a subsequent target presented in the same portion of space. If the adapter is bigger than the target, the latter will be perceived smaller than its physical size and vice versa, if the adaptation is performed with a small stimulus the target will be perceived bigger. This process is called size adaptation or size aftereffect (Pooresmaeili et al., 2013).

Visual perception is not just influenced by visual information, but can also be greatly altered by other sensory modalities, even at early stages of processing (for a review see Shams and Kim, 2010). Auditory or tactile information has an impact on several visual tasks, such as the visual detection task (Frassinetti et al., 2002; Bolognini et al., 2005), contrast sensitivity (Tanaka et al., 2005; Lippert et al., 2007), and motion perception (Meyer and Wuerger, 2001; Hidaka et al., 2009; Teramoto et al., 2010). Another example of such interaction across modalities is given by the bouncing illusion (Sekuler et al., 1997). When two identical objects move along the diagonal of a 2-dimensional display, the objects can be seen either bouncing against each other or crossing along a stream. However, when the two objects coincide and a brief sound is simultaneously presented, the visual perception of motion is biased toward the bouncing motion. The reason behind this perceptual switch is not clear, but this phenomenon can also occur with subliminal sounds, suggesting a perceptual level of processing (Dufour et al., 2008).

Adaptation can also be transferred between different sensory modalities, leading to cross-modal aftereffects by using a visual/tactile motion adaptation (Konkle et al., 2009), vestibular/proprioceptive motion adaptation (Cuturi and Macneilage, 2014) and visual/auditory depth motion adaptation (Kitagawa and Ichihara, 2002).

Similarly, audition can influence perception of size, showing cross-modal synesthetic association. Evidence of such interactions between sound frequencies and visual size show that irrelevant sounds could influence the speed with which participants judged the size of visual stimuli (Gallace and Spence, 2006; Parise and Spence, 2009). Takeshima and Gyoba (2013) tested which aspects between sound intensity, temporal windows of audio-visual interaction and eccentricity of the visual stimulus could been involved in the influence of auditory stimuli on visual perception of size. They found that high-intensity auditory stimuli increased the perceived size of an object and that this bias is limited to a narrow temporal window (−100 to +100 ms), and that this effect was greater in the peripheral visual field.

No studies to date have investigated whether similar associations are also present for the integration between vision and audition with regard to size aftereffect. Here we investigated this point by testing the influence of auditory cues on visual size aftereffect.

We used a similar paradigm of size adaptation as that used by Pooresmaeili et al. (2013). Participants had to perform a size discrimination task in three experimental conditions: visual-only, visual-sound at 100 Hz and visual-sound at 9,000 Hz. The participants had to judge which of the two stimuli was bigger. Visual stimuli were presented in the periphery of the visual field. In the main experiment, the auditory stimulus was concurrent to the test stimulus. For all conditions, we repeated the task, presenting an adapter (twice as big as the reference stimulus) before the test stimulus. In this way we had a congruent and an incongruent condition, i.e., the high pitch and adaptation would facilitate or increase the effect, that is, the test stimulus would be perceived as smaller, whereas the low pitch would decrease the effect induced by adaptation. We run also a control experiment with a reduced sample of participants in which there was a temporal delay between test and auditory stimuli. We wanted to test whether the effect was modulated by stimulus onset asynchrony.

Before the size discrimination task, the participants participated in a “listen and draw” task, in which they listened to sounds of different frequencies and had to draw on a screen a circle that matched the size expressed by the sound presented.

We hypothesized that in the “listen and draw” task, the participants would naturally match high frequency to small size and low frequency to big size. In the size discrimination task, we expected an increase in the aftereffect in the high pitch condition and a decrease in the aftereffect in the low pitch condition compared to the visual-only condition. Our results confirm our hypothesis: the low frequency sound decreased the effect of visual adaptation while the high frequency sound had the opposite effect.

Materials and Methods

Participants

Sixteen healthy adults (eight male; average age of 25, SD = 3) with normal or corrected to normal vision were recruited to participate in our experiment. All participants gave written informed consent before starting the test. The study was approved by the ethics committee of the local health service (Comitato Etico, ASL 3, Genova).

Apparatus and Stimuli

For the listen and draw task, the sounds were generated by means of Audacity software (http://audacity.sourceforge.net/). We used five sounds with different pitches: 100, 500, 1,000, 5,000, and 9,000 Hz. The duration of the auditory stimulus was 500 ms. Visual stimuli were generated in MATLAB, using the Psychophysics Toolbox version extensions. The monitor on which the participants had to draw was a 19-inch LCD color monitor (resolution 1,280 × 1,024).

For the size discrimination task, stimuli were displayed on a Barco monitor (resolution of 1,280 × 1,024 corresponding to 41° × 31.2° from subject viewing distance of 57 cm with a refresh rate of 100 Hz).

In the size discrimination task, visual stimuli were Craik-O'Brien-Cornsweet stimuli. The stimuli were constructed by high-pass Gaussian filters (with a 50% cutoff at spatial frequency of 0.5 cycles/°), with the advantage of eliciting localized activation of the visual cortex, limited to their edges (Perna et al., 2005; Boyaci et al., 2007; Pooresmaeili et al., 2013). All visual stimuli were displayed with an eccentricity of 10°. The test stimulus was always presented on the left of the screen, whereas the reference stimulus was presented with the same eccentricity on the right side. The adapter stimulus was always centered at the same eccentricity as the test stimulus. The size of the adapter and the reference stimuli were keep constant, at 10° and 5° respectively. The size of the test stimulus was determined by QUEST (Watson and Pelli, 1983), an adaptive algorithm which, based on the current estimation of the participant, estimates the best stimulus value to be presented in the subsequent trial. The range of variation of the test stimulus was between 0.5 to 1.5 times the size of reference stimulus. Therefore, the physical size of the test stimulus was always smaller than the adapter. The test and the reference stimuli were presented for 500 ms, while the adapter lasted 10 s with a top-up of 6 s. For the auditory conditions, we used two of the sounds used in the previous task: the lowest pitch (100 Hz) and the highest pitch (9,000 Hz).

In all tasks, sounds were heard through noise canceling headphones (Bose^® QuietComfort 25 Acoustic Noise Cancelling Headphones).

Procedure

First, each participant performed “the listen and draw” task. The participant sat in front of a screen and was asked to draw a circle as big as the sound he/she heard by pressing the left button of the mouse (Figure 1B). Participants performed 25 trials, 5 repetitions per pitch level.

FIGURE 1

Figure 1. (A) Procedure of the size adaptation paradigm: the first phase started with a gray screen and a fixation point in the middle of the screen, then the test stimulus and the reference stimulus were subsequently presented. Concurrent with the test stimulus, an acoustic sound (low or high frequency) could also be presented with the same duration of the test stimulus. In the second phase, the adapter was presented before the test stimulus. (B) Set up for the “listen and draw” task. First, the participant hears on of the sounds and then using the mouse has to draw a circle as big as the sound heard.

The size discrimination experiment was divided into two phases. In the first one the subject was presented with three blocks of trials, one for each condition, in a pseudo-randomized order. In the visual condition, we evaluated the perceptual bias of each participant in discriminating the size test and reference stimuli (Figure 1A). In the sound conditions, we simultaneously presented the low (100 Hz) or high (9,000 Hz) pitch sounds with the test stimulus. Each condition consisted of 100 trials.

The second phase was identical to the first, except that before the test stimulus we presented the adapter stimulus. Also, in this case the sequence of the blocks was pseudo-randomized. Each condition consisted of 75 trials.

In all phases, the participants had to fix the fixation point on the center of the screen and judge whether the test stimulus was bigger or smaller than the reference by pressing a key on the keyboard (L = bigger, S = smaller).

A reduced sample of participants was tested again to repeat the size discrimination task. In this case in the auditory conditions the test and the auditory stimuli were presented with an asynchrony of 80 ms (as in Jaekl et al., 2012), with the visual stimulus leading.

Results

About the size discrimination task, first we ran an analysis to all conditions to check for the presence of outliers in the sample. One participant had an adaptation effect higher than two standard deviation from the mean in the incongruent condition (M = 10.2%, SD = 3.4%; participant = 18%), so was excluded from further the analysis.

We replicated the behavioral results obtain by Pooresmaeili et al. (2013). The presentation of the adapter stimulus influenced the perceived size of the subsequent test stimulus. The adapter, always bigger than the test, produced a shift of the psychometric function compared to the baseline, thus showing a decrease in the perceived size of the test stimulus (Figure 2). This result was supported by the statistical analysis that confirmed a significant difference between the point of subject equality (PSE) of the baseline and the adaptation in the visual condition [two tailed pair-sample t-test, t₍₁₅₎ = −9.1, p < 0.0001].

FIGURE 2

Figure 2. Psychometric discrimination function of the baseline (cyan) and adaptation (blue) in the visual-only condition of two typical subjects: adaptation to a 10° stimulus with a reference of 5° produces a rightward shift of the psychometric curve, indicating that the adaptation reduces the perceived size of the test stimulus.

First, we analyzed only the effect of the low and high pitch sound on the perceived size of the visual stimuli (phase one). We ran a one-way ANOVA with factor Condition (visual-only, high and low pitch). The results are showed in Figure 3A did not have a significant outcome [F_{(3, 15)} = 1.21, p = 0.31].

FIGURE 3

Figure 3. (A) The scatter plot shows the percentage of the perceived size of the visual stimulus in the auditory conditions (on the ordinate) and the only-visual condition (on the abscissa). The filled squares represent the average of the low pitch condition (green) and high pitch condition (red). The empty symbols are the data of single participants. (B) The scatter plot represents the percentage of the perceived size of the visual stimulus in all conditions after the visual adaptation (on the ordinate) and the only-visual condition (on the abscissa) without adaptation. The filled squares represent the average of the low pitch (green), high pitch (red) and only-visual (blue) conditions. The empty symbols are the data of single participants.

Subsequently, we calculated the effect of adaptation for all three conditions (visual-only, low and high pitch). The only-visual condition acquired in the first phase was considered as baseline. We defined as Δ of the effect the difference of the PSEs of each adaptations and the baseline. The results are plotted in Figure 3B. The Δ of the effect for the sound conditions showed an opposite tendency: the low pitch sound presented simultaneously to the visual test made the test stimulus be perceived bigger compared to in the visual-only condition, whereas the high frequency sound made it be perceived even smaller. A one-way ANOVA with factor Condition (visual-only, high pitch, and low pitch) showed a main effect [F_{(3, 15)} = 5, p < 0.02]. Post-hoc analysis (two tailed pair-sample t-test) Bonferroni corrected for multiple comparison reflected a significant difference between the two sound conditions [t₍₁₅₎ = −3.61, p < 0.01] and between the visual-only and the high pitch conditions [t₍₁₅₎ = 2.96, p < 0.05]. Moreover, we wanted to test whether there is a correlation between the effect of adaptation and the pitch, that could explain the variability between participants. We run a Pearson correlation analysis between the Δ of the effect in the only-visual condition and the sound conditions. We have a positive correlation of 0.79 (p < 0.0001) between the only-visual and low pitch conditions and of 0.82 (p < 0.0001) between the only-visual and high pitch conditions, i.e., more a participant is affected by contextual visual information more this effect will increase by adding congruent auditory information.

To test whether the effect found was due to the synchrony between acoustic and visual stimuli, we analyzed the results of a reduced sample (5 participants) where we introduced an asynchrony of 80 ms between the test and the acoustic stimuli. Given the small size of the sample we used a no parametric test, i.e., Kruskal–Wallis test. As for the synchronous condition, we calculated the Δ of the effect by subtracting the baseline to the PSEs of the adaptation for each condition (Figure 4). The results did not highlight a significant difference between the conditions (χ² = 1.94, p = 0.37).

FIGURE 4

Figure 4. Average of the aftereffect for each condition with temporal asynchrony: visual-only (blue), low pitch (green), and high pitch (red) frequencies frequency. The empty symbols are the data of single participants.

For the “listen and draw” test, we calculated the area drawn on the screen by converting the area from pixels to cm². We ran a Lilliefors (Kolmogorov–Smirnov) test to check the normality of the sample for the “listen and draw” test. Results showed that data were not normally distributed for the area drawn.

We used non-parametric statistical analysis. To see if there was a difference in the areas drawn associated to each pitch, we performed a Kruskal–Wallis test analysis. Results (Figure 5) showed a main effect for the type of pitch heard (χ² = 26.14, p < 0.0001). A post-hoc Bonferroni correction for multiple comparisons revealed a significant difference between the areas of 100 and 9,000 Hz (p < 0.001), 100 and 5,000 Hz (p < 0.01), 500 and 9,000 Hz (p < 0.01) and 1,000 and 9,000 Hz (p < 0.01).

FIGURE 5

Figure 5. Results of the “listen and draw” task. Each bar represents the average area in cm² drawn for each frequency of the sound heard. ***Significant difference with p < 0.001. **Significant difference with p < 0.01.

Discussion

Several studies have shown cross-modal associations between sound frequencies and visual features of objects, such as spatial positions, brightness, size and sharpness (Carello et al., 1998; Gallace and Spence, 2006; Parise and Spence, 2009; for a review Spence, 2011; Bien et al., 2012; Jaekl et al., 2012; Takeshima and Gyoba, 2013). Here we showed that a similar association can also be obtained by using cross-modal stimulations: namely, an audio pitch cue can be used to modulate visual size adaptation.

The main hypothesis of this study was to see whether sound frequencies can influence size aftereffect. To test our hypothesis, we used the size adaptation paradigm of Pooresmaeili et al. (2013) to create a size aftereffect, in which, after adaptation, the stimulus test was perceived differently from its physical size. In our study, we presented a sound (low or high frequency pure tone) at the same time as the test stimulus to evaluate whether the effect of the adaptation was modified or not. Pooresmaeili et al. (2013) tested visual adaptation for both small and big size, i.e., in the first case the adapter was smaller than the reference, in the second case the adapter was bigger. Since they found a greater effect for the condition in which the adapter was bigger than the reference stimulus, we decided to focus on this condition in our study.

As we expected, the results confirmed our hypothesis. In the congruent condition, i.e., high pitch sound associated with adaptation to big size, the test stimulus was perceived even smaller compared to the visual-only condition. This might happen because both high pitch sound and adapter have the same effect on the test stimulus, i.e., to be perceived as smaller than the reference. The opposite effect was found for the incongruent condition, where we found a decrease of the aftereffect associated to the low pitch sound, due to opposite effects, i.e., the low pitch sound might increase the perceived size of the visual test, while adaptation to a bigger stimulus makes the test stimulus be perceived smaller, thus showing a reduction of the aftereffect.

Several studies have shown how size perception can be influenced by the pitch of sounds (Mondloch and Maurer, 2004; Gallace and Spence, 2006; Parise and Spence, 2009; Evans and Treisman, 2011; Bien et al., 2012) or loudness (Jaekl et al., 2012; Takeshima and Gyoba, 2013), i.e., low pitch (or loudness) sounds are associated with big objects, while high pitch (or loudness) sounds with small ones. The novelty of our work consists of showing that pitch has an effect not only on size perception but also on the perceived size after visual adaptation. These results contribute to corroborating how human primary visual cortex activity is modulated by sound as a result of auditory-visual integration. It has been shown that non-visual sensory information can modulate visual perception in a different domain at the behavioral level: in the last decade several pieces of evidence have shown how cross-modal stimulation can affect the activity of the visual cortex (Lewis et al., 2000; Arden et al., 2003; Scheef et al., 2009; Murray et al., 2015). For example Molholm et al. (2002) found an ERP response to audiovisual stimuli at parieto-occipital/occipital position. Moreover, there are findings suggesting that the modulation of the visual cortex by sound occurs at the early stages of information processing, directly involving the primary visual cortex (Shams et al., 2000, 2001; Noesselt et al., 2007; Feng et al., 2014). In this study we have deliberately used an experimental paradigm that we know acts directly on V1. Aftereffect, per-se, is a behavioral technique used to investigate adaptability of the neural mechanism tuned (Frisby, 1980), in this specific case, to size perception. Furthermore, Pooresmaeili et al. (2013) suggested a primary, not auxiliary, role of the striate cortex in size adaptation based on a response gain model on local excitation and inhibition within area V1. Moreover, the adaptation paradigm did not use perspective or vergence cues to create size illusion, because the distance and the depth from the participant were kept constant. Therefore, the apparent size should not be attributable to simultaneous spatial contextual effects. Looking to single participants data, it is present a variability in effect produced by the association between adaptation and sounds. We found that there is a positive correlation between the Δ effect of only-visual adaptation and the Δ of auditory condition, meaning that the subjects that are more affected by only-visual adaptation are the ones that are more influenced also in the auditory conditions.

Given the lack of spatial congruency between the test and the auditory stimuli, i.e., the test stimulus is always presented on the left, instead the sound is listen with both ears, the temporal synchronization appear to be the main factor that bounce together these two stimuli is. We wanted to highlight this point because it suggests that temporal coincidence plays a key role to find the effect. To test this hypothesis, we ran a control experiment with a reduced part of the initial sample, where we put a temporal delay between the presentation of the test and sounds stimuli. We found no significant difference between the Δ effect of the auditory (low and high pitch) and the Δ effect of the only-visual conditions, suggesting that the effects of auditory pitch vanish after its presentation. If this was not the case, we would have found a greater effect as in Jaekl et al. (2012), where the authors found an increase of the perceived size of a circle when the sound followed the visual stimulus. So it seems that temporal synchronization is needed when the perception has already been altered by visual adaptation.

Another evidence of the cross-modal association between sound frequencies and size perception, is given by the “listen and draw” task. In this task we left the participants free to draw a circle as big as the sound they heard, in order to make the association as natural and ecological as possible, unlike for example discrimination task, in which the person is forced to choose between two or more stimuli. Taking into account the lowest frequency (100 Hz), it is clear how this frequency is associated with an extremely large circle, generating a gap with the others sounds. On the other hand, the same significant strong effect was not found with the highest frequency (9,000 Hz); although the area drawn was smaller, the gap was not as consistent as for the 100 Hz sound, in comparison with the other frequencies used. A possible explanation of this discrepancy can be given by the curves of Fletcher and Munson (1933). According to this approach, the perceived intensity of a sound can remain unaltered if frequency and sound pressure vary following these curves. For example, 1,000 Hz at 40 dB will be perceived with the same intensity as 100 Hz at 54 dB. It should be highlighted that the steepness of the curve decreases with increasing frequency. In our experiment, we kept sound pressure constant, therefore by varying the sound's frequency, the perceived intensity also changed. Given the Fletcher and Munson curves steepness, the difference in intensity is higher for low frequencies than high frequencies. This might explain why we found a bigger gap between the area drawn associated with 100 Hz compared to the others, where the size of the area decreases steadily as the frequency increases.

In summary, we show at the behavioral level, using a size adaptation paradigm, that sounds of different pitch (high and low frequency) can alter a visual size aftereffect. A high pitch frequency sound increases the size aftereffect while a low pitch frequency sound reduces it. A possible cortical network for this cross-modal modulation could be the visual cortex, which has been shown to be involved in purely visual size adaptation (Sperandio et al., 2012; Pooresmaeili et al., 2013; Murray et al., 2015). A possible speculation is that the same network could be involved in this audio-visual interaction. Further neurophysiological studies will be performed to validate this statement.

Author Contributions

AT and LC: Conceived and designed the project. AT: Performed experiments. AT: Analyzed data. AT, LC, and MG: Wrote and edited the manuscript. All authors gave final approval for publication.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was supported by the European weDraw project (ICT-22-2016—Technologies for Learning and Skills, under Grant Agreement No. 732391). Thanks to Claudio Campus and Marco Jacono for technical support.

References

Aglioti, S., DeSouza, J. F. X., and Goodale, M. A. (1995). Size-contrast illusions deceive the eye but not the hand. Curr. Biol. 5, 679–685. doi: 10.1016/S0960-9822(95)00133-3

CrossRef Full Text | Google Scholar

Arden, G. B., Wolf, J. E., and Messiter, C. (2003). Electrical activity in visual cortex associated with combined auditory and visual stimulation in temporal sequences known to be associated with a visual illusion. Vision Res. 43, 2469–2478. doi: 10.1016/S0042-6989(03)00437-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Arnold, D. H., Birt, A., and Wallis, T. S. A (2008). Perceived size and spatial coding. J. Neurosci. 28, 5954–5958. doi: 10.1523/JNEUROSCI.0578-08.2008

PubMed Abstract | CrossRef Full Text | Google Scholar

Bien, N., ten Oever, S., Goebel, R., and Sack, A. T. (2012). The sound of size. Crossmodal binding in pitch-size synesthesia: a combined TMS, EEG and psychophysics study. Neuroimage 59, 663–672. doi: 10.1016/j.neuroimage.2011.06.095

PubMed Abstract | CrossRef Full Text | Google Scholar

Blakemore, C., and Sutton, P. (1969). Size adaptation: a new aftereffect. Science 166, 245–247. doi: 10.1126/science.166.3902.245

PubMed Abstract | CrossRef Full Text | Google Scholar

Bolognini, N., Frassinetti, F., Serino, A., and Làdavas, E. (2005). “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs. Exp. Brain Res. 160, 273–282. doi: 10.1007/s00221-004-2005-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Boyaci, H., Fang, F., Murray, S. O., and Kersten, D. (2007). Responses to lightness variations in early human visual cortex. Curr. Biol. 17, 989–993. doi: 10.1016/j.cub.2007.05.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Bradshaw, M. F., Glennerster, A., and Rogers, B. J. (1996). The effect of display size on disparity scaling from differential perspective and vergence cues. Vision Res. 36, 1255–1264. doi: 10.1016/0042-6989(95)00190-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Carello, C., Anderson, K. L., and Kunkler-Peck, A. J. (1998). Perception of object length by sound. Psychol. Sci. 9, 211–214. doi: 10.1111/1467-9280.00040

CrossRef Full Text | Google Scholar

Cuturi, L. F., and Macneilage, P. R. (2014). Optic flow induces nonvisual self-motion aftereffects. Curr. Biol. 24, 2817–2821. doi: 10.1016/j.cub.2014.10.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Dufour, A., Touzalin, P., Moessinger, M., Brochard, R., and Després, O. (2008). Visual motion disambiguation by a subliminal sound. Conscious. Cogn. 17, 790–797. doi: 10.1016/j.concog.2007.09.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Evans, K. K., and Treisman, A. (2011). Natural cross-modal mappings between visual and auditory features. J. Vis. 10, 6.1–6.12. doi: 10.1167/10.1.6

PubMed Abstract | CrossRef Full Text | Google Scholar

Feng, W., Stormer, V. S., Martinez, A., McDonald, J. J., and Hillyard, S. A. (2014). Sounds activate visual cortex and improve visual discrimination. J. Neurosci. 34, 9817–9824. doi: 10.1523/JNEUROSCI.4869-13.2014

PubMed Abstract | CrossRef Full Text | Google Scholar

Fletcher, H., and Munson, W. A. (1933). Loudness, its definition, measurement and calculation. Bell Syst. Tech. J. 12, 377–430. doi: 10.1002/j.1538-7305.1933.tb00403.x

CrossRef Full Text | Google Scholar

Frassinetti, F., Bolognini, N., and Làdavas, E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp. Brain Res. 147, 332–343. doi: 10.1007/s00221-002-1262-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Frisby, J. P. (1980). Seeing: Illusion, Mind and Brain. Oxford: Oxford University Press.

Gallace, A., and Spence, C. (2006). Multisensory synesthetic interactions in the speeded classification of visual size. Percept. Psychophys. 68, 1191–1203. doi: 10.3758/BF03193720

PubMed Abstract | CrossRef Full Text | Google Scholar

Haffenden, A. M., Schiff, K. C., and Goodale, M. A. (2001). The dissociation between perception and action in the Ebbinghaus illusion: nonillusory effects of pictorial cues on grasp. Curr. Biol. 11, 177–181. doi: 10.1016/S0960-9822(01)00023-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Hidaka, S., Manaka, Y., Teramoto, W., Sugita, Y., Miyauchi, R., Gyoba, J., et al. (2009). Alternation of sound location induces visual motion perception of a static object. PLoS ONE 4:e8188. doi: 10.1371/journal.pone.0008188

PubMed Abstract | CrossRef Full Text | Google Scholar

Jaekl, P., Soto-Faraco, S., and Harris, L. R. (2012). Perceived size change induced by audiovisual temporal delays. Exp. Brain Res. 216, 457–462. doi: 10.1007/s00221-011-2948-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Kitagawa, N., and Ichihara, S. (2002). Hearing visual motion in depth. Nature 416, 172–174. doi: 10.1038/416172a

PubMed Abstract | CrossRef Full Text | Google Scholar

Konkle, T., and Oliva, A. (2012). A real-world size organization of object responses in occipitotemporal cortex. Neuron 74, 1114–1122. doi: 10.1016/j.neuron.2012.04.036

PubMed Abstract | CrossRef Full Text | Google Scholar

Konkle, T., Wang, Q., Hayward, V., and Moore, C. I. (2009). Motion aftereffects transfer between touch and vision. Curr. Biol. 19, 745–750. doi: 10.1016/j.cub.2009.03.035

PubMed Abstract | CrossRef Full Text | Google Scholar

Leibowitz, H., Brislin, R., Perlmutter, L., and Hennessy, R. (1969). Ponzo perspective illusion as a manifestation of space perception. Science 166, 1174–1176. doi: 10.1126/science.166.3909.1174

PubMed Abstract | CrossRef Full Text | Google Scholar

Lewis, J., Beauchamp, M. S., and DeYoe, E. A. (2000). A comparison of visual and auditory motion processing in human cerebral cortex. Cereb. Cortex 10, 873–888. doi: 10.1093/cercor/10.9.873

PubMed Abstract | CrossRef Full Text | Google Scholar

Lippert, M., Logothetis, N. K., and Kayser, C. (2007). Improvement of visual contrast detection by a simultaneous sound. Brain Res. 1173, 102–119. doi: 10.1016/j.brainres.2007.07.050

PubMed Abstract | CrossRef Full Text | Google Scholar

Mondloch, C. J., and Maurer, D. (2004). Do small white balls squeak? Pitch-object correspondences in young children. Cogn. Affect. Behav. Neurosci. 4, 133–136. doi: 10.3758/CABN.4.2.133

PubMed Abstract | CrossRef Full Text | Google Scholar

Meyer, G. F., and Wuerger, S. M. (2001). Cross-modal integration of auditory and visual motion signals. Neuroreport 12, 2557–2560. doi: 10.1097/00001756-200108080-00053

PubMed Abstract | CrossRef Full Text | Google Scholar

Molholm, S., Ritter, W., Murray, M. M., Javitt, D. C., Schroeder, C. E., and Foxe, J. J. (2002). Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res. Cogn. Brain Res. 14, 115–128. doi: 10.1016/S.0926-6410(02)00066-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Murray, M. M., Thelen, A., Thut, G., Romei, V., Martuzzi, R., and Matusz, P. J. (2015). The multisensory function of the human primary visual cortex. Neuropsychologia 83, 161–169. doi: 10.1016/j.neuropsychologia.2015.08.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Noesselt, T., Rieger, J. W., Schoenfeld, M. A., Kanowski, M., Hinrichs, H., Heinze, H.-J., et al. (2007). Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J. Neurosci. 27, 11431–11441. doi: 10.1523/JNEUROSCI.2252-07.2007

PubMed Abstract | CrossRef Full Text | Google Scholar

Parise, C. V., and Spence, C. (2009). “When birds of a feather flock together”: synesthetic correspondences modulate audiovisual integration in non-synesthetes. PLoS ONE 4:e5664. doi: 10.1371/journal.pone.0005664

CrossRef Full Text | Google Scholar

Perna, A., Tosetti, M., Montanaro, D., and Morrone, M. C. (2005). Neuronal mechanisms for illusory brightness perception in humans. Neuron 47, 645–651. doi: 10.1016/j.neuron.2005.07.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Pooresmaeili, A., Arrighi, R., Biagi, L., and Morrone, M. C. (2013). Blood oxygen level-dependent activation of the primary visual cortex predicts size adaptation illusion. J. Neurosci. 33, 15999–16008. doi: 10.1523/JNEUROSCI.1770-13.2013

PubMed Abstract | CrossRef Full Text | Google Scholar

Scheef, L., Boecker, H., Daamen, M., Fehse, U., Landsberg, M. W., Granath, D. O., et al. (2009). Multimodal motion processing in area V5/MT: evidence from an artificial class of audio-visual events. Brain Res. 1252, 94–104. doi: 10.1016/j.brainres.2008.10.067

PubMed Abstract | CrossRef Full Text | Google Scholar

Sekuler, R., Sekuler, A. B., and Lau, R. (1997). Sound alters visual motion perception. Nature 385:308. doi: 10.1038/385308a0

PubMed Abstract | CrossRef Full Text | Google Scholar

Shams, L., and Kim, R. (2010). Crossmodal influences on visual perception. Phys. Life Rev. 7, 269–284. doi: 10.1016/j.plrev.2010.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Shams, L., Kamitani, Y., and Shimojo, S. (2000). What you see is what you hear. Nature 408:788. doi: 10.1038/35048669

PubMed Abstract | CrossRef Full Text | Google Scholar

Shams, L., Kamitani, Y., Thompson, S., and Shimojo, S. (2001). Sound alters visual evoked potentials in humans. Neuroreport 12, 3849–3852. doi: 10.1097/00001756-200112040-00049

PubMed Abstract | CrossRef Full Text | Google Scholar

Spence, C. (2011). Crossmodal correspondences: a tutorial review. Atten. Percept. Psychophys. 73, 971–995. doi: 10.3758/s13414-010-0073-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Sperandio, I., Chouinard, P. A., and Goodale, M. A. (2012). Retinotopic activity in V1 reflects the perceived and not the retinal size of an afterimage. Nat. Neurosci. 15, 540–542. doi: 10.1038/nn.3069

CrossRef Full Text | Google Scholar

Takeshima, Y., and Gyoba, J. (2013). High-intensity sound increases the size of visually perceived objects. Atten. Percept. Psychophys. 75, 501–507. doi: 10.3758/s13414-012-0403-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Tanaka, Y., Nogai, T., Munetsuna, S., and Bonneh, Y. (2005). Improvement of visual contrast sensitivity by auditory cueing signals. Percept. ECVP Abstr. 34.

Teramoto, W., Manaka, Y., Hidaka, S., Sugita, Y., Miyauchi, R., Sakamoto, S., et al. (2010). Visual motion perception induced by sounds in vertical plane. Neurosci. Lett. 479, 221–225. doi: 10.1016/j.neulet.2010.05.065

PubMed Abstract | CrossRef Full Text | Google Scholar

Thompson, P., and Burr, D. (2009). Visual aftereffects. Curr. Biol. 19, R11–R14. doi: 10.1016/j.cub.2008.10.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Watson, A. B., and Pelli, D. G. (1983). QUEST: a bayesian adaptive psychometric method. Attent. Percept. Psychophys. 33, 113–120.

PubMed Abstract | Google Scholar