In recent years there has been a dramatic progress in understanding how stimuli from different sensory modalities are integrated among each other. Multisensory integration results in a unitary representation of the world that strongly characterizes perception and cognition in humans. The body of knowledge acquired so far on multisensory integration has been gained through several research techniques and approaches, including neurophysiology, experimental psychology, neuropsychology, neuroimaging, and computational modeling. This special issue aims at presenting an up-to-date overview of the research on multisensory integration. In particular, the proposed collection of papers features state-of-the-art reviews or original articles about key themes in multisensory research, considering novel evidence on the physiological mechanism of multisensory integration at cell level, and on the behavioral effects of multisensory integration on perception and cognition in humans.
The issue starts with novel findings about the functional properties of multisensory integration in multisensory neurons of the superior colliculus (SC). Perrault et al. (2011) show how the state of the multisensory cells in SC and their integrative principles are modulated as a consequence of the exposure to a series of testing stimuli during an experiment, following the well-known “inverse effectiveness principle.” The evidence provided has both empirical and practical implications: first, it shows that multisensory integration in the SC is highly plastic and multisensory neurons can adapt to rapidly changing environmental events; second, the neuron studied at the beginning of an experiment is not the same at the end of it, and this should represent an important caveat in the interpretation of experimental data.
The importance of biologically inspired computational modeling of multisensory integration is well highlighted by Cuppini et al. (2011) who describe two neural network models. The first one is aimed at reproducing and explaining the main physiological features of multisensory integration in SC’s neurons, and its development during postnatal life depending on sensory experience. The second model tackles the problem of how tactile stimuli on a body part and visual (or auditory) stimuli close to the same body part are integrated in multimodal parietal neurons to form the perception of near peripersonal space. The proposed models are helpful for the interpretation and the integration of behavioral and physiological evidence, and they offer a novel perspective for guiding future experiments.
The issue of the effects of visual–auditory interactions on human perception is the topic of four behavioral experiments, and a review article. Romei et al. (2011) nicely demonstrate that visual duration discrimination can be enhanced by temporally congruent auditory stimuli, through changes in visual sensitivity. Alink et al. (2012) show that moving sounds can capture the direction of visual stimuli, especially when visual motion direction is ambiguous. Arrighi et al. (2011) focus on the effect of the attentional resources on crossmodal perceptual abilities, and show that vision and audition rely on separate attentional resources. Finally, Chen et al. (2011) provide novel evidence for the effect of the auditory semantic context on visual awareness by using the binocular rivalry paradigm. All together, these research findings provide original support to the view that multisensory (audio–visual) experience can impact unisensory processing. This proposal is deeply discussed by Shams et al. (2011). The authors review three different types of learning, namely perceptual learning, sensory recalibration, and associative learning. In all these cases, exposure to multisensory stimuli profoundly influences the subsequent unisensory processing, suggesting that the modification of unisensory representations by multisensory relationships may be a general learning strategy employed by the brain.
Visual–tactile interactions are investigated by two studies, and discussed in a review article. First, Gori et al. (2011) explore the mechanisms mediating crossmodal facilitation and summation between visual and tactile motion. The authors find that while summation produced a generic, non-specific improvement of thresholds, probably reflecting higher-order interaction of decision signals, facilitation reveals a strong, direction-specific interaction, likely reflecting low-level sensory interactions. Mancini et al. (2011) adopt a neuropsychological approach to investigate the integration between vision and haptics by adopting an original bisection task of a crossmodal illusion, the Judd variant of the Müller-Lyer illusion. The main finding is that, although a right-hemisphere damage my induce modality-specific deficits of spatial attention and representation, the ability of properly integrate visual and tactile sensory inputs is spared in right-brain-damaged patients. This evidence is in line with another type of crossmodal phenomenon that is maintained in right-brain-damaged patients, namely crossmodal extinction. Jacobs et al. (2011) provide a critical review of the research on crossmodal extinction, demonstrating how the study of this phenomenon has critically contributed to increase our understanding of how the integration of stimuli perceived in multiple sensory modalities is used by the human brain to build coherent representations of the space that directly surrounds us.
Moving to complex cognitive implications of sensory processing, Forgiarini et al. (2011) show the existence of a racial bias in the autonomic reaction to other people’s pain and its link with implicit racial biases as assessed through the well-known implicit association test. The authors show and discuss the role of others’ race in moderating empathic reactions, which represents a crucial clue for understanding to what extent social interactions may be influenced by deeply rooted automatic and uncontrollable processes.
The multisensory and sensorimotor basis of bodily self-consciousness is the topic of the review by Ionta et al. (2011), which focus on the multisensory role of the temporo-parietal junction in bodily self-consciousness, as highlighted by the study of neurological patients suffering from out-of-body experiences, and of healthy subjects undergoing experimentally induced multisensory conflicts.
This special issue ends with a review by Bolognini and Maravita (2011) that critically discuss why techniques of non-invasive brain stimulation can represent a unique and powerful approach to inform models of causal relations between specific brain regions and multisensory functions.
Overall, the challenge of this special issue is to provide an interdisciplinary context allowing to understand the basic principles of multisensory integration in animals and humans and the key issues that this fascinating field of study rises for future research. To conclude, we would like to deeply thank the many scientists who contributed so nicely to this special issue.
References
Alink, A., Euler, F., Galeano, E., Krugliak, A., Singer, W., and Kohler, A. (2012). Auditory motion capturing ambiguous visual motion. Front. Psychol. 2:391. doi: 10.3389/fpsyg.2011.00391
Arrighi, R., Lunardi, R., and Burr, D. (2011). Vision and audition do not share attentional resources in sustained tasks. Front. Psychol. 2:56. doi: 10.3389/fpsyg.2011.00056
Bolognini, N., and Maravita, A. (2011). Uncovering multisensory processing through non-invasive brain stimulation. Front. Psychol. 2:46. doi: 10.3389/fpsyg.2011.00046
Chen, Y., Yeh, S., and Spence, C. (2011). Crossmodal constraints on human perceptual awareness: auditory semantic modulation of binocular rivalry. Front. Psychol. 2:212. doi: 10.3389/fpsyg.2011.00212
Cuppini, C., Magosso, E., and Ursino, M. (2011). Organization, maturation, and plasticity of multisensory integration: insights from computational modeling studies. Front. Psychol. 2:77. doi: 10.3389/fpsyg.2011.00077
Forgiarini, M., Gallucci, M., and Maravita, A. (2011). Racism and the empathy for pain on our skin. Front. Psychol. 2:108. doi: 10.3389/fpsyg.2011.00108
Gori, M., Mazzilli, G., Sandini, G., and Burr, D. (2011). Cross-sensory facilitation reveals neural interactions between visual and tactile motion in humans. Front. Psychol. 2:55. doi: 10.3389/fpsyg.2011.00055
Ionta, S., Gassert, R., and Blanke, O. (2011). Multi-sensory and sensorimotor foundation of bodily self-consciousness–an interdisciplinary approach. Front. Psychol. 2:383. doi: 10.3389/fpsyg.2011.00383
Jacobs, S., Brozzoli, C., Hadj-Bouziane, F., Meunier, M., and Farnè, A. (2011). Studying multisensory processing and its role in the representation of space through pathological and physiological crossmodal extinction. Front. Psychol. 2:89. doi: 10.3389/fpsyg.2011.00089
Mancini, F., Bricolo, E., Mattioli, F. C., and Vallar, G. (2011). Visuo-haptic interactions in unilateral spatial neglect: the cross modal Judd illusion. Front. Psychol. 2:341. doi: 10.3389/fpsyg.2011.00341
Perrault, T. J. Jr., Stein, B. E., and Rowland, B. A. (2011). Non-stationarity in multisensory neurons in the superior colliculus. Front. Psychol. 2:144. doi: 10.3389/fpsyg.2011.00144
Romei, V., DeHaas, B., Mok, R. M., and Driver, J. (2011). Auditory stimulus timing influences perceived duration of co-occurring visual stimuli. Front. Psychol. 2:215. doi: 10.3389/fpsyg.2011.00215
Citation: Bolognini N and Maravita A (2012) Interactions between senses: updating on neural mechanisms and behavioral evidence. Front. Psychology 3:122. doi: 10.3389/fpsyg.2012.00122
Received: 22 March 2012; Accepted: 07 April 2012;
Published online: 03 May 2012.
Copyright: © 2012 Bolognini and Maravita. This is an open-access article distributed under the terms of the Creative Commons Attribution Non Commercial License, which permits non-commercial use, distribution, and reproduction in other forums, provided the original authors and source are credited.
*Correspondence: nadia.bolognini@unimib.it