Rafael Bretas
Banty Tia
Atsushi Iriki- Laboratory for Symbolic Cognitive Development, RIKEN Center for Biosystems Dynamics Research, Kobe, Japan
Introduction
Our first experience of the world originates from the information we receive through the senses, allowing us to make mental representations of the features that can be experienced from each part of the environment—be those objects, events, places, or beings. However, these parts are not perceived separately through each sense. Rather, sight, touch, smell, hearing, and taste are integrated early in life in multimodal areas in the brain (Lewis and Essen, 2000). While this process, together with memory, supports the formation of beliefs of increasing complexity, it is also constantly modified by those same beliefs. In this opinion paper we briefly describe some of the neural underpinnings of conscious perception and illustrate how a complex belief is formed from sensory information using the example of mirror self-recognition in macaques.
From when a sense is raised to awareness until when it is integrated into other senses, a separate process occurs. A relational association is established, one in which the codependency of these stimuli becomes their own defining characteristic. That is, an object or event is recognized by simultaneously eliciting different modalities of sensation (Crick and Koch, 1990; Deroy et al., 2016). It is important to clarify that multisensory integration does not necessarily induce a conscious process. However, unconscious integration seems to be possible in only limited conditions, such as in simple forms of visual adaptation or when a stimuli pair has been previously learned (Faivre et al., 2014; Mudrik et al., 2014).
In this sense, integration can be understood as an antecedent to a behavior, perhaps similarly to how attitudes or mindsets are modulated (Seitz and Angel, 2012) or as an empirical belief (Seitz and Angel, 2020). Nevertheless, becoming aware of a percept as an amalgam of sensations forms the basis for a conscious belief that can be expressed as a decision or action, or in declarative form as a conceptual belief (Seitz and Angel, 2020). From the point in which a sense is raised to awareness, recognition may take place. Although often understood as a single behavioral phenomenon, “recognition” arises from separate neurophysiological processes that can function independently of each other. Here, we will focus on two of these general processes that are often taken as determinators of recognition.
First, there is a memory component that locates the sensory information in the place and context where sensation occurs (Mandler, 1980), supported by connections between sensory areas in the neocortex, perirhinal cortex, and parahippocampal regions (Brown and Aggleton, 2001; Eichenbaum et al., 2007). This system, or systems, as it could be subdivided in two main components, is responsible for retrieval of contextual information and the recollection of the stimulus (Brown and Aggleton, 2001). Autobiographical memory also centers sensory experiences around oneself as the individual agent of sensation, perhaps mediated by connections between the posterior cingulate and medial parietal cortex (Rolls, 2022). The hippocampal and parahippocampal regions show extensive connections to sensory and motor areas, but despite playing a fundamental role in recollecting and situating the sensory information received in time and space, recognition itself seems to be formed independently from these regions. Moreover, lesion experiments confirm that memory is not necessary for simple object recognition (DeCoteau and Kesner, 1998; Burwell, 2000; Save and Poucet, 2000; Langston and Wood, 2010), instead it may represent the emotional contents and semantic information rather than the physical properties that allow the conscious perception of an object (Rolls, 2022; Rolls et al., 2022).
The second function that supports recognition is multisensory integration. The perception of simple physical features, such as shape, color, or texture, can be accomplished by unimodal tactile and visual processing streams without reaching awareness. Unconscious perception is also common in multimodal areas for the purposes of guiding motor control (Milner and Goodale, 2008; Mudrik et al., 2014). However, recognition (i.e., the conscious perceptual experience that allows the identification of an object or scene) recruits large, distributed networks that integrate different senses (Dijkerman and de Haan, 2007; Winters and Reid, 2010). The parietal cortex appears to be the source of this conscious perception process, being well interconnected with prefrontal, cingulate and primary sensory areas (Lewis and Essen, 2000; Vincent et al., 2006; Whitlock et al., 2008; Rolls et al., 2022). Furthermore, the parietal cortex, together with the prefrontal cortex, directs attention and modulates perception and the emotional significance of sensory events (Mesulam, 1998; Steinmetz et al., 2000; Culham and Kanwisher, 2001; Galletti et al., 2010). In humans, the parietal cortex is also functionally interconnected with language and declarative memory areas, a hallmark of conscious perception (Rolls et al., 2022).
This wide network that combines perception and memory to contextualize what is perceived may form a part of the broader consciousness, allowing one to recognize the world and the self (Dehaene and Changeux, 2011; Mudrik et al., 2014). The large-scale synchronization between sensory, motor and executive functions required for conscious perception could be mediated by the claustrum, as an area with reciprocal connections to most regions of the cortex, perhaps as a conductor of sensory experiences (Crick and Koch, 2005), with this combination process being permeated by different degrees of beliefs (Seitz and Angel, 2012).
Here, we would like to focus on the formation and modulation of some simple beliefs related to the self and the world. For example, when there is a mismatch between the senses, such as in the ventriloquist effect, localizing the source of a sound together with a movement (mouth movement, in this example) uses a combination of both auditory and visual senses, but viewers' sensations are distorted by beliefs of how reality should be according to previous experiences (Alais and Burr, 2004; Seitz and Angel, 2020). The viewer knows that the sound comes from the ventriloquist, not the puppet, but at the same time they also have experienced a reality in which sounds usually come from moving parts, and in particular, voices come from moving mouths.
This type of sensory conflict is clearer in the rubber hand illusion. Even when there is a stable, conscious belief that the hand seen is not one's own, a false belief associated with a recent sensation may override the belief. The effect is strong enough to cause the activation of somatosensory areas in the brain when the false hand is stroked (Ehrsson et al., 2005). Such cases could be an example of how a conscious declarative belief can be distorted by a sensory belief. In both cases the causes could maybe be reduced to the single, deeply rooted, unconscious belief intrinsic to the senses involved that sight has higher spatial accuracy than sound or touch. Therefore, sight should be more reliable when incongruent spatial judgements are involved.
There are also cases where sensation is not inconsistent but instead ambiguous, such as in multistable perception (i.e., when two concurrent percepts spontaneously change). In these situations, prior beliefs may act in harmony with newly acquired sensory information to guide attention and modulate perception (Sterzer et al., 2009; Conrad et al., 2012), with the speed and often mutually exclusiveness of these changes being noteworthy.
The self in the mirror
Self-recognition in the mirror was proposed by Gallup in 1977 as “a technique for providing empirical and operational substance to the existence of self-awareness” (Gallup, 1977). Despite debates over how much self-awareness intersects with mirror self-recognition, the mirror offers the perfect example of the development of different categories of belief and how they interfere with each other according to the criteria of Seitz and Angel (2020). Since humans appear to develop or be guided into mirror self-recognition very early in life, it may be difficult to imagine its emergence. Macaques however, although not having innate self-recognition, can acquire it through training and habituation (Chang et al., 2017; Bretas et al., 2021). Therefore, below we follow the trajectory of a hypothetical macaque acquiring such a skill based on our own experimental observations (Bretas et al., 2021).
At the lowest level, there is the visual stimulus imparted by the mirror as the sensory reality (i.e., raw information that contains the visual features of the subject in the mirror). Innate visual mechanisms combined with the memory mechanisms described before allow for the classification of this stimulus (The image in the mirror moves. It's a live being. It is a monkey.) and its valuation in terms of emotional loading (Is it a threat? Is it a partner?). This process can start before the stimulus is raised to conscious awareness and is an example of empirical beliefs (Figure 1).
Figure 1. When a macaque looks into a mirror, perception of the raw visual information from the mirror forms a first approximation of reality in the form of empirical beliefs (Seitz and Angel, 2020). This visual stimulus is integrated with other sensory modalities, such as the proprioception of the macaque's own limbs moving, and raised to conscious awareness. Recognition of the mirror image then takes place - initially as another macaque, not the self. Simultaneously, the novelty, emotional significance and other aspects associated with the stimuli are recollected from memory—previous experiences with other macaques, the novelty of the individual seen in the mirror, innate fear, etc. The associations between what is perceived and the environment form relational beliefs (Seitz and Angel, 2020). These beliefs are being updated constantly according to new sensory information received from the environment, which also feedbacks into perception through attention mechanisms. Finally, a complex belief that matches the perceived stimuli may evolve, the belief that the macaque in the mirror is a reflection of the self. This belief is expressed in the form of mirror self-recognition behaviors (Chang et al., 2017; Bretas et al., 2021), but in humans it could further develop into conceptual beliefs, discrete, language bound concepts (Seitz and Angel, 2020) (e.g.: “I am the person in the mirror and I appear to others are they appear to me”).
Before self-awareness, one must develop other-awareness, since both processes require the capacity of secondary representation (Asendorpf and Baudonnière, 1993). Both other beings and the self-body are directly accessible through the senses, but even accounting for physical similarities, there is a significant difference in perspective. At the most basic level, the spatial organization of the body will appear to be different, with one not being able to see their own face or back, for example. There is also a difference in the integration of the senses: visually the self and the other can be equally perceived, but touch is limited to the self. For example, while two people can see each other touching an object, the touch sensation can only be felt by the agent. In the same way that self-body awareness arises from the association between different unimodal sensations, such as vision and touch (Apps and Tsakiris, 2014), mirror-self recognition relies on these same sensory mechanisms as precursors.
When the macaque looks into the mirror, a third-person view of the self is shown. This view elicits new beliefs, both conflicting and ambiguous, as described in the previous section. While the body in the mirror appears from the same perspective as that of others, its movements can be precisely controlled, which is a sense of agency over a distant subject. The same could be said about the touch sensation, which now can be felt by both the agent and the mirror-image. The subject now may recollect that the primate in the mirror is always the same subject with the same face. It also does not feel like the primate's previous experiences or what is expected of the related visual stimuli. The mirror shows soft fur, but it is cold and hard to the touch; the primate in the mirror cannot be touched. These new sensorial experiences accumulate and generate relational beliefs about the relations with the environment. According to Sugiura et al. (2015), mental representations are formed through the association between an action and its consequential perception learned through repeated experience.
Multimodal visuo-somatosensory neurons are often spatially tunned to represent the space around the subject from an egocentric point of view, mapping the position of the own body and reachable objects nearby. These neurons estimate and guide limb movement as well as tracking objects moving toward or nearby the subject (Taoka et al., 2013; Hihara et al., 2015; Galletti et al., 2022). But when movement is performed in front of the mirror, the mismatch between the prediction and the actual sensation may no longer appropriately represent the actual bodily or environmental state (Sugiura et al., 2015). Perception may then be updated by this new incoming sensory data and combined with past outputs and decisions to account for this new state, in accordance with the free-energy principle (Apps and Tsakiris, 2014); relational beliefs can alter empirical beliefs. This update to sensation may lead to the mirror being ignored from that point on as a useless social cue, with no new attempts to interact with this primate in the mirror and no emotional reactions of fear or dominance upon seeing it. However, discrepancy from previous beliefs in this new information received from unimodal areas could be explained away by multisensory integration, giving rise to self-recognition (Apps and Tsakiris, 2014; Chang et al., 2017; Bretas et al., 2021). Indeed, binding different sensory aspects of and object in a mutually coherent way provides the experience of perceptual unity necessary to group the individual body parts in a concept of an indivisible self-body (Crick and Koch, 1990; Bretas et al., 2021).
The acknowledgment of the self in third-person promotes the belief that the other is like the self, with empathy and the emotional valence of the new beliefs further shaping both mirror-perception and own-perception (Gallup, 1998; Sugiura et al., 2015; Bretas et al., 2020). Primates are special in that their brains evolved with the addition of new functional subdivisions to the neocortex (Dooley and Krubitzer, 2013). Areas in the parietal cortex related to self-awareness and social-awareness may be essential to the development of mental models of introspectively based social strategies and language, forming the basis for conceptual beliefs and culture (Gallup, 1998; Sugiura et al., 2015; Bretas et al., 2020, 2021; Seitz and Angel, 2020; Seitz, 2022). Conceptual beliefs, thereupon, give support to meta-beliefs, elevating relational beliefs about the other to language-bound discrete concepts (e.g., “the other believes…”) to achieve a comprehensive notion of the world grounded in internal representations of the physical, social, and cultural environments (Angel and Seitz, 2016; Bretas et al., 2020; Seitz and Angel, 2020).
Author contributions
RB and AI contributed to conception and theory of the study. RB wrote the first draft of the manuscript. BT, YY, and AI wrote sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.
Funding
The authors declare that this study received funding from Dr. Rüdiger Seitz, via the Volkswagen Foundation, Siemens Healthineers, and the Betz Foundation. The funders were not involved in the study design, collection, analysis, interpretation of data, the writing of this article, or the decision to submit it for publication.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Alais, D., and Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257–262. doi: 10.1016/j.cub.2004.01.029
Angel, H.-F., and Seitz, R. J. (2016). Process of believing as fundamental brain function: the concept of credition. SFU Res. Bull. Nr. 1, 1–20. doi: 10.15135/2016.4.1.1-20
Apps, M. A. J., and Tsakiris, M. (2014). The free-energy self: a predictive coding account of self-recognition. Neurosci. Biobehav. Rev. 41, 85–97. doi: 10.1016/j.neubiorev.2013.01.029
Asendorpf, J. B., and Baudonnière, P.-M. (1993). Self-awareness and other-awareness: mirror self-recognition and synchronic imitation among unfamiliar peers. Dev. Psychol. 29, 88–95. doi: 10.1037/0012-1649.29.1.88
Bretas, R., Taoka, M., Hihara, S., Cleeremans, A., and Iriki, A. (2021). Neural evidence of mirror self-recognition in the secondary somatosensory cortex of macaque: observations from a single-cell recording experiment and implications for consciousness. Brain Sci. 11, 157. doi: 10.3390/brainsci11020157
Bretas, R., Taoka, M., Suzuki, H., and Iriki, A. (2020). Secondary somatosensory cortex of primates: Beyond body maps, toward conscious self-in-the-world maps. Exp. Brain Res. 238, 259–272. doi: 10.1007/s00221-020-05727-9
Brown, M. W., and Aggleton, J. P. (2001). Recognition memory: what are the roles of the perirhinal cortex and hippocampus? Nat. Rev. Neurosci. 2, 51–61. doi: 10.1038/35049064
Burwell, R. D. (2000). The parahippocampal region: corticocortical connectivity. Ann. N. Y. Acad. Sci. 911, 25–42. doi: 10.1111/j.1749-6632.2000.tb06717.x
Chang, L., Zhang, S., Poo, M., and Gong, N. (2017). Spontaneous expression of mirror self-recognition in monkeys after learning precise visual-proprioceptive association for mirror images. Proc. Natl. Acad. Sci. 114, 3258–3263. doi: 10.1073/pnas.1620764114
Conrad, V., Vitello, M. P., and Noppeney, U. (2012). Interactions between apparent motion rivalry in vision and touch. Psychol. Sci. 23, 940–948. doi: 10.1177/0956797612438735
Crick, F., and Koch, C. (1990). Towards a neurobiological theory of consciousness. Semin. Neurosci. 2, 263–275.
Crick, F. C., and Koch, C. (2005). What is the function of the claustrum? Philos. Trans. R. Soc. B Biol. Sci. 360, 1271–1279. doi: 10.1098/rstb.2005.1661
Culham, J. C., and Kanwisher, N. G. (2001). Neuroimaging of cognitive functions in human parietal cortex. Curr. Opin. Neurobiol. 11, 157–163. doi: 10.1016/S0959-4388(00)00191-4
DeCoteau, W., and Kesner, R. (1998). Effects of hippocampal and parietal cortex lesions on the processing of multiple-object scenes. Behav. Neurosci. 112, 68–82. doi: 10.1037/0735-7044.112.1.68
Dehaene, S., and Changeux, J.-P. (2011). Experimental and theoretical approaches to conscious processing. Neuron 70, 200–227. doi: 10.1016/j.neuron.2011.03.018
Deroy, O., Faivre, N., Lunghi, C., Spence, C., Aller, M., and Noppeney, U. (2016). The complex interplay between multisensory integration and perceptual awareness. Multisen. Res. 29, 585–606. doi: 10.1163/22134808-00002529
Dijkerman, H. C., and de Haan, E. H. F. (2007). Somatosensory processes subserving perception and action. Behav. Brain Sci. 30, 189–201. doi: 10.1017/S0140525X07001392
Dooley, J., and Krubitzer, L. (2013). Cortical plasticity within and across lifetimes: How can development inform us about phenotypic transformations? Front. Hum. Neurosci. 7, 620. doi: 10.3389/fnhum.2013.00620
Ehrsson, H. H., Holmes, N. P., and Passingham, R. E. (2005). Touching a rubber hand: Feeling of body ownership is associated with activity in multisensory brain areas. J. Neurosci. 25, 10564–10573. doi: 10.1523/JNEUROSCI.0800-05.2005
Eichenbaum, H., Yonelinas, A. R., and Ranganath, C. (2007). The Medial Temporal Lobe and Recognition Memory. Ann. Rev. Neurosci. 30, 123–152. doi: 10.1146/annurev.neuro.30.051606.094328
Faivre, N., Mudrik, L., Schwartz, N., and Koch, C. (2014). Multisensory integration in complete unawareness: evidence from audiovisual congruency priming. Psychol. Sci. 25, 2006–2016. doi: 10.1177/0956797614547916
Galletti, C., Breveglieri, R., Lappe, M., Bosco, A., Ciavarro, M., and Fattori, P. (2010). Covert shift of attention modulates the ongoing neural activity in a reaching area of the macaque dorsomedial visual stream. PLoS ONE 5, e15078. doi: 10.1371/journal.pone.0015078
Galletti, C., Gamberini, M., and Fattori, P. (2022). The posterior parietal area V6A: An attentionally-modulated visuomotor region involved in the control of reach-to-grasp action. Neurosci. Biobehav. Rev. 141, 104823. doi: 10.1016/j.neubiorev.2022.104823
Gallup, G. G. (1977). Self recognition in primates: A comparative approach to the bidirectional properties of consciousness. Am. Psychol. 32, 329–338. doi: 10.1037/0003-066X.32.5.329
Gallup, G. G. (1998). Self-awareness and the evolution of social intelligence. Behav. Processes 42, 239–247. doi: 10.1016/S0376-6357(97)00079-X
Hihara, S., Taoka, M., Tanaka, M., and Iriki, A. (2015). Visual responsiveness of neurons in the secondary somatosensory area and its surrounding parietal operculum regions in awake macaque monkeys. Cereb. Cortex 25, 4535–4550. doi: 10.1093/cercor/bhv095
Langston, R. F., and Wood, E. R. (2010). Associative recognition and the hippocampus: Differential effects of hippocampal lesions on object-place, object-context and object-place-context memory. Hippocampus 20, 1139–1153. doi: 10.1002/hipo.20714
Lewis, J. W., and Essen, D. C. V. (2000). Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey. J. Comp. Neurol. 428, 112–137. doi: 10.1002/1096-9861(20001204)428:1<112::aid-cne8>3.0.co;2-9
Mandler, G. (1980). Recognizing: the judgment of previous occurrence. Psychol. Rev. 87, 252–271. doi: 10.1037/0033-295X.87.3.252
Mesulam, M. (1998). From sensation to cognition. Brain 121, 1013–1052. doi: 10.1093/brain/121.6.1013
Milner, A. D., and Goodale, M. A. (2008). Two visual systems re-viewed. Neuropsychologia 46, 774–785. doi: 10.1016/j.neuropsychologia.2007.10.005
Mudrik, L., Faivre, N., and Koch, C. (2014). Information integration without awareness. Trends Cogn. Sci. 18, 488–496. doi: 10.1016/j.tics.2014.04.009
Rolls, E. T. (2022). The hippocampus, ventromedial prefrontal cortex, and episodic and semantic memory. Prog. Neurobiol. 217, 102334. doi: 10.1016/j.pneurobio.2022.102334
Rolls, E. T., Deco, G., Huang, C.-C., and Feng, J. (2022). The human posterior parietal cortex: Effective connectome, and its relation to function. Cereb. Cortex bhac266. doi: 10.1093/cercor/bhac266
Save, E., and Poucet, B. (2000). Involvement of the hippocampus and associative parietal cortex in the use of proximal and distal landmarks for navigation. Behav. Brain Res. 109, 195–206. doi: 10.1016/S0166-4328(99)00173-4
Seitz, R. J. (2022). Believing and beliefs—neurophysiological underpinnings. Front. Behav. Neurosci. 16, 880504. doi: 10.3389/fnbeh.2022.880504
Seitz, R. J., and Angel, H.-F. (2012). Processes of believing—a review and conceptual account. Rev. Neurosci. 23, 303–309. doi: 10.1515/revneuro-2012-0034
Seitz, R. J., and Angel, H.-F. (2020). Belief formation – a driving force for brain evolution. Brain Cogn. 140, 105548. doi: 10.1016/j.bandc.2020.105548
Steinmetz, P. N., Roy, A., Fitzgerald, P. J., Hsiao, S. S., Johnson, K. O., and Niebur, E. (2000). Attention modulates synchronized neuronal firing in primate somatosensory cortex. Nature 404, 187. doi: 10.1038/35004588
Sterzer, P., Kleinschmidt, A., and Rees, G. (2009). The neural bases of multistable perception. Trends Cogn. Sci. 13, 310–318. doi: 10.1016/j.tics.2009.04.006
Sugiura, M., Seitz, R. J., and Angel, H.-F. (2015). Models and neural bases of the believing process. J. Behav. Brain Sci. 5, 12. doi: 10.4236/jbbs.2015.51002
Taoka, M., Tanaka, M., Hihara, S., Ojima, H., and Iriki, A. (2013). Neural response to movement of the hand and mouth in the secondary somatosensory cortex of Japanese monkeys during a simple feeding task. Somatosens. Motor Res. 30, 140–152. doi: 10.3109/08990220.2013.779246
Vincent, J. L., Snyder, A. Z., Fox, M. D., Shannon, B. J., Andrews, J. R., Raichle, M. E., et al. (2006). Coherent spontaneous activity identifies a hippocampal-parietal memory network. J. Neurophysiol. 96, 3517–3531. doi: 10.1152/jn.00048.2006
Whitlock, J. R., Sutherland, R. J., Witter, M. P., Moser, M.-B., and Moser, E. I. (2008). Navigating from hippocampus to parietal cortex. Proc. Natl. Acad. Sci. 105, 14755–14762. doi: 10.1073/pnas.0804216105
Keywords: somatosensory integration, mirror self recognition, consciousness, credition, beliefs, parietal cortex, self recognition
Citation: Bretas R, Tia B, Yamazaki Y and Iriki A (2022) Multisensory integration and belief in the self. Front. Psychol. 13:983592. doi: 10.3389/fpsyg.2022.983592
Received: 04 July 2022; Accepted: 30 September 2022;
Published: 13 October 2022.
Edited by:
Rüdiger J. Seitz, Heinrich Heine University of Düsseldorf, GermanyReviewed by:
Julia U. Henschke, University Hospital Magdeburg, GermanyPatrizia Fattori, University of Bologna, Italy
Copyright © 2022 Bretas, Tia, Yamazaki and Iriki. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Rafael Bretas, rafael.bretasvieira@riken.jp; Atsushi Iriki, atsushi.iriki@riken.jp