Linda Drijvers1,2,3*†
James P. Trujillo1,2†- 1Centre for Language Studies, Radboud University, Nijmegen, Netherlands
- 2Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen, Netherlands
- 3Max Planck Institute for Psycholinguistics, International Max Planck Research School for Language Sciences, Nijmegen, Netherlands
A commentary on
Transcranial Magnetic Stimulation over Left Inferior Frontal and Posterior Temporal Cortex Disrupts Gesture-Speech Integration
by Zhao, W., Riggs, K., Schindler, I., and Holle, H. (2018). J. Neurosci. 10, 1748–1717. doi: 10.1523/JNEUROSCI.1748-17.2017
Listeners integrate speech with gestures in face-to-face communication. This semantic integration of auditory and visual information is thought to occur automatically (Kelly et al., 2010). Previous neuroimaging studies have found inconsistent results on the involvement and role of the left inferior frontal gyrus (LIFG, e.g., Willems et al., 2007, 2009) and posterior middle temporal gyrus (pMTG, e.g., Holle et al., 2008, 2010; Straube et al., 2011) in this process. For example, Willems et al. (2009) suggested that pMTG is mostly involved in the integration of pantomimes (re-enactments of actions that can be understood without speech) and speech, whereas LIFG is more involved in the integration of iconic gestures (gestures that are ambiguous without speech) and speech. Extending on this, Holle et al. (2010) suggested that pSTS/MTG is involved in the integration of iconic gestures and speech to perform an initial conceptual matching between the inputs, whereas LIFG might be involved in processes of modulation and revision. Others have argued that LIFG/pMTG are both involved in combining the semantic information from both inputs (Dick et al., 2014)
A recent study in The Journal of Neuroscience by Zhao et al. (2018) investigated whether LIFG/pMTG are causally involved in speech-gesture integration by perturbing activity in these regions using continuous theta burst stimulation (cTBS) (Experiment 1) and repetitive transcranial magnetic stimulation (rTMS) (Experiment 2) relative to a control region. Participants were presented with videos of an actor/actress uttering action verbs while producing a gesture. In these videos, gender congruency and semantic congruency were manipulated. Participants were asked to identify the gender of the speaker after every video. In Experiment 1, Zhao et al. (2018) used cTBS to investigate whether disrupting activity in LIFG/pMTG would lead to a reduction of the semantic congruency effect. A disruption of brain activity was expected to lead to less distraction or competition by a mismatching gesture when a brain area was causally involved in speech-gesture integration. In Experiment 1, a reduction of the semantic congruency effect was indeed found for LIFG, but not pMTG. In Experiment 2, Zhao et al. (2018) did find a reduction of the semantic congruency effect in pMTG, but LIFG was not stimulated. The authors conclude that LIFG and pMTG likely work together during speech-gesture integration, and postulate that LIFG regulates strategic semantic access on temporal storage areas, whereas pMTG is involved in accessing supramodal representations.
We suggest that these results could be explained by the coupling between pMTG and LIFG. As the authors state that the LIFG and pMTG are tightly anatomically coupled, it is possible that stimulating one area would also result in (an unquantified) stimulation of the other (Silvanto and Pascual-Leone, 2008). An alternative explanation for the non-significant effect of stimulating the pMTG in Experiment 1 is that only the LIFG is causally involved in the integration of speech and higher-level visual semantic information, whereas pMTG is integrating low-level information from the auditory and visual modality. In this case pMTG stimulation leads to distal suppression of the LIFG, leading to a weakening of the semantic congruency effect. The effect is then less pronounced due to stimulation of the causal region (LIFG) being indirect. While Zhao et al acknowledge that non-stimulated brain areas might also be affected, this is an important limiting factor for using TMS for functional localization, in particular without quantifying indirect stimulation effects (Kim et al., 2016). In order to rule out this possibility, neuroimaging could be used to assess not only activation in the targeted region, but also of the non-targeted region. This would clarify whether LIFG was unaffected in Experiment 2, and further elucidate the separate roles for pMTG and LIFG in speech-gesture integration.
We therefore argue that the current study has demonstrated that TMS over LIFG/MTG disrupted the semantic integration of speech and gestures, but that these results do not demonstrate the functional roles of these areas during speech-gesture integration. For example, Zhao et al. (2018) argue that the potential role of LIFG might be the strategic recovery of context-appropriate semantic information. However, we believe that the observed reduction of the semantic congruency effect after stimulating LIFG in Experiment 1 could equally well be explained as a disruption of the unification of the gesture and speech signal, which might be more taxing when semantic incongruency increases integration load. This explanation fits with the MUC model (Hagoort, 2013). We therefore argue that future work could consider a different behavioral task to probe speech-gesture integration that does not include incongruent gestures, as these gestures could cause additional integration processes that might not be natural in everyday multimodal communication.
An interesting follow-up would be to use a different behavioral task and neuroimaging, combined with a measure of directional connectivity, such as dynamic causal modeling (DCM) (Friston et al., 2003), to truly unravel the functional roles of LIFG/pMTG. Directional connectivity analysis could quantify how LIFG and MTG interact during speech gesture integration without any stimulation, and subsequently determine how the system is affected by stimulation over LIFG, MTG, or both. As DCM allows one to model how stimuli and task manipulations affect specific model parameters (i.e., activity within regions, and coupling strength between regions), combining DCM with TMS could be useful for making inferences regarding the specific roles of regions and their connections.
In conclusion, utilizing brain stimulation provides novel, interesting and relevant findings regarding the role of frontal and temporal regions in speech-gesture integration. However, we suggest that without additional measures such as neuroimaging, as well as an integration measure that does not manipulate semantic congruency, it is not possible to wholly disentangle the roles of the LIFG and MTG in terms of their causal involvement in speech-gesture integration. Overall, the findings of Zhao and colleagues highlight the importance of the LIFG and MTG in speech-gesture integration, and provide an important and exciting foundation for future work to further explore the contribution of the two areas.
Author Contributions
LD and JT discussed the original idea and wrote the manuscript together.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This research was supported by Gravitation Grant 024.001.006 of the Language in Interaction Consortium, of the Netherlands Organization for Scientific Research (NWO). We thank prof. Asli Ozyurek in helping us improve this commentary.
References
Dick, A. S., Mok, E. H., Raja Beharelle, A., Goldin-Meadow, S., and Small, S. L. (2014). Frontal and temporal contributions to understanding the iconic co-speech gestures that accompany speech. Hum. Brain Mapp. 35, 900–917. doi: 10.1002/hbm.22222
Friston, K. J., Harrison, L., and Penny, W. (2003). Dynamic causal modelling. Neuroimage 19, 1273–1302. doi: 10.1016/S1053-8119(03)00202-7
Hagoort, P. (2013). MUC (Memory, Unification, Control) and beyond. Front. Psychol. 4:416. doi: 10.3389/fpsyg.2013.00416
Holle, H., Gunter, T. C., Rüschemeyer, S. A., Hennenlotter, A., and Iacoboni, M. (2008). Neural correlates of the processing of co-speech gestures. Neuroimage 39, 2010–2024. doi: 10.1016/j.neuroimage.2007.10.055
Holle, H., Obleser, J., Rueschemeyer, S. A., and Gunter, T. C. (2010). Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions. Neuroimage 49, 875–884. doi: 10.1016/j.neuroimage.2009.08.058
Kelly, S. D., Creigh, P., and Bartolotti, J. (2010). Integrating speech and iconic gestures in a Stroop-like task: evidence for automatic processing. J. Cogn. Neurosci. 22, 683–694. doi: 10.1162/jocn.2009.21254
Kim, K., Ekstrom, A. D., and Tandon, N. (2016). A network approach for modulating memory processes via direct and indirect brain stimulation: toward a causal approach for the neural basis of memory. Neurobiol. Learn. Mem. 134, 162–177. doi: 10.1016/J.NLM.2016.04.001
Silvanto, J., and Pascual-Leone, A. (2008). State-dependency of transcranial magnetic stimulation. Brain Topogr. 21, 1–10. doi: 10.1007/s10548-008-0067-0
Straube, B., Green, A., Bromberger, B., and Kircher, T. (2011). The differentiation of iconic and metaphoric gestures: common and unique integration processes. Hum. Brain Mapp. 32, 520–533. doi: 10.1002/hbm.21041
Willems, R. M., Ozyürek, A., and Hagoort, P. (2009). Differential roles for left inferior frontal and superior temporal cortex in multimodal integration of action and language. Neuroimage 47, 1992–2004. doi: 10.1016/j.neuroimage.2009.05.066
Willems, R. M., Ozyürek, A., Hagoort, P., Ozyurek, A., and Hagoort, P. (2007). When language meets action: the neural integration of gesture and speech. Cereb. Cortex 17, 2322–2333. doi: 10.1093/cercor/bhl141
Keywords: speech, gesture, transcranial magnetic stimulation (TMS), semantic integration, speech-gesture integration
Citation: Drijvers L and Trujillo JP (2018) Commentary: Transcranial Magnetic Stimulation over Left Inferior Frontal and Posterior Temporal Cortex Disrupts Gesture-Speech Integration. Front. Hum. Neurosci. 12:256. doi: 10.3389/fnhum.2018.00256
Received: 08 April 2018; Accepted: 05 June 2018;
Published: 20 June 2018.
Edited by:
Paul Fredrick Sowman, Macquarie University, AustraliaReviewed by:
Mitchell Ryan Goldsworthy, University of Adelaide, AustraliaCopyright © 2018 Drijvers and Trujillo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Linda Drijvers, linda.drijvers@mpi.nl
†These authors have contributed equally to this work.