Listeners integrate speech with gestures in face-to-face communication. This semantic integration of auditory and visual information is thought to occur automatically (Kelly et al., 2010). Previous neuroimaging studies have found inconsistent results on the involvement and role of the left inferior frontal gyrus (LIFG, e.g., Willems et al., 2007, 2009) and posterior middle temporal gyrus (pMTG, e.g., Holle et al., 2008, 2010; Straube et al., 2011) in this process. For example, Willems et al. (2009) suggested that pMTG is mostly involved in the integration of pantomimes (re-enactments of actions that can be understood without speech) and speech, whereas LIFG is more involved in the integration of iconic gestures (gestures that are ambiguous without speech) and speech. Extending on this, Holle et al. (2010) suggested that pSTS/MTG is involved in the integration of iconic gestures and speech to perform an initial conceptual matching between the inputs, whereas LIFG might be involved in processes of modulation and revision. Others have argued that LIFG/pMTG are both involved in combining the semantic information from both inputs (Dick et al., 2014)
A recent study in The Journal of Neuroscience by Zhao et al. (2018) investigated whether LIFG/pMTG are causally involved in speech-gesture integration by perturbing activity in these regions using continuous theta burst stimulation (cTBS) (Experiment 1) and repetitive transcranial magnetic stimulation (rTMS) (Experiment 2) relative to a control region. Participants were presented with videos of an actor/actress uttering action verbs while producing a gesture. In these videos, gender congruency and semantic congruency were manipulated. Participants were asked to identify the gender of the speaker after every video. In Experiment 1, Zhao et al. (2018) used cTBS to investigate whether disrupting activity in LIFG/pMTG would lead to a reduction of the semantic congruency effect. A disruption of brain activity was expected to lead to less distraction or competition by a mismatching gesture when a brain area was causally involved in speech-gesture integration. In Experiment 1, a reduction of the semantic congruency effect was indeed found for LIFG, but not pMTG. In Experiment 2, Zhao et al. (2018) did find a reduction of the semantic congruency effect in pMTG, but LIFG was not stimulated. The authors conclude that LIFG and pMTG likely work together during speech-gesture integration, and postulate that LIFG regulates strategic semantic access on temporal storage areas, whereas pMTG is involved in accessing supramodal representations.
We suggest that these results could be explained by the coupling between pMTG and LIFG. As the authors state that the LIFG and pMTG are tightly anatomically coupled, it is possible that stimulating one area would also result in (an unquantified) stimulation of the other (Silvanto and Pascual-Leone, 2008). An alternative explanation for the non-significant effect of stimulating the pMTG in Experiment 1 is that only the LIFG is causally involved in the integration of speech and higher-level visual semantic information, whereas pMTG is integrating low-level information from the auditory and visual modality. In this case pMTG stimulation leads to distal suppression of the LIFG, leading to a weakening of the semantic congruency effect. The effect is then less pronounced due to stimulation of the causal region (LIFG) being indirect. While Zhao et al acknowledge that non-stimulated brain areas might also be affected, this is an important limiting factor for using TMS for functional localization, in particular without quantifying indirect stimulation effects (Kim et al., 2016). In order to rule out this possibility, neuroimaging could be used to assess not only activation in the targeted region, but also of the non-targeted region. This would clarify whether LIFG was unaffected in Experiment 2, and further elucidate the separate roles for pMTG and LIFG in speech-gesture integration.
We therefore argue that the current study has demonstrated that TMS over LIFG/MTG disrupted the semantic integration of speech and gestures, but that these results do not demonstrate the functional roles of these areas during speech-gesture integration. For example, Zhao et al. (2018) argue that the potential role of LIFG might be the strategic recovery of context-appropriate semantic information. However, we believe that the observed reduction of the semantic congruency effect after stimulating LIFG in Experiment 1 could equally well be explained as a disruption of the unification of the gesture and speech signal, which might be more taxing when semantic incongruency increases integration load. This explanation fits with the MUC model (Hagoort, 2013). We therefore argue that future work could consider a different behavioral task to probe speech-gesture integration that does not include incongruent gestures, as these gestures could cause additional integration processes that might not be natural in everyday multimodal communication.
An interesting follow-up would be to use a different behavioral task and neuroimaging, combined with a measure of directional connectivity, such as dynamic causal modeling (DCM) (Friston et al., 2003), to truly unravel the functional roles of LIFG/pMTG. Directional connectivity analysis could quantify how LIFG and MTG interact during speech gesture integration without any stimulation, and subsequently determine how the system is affected by stimulation over LIFG, MTG, or both. As DCM allows one to model how stimuli and task manipulations affect specific model parameters (i.e., activity within regions, and coupling strength between regions), combining DCM with TMS could be useful for making inferences regarding the specific roles of regions and their connections.
In conclusion, utilizing brain stimulation provides novel, interesting and relevant findings regarding the role of frontal and temporal regions in speech-gesture integration. However, we suggest that without additional measures such as neuroimaging, as well as an integration measure that does not manipulate semantic congruency, it is not possible to wholly disentangle the roles of the LIFG and MTG in terms of their causal involvement in speech-gesture integration. Overall, the findings of Zhao and colleagues highlight the importance of the LIFG and MTG in speech-gesture integration, and provide an important and exciting foundation for future work to further explore the contribution of the two areas.
Statements
Author contributions
LD and JT discussed the original idea and wrote the manuscript together.
Acknowledgments
This research was supported by Gravitation Grant 024.001.006 of the Language in Interaction Consortium, of the Netherlands Organization for Scientific Research (NWO). We thank prof. Asli Ozyurek in helping us improve this commentary.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1
DickA. S.MokE. H.Raja BeharelleA.Goldin-MeadowS.SmallS. L. (2014). Frontal and temporal contributions to understanding the iconic co-speech gestures that accompany speech. Hum. Brain Mapp.35, 900–917. 10.1002/hbm.22222
2
FristonK. J.HarrisonL.PennyW. (2003). Dynamic causal modelling. Neuroimage19, 1273–1302. 10.1016/S1053-8119(03)00202-7
3
HagoortP. (2013). MUC (Memory, Unification, Control) and beyond. Front. Psychol.4:416. 10.3389/fpsyg.2013.00416
4
HolleH.GunterT. C.RüschemeyerS. A.HennenlotterA.IacoboniM. (2008). Neural correlates of the processing of co-speech gestures. Neuroimage39, 2010–2024. 10.1016/j.neuroimage.2007.10.055
5
HolleH.ObleserJ.RueschemeyerS. A.GunterT. C. (2010). Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions. Neuroimage49, 875–884. 10.1016/j.neuroimage.2009.08.058
6
KellyS. D.CreighP.BartolottiJ. (2010). Integrating speech and iconic gestures in a Stroop-like task: evidence for automatic processing. J. Cogn. Neurosci.22, 683–694. 10.1162/jocn.2009.21254
7
KimK.EkstromA. D.TandonN. (2016). A network approach for modulating memory processes via direct and indirect brain stimulation: toward a causal approach for the neural basis of memory. Neurobiol. Learn. Mem.134, 162–177. 10.1016/J.NLM.2016.04.001
8
SilvantoJ.Pascual-LeoneA. (2008). State-dependency of transcranial magnetic stimulation. Brain Topogr.21, 1–10. 10.1007/s10548-008-0067-0
9
StraubeB.GreenA.BrombergerB.KircherT. (2011). The differentiation of iconic and metaphoric gestures: common and unique integration processes. Hum. Brain Mapp.32, 520–533. 10.1002/hbm.21041
10
WillemsR. M.OzyürekA.HagoortP. (2009). Differential roles for left inferior frontal and superior temporal cortex in multimodal integration of action and language. Neuroimage47, 1992–2004. 10.1016/j.neuroimage.2009.05.066
11
WillemsR. M.OzyürekA.HagoortP.OzyurekA.HagoortP. (2007). When language meets action: the neural integration of gesture and speech. Cereb. Cortex17, 2322–2333. 10.1093/cercor/bhl141
12
ZhaoW.RiggsK.SchindlerI.HolleH. (2018). Transcranial magnetic stimulation over left inferior frontal and posterior temporal cortex disrupts gesture-speech integration. J. Neurosci.10, 1748–1717. 10.1523/JNEUROSCI.1748-17.2017
Summary
Keywords
speech, gesture, transcranial magnetic stimulation (TMS), semantic integration, speech-gesture integration
Citation
Drijvers L and Trujillo JP (2018) Commentary: Transcranial Magnetic Stimulation over Left Inferior Frontal and Posterior Temporal Cortex Disrupts Gesture-Speech Integration. Front. Hum. Neurosci. 12:256. doi: 10.3389/fnhum.2018.00256
Received
08 April 2018
Accepted
05 June 2018
Published
20 June 2018
Volume
12 - 2018
Edited by
Paul Fredrick Sowman, Macquarie University, Australia
Reviewed by
Mitchell Ryan Goldsworthy, University of Adelaide, Australia
Updates
Copyright
© 2018 Drijvers and Trujillo.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Linda Drijvers linda.drijvers@mpi.nl
†These authors have contributed equally to this work.
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.