Sven Hoffmann1*
Uirassu Borges1
Laura Bröker1
Sylvain Laborde1,2
Roman Liepelt1
Babett H. Lobinger1
Jonna Löffler1
Lisa Musculus1
Markus Raab1,3- 1Department of Performance Psychology, Institute of Psychology, German Sport University Cologne, Cologne, Germany
- 22EA 4260 Normandie Université, Caen, France
- 3School of Applied Sciences, London Southbank University, London, United Kingdom
There is a vast amount of literature concerning the integration of action and cognition. Although this broad research area is of great interest for many disciplines like sports, psychology and cognitive neuroscience, only a few attempts tried to bring together different perspectives so far. Our goal is to provide a perspective to spark a debate across theoretical borders and integration of different disciplines via psychophysiology. In order to boost advances in this research field it is not only necessary to become aware of the different areas that are relevant but also to consider methodological aspects and challenges. We briefly describe the most relevant theoretical accounts to the question of how internal and external information processes or factors interact and, based on this, argue that research programs should consider the three dimensions: (a) dynamics of movements; (b) multivariate measures and; (c) dynamic statistical parameters. Only with an extended perspective on theoretical and methodological accounts, one would be able to integrate the dynamics of actions into theoretical advances.
A Brief Definition of Action
Imagine Wladimir Klitschko, one of the most successful heavyweight boxing champions of all times, aiming at winning his next boxing match. To win the fight he has to plan and execute different actions that are embedded in a continuous stream of ongoing behavior. A precise definition of action is difficult given its manifold manifestations. Some actions can be characterized as ephemeral, like a hook during the fight. Other actions are more complex and enduring, like “winning the match,” and could be seen as a sum of sequential actions. The question of how to separate single actions is often solved by determining their underlying goals (Herwig et al., 2013; Künzell et al., 2017). In this vein, goals can either be within the perceptual range (e.g., hitting the opponent’s rib cage with the fist) or in anticipated future states that cannot yet be perceived in the environment (e.g., winning this match to become the leader of one’s weight division). According to Prinz (2013), goals can be “hot” and require active intervention to achieve them (e.g., sidestep to a counter strike) or they can be “cold” and temporally uncritical (e.g., increasing one’s stamina). Herwig et al. (2013) stated that “an action starts with the first behavioral activity directed toward a particular goal and ends with the achievement of the goal” (p. 106). To achieve a goal, external and internal factors must be geared toward the prospective goal state. For our Klitschko example, this would mean that actions might be influenced by internal factors like fitness level, accuracy, and speed of punches or by external factors such as the opponent being southpaw or noise and lighting conditions in the arena. In extension to the definition of Herwig et al. (2013), we propose that the dynamics of movements and behavioral activity, defined as one element of action besides goals, need to be further specified and partitioned. We suggest that a psychophysiological perspective on actions extends the existing definition by offering an explanation of how internal and external factors interact. Thus, a perspective on integrating action and cognition has to ask the questions: Which are the relevant mechanisms that initiate, guide, and evaluate action? And which methodological challenges arise when the aim is to integrate mind and motion? Answering these questions requires a consideration of different theoretical accounts, but also of new technical developments and methodological advances.
Core Theoretical Accounts Related to the Interplay of Action and Cognition
Multi-Sensory Integration
Most action is embedded in a rich perceptual environment and perceptual inputs coming from different input modalities have to be integrated to allow proper action selection. Multi-sensory integration is a crucial physiological theory that has, for instance, explained psycho-physiological phenomena such as the rubber hand illusion (Botvinick and Cohen, 1998) or the McGurk effect (McGurk and MacDonald, 1976). Multi-sensory integration addresses the perceptual binding problem (Milner, 1974; Mioche and Singer, 1989), which considers the question of how neural inputs from different modalities (e.g., vision, audition, olfaction) are integrated into one coherent, valid, and robust perceptual experience (Spence, 2011). The neurobiological bases of multi-sensory integration are bi- or multimodal neurons that are excited by more than one input modality existing in a large number of brain areas, such as the somatosensory cortex (Stein et al., 2009). Various complementing principles have been formulated for multi-sensory integration. The principle of visual dominance postulates that vision has a greater influence on the other senses than vice versa (Witten and Knudsen, 2005) the principle of modality appropriateness claims that cross-modal influences depend on the modality’s appropriateness for a given task (Welch and Warren, 1980). Finally, the principle of Bayesian integration postulates that the brain uses a form of Bayesian inference to integrate multimodal information into a coherent perception of the world (Deneve and Pouget, 2004).
Embodied Cognition
The concept of embodied cognition assumes strong interactions between cognition, perception, and movement (Shapiro, 2010; Fischer and Coello, 2015), implying they cannot be studied independently of each other. In recent years, more and more evidence pointed to a considerable plasticity of the brain with respect to the integration of body parts (Blanke, 2012). Embodied cognition theories assume that the “self” emerges from the integration of bodily and/or environmental information. This information consists of visual, tactile, proprioceptive, vestibular, auditory, olfactory, visceral, and motor inputs (Blanke, 2012). Thus, embodied cognition is, at least from a neuroscience perspective, closely connected to multi-sensory integration. A recent study by Collins et al. (2016) showed that synchronizing stimulation of the hand somatosensory cortex and observed touches of a corresponding prosthetic hand created the perception of ownership of the artificial limb. From a psychological perspective, there is agreement that cognitive functions can only be understood when considering their relevance for actions (Wilson, 2002). To sum up, embodied cognition approaches postulate that cognitive processes are embodied and hypothesize that an individual’s bodily state (and its capacities and skills) and the environment interact (Rowlands, 2010).
Decision Making
Prior to acting, one needs to decide to do so. However, theoretical concepts of decision making often neglect the fact that cognitive and motor processes are intertwined. Some approaches describe decisions as heuristics, with strategies formally defined as building blocks, i.e., a search-, stop-, and decision-rule. The mere composition of cognitive building blocks reveals that the involvement of the sensorimotor system is widely ignored. An exception to this is the simple heuristic approach, which claims that a simple heuristic applied to option generation and decision making in complex sports behavior considers the building block of execution (de Oliveira et al., 2014). The execution rule addresses the question of which action to carry out and, more importantly, how to execute it with the motor system (Raab et al., 2005; de Oliveira et al., 2014). In our opinion, the intertwined motor and cognitive components of decisions, which have been theoretically discussed and coined in the light of motor control (Wolpert and Landy, 2012) or “embodied choices” (Raab, 2017), should be empirically studied and incorporated into theories of the psychophysiology of movement. To reach this goal, psychophysiological models of motor heuristics and embodied choices have to be formulated. Different models of the psychophysiology of movement have been studied in animals (Cisek and Kalaska, 2010) and humans (Wichary and Smolen, 2016) and can serve as a starting point for theory testing, integration, and development. Now, neuroscientific studies integrating different methodological challenges (Kyathanahally et al., 2017) target decision making (Chand and Dhamala, 2017; Muraskin et al., 2017), and find correlations of decision making and somatosensory networks (Harris and Lim, 2016), however, the aspect of movement or “real” (i.e., dynamic) action is still rarely taken into account. We return to this point when outlining current methodological developments.
Psychophysiology of Action and Cognitive Control
The literature concerning this topic is far too vast and complex to be laid down in detail here. Thus, we stick to the core mechanisms that seem relevant in this regard. From a neurophysiological point of view prefrontal cortex (PFC), motor cortex and basal ganglia networks and their corresponding neurotransmitter systems are crucial for action and cognition. More specifically, for task performance (“blocking and punching” to stick to the initial example) the stability of information in the PFC is crucial. Within these networks, there are structures and functions relevant for the evaluation of goal states. That is, for example, the allocation of attention to task-relevant features. On a rather simple level, the establishment of goals refers to efficient stimulus-response mappings concerning the task at hand, or, coming back to the initial example: “Move right arm up if left shoulder of the opponent indicates a punch.” But how do we keep track of our action goals on a neurophysiological level, and how are decisions implemented while acting? Working memory is crucial: the before-mentioned task-goal representations are being kept “online” within PFC networks (Jonides et al., 2008) via dopaminergic mediated activity states (Seamans and Yang, 2004). These assumptions match another idea: the reinforcement learning hypothesis (Holroyd and Coles, 2002). It assumes that within the PFC, dopamine (DA) is a kind of “gating signal” that is involved in keeping and adapting goal-state relations via reinforcement learning mechanisms (Miller, 2000). There exists considerable literature concerning connections of the PFC with parietal junctions (dorsolateral-prefrontal network) and how these networks control a variety of cognitive functions like motor planning, working memory, or allocation of attention. Ptak et al. (2017) have claimed that the core mechanism for motor planning (i.e., action planning) is action emulation. They argue that this emulation consists of a dynamic representation of abstract movement kinematics that sustains its internal manipulation. Thereby, it ensures its maintenance over short time periods. Further, it can be assumed that this dorsolateral prefrontal network has evolved from a motor control network to a general system supporting motor and cognitive functions. Related to the impact of actions, a recent account by Peterburs and Desmond (2016) suggests the cerebellum, the core structure for movement execution and motor adaptation, to play a crucial role in sensory prediction, error and conflict processing, response inhibition, as well as feedback learning. Here we return to the idea of embodied cognition: the key aspect of understanding cognition might be, not only from a psychological but also from a psychophysiological perspective, action. The neurophysiological mechanisms of action and the corresponding mechanisms related to the interaction with the environment are at the core of cognition.
Interoceptive and Exteroceptive Changes
Finally, using our example with Klitschko, interoceptive and exteroceptive changes produced by actions can be considered dynamically interacting. For instance, muscle tension cannot be maintained after a series of punches (interoceptive change), and Klitschko’s opponent can positively influence the estimations of his chances to win by falling after a good punching sequence (exteroceptive change). One can argue that actions could be considered body movements which depend on external and internal factors. Such actions produce re-afferent feedback to interoceptive (generated within the body) and exteroceptive (generated outside the body) change. For instance, in a model of Schubotz (2007) these changes are meant to appear dynamically during movements. She defines interoceptive changes as proprioception (sense of the relative position of the body parts), visceroception (sense of the inner organs), equilibrioception (sense of balance) and nociception (sense from organs, joints, and bones). For complex actions, vision, audition, haptic and other senses are used to detect exteroceptive changes. Whereas we see a current trend in multi-sensory integration research (Greenlee, 2017), less knowledge is gained for the complex interaction of combining interoceptive and exteroceptive changes and their respective measures in one research program (Suzuki et al., 2013).
Methodological Challenges Related to Integrating Action and Cognition
Besides the theoretical challenges that arise when taking the dynamics of actions into account, one of the core problems is the operationalization of the corresponding constructs. Nowadays there is a considerable amount of technical possibilities, however, only a few are used in dynamic contexts, or only in isolated paradigms measuring certain aspects. We think that more than static measurement of actions is required. For instance, in most classical experimental settings response times are measured or areas under the curve of a movement pattern are extracted, but temporal information provided by modern hard- and software is rarely taken into account. Further, these “static” measurements are often being analyzed independently of each other, e.g., response times are analyzed separate from EEG parameters, and are rarely integrated into attempts to solve theoretical questions (Debener, 2005; Hoffmann and Falkenstein, 2010; Plewan et al., 2016). Finally, many studies focus on central parameters like mean values or compare variances. However, in cognitive neuroscience a lot of measures have been developed which make it possible to describe the temporal dynamics of signals, e.g., time-frequency analysis, non-linear dynamics, or cross-coherence. We suggest a three-dimensional perspective integrating static and dynamic measurements, single and multiple measurements, and dynamics of statistics used. Anyway, the attempt to provide a comprehensive description of all possible methods would be too ambitious. In the following we will therefore describe essential aspects of this three-dimensional perspective and exemplify each aspect by selected methods.
From Static to a Dynamic Measurement of Action
The use of “static” measurements has a long tradition in psychology and is closely connected to the concept of the measurement error (Lord, 1959; Novick, 1966; Rozeboom et al., 1969). In experimental research (as in any empirical research) two types of errors, systematic and random error, might arise. By controlling the experimental setup, one hopes to control for errors or at least keep them constant. This has been achieved with rather static devices: The measurement error (of the device) in simple button presses can be easily controlled. Also, measuring psychophysiological variables can be quite tricky due to the sensitivity of the systems to movement artifacts or other artifact sources. In psychophysiology the term “signal-to-noise ratio” describes this relation of true value and error. To keep this ratio at optimum, the influence of artifacts can be kept low if the participant does not move strongly. Related to the aspect of how to deal with artifacts and how to report them, precise recommendations exist (Keil et al., 2013). Anyway, in former days, measurement instruments did not have the same high precision modern systems have, e.g., eye-tracking systems with high. Also, it was not possible to measure “online,” that is, in more dynamic settings. This was due to the before mentioned problems, but also because it was not possible to measure remotely electromyographic or EEG data. In recent years solutions to these problems, for example, mobile EEG (Vos and Debener, 2014; Wascher et al., 2014) or even mobile brain/body imaging (Makeig, 2009; Banaei et al., 2017) have been put forward. Some of these systems show an excellent signal-to-noise ratio (Radüntz, 2018). However, many studies rather focus on “static” measurements like button presses to indicate choices. Only a few researchers integrate dynamic measures like eye-movements (Krajbich and Rangel, 2011). These dynamic measures could be highly relevant in order to model for example “true” action adaptation, like the change of direction of a movement during execution. We argue that due to this technical evolution more dynamic measurements should be taken into account since they provide a more fine-graded analysis of the mechanisms involved.
From Single to Multiple Measurements
If one takes into account dynamic measurements, it seems straightforward to make corresponding inferences concerning the number of dependent variables that are being taken into account. Obviously, one has to be aware of the problem that: (a) one has to carefully select variables meaningful for the research question at hand and (b) to control for the problem of alpha error inflation. However, recent advances in statistics and neurosciences promise to integrate measures that might not only bring together different approaches but also provide new insights. One of these developments is, for example, the increasing usage of multivariate statistics to model artifact and neural sources (Jung et al., 2000). With these techniques it is even possible to integrate different measurements like simultaneous EEG-fRMI (Debener et al., 2006; Diukova et al., 2008; Sajda, 2009; Hoffmann et al., 2013; Dizaji and Soltanian-Zadeh, 2017). For a detailed and extended overview of methodological advances, the reader might refer to a recent research topic edited by Gramann et al. (2014).
From Static to Dynamic Parameters
With “static” we refer to the assumption that the mean is an estimate of the “true” value of some operationalized concept. This assumption holds when any non-systematic variation is distributed normally around the “true” value and that with increasing sample size such variation approximates zero. Shortly described it is aimed to minimize the measurement error. For instance in response times experiments, the mean response times are assumed to estimate the “true response time” with respect to some experimental condition. The logic behind that is: If the cognitive (or neural) system responds to a stimulus, the response consists of the “true” response and the error. With increasing number of trials, this error is being statistically minimized. However, a vast amount of literature presenting alternative statistical parameters, capturing the variability of responses, has emerged. Indeed, this variability of responses might be modulated and might indicate variations in the processes of interest. One way to capture it are for example ex-Gaussians. They provide good fits of empirical response time distributions (Spieler et al., 1996; Matzke and Wagenmakers, 2009) and they allow descriptions of differences between conditions reflected in shifting and/or skewing of the RT distribution. Group differences can be more easily detected compared to classical approaches using Gaussian parameters like the mean. Other important approaches are drift-diffusion models (Ratcliff, 2013) and hierarchical drift-diffusion models (Lee and Wagenmakers, 2009; Wiecki et al., 2013). With drift-diffusion models one can basically model psychological parameters, e.g., information accumulation and decision threshold, by taking into account statistical parameters of response time distributions (Ratcliff, 2013). In sum, they have a long tradition in psychology, and recent advances integrate these accounts into neuroscience (Mulder et al., 2013; Forstmann et al., 2016).
A Perspective on Integrating Cognition and Action
This headline promises much, and for sure a single, broad perspective will not provide a solution for “everything.” However, we believe that a research program related to the integration of action and cognition, be it related to embodiment or cognitive control, should at least consider the literature of the fields mentioned herein but also consider methodological and theoretical advances from other fields. Indeed, more and more studies integrate concepts and methods from different fields (Cohen and Cavanagh, 2011; Cavanagh et al., 2011; Schneider et al., 2014; Stock et al., 2016). For instance, Kiverstein and Miller (2015) described how an integration of neuroscience and embodiment could be laid down but there was a focus on the integration of emotion and cognition. Hence, the core ideas of embodiment, that the body and actions are relevant for cognition (Wilson, 2002), has not been derived in detail yet. Coming back to the initial boxing example, it gets clear that the question of how Klitschko got to some strategic decisions cannot be answered without considering the interaction of interoceptive and exteroceptive information. Also, the effect of cognition during the fight cannot be investigated without using dynamic and multivariate measures, since the interaction of these intero- and exteroceptive changes might depend exactly on that setting. Also, this situation may affect neural mechanisms of cognitive control. Having a look at the different streams of literature in the research areas of multi-sensory integration, embodied cognition, decision making, and the neural base of cognitive control, one can only conclude that the fields have to become aware of advances in the respective other fields. For example, there is a huge amount of research related to what decision making is and why it works as it does but the investigation of how our neural system processes decisions and how all the processes are integrated “online” has not yet been developed in depth. We think that a psychophysiological perspective on actions extends existing definitions by offering the opportunity to find an explanation of how internal and external factors interact. Any perspective on integrating action and cognition should strive to find answers to the question about the relevant mechanisms that initiate, guide, and evaluate action.
Author Contributions
This perspective has been developed during a group retreat by the “Performance Psychology” group in 2017. Note that though the corresponding author coordinated, integrated, and wrote the manuscript, all other authors contributed equally with respect to conceptualization of the manuscript, as well as revising it. Thus, the order of the authors is, except the corresponding author, in alphabetical order.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
We would like to thank all group members and in particular Clara Schweinitz for her input during the retreat!
References
Banaei, M., Hatami, J., Yazdanfar, A., and Gramann, K. (2017). Walking through architectural spaces: the impact of interior forms on human brain dynamics. Front. Hum. Neurosci. 11:477. doi: 10.3389/fnhum.2017.00477
Blanke, O. (2012). Multisensory brain mechanisms of bodily self-consciousness. Nat. Rev. Neurosci. 13, 556–571. doi: 10.1038/nrn3292
Botvinick, M., and Cohen, J. (1998). Rubber hands ‘feel’ touch that eyes see. Nature 391, 756–756. doi: 10.1038/35784.4
Cavanagh, J. F., Wiecki, T. V., Cohen, M. X., Figueroa, C. M., Samanta, J., Sherman, S. J., et al. (2011). Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold. Nat. Neurosci. 14, 1462–1467. doi: 10.1038/nn.2925
Chand, G. B., and Dhamala, M. (2017). Interactions between the anterior cingulate-insula network and the fronto-parietal network during perceptual decision-making. Neuroimage 152, 381–389. doi: 10.1016/j.neuroimage.2017.03.014
Cisek, P., and Kalaska, J. F. (2010). Neural mechanisms for interacting with a world full of action choices. Annu. Rev. Neurosci. 33, 269–298. doi: 10.1146/annurev.neuro.051508.135409
Cohen, M. X., and Cavanagh, J. F. (2011). Single-trial regression elucidates the role of prefrontal theta oscillations in response conflict. Front. Psychol. 2:30. doi: 10.3389/fpsyg.2011.00030
Collins, K. L., Guterstam, A., Cronin, J., Olson, J. D., Ehrsson, H. H., and Ojemann, J. G. (2016). Ownership of an artificial limb induced by electrical brain stimulation. Proc. Natl. Acad. Sci. U.S.A. 114, 166–171. doi: 10.1073/pnas.1616305114
de Oliveira, R. F., de Lobinger, B. H., and Raab, M. (2014). An adaptive toolbox approach to the route to expertise in sport. Front. Psychol. 5:709. doi: 10.3389/fpsyg.2014.00709
Debener, S. (2005). Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. J. Neurosci. 25, 11730–11737. doi: 10.1523/jneurosci.3286-05.2005
Debener, S., Ullsperger, M., Siegel, M., and Engel, A. K. (2006). Single-trial EEGfMRI reveals the dynamics of cognitive function. Trends Cogn. Sci. 10, 558–563. doi: 10.1016/j.tics.2006.09.010
Deneve, S., and Pouget, A. (2004). Bayesian multisensory integration and cross-modal spatial links. J. Physiol. Paris 98, 249–258. doi: 10.1016/j.jphysparis.2004.03.011
Diukova, A., Mallikarjun, P., Auer, D., and Liddle, P. (2008). Trial-by-trial analysis of combined EEG and fMRI shows dynamics of cognitive function. Front. Hum. Neurosci. 2:1. doi: 10.3389/conf.neuro.09.2009.01.332
Dizaji, A. S., and Soltanian-Zadeh, H. (2017). A change-point analysis method for single-trial. Study of simultaneous EEG-fMRI of auditory/visual oddball task. bioRxiv [Preprint]. doi: 10.1101/100487
Fischer, M. H., and Coello, Y. (2015). Conceptual and Interactive Embodiment: Foundations of embodied Cognition. Abingdon: Routledge.
Forstmann, B. U., Ratcliff, R., and Wagenmakers, E.-J. (2016). Sequential sampling models in cognitive neuroscience: advantages applications, and extensions. Annu. Rev. Psychol. 67, 641–666. doi: 10.1146/annurev-psych-122414-033645
Gramann, K., Jung, T.-P., Ferris, D. P., Lin, C.-T., and Makeig, S. (2014). Towards a new cognitive neuroscience: modeling natural brain dynamics. Front. Hum. Neurosci. 8:444. doi: 10.3389/fnhum.2014.00444
Greenlee, M. W. (2017). Self-motion perception: ups and downs of multisensory integration and conflict detection. Curr. Biol. 27, R1006–R1007. doi: 10.1016/j.cub.2017.07.050
Harris, A., and Lim, S.-L. (2016). Temporal dynamics of sensorimotor networks in effort-based cost-benefit valuation: early emergence and late net value integration. J. Neurosci. 36, 7167–7183. doi: 10.1523/jneurosci.4016-15.2016
Herwig, A., Beisert, M., and Prinz, W. (2013). Action Science Emerging: Introduction and Leitmotifs in Action Science. Cambridge MA: The MIT Press. doi: 10.7551/mitpress/9780262018555.003.0001
Hoffmann, S., and Falkenstein, M. (2010). Independent component analysis of erroneous and correct responses suggests online response control. Hum. Brain Mapp. 31, 1305–1315. doi: 10.1002/hbm.20937
Hoffmann, S., Labrenz, F., Themann, M., Wascher, E., and Beste, C. (2013). Crosslinking EEG timefrequency decomposition and fMRI in error monitoring. Brain Struct. Funct. 219, 595–605. doi: 10.1007/s00429-013-0521-y
Holroyd, C. B., and Coles, M. G. H. (2002). The neural basis of human error processing: reinforcement learning dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709. doi: 10.1037/0033-295x.109.4.679
Jonides, J., Lewis, R. L., Nee, D. E., Lustig, C. A., Berman, M. G., and Moore, K. S. (2008). The mind and brain of short-term memory. Annu. Rev. Psychol. 59, 193–224. doi: 10.1146/annurev.psych.59.103006.093615
Jung, T.-P., Makeig, S., Humphries, C., Lee, T.-W., McKeown, M. J., Iragui, V., et al. (2000). Removing electroencephalographic artifacts by blind source separation. Psychophysiology 37, 163–178. doi: 10.1111/1469-8986.3720163
Keil, A., Debener, S., Gratton, G., Junghöfer, M., Kappenman, E. S., Luck, S. J., et al. (2013). Committee report: publication guidelines and recommendations for studies using electroencephalography and magnetoencephalography. Psychophysiology 51, 1–21. doi: 10.1111/psyp.12147
Kiverstein, J., and Miller, M. (2015). The embodied brain: towards a radical embodied cognitive neuroscience. Front. Hum. Neurosci. 9:237. doi: 10.3389/fnhum.2015.00237
Krajbich, I., and Rangel, A. (2011). Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions. Proc. Natl. Acad. Sci. U.S.A. 108, 13852–13857. doi: 10.1073/pnas.1101328108
Künzell, S., Broeker, L., Dignath, D., Ewolds, H., Raab, M., and Thomaschke, R. (2017). What is a task? An ideomotor perspective. Psychol. Res. 82, 4–11. doi: 10.1007/s00426-017-0942-y
Kyathanahally, S. P., Franco-Watkins, A., Zhang, X., Calhoun, V. D., and Deshpande, G. (2017). A realistic framework for investigating decision making in the brain with high spatiotemporal resolution using simultaneous EEG/fMRI and joint ICA. IEEE J. Biomed. Health Inform. 21, 814–825. doi: 10.1109/jbhi.2016.2590434
Lee, M. D., and Wagenmakers, E.-J. (2009). Bayesian Cognitive Modeling. Cambridge, MA: Cambridge University Press, doi: 10.1017/cbo9781139087759
Lord, F. M. (1959). Problems in mental test theory arising from errors of measurement. J. Am. Stat. Assoc. 54, 472–479. doi: 10.1080/01621459.1959.10501991
Makeig, S. (2009). Mobile brain/body imaging (mobi) of active cognition. Front. Comput. Neurosci. 3:1. doi: 10.3389/conf.neuro.10.2009.14.140
Matzke, D., and Wagenmakers, E.-J. (2009). Psychological interpretation of the ex-gaussian and shifted wald parameters: a diffusion model analysis. Psychon. Bull. Rev. 16, 798–817. doi: 10.3758/pbr.16.5.798
McGurk, H., and MacDonald, J. (1976). Hearing lips and seeing voices. Nature 264, 746–748. doi: 10.1038/264746a0
Miller, E. K. (2000). The prefrontal cortex and cognitive control. Nat. Rev. Neurosci. 1, 59–65. doi: 10.1038/35036228
Milner, P. M. (1974). A model for visual shape recognition. Psychol. Rev. 81, 521–535. doi: 10.1037/h0037149
Mioche, L., and Singer, W. (1989). Chronic recordings from single sites of kitten striate cortex during experience-dependent modifications of receptive-field properties. J. Neurophysiol. 62, 185–197. doi: 10.1152/jn.1989.62.1.185
Mulder, M. J., Boekel, W., Ratcliff, R., and Forstmann, B. U. (2013). Cortico-subthalamic connection predicts individual differences in value-driven choice bias. Brain Struct. Funct. 219, 1239–1249. doi: 10.1007/s00429-013-0561-3
Muraskin, J., Brown, T. R., Walz, J. M., Tu, T., Conroy, B., Goldman, R. I., et al. (2017). A multimodal encoding model applied to imaging decision-related neural cascades in the human brain. Neuroimage doi: 10.1016/j.neuroimage.2017.06.059 [Epub ahead of print].
Novick, M. R. (1966). The axioms and principal results of classical test theory. J. Math. Psychol. 3, 1–18. doi: 10.1016/0022-2496(66)90002-2
Peterburs, J., and Desmond, J. E. (2016). The role of the human cerebellum in performance monitoring. Curr. Opin. Neurobiol. 40, 38–44. doi: 10.1016/j.conb.2016.06.011
Plewan, T., Wascher, E., Falkenstein, M., and Hoffmann, S. (2016). Classifying response correctness across different task sets: a machine learning approach. PLoS One 11:e0152864. doi: 10.1371/journal.pone.0152864
Prinz, W. (2013). Action Science, eds W. Prinz, M. Beisert, and A. Herwig. Cambridge, MA: The MIT Press. doi: 10.7551/mitpress/9780262018555.001.0001
Ptak, R., Schnider, A., and Fellrath, J. (2017). The dorsal frontoparietal network: a core system for emulated action. Trends Cogn. Sci. 21, 589–599. doi: 10.1016/j.tics.2017.05.002
Raab, M. (2017). Motor heuristics and embodied choices: how to choose and act. Curr. Opin. Psychol. 16, 34–37. doi: 10.1016/j.copsyc.2017.02.029
Raab, M., Masters, R. S. W., and Maxwell, J. P. (2005). Improving the ‘how’ and ‘what’ decisions of elite table tennis players. Hum. Mov. Sci. 24, 326–344. doi: 10.1016/j.humov.2005.06.004
Radüntz, T. (2018). Signal quality evaluation of emerging EEG devices. Front. Physiol. 9:98. doi: 10.3389/fphys.2018.00098
Ratcliff, R. (2013). Parameter variability and distributional assumptions in the diffusion model. Psychol. Rev. 120, 281–292. doi: 10.1037/a0030775
Rowlands, M. (2010). The New Science of the Mind: From Extended Mind to Embodied Phenomenology. Cambridge, MA: Mit Press. doi: 10.7551/mitpress/9780262014557.001.0001
Rozeboom, W. W., Lord, F. M., Novick, M. R., and Birnbaum, A. (1969). Statistical theories of mental test scores. Am. Educ. Res. J. 6:112. doi: 10.2307/1162101
Sajda, P. (2009). “Signal processing challenges for single-trial analysis of simultaneous EEG/fMRI. Paper Presented at the 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE), Piscataway, NJ. doi: 10.1109/iembs.2009.5335024
Schneider, D., Hoffmann, S., and Wascher, E. (2014). Sustained posterior contralateral activity indicates re-entrant target processing in visual change detection: an EEG study. Front. Hum. Neurosci. 8:247. doi: 10.3389/fnhum.2014.00247
Schubotz, R. I. (2007). Prediction of external events with our motor system: towards a new framework. Trends Cogn. Sci. 11, 211–218. doi: 10.1016/j.tics.2007.02.006
Seamans, J. K., and Yang, C. R. (2004). The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Prog. Neurobiol. 74, 1–58. doi: 10.1016/j.pneurobio.2004.05.006
Spence, C. (2011). Crossmodal correspondences: a tutorial review. Atten. Percept. Psychophys. 73, 971–995. doi: 10.3758/s13414-010-0073-7
Spieler, D. H., Balota, D. A., and Faust, M. E. (1996). Stroop performance in healthy younger and older adults and in individuals with dementia of the Alzheimers type. J. Exp. Psychol. Hum. Percept. Perform. 22, 461–479. doi: 10.1037/0096-1523.22.2.461
Stein, B. E., Stanford, T. R., and Rowland, B. A. (2009). The neural basis of multisensory integration in the midbrain: its organization and maturation. Hear. Res. 258, 4–15. doi: 10.1016/j.heares.2009.03.012
Stock, A.-K., Hoffmann, S., and Beste, C. (2016). Effects of binge drinking and hangover on response selection sub-processes-a study using EEG and drift diffusion modeling. Addict. Biol. 22, 1355–1365. doi: 10.1111/adb.12412
Suzuki, K., Garfinkel, S. N., Critchley, H. D., and Seth, A. K. (2013). Multisensory integration across exteroceptive and interoceptive domains modulates self-experience in the rubber-hand illusion. Neuropsychologia 51, 2909–2917. doi: 10.1016/j.neuropsychologia.2013.08.014
Vos, M. D., and Debener, S. (2014). Mobile EEG: towards brain activity monitoring during natural action and cognition. Int. J. Psychophysiol. 91, 1–2. doi: 10.1016/j.ijpsycho.2013.10.008
Wascher, E., Heppner, H., and Hoffmann, S. (2014). Towards the measurement of event-related EEG activity in real-life working environments. Int. J. Psychophysiol. 91, 3–9. doi: 10.1016/j.ijpsycho.2013.10.006
Welch, R. B., and Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychol. Bull. 88, 638–667. doi: 10.1037/0033-2909.88.3.638
Wichary, S., and Smolen, T. (2016). Neural underpinnings of decision strategy selection: a review and a theoretical model. Front. Neurosci. 10:500. doi: 10.3389/fnins.2016.00500
Wiecki, T. V., Sofer, I., and Frank, M. J. (2013). HDDM: hierarchical bayesian estimation of the drift-diffusion model in python. Front. Neuroinform. 7:14. doi: 10.3389/fninf.2013.00014
Wilson, M. (2002). Six views of embodied cognition. Psychon. Bull. Rev. 9, 625–636. doi: 10.3758/bf03196322
Witten, I. B., and Knudsen, E. I. (2005). Why seeing is believing: merging auditory and visual worlds. Neuron 48, 489–496. doi: 10.1016/j.neuron.2005.10.020
Keywords: psychophysiology, cognition, action, decision making, performance monitoring, embodiment
Citation: Hoffmann S, Borges U, Bröker L, Laborde S, Liepelt R, Lobinger BH, Löffler J, Musculus L and Raab M (2018) The Psychophysiology of Action: A Multidisciplinary Endeavor for Integrating Action and Cognition. Front. Psychol. 9:1423. doi: 10.3389/fpsyg.2018.01423
Received: 26 March 2018; Accepted: 20 July 2018;
Published: 29 August 2018.
Edited by:
Aaron Williamon, Royal College of Music, United KingdomReviewed by:
Miguel Molina-Solana, Imperial College London, United KingdomJames George Jackson, Leeds Trinity University, United Kingdom
Copyright © 2018 Hoffmann, Borges, Bröker, Laborde, Liepelt, Lobinger, Löffler, Musculus and Raab. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Sven Hoffmann, s.hoffmann@dshs-koeln.de