Frank Krueger
Eva Wiese- 1School of Systems Biology, George Mason University, Fairfax, VA, United States
- 2Institute of Psychology and Ergonomics, Berlin, Germany
Introduction
Social Neuroergonomics (SNE) is a transdisciplinary field (including psychology, human factors, engineering, social neuroscience) devoted to the application of knowledge of the neurobiological underpinnings of social processes and behaviors (ranging from neural to hormonal to cellular and genetic mechanisms) to the design, engineering, and evaluation of human-machine systems. The tremendous advances that have been accomplished in each of these fields can inspire data-driven hypotheses, foster experimental designs, and increase neuroergonomic theories' relevance, which is not achievable from only a single-field perspective. Based on a profound understanding of both the social nature of human brains and the principles of human-centered design, SNE has the unique potential to advance our understanding of the psychoneurobiological basis of whether and how humans engage in social interactions with technology (ranging from automated technical systems to autonomous robots) and to use these insights to foster more efficient and satisfying human-machine interactions (HMIs) (e.g., reduction of human errors, increase of trust and productivity, enhancement of safety) in everyday settings (Hancock et al., 2011). Like any emerging research field, SNE will face challenges in the upcoming years that have to be acknowledged, addressed, and resolved. This Special Grand Challenge highlights some of these challenges to ensure the success of SNE as a multi-level integrative field with high relevance and impact on our everyday lives as well as practical implications in diverse sectors such as healthcare, education, therapy, and entertainment (Parasuraman, 2011; Ayaz and Dehais, 2019).
Challenge 1: From Automation to Autonomy
One challenge for SNE will be to optimally foster the shift from automated (i.e., executing a pre-defined task by rule-based responses in reasonably well-known and structured environments) to fully autonomous (i.e., executing a task by adaptive-based learning or artificial intelligence-based capabilities in unknown and changing environments) systems, such as embodied social robots, that need to be adaptive, personified, independent, socially intelligent, and indeterminate to satisfy users' needs (Schaefer et al., 2016). Our future will inevitably be shaped by a co-existence of humans as social, biological agents and social, artificially intelligent machine agents. Social embodied robots are already increasingly taking on numerous roles in our society in various domains, which highlights the necessity of a scientific investigation of the psychoneurobiological basis of HMIs as well as the formulation of empirically derived guidelines for the design and evaluation of such systems (Fujitam, 2004). However, we are still confronted with the reality that current social robot systems fall short of expectations humans have about intuitive, efficient, and rewarding mutual interactions (Dautenhahn et al., 2002). One critical psychological factor for successful social interactions is that we are more motivated to engage in interactions with entities that are believed “to have a mind of their own” —with internal states like intentions, emotions, or preferences (i.e., mind perception, Gray et al., 2007). Mind perception is an attributional process by which a perceiver imbues a human-like mind status to a perceived agent based on certain features of the perceiver (e.g., state and/or trait loneliness, knowledge about technology), as well as certain features of the perceived (e.g., human appearance, biological motion, similarity-to-self, Waytz et al., 2010). Once the mind is attributed to agents—human or non-human—their behaviors and actions tend to be interpreted in mentalistic terms (i.e., driven by human-like internal states) and interacting with such agents triggers motivational, affective, and cognitive processes in our social brains (Wiese et al., 2017). However, anthropomorphic appearance does not always lead to more social human-robot interactions, as subtle imperfections of robots that physically appear extremely—but not perfectly—human-like (e.g., androids, geminoids) can become disturbing and hinder social HMIs (i.e., “uncanny valley” hypothesis, Mori, 1970). Over the last decades, social neuroscience has shed some light on which perceptual-motor features activate social brain areas (e.g., goal-directed actions for the mirror-neuron system) during human-human interactions; building upon these insights, a minimal set of design features can be determined to evoke similar processes during HMIs (Yamaoka et al., 2007)—harvesting behavioral, psychological, and neurobiological methodologies. Therefore, to facilitate the adjustment for the transition from automated to fully autonomous social agents, SNE can help us to formulate a minimal set of design features triggering social brain networks in a diverse set of users to develop social robots that maximize the likelihood of human engagement in satisfying social interactions with these agents in everyday environments (Fong et al., 2003).
Challenge 2: From Observation to Interaction
Another challenge for the growing field of SNE is to shed light on the multifaceted foundation of social cognition during HMIs utilizing ecologically-sound experimental paradigms that investigate dynamically unfolding interactions over time instead of static observations of the social situation. So far, a plethora of studies dominantly examine the brain in isolation by employing social observation paradigms instead of imposing interactive paradigms where social agents adopt complementary and alternating roles during the progression of social interactions that affect immediate and upcoming occurrences between humans and robotic agents. A “second-person” neuroscientific framework (Schilbach et al., 2013) or “two-person neuroscience” approach (Hari and Kujala, 2009) for SNE is needed for which neural mechanisms of the human's brain are evaluated during real-time reciprocal social interactions—imposing either turn-based or mutual realistic HMIs. Social interactions are more than just tapping into an agent's social knowledge representations; instead, they initiate shared processing and knowledge between agents (Redcay and Schilbach, 2019). Neuroscience has proven that variations in the configuration of brain networks exist when we are interacting in real-time with other agents (i.e., “online” social cognition mirroring the dynamics between agents) in comparison to observing other agents (i.e., “offline” social cognition mirroring the statics inside the agent) (Pfeiffer et al., 2013). Overall, the neural mechanisms of dynamic real-time social HMI are largely unexplored and can be considered the “dark matter” in studying HMI (Schilbach et al., 2013). To close this gap, SNE should develop, test, and validate empirical paradigms that allow the measurement of neuropsychological indices employing hyperscanning techniques in real-time dynamic social HMI interactions. These measurements should not be confined to short one-time interactions between humans and expensive, difficult-to-handle robot platforms in research laboratories, instead should be obtained in everyday environments with easy-to-use social robot platforms (e.g., Cozmo) over longer periods. To make long-term social interactions with robots motivating to the general public, SNE will need input from computer science to develop social artificially intelligent systems that can attune to human input and adapt flexibly in dynamic interactions.
Challenge 3: From Explicit to Implicit Measures
Another challenge is to widen the spectrum of examining psychological processes from explicit (occurring controlled and slow with conscious access and control) to implicit (occurring automatically and fast without conscious awareness or control). If studying social interaction (as opposed to observation) is seen as the “dark matter” of SNE, then investigating implicit social processes (as opposed to explicit processes) can be regarded the “dark energy” of SNE. Like dark energy is taking up most of our universe, implicit processes shape most of our social interactions. As two levels of social cognition, implicit and explicit processes often interplay and influence social perception, cognition, and interaction—sometimes complementary and other times oppositional (Frith and Frith, 2008). Studying how brain networks recruited for implicit processes (e.g., salience network) interact with brain networks recruited for explicit processes (e.g., central-executive network) (Forbes and Grafman, 2013) can provide us with valuable insights on how to understand better and design mutual, dynamic HMI. Despite the fast growth in the field of HMI, reliable and innovative implicit measures predicting outcomes in HMI (e.g., social bonding, joint performance) are missing. Most studies employ primarily self-report or measures of explicit processes (e.g., subjective ratings) to evaluate trust in HMIs (Sanders et al., 2016). However, research has shown that assessing measures of implicit attitudes toward machines (e.g., via Implicit Association Test) can be more predictive of trust behavior than objective measures of explicit processes (Merritt et al., 2013). In this vein, it will be important for SNE to advance the field of social human-technology interaction by proposing innovative paradigms that combine measures of implicit and explicit processes with neuroscience methods to ultimately develop empirically derived theories of when and how critical operative social processes shape dynamic HMI.
Challenge 4: From Dyads to Groups
Another challenge is to broaden the research focus on studying HMIs ranging from dyads over groups to cultures and their underpinning neural mechanisms to understand better how these psychoneurobiological processes contribute to more beneficial HMIs. On the one hand, dyads are different from groups: dyads can both form and dissolve faster, dyads experience stronger and sometimes different emotions than people in groups, and dyads are less complex than groups as they lack group-specific manifestations (e.g., socialization, coalition formation, and majority/minority influence) (Moreland, 2010). Robots are most commonly expected to assist groups of people, such as in public places (e.g., shopping malls, airports, museums) (Burgard et al., 1999). However, to date, little is known about the environmental and psychological (i.e., motivational, affective, and cognitive) factors that foster social integration, group formation, and interaction within mixed human-robot groups—as current research primarily focuses on dyadic interactions with social agents. On the other hand, we often assume that our research findings are universal, but comparative studies report differences, for example, between industrialized and small-scale societies and Western and non-Western cultures. However, most participants in neuroergonomic studies are from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) populations; therefore, the question of the representativeness of those results should be kept in mind (Falk et al., 2013). For example, various cultures exhibit substantial disparities in cognitive (e.g., perception, categorization, deduction) and social (e.g., fairness, cooperation, morality) mechanisms (Henrich et al., 2010). Furthermore, differences in social interactions due to dyadic, group, and cultural variations evoke differential psychological processes that influence neural processing, and recent neuroimaging experiments have highlighted the neural networks underlying those variances (Ames and Fiske, 2010). Therefore, the next phase of SNE research should focus on expanding neuroergonomic research to groups (besides dyads) and diverse cultural populations (besides WEIRD populations) (Kedia et al., 2017)—studying the neurophysiological correlates of human-machine/human-AI teaming in comparision to human-human teaming by utilizing hyperscanning techniques.
Challenge 5: From Laboratory to Natural Environments
A final challenge for SNE is implementing a systematic experimental approach to develop interactive human-robot paradigms and procedures by starting in the laboratory and adjusting them for field and finally natural environments combining psychobehavioral with neurobiological measures. Laboratory experiments control independent variables under highly standardized conditions; field experiments are performed in an everyday environment but still manipulate independent variables; whereas natural experiments are conducted with no control over independent variables as they occur naturally in real life. Applying concurrent neuroimaging techniques—combining functional magnetic resonance imaging (fMRI) with functional near-infrared spectroscopy (fNIRS) or electroencephalography (EEG) to optimize spatial and temporal resolution—while comparing dynamic HMI with human-human interactions under laboratory conditions will enable us to identify target regions within social networks for subsequent field and natural experiments. Most neuroscience methods such as fMRI impose heavy constraints on and exhaust participants (e.g., constrainment of movements, awareness of being observed, multiplicity of trials within the same task) that results in biases of psychological processes and prevent them from reacting naturally as in real-world situations (Kedia et al., 2017). Identified regions within necessary social networks via fMRI can be later targeted with mobile and portable functional neuroimaging tools (e.g., fNIRS, EEG) in field or natural settings to explore mutual moment-to-moment dynamic HMI in more ecological environments (e.g., development of HR relationships and attachment, Krueger et al., 2021). These methodological developments should be combined with a wider use and creation of analysis methods (e.g., computational methods) to capture inter-individual variations at the structural and functional brain level during real-world HMI. SNE should concentrate on the problem of the generalizability and reproducibility of research findings—including under-powered neuroimaging experiments, dependency on flexible statistical analyses, and post-hoc modification of hypotheses—which can lead to a distorted characterization of brain activity. Such challenges have to be addressed to avoid a replication crisis and before making any claim regarding practical applications. Moreover, to implement HMI studies “in the wild,” researchers need to identify social robot platforms that are easy-to-use, affordable, programmable, and AI-compatible (e.g., Cozmo) and explore their utility for the empirical investigation of HMI (including code-sharing on open science platforms or platforms like Github) in everyday environments. In the same vein, SNE should also explore the utility of neurophysiological measurement tools that are user-friendly and can be used by non-scientists at home (e.g., Muse headband).
Conclusion
While SNE has started to illuminate the behavioral, psychological, and neurobiological processes of HMIs, this newly emerging research area is still at its inception. However, the identified five challenges—development from automatic to autonomous systems, the progress from observation to interaction, the expansion from extrinsic to intrinsic measures, the extension from dyads to groups, and the move from the laboratory to natural experiments—will allow both conceptually and methodologically ground-breaking insights that point to a bright future for this field. With the overlapping and similarly rapidly developing sister fields of neuroergonomics and other related fields, SNE holds a great deal of promise in discovering the neurobiological processes that underlie people's emotions, feelings, and behaviors to enhance current and future HMIs, especially in daily life and natural environments. We hope that the outlined grand challenges will lead neuroscientists, psychologists, and engineers to work together more consistently, with a shared assumption that the comprehension of HMIs will be improved by an integrative analysis that incorporates levels of investigations spanning from genes over brains to behaviors and cultures. Moreover, we expect that SNE will attract the attention of researchers, professionals, and the public, who are increasingly building on SNE's insights and methodologies to explore HMIs successfully in the future. Eventually, understanding HMIs will help us to understand ourselves better.
Author Contributions
FK and EW handled the entirety of this paper.
Funding
The work by FK was supported by a MINERVA Grant (FA9550-18-1–0455).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Ames, D. L., and Fiske, S. T. (2010). Cultural Neuroscience. Asian J. Soc. Psychol. 13, 72–82. doi: 10.1111/j.1467-839X.2010.01301.x
Ayaz, H., and Dehais, F. (eds). (2019). Neuroergonomics: the Brain at Work and in Everyday Life. London: Academic Press.
Burgard, W., Cremers, A., Fox, D., Hähnel, D., Lakemeyer, G., Schulz, D., et al. (1999). Experiences with an interactive museum tour-guide robot. Artificial Intelligence 114, 3–55. doi: 10.1016/S0004-3702(99)00070-3
Dautenhahn, K., Ogden, B., and Quick, T. (2002). From embodied to socially embedded agents – Implications for interaction-aware robots. Cognit. Syst. Res. 3, 397–428. doi: 10.1016/S1389-0417(02)00050-5
Falk, E. B., Hyde, L. W., Mitchell, C., Faul, J., Gonzalez, R., Heitzeg, M. M., et al. (2013). What is a representative brain? Neuroscience meets population science. Proc. Natl. Acad. Sci. U.S.A. 110, 17615–17622. doi: 10.1073/pnas.1310134110
Fong, T., Nourbakhsh, I., and Dautenhahn, K. (2003). A survey of socially interactive robots. Robotics Autonomous Syst. 42, 143–166. doi: 10.1016/S0921-8890(02)00372-X
Forbes, C. E., and Grafman, J. (2013). Social neuroscience: the second phase. Front. Hum. Neurosci. 7:20. doi: 10.3389/fnhum.2013.00020
Frith, C. D., and Frith, U. (2008). Implicit and explicit processes in social cognition. Neuron 60, 503–510. doi: 10.1016/j.neuron.2008.10.032
Fujitam, M. (2004). On activating human communications with pet-type robot AIBO. Proc. IEEE 92, 1804–1813. doi: 10.1109/JPROC.2004.835364
Gray, H. M., Gray, K., and Wegner, D. M. (2007). Dimensions of mind perception. Science 315:619. doi: 10.1126/science.1134475
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y., De Visser, E. J., and Parasuraman, R. (2011). A meta-analysis of factors affecting trust in human-robot interaction. Hum. Factors 53, 517–527. doi: 10.1177/0018720811417254
Hari, R., and Kujala, M. V. (2009). Brain basis of human social interaction: from concepts to brain imaging. Physiol. Rev. 89, 453–479. doi: 10.1152/physrev.00041.2007
Henrich, J., Heine, S. J., and Norenzayan, A. (2010). The weirdest people in the world? Behav Brain Sci 33, 61–83; discussion 83–135. doi: 10.1017/S0140525X0999152X
Kedia, G., Harris, L., Lelieveld, G. J., and Van Dillen, L. (2017). From the brain to the field: the applications of social neuroscience to economics, health and law. Brain Sci. 7:94. doi: 10.3390/brainsci7080094
Krueger, F., Mitchell, K., Katz, J., and Deshpande, G. (2021). Human-dog relationships as a working framework for exploring human-robot attachment: a multidisciplinary review. Anim. Cognit. doi: 10.1007/s10071-021-01472-w
Merritt, S. M., Heimbaugh, H., Lachapell, J., and Lee, D. (2013). I trust it, but I don'tknow why: effects of implicit attitudes toward automation on trust in an automated system. Hum. Fact. 55, 520–534. doi: 10.1177/0018720812465081
Moreland, R. (2010). Are dyads really groups? Small Group Res. 41, 251–267. doi: 10.1177/1046496409358618
Parasuraman, R. (2011). Neuroergonomics: brain, cognition, and performance at work. Curr. Direct. Psychol. Sci. 20, 181–186. doi: 10.1177/0963721411409176
Pfeiffer, U. J., Timmermans, B., Vogeley, K., Frith, C. D., and Schilbach, L. (2013). Towards a neuroscience of social interaction. Front Hum. Neurosci. 7:22. doi: 10.3389/fnhum.2013.00022
Redcay, E., and Schilbach, L. (2019). Using second-person neuroscience to elucidate the mechanisms of social interaction. Nat. Rev. Neurosci. 20, 495–505. doi: 10.1038/s41583-019-0179-4
Sanders, T. L., Schafer, K., Volante, W., Reardon, A., and Hancock, P. (2016). “Implicit attitudes toward robots,” in Proceedings of the Human Factors and Ergonomics Society 2016 Annual Meeting. Washington, DC. doi: 10.1177/1541931213601400
Schaefer, K. E., Chen, J. Y., Szalma, J. L., and Hancock, P. A. (2016). A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Hum. Factors 58, 377–400. doi: 10.1177/0018720816634228
Schilbach, L., Timmermans, B., Reddy, V., Costall, A., Bente, G., Schlicht, T., et al. (2013). Toward a second-person neuroscience. Behav. Brain Sci. 36, 393–414. doi: 10.1017/S0140525X12000660
Waytz, A., Gray, K., Epley, N., and Wegner, D. M. (2010). Causes and consequences of mind perception. Trends Cogn. Sci. 14, 383–388. doi: 10.1016/j.tics.2010.05.006
Wiese, E., Metta, G., and Wykowska, A. (2017). Robots as intentional agents: using neuroscientific methods to make robots appear more social. Front. Psychol. 8:1663. doi: 10.3389/fpsyg.2017.01663
Keywords: from automation to autonomy, from observation to interaction, from explicit to implicit measures, from dyads to groups, from laboratory to natural environments
Citation: Krueger F and Wiese E (2021) Specialty Grand Challenge Article- Social Neuroergonomics. Front. Neuroergon. 2:654597. doi: 10.3389/fnrgo.2021.654597
Received: 16 January 2021; Accepted: 15 March 2021;
Published: 07 April 2021.
Edited by:
Frederic Dehais, Institut Supérieur de l'Aéronautique et de l'Espace (ISAE-SUPAERO), FranceReviewed by:
Riccardo Poli, University of Essex, United KingdomCopyright © 2021 Krueger and Wiese. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Frank Krueger, fkrueger@gmu.edu