M. Ege Cansev
Alexandra J. Miller
Jeremy D. Brown
Philipp Beckerle- 1Chair of Autonomous Systems and Mechatronics, Department of Electrical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
- 2Haptics and Medical Robotics Laboratory, Johns Hopkins University, Department of Mechanical Engineering, Baltimore, MD, United States
- 3Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
In this paper, we discuss the potential contribution of affective touch to the user experience and robot performance in human-robot interaction, with an in-depth look into upper-limb prosthesis use as a well-suited example. Research on providing haptic feedback in human-robot interaction has worked to relay discriminative information during functional activities of daily living, like grasping a cup of tea. However, this approach neglects to recognize the affective information our bodies give and receive during social activities of daily living, like shaking hands. The discussion covers the emotional dimensions of affective touch and its role in conveying distinct emotions. In this work, we provide a human needs-centered approach to human-robot interaction design and argue for an equal emphasis to be placed on providing affective haptic feedback channels to meet the social tactile needs and interactions of human agents. We suggest incorporating affective touch to enhance user experience when interacting with and through semi-autonomous systems such as prosthetic limbs, particularly in fostering trust. Real-time analysis of trust as a dynamic phenomenon can pave the way towards adaptive shared autonomy strategies and consequently enhance the acceptance of prosthetic limbs. Here we highlight certain feasibility considerations, emphasizing practical designs and multi-sensory approaches for the effective implementation of affective touch interfaces.
1 Introduction
Touch is vital for a human to live a healthy life (Field, 2014). According to Maslow’s Hierarchy of Needs, to lead a healthy life, one must have physiological health, safety and security, love and belonging, self-esteem, and self-actualization (Maslow, 1943). Many of these are emotional needs, and as sentient beings, we must account for our emotions in human-robot interaction. One way emotions are influenced is through affective and social touch (McGlone et al., 2014; Schirmer et al., 2023). The ideal human-robot interaction scenario should consider this and provide avenues for haptic communication.
In addition to claiming that human-human interactions benefit from the affective components of touch, we also envision that both the user and the robot can benefit from affective communication during human-robot interaction. On one hand, affective touch can trigger a wide range of emotional responses, including pleasantness through gentle touch (Essick et al., 1999), embodiment of artificial limbs (Crucianelli et al., 2013), and calming effects under stressful conditions (Morrison, 2016). On the other hand, certain psychological factors, such as trust and cognitive load, significantly shape the nature of interaction with a (semi-)autonomous system (Lee and See, 2004; Hancock et al., 2011). At this point, we seek to answer the question: “Can we propose a conceptual method to incorporate affective touch into human-robot interaction to improve the user experience and accordingly enhance the nature and efficiency of the interaction?”
In the unique case where the robot becomes a part of the human body, as in an upper-limb prosthesis, social touch is inherently intertwined in the interaction. Affective communication could be included to improve the experience of using an upper-limb prosthesis because our hands are more than just functional tools. They are mediums through which social connection occurs. Thus, people who wear prostheses and do not experience the same tactile sensations as those who do not, must be included in our research on social touch interactions. Despite significant improvements in prosthesis technology, 44% of wearers still reject their clinical devices (Salminger et al., 2022). At the same time, it has been shown that those wearers who keep their devices largely use them for non-prehensile (grasping) manipulations (Spiers et al., 2021). In addition to excessive prosthesis weight (Biddiss et al., 2007), improper socket fit (Daly et al., 2014), and unreliable function (Salminger et al., 2022), researchers have suggested that a lack of haptic feedback has contributed to prosthesis rejection and disuse (Niedernhuber et al., 2018; Stephens-Fripp et al., 2018). To remedy this, research has explored both invasive and non-invasive approaches to relaying discriminative haptic feedback to aid prosthesis users in functional task execution (Bensmaia et al., 2020; Sensinger and Dosen, 2020). For example, researchers have demonstrated that vibrotactile feedback of grip force combined with autonomous grasp control leads to improved task execution and lower mental effort (Thomas et al., 2023a; b). Likewise, researchers have demonstrated that invasive peripheral nerve feedback of grasp pressure leads to both functional and psychosocial improvements for prosthesis wearers (Chee et al., 2022). More recently, researchers have demonstrated that providing non-invasive thermal feedback to prosthesis wearers also provides functional utility (Iberite et al., 2023; Osborn et al., 2023). While these advances have significantly advanced our understanding of the importance of haptic feedback in dexterous task execution, our understanding of their utility in social interactions, in particular those involving affective touch are not well understood.
Figure 1 provides an overview of one of the potential applications of social touch in an exemplary prosthesis handshaking scenario. In later sections, we use the terms social touch and affective touch. While affective touch primarily aims to elicit emotional responses and foster bonding and comfort with a stronger emphasis on its neurophysiological background, social touch serves a broader range of social functions and conveys messages related to social interactions and relationships (Cascio et al., 2019). Given the critical roles both types of touch play in human communication, bonding, and wellbeing, existing literature approaches these concepts from distinct perspectives. We intentionally use both terms in their respective contexts to differentiate between broad social touch interactions, potentially experienced by prosthesis users, and affective touch that describes touch that stimulates CT afferents.
Figure 1. An overview of the presented perspectives exemplified in a handshake scenario involving an individual with a prosthetic arm. In this setup, the prosthetic hand is equipped with sensorized skin to detect social touch. The haptic interface, seamlessly integrated into the socket of the prosthetic arm, then transmits the detected touch to the residual limb of the prosthesis wearer. The wearer’s emotional state may undergo positive changes during social touch. Additionally, the haptic interface can autonomously mediate this touch, stepping in to restore trust when the wearer’s confidence in the prosthesis diminishes, in this or other appropriate scenarios. This approach holds the potential to significantly enhance both user experience and prosthesis performance, especially when integrated into shared autonomy strategies.
2 Affective touch, trust, and shared autonomy
Considering affective touch as a haptic communication channel, which has been previously explored primarily in its hedonic aspects, may contribute significantly to fostering trust between collaborating agents engaged even in tasks defined solely by functionality. In semi-autonomous systems, trust is a key psychological factor, influencing user interactions and task performance (Hancock et al., 2011). Hence, we inquire whether trust can serve as a link that improves interactions with semi-autonomous systems through the incorporation of affective touch. We posit that establishing such a connection during human-robot interaction is feasible, especially if wearable haptic interfaces, such as prosthetic arms, can autonomously convey affective touch or realistically transfer emotional cues from another individual.
2.1 The emotional impact of affective touch
Humans have multiple means to express their emotions, verbally through words and tone of voice or non-verbally through gestures, facial expressions and touch. Hertenstein et al. (2009) showed that eight distinct emotions, i.e., anger, fear, disgust, sadness, happiness, love, gratitude, and sympathy, can be communicated via touch. Although affective touch relates to the tactile communication of any emotional and social information (McGlone et al., 2014), pleasantness is one of the strongly elicited emotions that is linked to affective touch (Essick et al., 1999; Ackerley et al., 2014; Pawling et al., 2017). Due to this link, it is even mentioned as pleasant touch by Löken et al. (2009); Essick et al. (2010). Beyond being perceived as pleasant, affective touch enhanced perceived embodiment of a rubber hand (Crucianelli et al., 2013; Van Stralen et al., 2014) in variations of the classical rubber hand illusion experiment (Botvinick and Cohen, 1998). From a neurological perspective, Morrison (2016) claims that affective touch acts as a stress buffer, i.e., calibrates the stress responses of the body. Affective touch also reduces anxiety levels and autonomic responses, e.g., skin conductance and heart rate variability, which indicates a calming effect, when administered to a partner (Mazza et al., 2023). Therefore, including affective touch in human-robot interaction may be beneficial to the human’s experience.
2.2 Human-centered perspective on shared autonomy
Recently, semi-autonomous systems have been playing an important role in daily and professional life. They serve as a bridge towards fully autonomous systems, aiming to alleviate both physical and cognitive burdens on humans. Besides, semi-autonomous systems can be more favorable than autonomous systems in certain scenarios. Despite significant progress in machine learning and artificial intelligence, human expertise remains indispensable for complex tasks like surgical procedures involving medical robots, potentially enhancing patient trust Moustris et al. (2011). Moreover, human-in-the-loop systems offer flexibility and adaptability, crucial in dynamic and uncertain environments, such as semi-autonomous vehicles navigating through traffic congestion (Trautman et al., 2015; Schwarting et al., 2018).
Human-in-the-loop shared control approaches can also be useful in upper-limb prosthetics given the high dexterity required. Two main challenges in control of upper-limb prostheses are high cognitive burden of prosthesis use (Thomas et al., 2023b) and unnatural grasp movements that are incompatible with the motion of the intact limb (Guo et al., 2023). They reviewed various semi-autonomous control strategies that can reduce the workload of users and nonhuman behavior of prosthesis joints.
We claim that semi-autonomous systems, capable of adjusting their behavior while considering user experience, can enhance both performance and overall user satisfaction. Specifically, we propose the use of affective touch as a means to elevate the user experience during human-robot interaction. Achieving this goal requires a thorough understanding of the psychological factors influencing shared autonomy.
2.3 Psychological factors shaping shared autonomy
Enhancing the quality of interaction, whether it involves human-human, human-computer, or human-robot interaction, requires thoughtful consideration of the psychological states and needs of individuals involved (Cansev et al., 2021). The study of user experience (UX) in interactions with intelligent systems has gained attention in recent years (Shneiderman et al., 2017; Beckerle et al., 2018; Alenljung et al., 2019; Torricelli et al., 2020). The primary objective is to apprehend how technology can enhance the overall experience for its users. The subsequent challenge is considering the human factors as design criteria for interactive systems (Prati et al., 2021; Hao et al., 2022). By combining these approaches, developing human-centered designs will lead to enhanced UX, such as, enhanced embodiment of interactive systems (Schofield et al., 2021), which in turn fosters the research and advancements on human-centered designs including semi-autonomous systems.
Trust of a user on a semi-autonomous system is one of the most frequently considered psychological factors in discussions about shared autonomy. According to Lee and See (2004), trust is a multidimensional concept influenced by analytic, analogical, and affective interpretations of automated systems. They posit that emotions play a pivotal role, as affective processes significantly influence both analytic and analogical responses, emphasizing that trust is not only a cognitive consideration but also an emotional experience (Fine and Holyfield, 1996; Lee and See, 2004). At this point, we consider the potential impact of affective touch on eliciting emotional responses for trust-building in (semi-)autonomous systems while acknowledging the contribution of task-related analytic and analogical cognitive processes.
Analyzing and generalizing trust in automation is challenging as trust can fluctuate due to many human-related, robot-related, and environmental factors (Hancock et al., 2011; Schaefer et al., 2016). As shown in the review by Kohn et al. (2021), self-report measures along with behavioral measures are preferred over physiological measures (e.g., electrodermal activity, eye gaze tracking, and heart rate change and variability) in interpersonal trust analysis, because they are easy to validate and integrate into interaction. We believe incorporating physiological measures more often in trust analysis is important for avoiding biased results between subjective and objective measures as well as enabling real-time trust analysis. We agree with Hancock et al. (2011) that potential discrepancies between individual’s self-report and behavior can only be monitored when subjective measures are combined with objective measures. We posit that objective physiological measurements must be considered to accurately picture the state of trust. Additionally, we argue that real-time measurement of trust appears essential to create shared autonomy strategies that can adapt the behavior of semi-autonomous systems based on the trust of the user. To the best of our knowledge, the self-reported trust measurement with the most real-time capability assesses trust every 25 s via gamepad buttons (Desai et al., 2013). Such a measurement frequency would not suffice for responsive interaction in dynamically changing conditions. Among physiological measurements, both electrodermal activity and heart rate positively correlated with stress, anxiety, and cognitive workload (Caplan and Jones, 1975; Payne and Rick, 1986; Jacobs et al., 1994), emerge as valuable indicators of trust (Waytz et al., 2014; Akash et al., 2018). Since affective touch has soothing effects, e.g., acting as a stress buffer (Morrison, 2016) and reducing anxiety levels and autonomic responses under certain conditions (Mazza et al., 2023), as explained in Section 1, we believe that it can improve the trust of a person on a semi-autonomous system. For example, a prosthetic arm can autonomously mediate affective touch on the residual limb to soothe the wearer when trust on the prosthesis is decreased. This is why the real-time and accurate estimation of trust is important.
While trust is highlighted as the principal motivator and modulator of shared autonomy in this paper, we anticipate that agency, i.e., the feeling of being in control of an object’s movements (Braun et al., 2018), constitutes another psychological factor, aligning with the task-related analytic cognitive processes as defined by Lee and See (2004), in our perspective. Crucianelli et al. (2013) demonstrated that affective, slow stimuli not only enhances perceived ownership, but also improves the sense of agency, a sub-factor of embodiment (Schettler et al., 2019). The roles of other affect-related factors during semi-autonomous interaction is still to be investigated.
3 Social touch for upper-limb prosthetic users
We believe prosthesis wearers should experience the richness and benefits of affective touch during tactile interactions. Holding hands with a partner or spouse regulated neural and physiological responses to receiving a small electrical shock (Coan et al., 2006). Among couples who were separated by distance, a haptic bracelet that gave a squeeze on the wrist enhanced their feelings of social connection (van Hattum et al., 2022). Verbal and visual cues of affection could still be expressed, but the inclusion of touch made a significant difference in promoting emotional wellbeing. This section discusses the need to understand social interactions involving prostheses and the potential benefits of providing social haptic feedback to upper-limb prosthetics users.
3.1 Studying the behavioral impact of social touch: the Midas touch
Social touch can induce changes in human behavior to an extent measurable by researchers in “real-world” scenarios. These changes in behavior are referred to as the Midas Touch Effect. Brief, light touches on the hand and/or arm compared to no tactile contact during social exchanges resulted in people tipping more at a restaurant (Crusco and Wetzel, 1984), being more likely to return and lend money (Kleinke, 1977), and spending more time and money inside a store (Hornik, 2014). Social touch not only affects human behavior in an altruistic manner, but it also impacts one’s attitude toward and evaluation of the toucher or environment. Students who were touched on the hand by library clerks when returning the library cards rated the library personnel and facilities more favorably than students who did not receive a touch (Fisher et al., 1976). People who received a light touch from a store employee while shopping or from a waiter in a restaurant rated the environments as more friendly than those who were not (Hornik, 2014). Social touch increased people’s willingness to participate in mall interviews (Hornik and Ellis, 1988). In a classroom setting, students were given the opportunity to write the solution to a statistical problem on the board. During the exercise, the teacher briefly touched a handful of students on the forearm. Results showed that being touched correlated with increased volunteering among the students (Guéguen, 2004). Additionally, bus drivers were more inclined to give passengers a free ride when they were touched on the arm while being asked the question (Guéguen and Fischer-Lokou, 2003). Finally, researchers have shown that people are more willing to give strangers a free cigarette if they were touched together with the request (Joule and Guéguen, 2007). Social touch is understated but plays an important role in our decision making and social behavior.
Current clinical prosthetics provide no cutaneous haptic feedback to the wearer, thus reducing the tactile interaction to a visual or purely kinesthetic stimulus: the person will see the touch or feel the force via the prosthetic socket on the residual limb from the touch. This begs the question, “What happens when one’s sense of touch is missing, such as for an upper-limb prosthetic user? Will the Midas Touch Effect remain if a person receives a touch on a prosthetic limb?” We hypothesize that prosthesis wearers do not experience it in the same manner, but an emotional change may still occur. Therefore, we advocate for researchers to investigate the social haptic needs of upper-limb prosthetics wearers. This supports placing equal emphasis on improving social touch experiences for prosthetics users and can enhance prosthesis acceptance and improve quality of life.
3.2 Studying social touch in upper-limb prostheses
We suggest that researchers investigate if and to what extent social touch enhances upper-limb prosthetics user experience. This may include distributing questionnaires that ask what their social touch experiences are in everyday life. Furthermore, user-studies that explore prosthesis wearer reaction in social touch scenarios, both in a lab and real-world setting, could be conducted. The Trier Social Stress Test (Kirschbaum et al., 1993), and other laboratory psychological experimental paradigms (McCarthy and Elson, 2018) have been used for decades to uncover underlying emotional experiences. For upper-limb prosthetics, the Midas Touch paradigm provides an appropriate framework for this, as previous studies without prostheses have been able to obtain quantitative data describing the effects of social touch.
To relay affective touch to the user, appropriate hardware must be used such as touch sensors on the prosthesis that signal haptic feedback displays to provide the cue to the wearer. Researchers have developed sensory skins specifically for prosthetics (Chortos et al., 2016; Yang et al., 2019; Chen et al., 2021), and sensors for social touch mediation (Eid and Al Osman, 2016; Huisman, 2017). Haptics researchers have created haptic displays that can relay social touch to the wearer (Huisman et al., 2013; Casini et al., 2015; Culbertson et al., 2018), which could be modified for prosthesis users. Furthermore, thermal displays can be incorporated to provide the realistic warmth felt during skin-to-skin contact in social touch studies (Iberite et al., 2023; Muheim et al., 2024).
To address the emotional and social aspects of the multi-faceted field of human-robot interaction, especially in prosthetics research, we suggest taking a holistic human approach and setting design requirements based on human needs. Maslow’s hierarchy of human needs involves social and emotional wellbeing, which are met through social haptic interaction. Recognizing social and psychological needs of a person, in addition to the physical functional requirements needed to lead a healthy life, will shift human-robot interaction research from a narrow focus on device advancement to the overall promotion of human wellbeing.
4 Operational considerations
Our perspectives are articulated primarily through insights derived from social affective touch. As shown in Figure 2, a variety of interfaces with different sensing and stimulation options, such as vibrotactile (Kindel et al., 2024), pneumatic (van Beek et al., 2023), and linear actuation (Ferguson et al., 2022) as well as tactile-sensitive skins (Massari et al., 2022), can be utilized to communicate social affective touch. Nonetheless, the current capabilities of interfaces designed to facilitate affective touch prompt the question of whether these interfaces can convincingly replicate human touch. The perception of affective touch may vary based on the chosen stimulation modality and conditions, with the possibility of it being perceived as unpleasant or even unsettling (Culbertson et al., 2018). Consequently, adherence to design guidelines is imperative to prevent any discomfort for prosthesis wearers. Additionally, the interfaces need to be wearable and sufficiently compact for seamless integration into the prosthesis socket.
Figure 2. Examples of haptic interfaces for social affective touch applications. Top left: A 2D array of linear actuators creating indentations on the skin. Top right: A linear array of pneumatic stimulators. Bottom left: A design update on the array of linear actuators combined with sensorized skin. Bottom right: A vibrotactile bracelet with an inertial measurement unit for bi-directional haptic interaction.
Incorporating an additional haptic channel for affective interaction inevitably raises concerns about increased electronics, costs, design complexity, computational load, and further technical challenges due to the requirements of affective touch, potentially leading to performance degradation (Ege Cansev et al., 2021). We believe multisensory integration, which is described as the collaboration of sensory modalities and the integration of their informational content by Stein and Stanford (2008), via pseudo-haptic feedback, i.e., conveying haptic information just based on visual feedback (Lécuyer et al., 2000) can mitigate these potential issues. Pseudo-haptic feedback is used to represent multiple discriminative haptic properties, such as weight (Jauregui et al., 2014), stiffness (Lécuyer et al., 2001), friction and roughness (Costes et al., 2019). While implementing this solution in a virtual reality environment may not be suitable for scenarios such as a handshake via a prosthesis, the use of augmented reality to generate pseudo-haptic feedback can, at least partially, facilitate the communication of functional haptic information. This approach allows for the conservation of haptic resources, ensuring their availability for affective haptic exchanges.
5 Conclusion
We explored how affective touch can enrich the user experience in human-robot interaction, particularly focusing on upper-limb prosthetics and the impact of social touch. We advocate for further research into the broader implications of social touch in human-robot interaction and the potential benefits of integrating affective communication channels. By evoking positive emotions, affective touch can enhance user experience and foster trust, thereby improving collaborative tasks. Affective touch holds promise for influencing trust in shared autonomy scenarios, urging researchers to delve into how individuals using upper-limb prosthetics perceive affective touch and its effects on device usage. Wearable haptic interfaces capable of autonomously conveying affective touch could revolutionize human-robot interaction, facilitating more meaningful and emotionally connected interactions.
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
MC: Conceptualization, Writing–original draft, Writing–review and editing. AM: Conceptualization, Writing–original draft, Writing–review and editing. JB: Supervision, Writing–review and editing. PB: Supervision, Writing–review and editing.
Funding
The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work received support from the Mercator Research Center Ruhr (MERCUR) (Grant Number: An-2019-0032).
Acknowledgments
We acknowledge the support of the Dynamics and Control Group at the Department of Mechanical Engineering of Eindhoven University of Technology, Eindhoven, Netherlands, for supplying the pneumatic stimulator, and the support of Neuro-Robotic Touch Lab of the BioRobotics Institute of Sant’Anna School of Advanced Studies, Pisa, Italy, for supplying artificial skin in the framework of a scientific collaboration.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Ackerley, R., Wasling, H. B., Liljencrantz, J., Olausson, H., Johnson, R. D., and Wessberg, J. (2014). Human c-tactile afferents are tuned to the temperature of a skin-stroking caress. J. Neurosci. 34, 2879–2883. doi:10.1523/jneurosci.2847-13.2014
Akash, K., Hu, W.-L., Jain, N., and Reid, T. (2018). A classification model for sensing human trust in machines using eeg and gsr. ACM Trans. Interact. Intelligent Syst. (TiiS) 8, 1–20. doi:10.1145/3132743
Alenljung, B., Lindblom, J., Andreasson, R., and Ziemke, T. (2019). “User experience in social human-robot interaction,” in Rapid automation: concepts, methodologies, tools, and applications (IGI Global), 1468–1490.
Beckerle, P., Kõiva, R., Kirchner, E. A., Bekrater-Bodmann, R., Dosen, S., Christ, O., et al. (2018). Feel-good robotics: requirements on touch for embodiment in assistive robotics. Front. neurorobotics 12, 84. doi:10.3389/fnbot.2018.00084
Bensmaia, S. J., Tyler, D. J., and Micera, S. (2020). Restoration of sensory information via bionic hands. Nat. Biomed. Eng. 7 (4), 443–455. doi:10.1038/s41551-020-00630-8
Biddiss, E., Beaton, D., and Chau, T. (2007). Consumer design priorities for upper limb prosthetics. Disabil. Rehabilitation Assistive Technol. 2, 346–357. doi:10.1080/17483100701714733
Botvinick, M., and Cohen, J. (1998). Rubber hands ‘feel’touch that eyes see. Nature 391, 756. doi:10.1038/35784
Braun, N., Debener, S., Spychala, N., Bongartz, E., Sörös, P., Müller, H. H., et al. (2018). The senses of agency and ownership: a review. Front. Psychol. 9, 535. doi:10.3389/fpsyg.2018.00535
Cansev, M. E., Xue, H., Rottmann, N., Bliek, A., Miller, L. E., Rueckert, E., et al. (2021). Interactive human–robot skill transfer: a review of learning methods and user experience. Adv. Intell. Syst. 3, 2000247. doi:10.1002/aisy.202000247
Caplan, R. D., and Jones, K. W. (1975). Effects of work load, role ambiguity, and type a personality on anxiety, depression, and heart rate. J. Appl. Psychol. 60, 713–719. doi:10.1037//0021-9010.60.6.713
Cascio, C. J., Moore, D., and McGlone, F. (2019). Social touch and human development. Dev. Cogn. Neurosci. 35, 5–11. doi:10.1016/j.dcn.2018.04.009
Casini, S., Morvidoni, M., Bianchi, M., Catalano, M., Grioli, G., and Bicchi, A. (2015). Design and realization of the CUFF - clenching upper-limb force feedback wearable device for distributed mechano-tactile stimulation of normal and tangential skin forces. IEEE Int. Conf. Intelligent Robots Syst. 2015-Decem, 1186–1193. doi:10.1109/IROS.2015.7353520
Chee, L., Valle, G., Preatoni, G., Basla, C., Marazzi, M., and Raspopovic, S. (2022). Cognitive benefits of using non-invasive compared to implantable neural feedback. Sci. Rep. 12, 16696. doi:10.1038/s41598-022-21057-y
Chen, H., Dejace, L., and Lacour, S. P. (2021). Electronic skins for healthcare monitoring and smart prostheses. Robotics, Aut. Syst. Annu. Rev. Control Robot. Aut. Syst 4, 629–650. doi:10.1146/annurev-control-071320-101023
Chortos, A., Liu, J., and Bao, Z. (2016). Pursuing prosthetic electronic skin. Nat. Mater. 15, 937–950. doi:10.1038/nmat4671
Coan, J. A., Schaefer, H. S., and Davidson, R. J. (2006). Lending a hand social regulation of the neural response to threat. Tech. Rep. 17, 1032–1039. doi:10.1111/j.1467-9280.2006.01832.x
Costes, A., Argelaguet, F., Danieau, F., Guillotel, P., and Lécuyer, A. (2019). Touchy: a visual approach for simulating haptic effects on touchscreens. Front. ICT 6, 1. doi:10.3389/fict.2019.00001
Crucianelli, L., Metcalf, N. K., Fotopoulou, A., and Jenkinson, P. M. (2013). Bodily pleasure matters: velocity of touch modulates body ownership during the rubber hand illusion. Front. Psychol. 4, 703. doi:10.3389/fpsyg.2013.00703
Crusco, A. H., and Wetzel, C. G. (1984). The Midas touch: the effects of interpersonal touch on restaurant tipping. Personality Soc. Psychol. Bull. 10, 512–517. doi:10.1177/0146167284104003
Culbertson, H., Nunez, C. M., Israr, A., Lau, F., Abnousi, F., and Okamura, A. M. (2018). A social haptic device to create continuous lateral motion using sequential normal indentation. IEEE Haptics Symp. HAPTICS 2018-March, 32–39. doi:10.1109/HAPTICS.2018.8357149
Daly, W., Voo, L., Rosenbaum-Chou, T., Arabian, A., and Boone, D. (2014). Socket pressure and discomfort in upper-limb prostheses: a preliminary study. J. Prosthetics Orthot. 26, 99–106. doi:10.1097/JPO.0000000000000021
Desai, M., Kaniarasu, P., Medvedev, M., Steinfeld, A., and Yanco, H. (2013). “Impact of robot failures and feedback on real-time trust,” in 2013 8th ACM/IEEE international Conference on human-robot interaction (HRI) (IEEE), 251–258.
Ege Cansev, M., Nordheimer, D., Andrea Kirchner, E., and Beckerle, P. (2021). Feel-good requirements: neurophysiological and psychological design criteria of affective touch for (assistive) robots. Front. Neurorobotics 15, 661207. doi:10.3389/fnbot.2021.661207
Eid, M. A., and Al Osman, H. (2016). Affective haptics: current research and future directions. IEEE Access 4, 26–40. doi:10.1109/ACCESS.2015.2497316
Essick, G. K., James, A., and McGlone, F. P. (1999). Psychophysical assessment of the affective components of non-painful touch. Neuroreport 10, 2083–2087. doi:10.1097/00001756-199907130-00017
Essick, G. K., McGlone, F., Dancer, C., Fabricant, D., Ragin, Y., Phillips, N., et al. (2010). Quantitative assessment of pleasant touch. Neurosci. & Biobehav. Rev. 34, 192–203. doi:10.1016/j.neubiorev.2009.02.003
Ferguson, N., Cansev, M. E., Dwivedi, A., and Beckerle, P. (2022). “Design of a wearable haptic device to mediate affective touch with a matrix of linear actuators,” in International conference on system-integrated intelligence (Springer), 507–517.
Field, T. (2014). engTouch/tiffany field. (Cambridge, Massachusetts: The MIT Press), second edition. edn.
Fine, G. A., and Holyfield, L. (1996). Secrecy, trust, and dangerous leisure: generating group cohesion in voluntary organizations. Soc. Psychol. Q. 59, 22–38. doi:10.2307/2787117
Fisher, J. D., Rytting, M., and Heslin, R. (1976). Hands touching hands: affective and evaluative effects of an interpersonal touch. Sociometry 39, 416–421. doi:10.2307/3033506
Guéguen, N. (2004). Nonverbal encouragement of participation in a course: the effect of touching. Soc. Psychol. Educ. 7, 89–98. doi:10.1023/b:spoe.0000010691.30834.14
Guéguen, N., and Fischer-Lokou, J. (2003). Another evaluation of touch and helping behavior. Psychol. Rep. 92, 62–64. doi:10.2466/PR0.2003.92.1.62
Guo, W., Xu, W., Zhao, Y., Shi, X., Sheng, X., and Zhu, X. (2023). Toward human-in-the-loop shared control for upper-limb prostheses: a systematic analysis of state-of-the-art technologies. IEEE Trans. Med. Robotics Bionics 5, 563–579. doi:10.1109/tmrb.2023.3292419
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y., De Visser, E. J., and Parasuraman, R. (2011). A meta-analysis of factors affecting trust in human-robot interaction. Hum. factors 53, 517–527. doi:10.1177/0018720811417254
Hao, C., Dwivedi, A., and Beckerle, P. (2022). “A literature-based perspective on human-centered design and evaluation of interfaces for virtual reality in robotics,” in International workshop on human-friendly robotics (Springer), 1–13.
Hertenstein, M. J., Holmes, R., McCullough, M., and Keltner, D. (2009). The communication of emotion via touch. Emotion 9, 566–573. doi:10.1037/a0016108
Hornik, J., and Ellis, S. (1988). Strategies to secure compliance for a mall intercept interview. Public Opin. Q. 52, 539–551. doi:10.1086/269129
Huisman, G. (2017). Social touch technology: a survey of haptic technology for social touch. IEEE Trans. Haptics 10, 391–408. doi:10.1109/TOH.2017.2650221
Huisman, G., Darriba Frederiks, A., Van Dijk, B., Hevlen, D., and Krose, B. (2013) “The TaSSt: tactile sleeve for social touch,” in 2013 world haptics conference, WHC 2013, 211–216doi. doi:10.1109/WHC.2013.6548410
Iberite, F., Muheim, J., Akouissi, O., Gallo, S., Rognini, G., Morosato, F., et al. (2023). Restoration of natural thermal sensation in upper-limb amputees. Science 380, 731–735. doi:10.1126/science.adf6121
Jacobs, S. C., Friedman, R., Parker, J. D., Tofler, G. H., Jimenez, A. H., Muller, J. E., et al. (1994). Use of skin conductance changes during mental stress testing as an index of autonomic arousal in cardiovascular research. Am. heart J. 128, 1170–1177. doi:10.1016/0002-8703(94)90748-x
Jauregui, D. A. G., Argelaguet, F., Olivier, A.-H., Marchal, M., Multon, F., and Lecuyer, A. (2014). Toward pseudo-haptic avatars: modifying the visual animation of self-avatar can simulate the perception of weight lifting. IEEE Trans. Vis. Comput. Graph. 20, 654–661. doi:10.1109/tvcg.2014.45
Joule, R.-V., and Guéguen, N. (2007). Touch, compliance, and awareness of tactile contact. Percept. Mot. Skills 104, 581–588. doi:10.2466/pms.104.2.581-588
Kindel, J., Andreas, D., Hou, Z., Dwivedi, A., and Beckerle, P. (2024). A wearable bidirectional human–machine interface: merging motion capture and vibrotactile feedback in a wireless bracelet. Multimodal Technol. Interact. 8, 44. doi:10.3390/mti8060044
Kirschbaum, C., Pirke, K.-M., and Hellhammer, D. H. (1993). The ‘trier social stress test’ – a tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology 28, 76–81. doi:10.1159/000119004
Kleinke, C. L. (1977). Compliance to requests made by gazing and touching experimenters in field settings. Tech. Rep. 13, 218–223. doi:10.1016/0022-1031(77)90044-0
Kohn, S. C., de Visser, E. J., Wiese, E., Lee, Y.-C., and Shaw, T. H. (2021). Measurement of trust in automation: a narrative review and reference guide. Front. Psychol. 12, 604977. doi:10.3389/fpsyg.2021.604977
Lécuyer, A., Burkhardt, J.-M., Coquillart, S., and Coiffet, P. (2001). ““boundary of illusion”: an experiment of sensory integration with a pseudo-haptic system,” in Proceedings IEEE virtual reality 2001 (IEEE), 115–122.
Lécuyer, A., Coquillart, S., Kheddar, A., Richard, P., and Coiffet, P. (2000). “Pseudo-haptic feedback: can isometric input devices simulate force feedback?,” in Proceedings IEEE virtual reality 2000 (cat. No. 00CB37048) (IEEE), 83–90.
Lee, J. D., and See, K. A. (2004). Trust in automation: designing for appropriate reliance. Hum. factors 46, 50–80. doi:10.1518/hfes.46.1.50.30392
Löken, L. S., Wessberg, J., McGlone, F., and Olausson, H. (2009). Coding of pleasant touch by unmyelinated afferents in humans. Nat. Neurosci. 12, 547–548. doi:10.1038/nn.2312
Massari, L., Fransvea, G., D’Abbraccio, J., Filosa, M., Terruso, G., Aliperta, A., et al. (2022). Functional mimicry of ruffini receptors with fibre bragg gratings and deep neural networks enables a bio-inspired large-area tactile-sensitive skin. Nat. Mach. Intell. 4, 425–435. doi:10.1038/s42256-022-00487-3
Mazza, A., Cariola, M., Capiotto, F., Diano, M., Schintu, S., Pia, L., et al. (2023). Hedonic and autonomic responses in promoting affective touch. Sci. Rep. 13, 11201. doi:10.1038/s41598-023-37471-9
McCarthy, R. J., and Elson, M. (2018). A conceptual review of lab-based aggression paradigms. Collabra Psychol. 4. doi:10.1525/collabra.104
McGlone, F., Wessberg, J., and Olausson, H. (2014). Discriminative and affective touch: sensing and feeling. Neuron 82, 737–755. doi:10.1016/j.neuron.2014.05.001
Morrison, I. (2016). Keep calm and cuddle on: social touch as a stress buffer. Adapt. Hum. Behav. Physiology 2, 344–362. doi:10.1007/s40750-016-0052-x
Moustris, G. P., Hiridis, S. C., Deliparaschos, K. M., and Konstantinidis, K. M. (2011). Evolution of autonomous and semi-autonomous robotic surgical systems: a review of the literature. Int. J. Med. robotics Comput. assisted Surg. 7, 375–392. doi:10.1002/rcs.408
Muheim, J., Iberite, F., Akouissi, O., Monney, R., Morosato, F., Gruppioni, E., et al. (2024). A sensory-motor hand prosthesis with integrated thermal feedback. Med 5, 118–125.e5. doi:10.1016/J.MEDJ.2023.12.006
Niedernhuber, M., Barone, D. G., and Lenggenhager, B. (2018). Prostheses as extensions of the body: progress and challenges. Neurosci. & Biobehav. Rev. 92, 1–6. doi:10.1016/J.NEUBIOREV.2018.04.020
Osborn, L. E., Venkatasubramanian, R., Himmtann, M., Moran, C. W., Pierce, J. M., Gajendiran, P., et al. (2023). Evoking natural thermal perceptions using a thin-film thermoelectric device with high cooling power density and speed. Nat. Biomed. Eng., 1–14. doi:10.1038/s41551-023-01070-w
Pawling, R., Cannon, P. R., McGlone, F. P., and Walker, S. C. (2017). C-tactile afferent stimulating touch carries a positive affective value. PloS one 12, e0173457. doi:10.1371/journal.pone.0173457
Payne, R., and Rick, J. (1986). Heart rate as an indicator of stress in suegeons and anaesthetists. J. psychosomatic Res. 30, 411–420. doi:10.1016/0022-3999(86)90080-2
Prati, E., Peruzzini, M., Pellicciari, M., and Raffaeli, R. (2021). How to include user experience in the design of human-robot interaction. Robotics Computer-Integrated Manuf. 68, 102072. doi:10.1016/j.rcim.2020.102072
Salminger, S., Stino, H., Pichler, L. H., Gstoettner, C., Sturma, A., Mayer, J. A., et al. (2022). Current rates of prosthetic usage in upper-limb amputees–have innovations had an impact on device acceptance? Disabil. Rehabilitation 44, 3708–3713. doi:10.1080/09638288.2020.1866684
Schaefer, K. E., Chen, J. Y., Szalma, J. L., and Hancock, P. A. (2016). A meta-analysis of factors influencing the development of trust in automation: implications for understanding autonomy in future systems. Hum. factors 58, 377–400. doi:10.1177/0018720816634228
Schettler, A., Raja, V., and Anderson, M. L. (2019). The embodiment of objects: review, analysis, and future directions. Front. Neurosci. 13, 1332. doi:10.3389/fnins.2019.01332
Schirmer, A., Croy, I., and Ackerley, R. (2023). What are C-tactile afferents and how do they relate to “affective touch”. Neurosci. & Biobehav. Rev. 151, 105236. doi:10.1016/J.NEUBIOREV.2023.105236
Schofield, J. S., Battraw, M. A., Parker, A. S., Pilarski, P. M., Sensinger, J. W., and Marasco, P. D. (2021). Embodied cooperation to promote forgiving interactions with autonomous machines. Front. Neurorobotics 15, 661603. doi:10.3389/fnbot.2021.661603
Schwarting, W., Alonso-Mora, J., and Rus, D. (2018). Planning and decision-making for autonomous vehicles. Annu. Rev. Control, Robotics, Aut. Syst. 1, 187–210. doi:10.1146/annurev-control-060117-105157
Sensinger, J. W., and Dosen, S. (2020). A review of sensory feedback in upper-limb prostheses from the perspective of human motor control. Front. Neurosci. 14. doi:10.3389/fnins.2020.00345
Shneiderman, B., Cohen, M., Jacobs, S., Plaisant, C., Diakopoulos, N., and Elmqvist, N. (2017). Designing the user interface: strategies for effective human-computer interaction. Global Edn. Pearson Deutschland.
Spiers, A. J., Cochran, J., Resnik, L., and Dollar, A. M. (2021). Quantifying prosthetic and intact limb use in upper limb amputees via egocentric video: an unsupervised, at-home study. IEEE Trans. Med. Robotics Bionics 3, 463–484. doi:10.1109/TMRB.2021.3072253
Stein, B. E., and Stanford, T. R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9, 255–266. doi:10.1038/nrn2331
Stephens-Fripp, B., Alici, G., and Mutlu, R. (2018). A review of non-invasive sensory feedback methods for transradial prosthetic hands. IEEE Access 6, 6878–6899. doi:10.1109/ACCESS.2018.2791583
Thomas, N., Fazlollahi, F., Kuchenbecker, K. J., and Brown, J. D. (2023a). The utility of synthetic reflexes and haptic feedback for upper-limb prostheses in a dexterous task without direct vision. IEEE Trans. Neural Syst. Rehabilitation Eng. 31, 169–179. doi:10.1109/TNSRE.2022.3217452
Thomas, N., Miller, A. J., Ayaz, H., and Brown, J. D. (2023b). Haptic shared control improves neural efficiency during myoelectric prosthesis use. Sci. Rep. 13 (1), 484. doi:10.1038/s41598-022-26673-2
Torricelli, D., Rodriguez-Guerrero, C., Veneman, J. F., Crea, S., Briem, K., Lenggenhager, B., et al. (2020). Benchmarking wearable robots: challenges and recommendations from functional, user experience, and methodological perspectives. Front. Robotics AI 7, 561774. doi:10.3389/frobt.2020.561774
Trautman, P., Ma, J., Murray, R. M., and Krause, A. (2015). Robot navigation in dense human crowds: statistical models and experimental studies of human–robot cooperation. Int. J. Robotics Res. 34, 335–356. doi:10.1177/0278364914557874
van Beek, F. E., Bisschop, Q. P., and Kuling, I. A. (2023). Validation of a soft pneumatic unit cell (puc) in a vr experience: a comparison between vibrotactile and soft pneumatic haptic feedback. IEEE Trans. Haptics 17, 191–201. doi:10.1109/toh.2023.3307872
van Hattum, M. T., Huisman, G., Toet, A., and van Erp, J. B. (2022). Connected through mediated social touch: “better than a like on facebook.” A longitudinal explorative field study among geographically separated romantic couples. Front. Psychol. 13, 817787. doi:10.3389/fpsyg.2022.817787
Van Stralen, H. E., van Zandvoort, M. J., Hoppenbrouwers, S. S., Vissers, L. M., Kappelle, L. J., and Dijkerman, H. C. (2014). Affective touch modulates the rubber hand illusion. Cognition 131, 147–158. doi:10.1016/j.cognition.2013.11.020
Waytz, A., Heafner, J., and Epley, N. (2014). The mind in the machine: anthropomorphism increases trust in an autonomous vehicle. J. Exp. Soc. Psychol. 52, 113–117. doi:10.1016/j.jesp.2014.01.005
Keywords: human-robot interaction, user experience, affective touch, trust, semi-autonomous systems, upper-limb prosthetics, haptic feedback
Citation: Cansev ME, Miller AJ, Brown JD and Beckerle P (2024) Implementing social and affective touch to enhance user experience in human-robot interaction. Front. Robot. AI 11:1403679. doi: 10.3389/frobt.2024.1403679
Received: 19 March 2024; Accepted: 26 July 2024;
Published: 12 August 2024.
Edited by:
Leimin Tian, Monash University, AustraliaReviewed by:
Paul D. Marasco, Cleveland Clinic, United StatesChuang Yu, University College London, United Kingdom
Copyright © 2024 Cansev, Miller, Brown and Beckerle. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: M. Ege Cansev, ege.cansev@fau.de
†These authors have contributed equally to this work and share first authorship