Skip to main content

ORIGINAL RESEARCH article

Front. Robot. AI
Sec. Human-Robot Interaction
Volume 11 - 2024 | doi: 10.3389/frobt.2024.1410957
This article is part of the Research Topic Perceiving, Generating, and Interpreting Affect in Human-Robot Interaction (HRI) View all 7 articles

Modeling Interpersonal Perception in Dyadic Interactions: Towards Robot-Assisted Social Mediation in the Real World

Provisionally accepted
Hifza Javed Hifza Javed 1*Weinan Wang Weinan Wang 1,2Affan Bin Usman Affan Bin Usman 1Nawid Jamali Nawid Jamali 1
  • 1 Honda Research Institute USA Mountain View, Mountain View, United States
  • 2 Department of Electrical and Computer Engineering, School of Engineering, Rutgers, The State University of New Jersey, Piscataway, United States

The final, formatted version of the article will be published soon.

    Social mediator robots have shown potential in facilitating human interactions by improving communication, fostering relationships, providing support, and promoting inclusivity. However, for these robots to effectively shape human interactions, they must understand the intricacies of interpersonal dynamics. This necessitates models of human understanding that capture interpersonal states and the relational affect arising from interactions. Traditional affect recognition methods, primarily focus on individual affect, and may fall short in capturing interpersonal dynamics crucial for social mediation. To address this gap, we propose a multimodal, multiperspective model of relational affect, utilizing a conversational dataset collected in uncontrolled settings. Our model extracts features from audiovisual data to capture affective behaviors indicative of relational affect. By considering the interpersonal perspectives of both interactants, our model predicts relational affect, enabling real-time understanding of evolving interpersonal dynamics. We discuss our model's utility for social mediation applications and compare it with existing approaches, highlighting its advantages for real-world applicability. Despite the complexity of human interactions and subjective nature of affect ratings, our model demonstrates early capabilities to enable proactive intervention in negative interactions, enhancing neutral exchanges, and respecting positive dialogues. We discuss implications for real-world deployment and highlight the limitations of current work. Our work represents a step towards developing computational models of relational affect tailored for real-world social mediation, offering insights into effective mediation strategies for social mediator robots.

    Keywords: relational affect, multimodal, Multi-perspective, dyadic interaction, Group dynamics, Interpersonal dynamics, social mediation

    Received: 02 Apr 2024; Accepted: 30 Oct 2024.

    Copyright: © 2024 Javed, Wang, Usman and Jamali. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Hifza Javed, Honda Research Institute USA Mountain View, Mountain View, United States

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.