Skip to main content

BRIEF RESEARCH REPORT article

Front. Robot. AI
Sec. Human-Robot Interaction
Volume 11 - 2024 | doi: 10.3389/frobt.2024.1346714

A Pipeline for Estimating Human Attention towards Objects with On-Board Cameras on the iCub Humanoid Robot

Provisionally accepted
Shiva Hanifi Shiva Hanifi *Elisa Maiettini Elisa Maiettini Maria Lombardi Maria Lombardi Lorenzo Natale Lorenzo Natale
  • Italian Institute of Technology (IIT), Genova, Italy

The final, formatted version of the article will be published soon.

    This research report introduces a learning system designed to detect the object that humans are gazing at, using solely visual feedback. By incorporating face detection, human attention prediction, and online object detection the system enables the robot to perceive and interpret human gaze accurately, thereby facilitating the establishment of joint attention with human partners. Additionally, a novel dataset collected with the humanoid robot iCub is introduced, comprising over 22, 000 images from ten participants gazing at different annotated objects. This dataset serves as a benchmark for human gaze estimation in table-top human-robot interaction (HRI) contexts. In this work, we use it to assess the proposed pipeline's performance and examine each component's effectiveness. Furthermore, the developed system is deployed on the iCub, and a supplementary video showcases its functionality. The results demonstrate the potential of the proposed approach as a first step to enhance social awareness and responsiveness in social robotics. This advancement can enhance assistance and support in collaborative scenarios, promoting more efficient human-robot collaborations.

    Keywords: Attention, Gaze estimation, Learning architecture, humanoid robot, Computer Vision, human-robot scenario

    Received: 29 Nov 2023; Accepted: 23 Sep 2024.

    Copyright: © 2024 Hanifi, Maiettini, Lombardi and Natale. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Shiva Hanifi, Italian Institute of Technology (IIT), Genova, Italy

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.