Skip to main content

ORIGINAL RESEARCH article

Front. Neurosci.
Sec. Perception Science
Volume 18 - 2024 | doi: 10.3389/fnins.2024.1416719

Brain Functional Connectivity under Teleoperation Latency: a fNIRS Study

Provisionally accepted
  • 1 University of Florida, Gainesville, United States
  • 2 National Institute of Standards and Technology (NIST), Gaithersburg, Maryland, United States

The final, formatted version of the article will be published soon.

    This study aims to understand the cognitive impact of latency in teleoperation and the related mitigation methods, using functional Near-Infrared Spectroscopy (fNIRS) to analyze functional connectivity. The latency between command, execution, and feedback in teleoperation can impair performance and affect operators' mental state. The neural underpinnings of these effects are not well understood. A human-subject experiment (n = 41) of a simulated remote robot manipulation task was performed. Three conditions were tested: no latency, with visual and haptic latency, with visual latency and no haptic latency. fNIRS and performance data were recorded and analyzed. The presence of latency in teleoperation significantly increased functional connectivity within and between prefrontal and motor cortexes. Maintaining visual latency while providing real-time haptic feedback reduced the average functional connectivity in all cortical networks and showed a significantly different connectivity ratio within prefrontal and motor cortical networks. The performance results showed the worst performance in the all-delayed condition and the best performance in no latency condition, which echoes the neural activity patterns. The study provides neurological evidence that latency in teleoperation increases cognitive load, anxiety, and challenges in motion planning and control. Real-time haptic feedback, however, positively influences neural pathways related to cognition, decision-making, and sensorimotor processes. This research can inform the design of ergonomic teleoperation systems that mitigate the effects of latency.

    Keywords: human-robot interaction, Sensory manipulation, Haptic Feedback, latency mitigation, motor functions, functional near-infrared spectroscopy (fNIRS)

    Received: 13 Apr 2024; Accepted: 28 Oct 2024.

    Copyright: © 2024 Ye, Zhou, Zhu, Vann and Du. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Jing Du, University of Florida, Gainesville, United States

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.