Yang Ye ¹ Tianyu Zhou ¹ Qi Zhu ² William Vann ¹ Jing Du ^1*

• ¹ University of Florida, Gainesville, United States
• ² National Institute of Standards and Technology (NIST), Gaithersburg, Maryland, United States

This study aims to understand the cognitive impact of latency in teleoperation and the related mitigation methods, using functional Near-Infrared Spectroscopy (fNIRS) to analyze functional connectivity. The latency between command, execution, and feedback in teleoperation can impair performance and affect operators' mental state. The neural underpinnings of these effects are not well understood. A human-subject experiment (n = 41) of a simulated remote robot manipulation task was performed. Three conditions were tested: no latency, with visual and haptic latency, with visual latency and no haptic latency. fNIRS and performance data were recorded and analyzed. The presence of latency in teleoperation significantly increased functional connectivity within and between prefrontal and motor cortexes. Maintaining visual latency while providing real-time haptic feedback reduced the average functional connectivity in all cortical networks and showed a significantly different connectivity ratio within prefrontal and motor cortical networks. The performance results showed the worst performance in the all-delayed condition and the best performance in no latency condition, which echoes the neural activity patterns. The study provides neurological evidence that latency in teleoperation increases cognitive load, anxiety, and challenges in motion planning and control. Real-time haptic feedback, however, positively influences neural pathways related to cognition, decision-making, and sensorimotor processes. This research can inform the design of ergonomic teleoperation systems that mitigate the effects of latency.