Skip to main content

REVIEW article

Front. Surg.
Sec. Neurosurgery
Volume 12 - 2025 | doi: 10.3389/fsurg.2025.1528362
This article is part of the Research Topic Innovations and Challenges in Surgical Education View all 9 articles

Artificial intelligence integration in surgery through hand and instrument tracking: a systematic literature review

Provisionally accepted
Kivanc Yangi Kivanc Yangi 1Thomas J. On Thomas J. On 1Yuan Xu Yuan Xu 1Arianna S. Gholami Arianna S. Gholami 1Jinpyo Hong Jinpyo Hong 1Alexander G. Reed Alexander G. Reed 1Pravarakhya Puppalla Pravarakhya Puppalla 1Jiuxu Chen Chen Jiuxu Chen Chen 1Jonathan A. Tangsrivimol Jonathan A. Tangsrivimol 1Baoxin Li Baoxin Li 2Marco Santello Marco Santello 2Michael T Lawton Michael T Lawton 1Mark Preul Mark Preul 1*
  • 1 Division of Neurological Surgery, Barrow Neurological Institute (BNI), Phoenix, Arizona, United States
  • 2 Arizona State University, Tempe, Arizona, United States

The final, formatted version of the article will be published soon.

    Objective: This systematic literature review of the integration of artificial intelligence (AI) applications in surgical practice through hand and instrument tracking provides an overview of recent advancements and analyzes current literature on the intersection of surgery with AI. Distinct AI algorithms and specific applications in surgical practice are also examined.Methods: An advanced search using medical subject heading terms was conducted in Medline (via PubMed), SCOPUS, and Embase databases for articles published in English. A strict selection process was performed, adhering to PRISMA guidelines.Results: A total of 225 articles were retrieved. After screening, 77 met inclusion criteria and were included in the review. Use of AI algorithms in surgical practice was uncommon during 2013–2017 but has gained significant popularity since 2018. Deep learning algorithms (n=62) are increasingly preferred over traditional machine learning algorithms (n=15). These technologies are used in surgical fields such as general surgery (n=19), neurosurgery (n=10), and ophthalmology (n=9). The most common functional sensors and systems used were prerecorded videos (n=29), cameras (n=21), and image datasets (n=7). The most common applications included laparoscopic (n=13), robotic-assisted (n=13), basic (n=12), and endoscopic (n=8) surgical skills training, as well as surgical simulation training (n=8).Conclusion: AI technologies can be tailored to address distinct needs in surgical education and patient care. The use of AI in hand and instrument tracking improves surgical outcomes by optimizing surgical skills training. It is essential to acknowledge the current technical and social limitations of AI and work toward filling those gaps in future studies.

    Keywords: artificial intelligence, deep learning, Education, Hand-tracking, Instrument tracking, machine learning, Surgery, surgical motion ABBREVIATIONS: AI, artificial intelligence

    Received: 14 Nov 2024; Accepted: 31 Jan 2025.

    Copyright: © 2025 Yangi, On, Xu, Gholami, Hong, Reed, Puppalla, Chen, Tangsrivimol, Li, Santello, Lawton and Preul. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Mark Preul, Division of Neurological Surgery, Barrow Neurological Institute (BNI), Phoenix, 85013, Arizona, United States

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.