Skip to main content

ORIGINAL RESEARCH article

Front. Virtual Real.
Sec. Augmented Reality
Volume 6 - 2025 | doi: 10.3389/frvir.2025.1499830
This article is part of the Research Topic A Metaverse for the Good: Design, Application and Understanding View all 13 articles

Enhancing Augmented Reality with Machine Learning for Hands-On Origami Training

Provisionally accepted
Piotr Skrzypczynski Piotr Skrzypczynski 1*MIkolaj Lysakowski MIkolaj Lysakowski 1*Jakub Gapsa Jakub Gapsa 1*Slawomir Konrad Tadeja Slawomir Konrad Tadeja 2*Chenxu Lyu Chenxu Lyu 2Thomas Bohné Thomas Bohné 2*
  • 1 Poznań University of Technology, Poznań, Poland
  • 2 Department of Engineering, University of Cambridge, Cambridge, United Kingdom

The final, formatted version of the article will be published soon.

    This research explores integrating augmented reality (AR) with machine learning (ML) to enhance hands-on skill acquisition through origami folding. We developed an AR system using the YOLOv8 model to provide real-time feedback and automatic validation of each folding step, offering step-bystep guidance to users. A novel approach to training dataset preparation was introduced, which improves the accuracy of detecting and assessing origami folding stages. In a formative user study involving 16 participants tasked with folding multiple origami models, the results revealed that while the ML-driven feedback increased task completion times, it also made participants feel more confident throughout the folding process. However, they also reported that the feedback system added cognitive load, slowing their progress, though it provided valuable guidance. These findings suggest that while ML-supported AR systems can enhance the user experience, further optimization is required to streamline the feedback process and improve efficiency in complex manual tasks.

    Keywords: augmented reality, machine learning, Edge computing, assembly task, Education

    Received: 21 Sep 2024; Accepted: 07 Jan 2025.

    Copyright: © 2025 Skrzypczynski, Lysakowski, Gapsa, Tadeja, Lyu and Bohné. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence:
    Piotr Skrzypczynski, Poznań University of Technology, Poznań, Poland
    MIkolaj Lysakowski, Poznań University of Technology, Poznań, Poland
    Jakub Gapsa, Poznań University of Technology, Poznań, Poland
    Slawomir Konrad Tadeja, Department of Engineering, University of Cambridge, Cambridge, United Kingdom
    Thomas Bohné, Department of Engineering, University of Cambridge, Cambridge, United Kingdom

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.