Deep Learning-Based Action Recognition for Joining and Welding Processes of Dissimilar Materials

Yiming Zou ^*

• Qingdao University of Science and Technology, Qingdao, China

Joining and welding processes for dissimilar materials present unique challenges due to the need for precise monitoring and analysis of complex physical and chemical interactions. These processes are influenced by variations in material behavior, dynamic changes in process parameters, and environmental factors, making real-time action recognition a critical tool for ensuring consistent quality, efficiency, and reliability. Traditional methods for analyzing such processes often fail to effectively capture the multi-scale spatiotemporal dependencies and adapt to the inherent variability of these operations. To address these limitations, we propose a novel deep learning-based framework specifically designed for action recognition in joining and welding tasks involving dissimilar materials. Our proposed model, the Multi-Scale Spatiotemporal Attention Network (MS-STAN), leverages advanced hierarchical feature extraction techniques and attention mechanisms to capture fine-grained spatiotemporal patterns across varying scales, while simultaneously suppressing irrelevant or noisy regions within the input data. The framework integrates adaptive frame sampling and lightweight temporal modeling to ensure computational efficiency, making it practical for real-time applications without sacrificing accuracy. Domainspecific knowledge is embedded into the framework, enhancing its interpretability and improving its ability to generalize across diverse joining and welding scenarios. Experimental results highlight the model's superior performance in recognizing critical process actions, offering a promising solution for robust real-time monitoring, quality assurance, and optimization of joining and welding workflows. This work paves the way for more intelligent and adaptive manufacturing systems.