Transformer network enhanced by dual convolutional neural network and Cross-attention for wheelset bearing fault diagnosis

Jing Zhao ¹ Junfeng Li ² Zengqiang Ma ^1*

• ¹ Shijiazhuang Tiedao University, Shijiazhuang, China
• ² Guangdong University of Foreign Studies, Guangzhou, Guangdong Province, China

Advances in deep learning methods have demonstrated remarkable development in wheelset fault diagnosis. The currently popular deep neural networks suffer from design flaws in their network structure, including low accuracy, large computational complexity, limitations in frequency domain analysis and long time series feature encoding. To tackle the aforementioned issues, a Transformer network model based on dual convolutional neural network and cross-attention enhancement (Trans-DCC) is proposed for wheelset bearing fault diagnosis. Firstly, a dual feature fusion mechanism is designed in the first layer of the Transformer encoder. The dual feature fusion mechanism consists of dual convolutional neural networks(CNNs), which fully extracts low-level features from the timefrequency signals and reduces the complexity of the subsequent attention calculations. Secondly, a cross-attention model is established to the last layer in the Transformer encoder, which integrates the multihead attention mechanism, the time-domain feature and frequency-domain feature extracted by feedforward connection layer. By calculating the attention weights, the model focuses more on important features before performing feature enhancement fusion; Finally, fully connected layers and softmax functions are used as classifier to realize the fualt diagnosis of the wheelset bearings. By analyzing the train wheelset bearing dataset, the examination demonstrates the accuracy and effectiveness of the Trans-DCC.