LIGHTWEIGHT MULTI-SCALE CONVOLUTIONAL TRANSFORMER FOR AIRCRAFT FAULT DIAGNOSIS USING VIBRATION ANALYSIS

Authors

  • Andrii Y. Didenko National University “Zaporizhzhia Polytechnic”, Zaporizhzhia, Ukraine
  • Artem Y. Didenko National University “Zaporizhzhia Polytechnic”, Zaporizhzhia, Ukraine
  • S. A. Subbotin National University “Zaporizhzhia Polytechnic”, Zaporizhzhia, Ukraine

DOI:

https://doi.org/10.15588/1607-3274-2025-1-7

Keywords:

fault detection, deep learning, rotating machinery, signal processing, transformer, neural networks

Abstract

Context. Fault diagnosis in rotating machinery, especially in aircraft, plays an important role in health monitoring systems. Early and accurate fault detection can significantly reduce the cost of repair and increase the lifetime of the mechanism. To detect the fault efficiently, intelligent methods based on traditional machine learning and deep learning techniques are used. The object of the research is the process of detecting faults in aircraft based on vibration analysis.
Objective of the work is the development of a deep learning method for fault diagnosis in rotating machinery with a high accuracy rate.
Method. The proposed method employs Transformer architecture. The first stage of processing the vibration signal is the multiscale feature extractor. This stage allows the model to examine input signals in different scales and reduce the impact of the noise.
The second stage is the Convolutional Transformer neural network. The convolution was introduced to the Transformer to combine locality and long-range dependencies feature extraction. The Self-attention mechanism of the Transformer was changed to Channel Attention, which reduces the number of parameters but maintains the strength of the attention. To maintain this idea, similar changes were made in the position-wise feed-forward network.
Results. The proposed method is tested on the aircraft vibration dataset. Two conditions were chosen for testing: limited data and noisy environment. The limited data condition is simulated by selecting a small number of samples into the training set (a maximum of 10 per class). The noisy environment condition is simulated by adding Gaussian noise to the raw signal. According to the obtained results, the proposed method achieves a high average precision metric rate with a small number of parameters. The experiments also show the importance of the proposed modules and changes, confirming the assumptions about the process of feature extraction.
Conclusion. The results of the conducted experiments show that the proposed model can detect faults with almost perfect accuracy, even with a small number of parameters. The proposed lightweight model is robust in limited data conditions and noisy environment conditions. The prospects for further research are the development of fast and accurate neural networks for fault diagnosis and the development of limited data training techniques.

Author Biographies

Andrii Y. Didenko, National University “Zaporizhzhia Polytechnic”, Zaporizhzhia

Postgraduate student of the Department of Software Tools

Artem Y. Didenko, National University “Zaporizhzhia Polytechnic”, Zaporizhzhia

Postgraduate student of the Department of Software Tools

S. A. Subbotin, National University “Zaporizhzhia Polytechnic”, Zaporizhzhia

Dr. Sc., Professor, Head of the Department of Software Tools

References

Baccarini L. M. R., Silva V. V. R., de Menezes B. R. et al. SVM practical industrial application for mechanical faults diagnostic, Expert Systems with Applications, 2011, Vol. 38, № 6, pp. 6980–6984. DOI: 10.1016/j.eswa.2010.12.017

Lu Q., Shen X., Wang X. et al. Fault diagnosis of rolling bearing based on improved VDM and KNN, Mathematical Problems in Engineering, 2021, Vol. 2021, Issue 1, №2530315. DOI: 10.1155/2021/2530315

Sun W., Chen J., Li J.Decision tree and PCA-based fault diagnosis of rotating machinery, Mechanical Systems and Signal Processing, 2007, Vol. 21, Issue 3, pp. 1300–1317. DOI: 10.1016/j.ymssp.2006.06.010

Zhao R., Yan R., Chen Z. et al. Deep learning and its applications to machine health monitoring, Mechanical Systems and Signal Processing, 2019, Vol. 115, pp. 213–237. DOI: 10.1016/j.ymssp.2018.05.050

Samanta B., Al-Balushi K., and Al-Araimi S. Artificial neural networks and genetic algorithm for bearing fault detection, Soft Computing, 2006, Vol. 10, pp. 264–271. DOI: 10.1007/s00500-005-0481-0

Wang X., Mao D., Li X. Bearing fault diagnosis based on vibro-acoustic data fusion and 1D-CNN network, Measurement, 2021, Vol. 173, №108518.DOI: 10.1016/j.measurement.2020.108518

Pinedo-Sanchez L. A., Mercado-Ravell D. A., CarballoMonsivais C. A. Vibration analysis in bearings for failure using CNN, Journal of the Brazilian Society of Mechanical Sciences and Engineering, 2020, Vol. 42, № 12, № 628. DOI: 10.1007/s40430-020-02711-w

Yu Z., Zhang C., Liu J. et al. SKND-TSACNN: A novel time-scale adaptive CNN framework for fault diagnosis of rotating machinery, Knowledge-Based Systems, 2023, Vol. 275, P. 110682. DOI: 10.1016/j.knosys.2023.110682;

Huang Y., Chen C. H., Huang C. J. Motor fault detection and feature extraction using RNN-based variational autoencoder, IEEE Access, 2019, Vol. 7, pp. 139086–139096. DOI: 10.1109/ACCESS.2019.2940769

Liu H., Zhou J., Zheng Y. et al. Fault diagnosis of rolling bearings with recurrent neural network-based autoencoders, ISA Transactions, 2018, Vol. 77, pp. 167–178. DOI: 10.1016/j.isatra.2018.04.005

Saghi T., Bustan D., Aphale S. S. Bearing fault diagnosis based on multi-scale CNN and bidirectional GRU, Vibration, 2023, Vol. 6, № 1, pp. 11–28. DOI: 10.3390/vibration6010002

Qiao M., Yan S., Tang X. et al. Deep Convolutional and LSTM Recurrent Neural Networks for Rolling Bearing Fault Diagnosis Under Strong Noises and Variable Loads, IEEE Access, 2020, Vol. 8, pp. 66257–66269. DOI: 10.1109/ACCESS.2020.2985617

Vaswani A., Shazeer N., Parmar N. et al. Attention is All you Need [Electronic resource], Semantic Schoolar, 2017,Access mode: https://www.semanticscholar.org/paper/Attention-is-AllyouNeed-VaswaniShazeer/204e3073870fae3d05bcbc2f6a8e263d9b72e776. DOI: 10.48550/arXiv.1706.03762

Xu J., L. Zhou, W. Zhao et al. Zero-shot learning for compound fault diagnosis of bearings, Expert Systems with Applications, 2022, Vol. 190, №116197. DOI: 10.1016/j.eswa.2021.116197

Zhang A., Li S., Cui Y. et al. Limited Data Rolling Bearing Fault Diagnosis With Few-Shot Learning, IEEE Access, 2019, Vol. 7, pp. 110895–110904. DOI: 10.1109/ACCESS.2019.2934233

Chen C. C., Liu Z., Yang G. et al. An improved fault diagnosis using 1D-convolutional neural network model, Electronics, 2021, Vol. 10, Issue 1, pp. 1–19. DOI: 10.3390/electronics10010059

Zhang W., Peng G., Li C. et al. A New Deep Learning Model for Fault Diagnosis with Good Anti-Noise and Domain Adaptation Ability on Raw Vibration Signals, Sensors, 2017, Vol. 17, Issue 2, №425. DOI: 10.3390/s17020425

Zhao M., Zhong S., Fu X. et al. Deep Residual Shrinkage Networks for Fault Diagnosis, IEEE Transactions on Industrial Informatics, 2020, Vol. 16, Issue 7, pp. 4681–4690. DOI: 10.1109/TII.2019.2943898

Zhang W., Li X., Ding Q. Deep residual learning-based fault diagnosis method for rotating machinery, ISA Transactions, 2019, Vol. 95, pp. 295–305. DOI: 10.1016/j.isatra.2018.12.025

Li X., Wan S., Liu S. et al. Bearing fault diagnosis method based on attention mechanism and multilayer fusion network, ISA Transactions, 2021, Vol. 128, Part B, pp. 550–564. DOI: 10.1016/j.isatra.2021.11.020

Verstraete D., Ferrada A., Droguett E. et al. Deep Learning Enabled Fault Diagnosis Using Time-Frequency Image Analysis of Rolling Element Bearings, Shock and Vibration, 2017, Vol. 2017, Issue 1, №5067651. DOI: 10.1155/2017/5067651

Pham M. T., Kim J.-M., and Kim C. H. Accurate Bearing Fault Diagnosis under Variable Shaft Speed using Convolutional Neural Networks and Vibration Spectrogram, Applied Sciences, 2020, Vol. 10, Issue 18, № 6385. DOI:10.3390/app10186385;

Dosovitskiy A., Beyer L., Kolesnikov A. et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale [Electronic resource], ICLR, 2021. Access mode: https://arxiv.org/abs/2010.11929. DOI:10.48550/arXiv.2010.11929

Zhou H., Zhang S., Peng J. et al. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting, Proceedings of the AAAI conference on artificial intelligence, 2021, Vol. 35, Issue 12, pp. 11106–11115. DOI: 10.48550/arXiv.2012.07436

Weng C., Lu B., Yao J. A One-Dimensional Vision Transformer with Multiscale Convolution Fusion for Bearing Fault Diagnosis, Global Reliability and Prognostics and Health Management. N., IEEE, 2021, pp. 1–6. DOI: 10.1109/PHM-Nanjing52125.2021.9612919

Jin Y., Hou L., Chen Y. A Time Series Transformer based method for the rotating machinery fault diagnosis, Neurocomputing, 2022, Vol. 494, pp. 379–395. DOI: 10.1016/j.neucom.2022.04.111

Zim A. H., Ashraf A., Iqbal A. et al. A Vision TransformerBased Approach to Bearing Fault Classification via Vibration Signals, Asia-Pacific Signal and Information Processing Association Annual Summit and Conference. C., IEEE, 2022, pp. 1321–1326. DOI: 10.48550/arXiv.2208.07070

Liu W., Zhang Z., Zhang J. et al. A Novel Fault Diagnosis Method of Rolling Bearings Combining Convolutional Neural Network and Transformer, Electronics, 2023,Vol. 12, Issue 8, №1838. DOI: 10.3390/electronics12081838

Tang X., Xu Z., Wang Z. A Novel Fault Diagnosis Method of Rolling Bearing Based on Integrated Vision Transformer Model, Sensors, 2022, Vol. 22, Issue 10, №3878. DOI: 10.3390/s22103878

Xu Y., Zhang Q., Zhang J. et al. ViTAE: vision transformer advanced by exploring intrinsic inductive bias, NIPS'21 : 35th International Conference on Neural Information Processing Systems 6–14 December 2021 : proceedings. Red Hook, NY, Curran Associates Inc., 2021, Vol. 34, pp. 28522–28535. DOI: 10.48550/arXiv.2106.03348

Sun W., Yan R., Jin R. et al. LiteFormer: A Lightweight and Efficient Transformer for Rotating Machine Fault Diagnosis, IEEE Transactions on Reliability, 2024, Vol. 73, Issue 2, pp. 1258–1269. DOI: 10.1109/TR.2023.3322860

Fang X., Deng X., Chen J. et al. Convolution Transformer Based Fault Diagnosis Method For Aircraft Engine Bearings, 43rd Chinese Control Conference, Kunming, 28–31 July 2024 : proceedings. IEEE, 2024, pp. 5038–5042. DOI: 10.23919/CCC63176.2024.10662769

Pei X., Zheng X., Wu J. Rotating Machinery Fault Diagnosis Through a Transformer Convolution Network Subjected to Transfer Learning, IEEE Transactions on Instrumentation and Measurement, 2021, Vol. 70, №2515611. DOI: 10.1109/TIM.2021.3119137

Jin Y., Qin C., Zhang Z. et al. A multi-scale convolutional neural network for bearing compound fault diagnosis under various noise conditions, Science China Technological Sciences, 2022, Vol. 65, Issue 11, pp. 2551–2563. DOI: 10.1007/s11431-022-2109-4

Chen X., Zhang B., Gao D. Bearing fault diagnosis base on multi-scale CNN and LSTM model, Journal of Intelligent Manufacturing, 2021, Vol. 32, Issue 4, pp. 971–987. DOI: 10.1007/s10845-020-01600-2

Hu J., Shen L., Sun G. Squeeze-and-Excitation Networks, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 18–23 June 2018 : preceedings. IEEE, 2018, pp. 7132–7141. DOI: 10.1109/CVPR.2018.00745

Jiang G., He H., Yan J. et al. Multiscale Convolutional Neural Networks for Fault Diagnosis of Wind Turbine Gearbox, IEEE Transactions on Industrial Electronics, 2019, Vol. 66, Issue 4, pp. 3196–3207. DOI: 10.1109/TIE.2018.2844805;

Jia L., Chow T. W. S., Wang Y. et al. Multiscale Residual Attention Convolutional Neural Network for Bearing Fault Diagnosis, IEEE Transactions on Instrumentation and Measurement, 2022, Vol. 71, №3519413. DOI: 10.1109/TIM.2022.3196742

Wang H., Liu Z., Peng D. et al. Understanding and Learning Discriminant Features based on Multiattention 1DCNN for Wheelset Bearing Fault Diagnosis, IEEE Transactions on Industrial Informatics, 2020, Vol. 16, Issue 9, pp. 5735– 5745. DOI: 10.1109/TII.2019.2955540

He K., Zhang X., Ren S. et al. Deep Residual Learning for Recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 27–30 June 2016. IEEE, 2016, pp. 770–778, DOI: 10.1109/CVPR.2016.90

Downloads

Published

2025-04-10

How to Cite

Didenko, A. Y., Didenko, A. Y., & Subbotin, S. A. (2025). LIGHTWEIGHT MULTI-SCALE CONVOLUTIONAL TRANSFORMER FOR AIRCRAFT FAULT DIAGNOSIS USING VIBRATION ANALYSIS. Radio Electronics, Computer Science, Control, (1), 75–83. https://doi.org/10.15588/1607-3274-2025-1-7

Issue

Section

Neuroinformatics and intelligent systems