LIGHTWEIGHT MULTI-SCALE CONVOLUTIONAL TRANSFORMER FOR AIRCRAFT FAULT DIAGNOSIS USING VIBRATION ANALYSIS
DOI:
https://doi.org/10.15588/1607-3274-2025-1-7Keywords:
fault detection, deep learning, rotating machinery, signal processing, transformer, neural networksAbstract
Context. Fault diagnosis in rotating machinery, especially in aircraft, plays an important role in health monitoring systems. Early and accurate fault detection can significantly reduce the cost of repair and increase the lifetime of the mechanism. To detect the fault efficiently, intelligent methods based on traditional machine learning and deep learning techniques are used. The object of the research is the process of detecting faults in aircraft based on vibration analysis.
Objective of the work is the development of a deep learning method for fault diagnosis in rotating machinery with a high accuracy rate.
Method. The proposed method employs Transformer architecture. The first stage of processing the vibration signal is the multiscale feature extractor. This stage allows the model to examine input signals in different scales and reduce the impact of the noise.
The second stage is the Convolutional Transformer neural network. The convolution was introduced to the Transformer to combine locality and long-range dependencies feature extraction. The Self-attention mechanism of the Transformer was changed to Channel Attention, which reduces the number of parameters but maintains the strength of the attention. To maintain this idea, similar changes were made in the position-wise feed-forward network.
Results. The proposed method is tested on the aircraft vibration dataset. Two conditions were chosen for testing: limited data and noisy environment. The limited data condition is simulated by selecting a small number of samples into the training set (a maximum of 10 per class). The noisy environment condition is simulated by adding Gaussian noise to the raw signal. According to the obtained results, the proposed method achieves a high average precision metric rate with a small number of parameters. The experiments also show the importance of the proposed modules and changes, confirming the assumptions about the process of feature extraction.
Conclusion. The results of the conducted experiments show that the proposed model can detect faults with almost perfect accuracy, even with a small number of parameters. The proposed lightweight model is robust in limited data conditions and noisy environment conditions. The prospects for further research are the development of fast and accurate neural networks for fault diagnosis and the development of limited data training techniques.
References
Baccarini L. M. R., Silva V. V. R., de Menezes B. R. et al. SVM practical industrial application for mechanical faults diagnostic, Expert Systems with Applications, 2011, Vol. 38, № 6, pp. 6980–6984. DOI: 10.1016/j.eswa.2010.12.017
Lu Q., Shen X., Wang X. et al. Fault diagnosis of rolling bearing based on improved VDM and KNN, Mathematical Problems in Engineering, 2021, Vol. 2021, Issue 1, №2530315. DOI: 10.1155/2021/2530315
Sun W., Chen J., Li J.Decision tree and PCA-based fault diagnosis of rotating machinery, Mechanical Systems and Signal Processing, 2007, Vol. 21, Issue 3, pp. 1300–1317. DOI: 10.1016/j.ymssp.2006.06.010
Zhao R., Yan R., Chen Z. et al. Deep learning and its applications to machine health monitoring, Mechanical Systems and Signal Processing, 2019, Vol. 115, pp. 213–237. DOI: 10.1016/j.ymssp.2018.05.050
Samanta B., Al-Balushi K., and Al-Araimi S. Artificial neural networks and genetic algorithm for bearing fault detection, Soft Computing, 2006, Vol. 10, pp. 264–271. DOI: 10.1007/s00500-005-0481-0
Wang X., Mao D., Li X. Bearing fault diagnosis based on vibro-acoustic data fusion and 1D-CNN network, Measurement, 2021, Vol. 173, №108518.DOI: 10.1016/j.measurement.2020.108518
Pinedo-Sanchez L. A., Mercado-Ravell D. A., CarballoMonsivais C. A. Vibration analysis in bearings for failure using CNN, Journal of the Brazilian Society of Mechanical Sciences and Engineering, 2020, Vol. 42, № 12, № 628. DOI: 10.1007/s40430-020-02711-w
Yu Z., Zhang C., Liu J. et al. SKND-TSACNN: A novel time-scale adaptive CNN framework for fault diagnosis of rotating machinery, Knowledge-Based Systems, 2023, Vol. 275, P. 110682. DOI: 10.1016/j.knosys.2023.110682;
Huang Y., Chen C. H., Huang C. J. Motor fault detection and feature extraction using RNN-based variational autoencoder, IEEE Access, 2019, Vol. 7, pp. 139086–139096. DOI: 10.1109/ACCESS.2019.2940769
Liu H., Zhou J., Zheng Y. et al. Fault diagnosis of rolling bearings with recurrent neural network-based autoencoders, ISA Transactions, 2018, Vol. 77, pp. 167–178. DOI: 10.1016/j.isatra.2018.04.005
Saghi T., Bustan D., Aphale S. S. Bearing fault diagnosis based on multi-scale CNN and bidirectional GRU, Vibration, 2023, Vol. 6, № 1, pp. 11–28. DOI: 10.3390/vibration6010002
Qiao M., Yan S., Tang X. et al. Deep Convolutional and LSTM Recurrent Neural Networks for Rolling Bearing Fault Diagnosis Under Strong Noises and Variable Loads, IEEE Access, 2020, Vol. 8, pp. 66257–66269. DOI: 10.1109/ACCESS.2020.2985617
Vaswani A., Shazeer N., Parmar N. et al. Attention is All you Need [Electronic resource], Semantic Schoolar, 2017,Access mode: https://www.semanticscholar.org/paper/Attention-is-AllyouNeed-VaswaniShazeer/204e3073870fae3d05bcbc2f6a8e263d9b72e776. DOI: 10.48550/arXiv.1706.03762
Xu J., L. Zhou, W. Zhao et al. Zero-shot learning for compound fault diagnosis of bearings, Expert Systems with Applications, 2022, Vol. 190, №116197. DOI: 10.1016/j.eswa.2021.116197
Zhang A., Li S., Cui Y. et al. Limited Data Rolling Bearing Fault Diagnosis With Few-Shot Learning, IEEE Access, 2019, Vol. 7, pp. 110895–110904. DOI: 10.1109/ACCESS.2019.2934233
Chen C. C., Liu Z., Yang G. et al. An improved fault diagnosis using 1D-convolutional neural network model, Electronics, 2021, Vol. 10, Issue 1, pp. 1–19. DOI: 10.3390/electronics10010059
Zhang W., Peng G., Li C. et al. A New Deep Learning Model for Fault Diagnosis with Good Anti-Noise and Domain Adaptation Ability on Raw Vibration Signals, Sensors, 2017, Vol. 17, Issue 2, №425. DOI: 10.3390/s17020425
Zhao M., Zhong S., Fu X. et al. Deep Residual Shrinkage Networks for Fault Diagnosis, IEEE Transactions on Industrial Informatics, 2020, Vol. 16, Issue 7, pp. 4681–4690. DOI: 10.1109/TII.2019.2943898
Zhang W., Li X., Ding Q. Deep residual learning-based fault diagnosis method for rotating machinery, ISA Transactions, 2019, Vol. 95, pp. 295–305. DOI: 10.1016/j.isatra.2018.12.025
Li X., Wan S., Liu S. et al. Bearing fault diagnosis method based on attention mechanism and multilayer fusion network, ISA Transactions, 2021, Vol. 128, Part B, pp. 550–564. DOI: 10.1016/j.isatra.2021.11.020
Verstraete D., Ferrada A., Droguett E. et al. Deep Learning Enabled Fault Diagnosis Using Time-Frequency Image Analysis of Rolling Element Bearings, Shock and Vibration, 2017, Vol. 2017, Issue 1, №5067651. DOI: 10.1155/2017/5067651
Pham M. T., Kim J.-M., and Kim C. H. Accurate Bearing Fault Diagnosis under Variable Shaft Speed using Convolutional Neural Networks and Vibration Spectrogram, Applied Sciences, 2020, Vol. 10, Issue 18, № 6385. DOI:10.3390/app10186385;
Dosovitskiy A., Beyer L., Kolesnikov A. et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale [Electronic resource], ICLR, 2021. Access mode: https://arxiv.org/abs/2010.11929. DOI:10.48550/arXiv.2010.11929
Zhou H., Zhang S., Peng J. et al. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting, Proceedings of the AAAI conference on artificial intelligence, 2021, Vol. 35, Issue 12, pp. 11106–11115. DOI: 10.48550/arXiv.2012.07436
Weng C., Lu B., Yao J. A One-Dimensional Vision Transformer with Multiscale Convolution Fusion for Bearing Fault Diagnosis, Global Reliability and Prognostics and Health Management. N., IEEE, 2021, pp. 1–6. DOI: 10.1109/PHM-Nanjing52125.2021.9612919
Jin Y., Hou L., Chen Y. A Time Series Transformer based method for the rotating machinery fault diagnosis, Neurocomputing, 2022, Vol. 494, pp. 379–395. DOI: 10.1016/j.neucom.2022.04.111
Zim A. H., Ashraf A., Iqbal A. et al. A Vision TransformerBased Approach to Bearing Fault Classification via Vibration Signals, Asia-Pacific Signal and Information Processing Association Annual Summit and Conference. C., IEEE, 2022, pp. 1321–1326. DOI: 10.48550/arXiv.2208.07070
Liu W., Zhang Z., Zhang J. et al. A Novel Fault Diagnosis Method of Rolling Bearings Combining Convolutional Neural Network and Transformer, Electronics, 2023,Vol. 12, Issue 8, №1838. DOI: 10.3390/electronics12081838
Tang X., Xu Z., Wang Z. A Novel Fault Diagnosis Method of Rolling Bearing Based on Integrated Vision Transformer Model, Sensors, 2022, Vol. 22, Issue 10, №3878. DOI: 10.3390/s22103878
Xu Y., Zhang Q., Zhang J. et al. ViTAE: vision transformer advanced by exploring intrinsic inductive bias, NIPS'21 : 35th International Conference on Neural Information Processing Systems 6–14 December 2021 : proceedings. Red Hook, NY, Curran Associates Inc., 2021, Vol. 34, pp. 28522–28535. DOI: 10.48550/arXiv.2106.03348
Sun W., Yan R., Jin R. et al. LiteFormer: A Lightweight and Efficient Transformer for Rotating Machine Fault Diagnosis, IEEE Transactions on Reliability, 2024, Vol. 73, Issue 2, pp. 1258–1269. DOI: 10.1109/TR.2023.3322860
Fang X., Deng X., Chen J. et al. Convolution Transformer Based Fault Diagnosis Method For Aircraft Engine Bearings, 43rd Chinese Control Conference, Kunming, 28–31 July 2024 : proceedings. IEEE, 2024, pp. 5038–5042. DOI: 10.23919/CCC63176.2024.10662769
Pei X., Zheng X., Wu J. Rotating Machinery Fault Diagnosis Through a Transformer Convolution Network Subjected to Transfer Learning, IEEE Transactions on Instrumentation and Measurement, 2021, Vol. 70, №2515611. DOI: 10.1109/TIM.2021.3119137
Jin Y., Qin C., Zhang Z. et al. A multi-scale convolutional neural network for bearing compound fault diagnosis under various noise conditions, Science China Technological Sciences, 2022, Vol. 65, Issue 11, pp. 2551–2563. DOI: 10.1007/s11431-022-2109-4
Chen X., Zhang B., Gao D. Bearing fault diagnosis base on multi-scale CNN and LSTM model, Journal of Intelligent Manufacturing, 2021, Vol. 32, Issue 4, pp. 971–987. DOI: 10.1007/s10845-020-01600-2
Hu J., Shen L., Sun G. Squeeze-and-Excitation Networks, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 18–23 June 2018 : preceedings. IEEE, 2018, pp. 7132–7141. DOI: 10.1109/CVPR.2018.00745
Jiang G., He H., Yan J. et al. Multiscale Convolutional Neural Networks for Fault Diagnosis of Wind Turbine Gearbox, IEEE Transactions on Industrial Electronics, 2019, Vol. 66, Issue 4, pp. 3196–3207. DOI: 10.1109/TIE.2018.2844805;
Jia L., Chow T. W. S., Wang Y. et al. Multiscale Residual Attention Convolutional Neural Network for Bearing Fault Diagnosis, IEEE Transactions on Instrumentation and Measurement, 2022, Vol. 71, №3519413. DOI: 10.1109/TIM.2022.3196742
Wang H., Liu Z., Peng D. et al. Understanding and Learning Discriminant Features based on Multiattention 1DCNN for Wheelset Bearing Fault Diagnosis, IEEE Transactions on Industrial Informatics, 2020, Vol. 16, Issue 9, pp. 5735– 5745. DOI: 10.1109/TII.2019.2955540
He K., Zhang X., Ren S. et al. Deep Residual Learning for Recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 27–30 June 2016. IEEE, 2016, pp. 770–778, DOI: 10.1109/CVPR.2016.90
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Andrii Y. Didenko, Artem Y. Didenko, S. A. Subbotin
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Creative Commons Licensing Notifications in the Copyright Notices
The journal allows the authors to hold the copyright without restrictions and to retain publishing rights without restrictions.
The journal allows readers to read, download, copy, distribute, print, search, or link to the full texts of its articles.
The journal allows to reuse and remixing of its content, in accordance with a Creative Commons license СС BY -SA.
Authors who publish with this journal agree to the following terms:
-
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License CC BY-SA that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
-
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
-
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
