ENSEMBLE OF SIMPLE SPIKING NEURAL NETWORKS AS A CONCEPT DRIFT DETECTOR
DOI:
https://doi.org/10.15588/1607-3274-2024-4-8Keywords:
machine learning, online learning, spiking neural networks, concept drift, drift detector, artificial neural networks, data stream mining, artificial intelligence, leaky integrate-and-fire neuronAbstract
Context. This paper provides a new approach in concept drift detection using an ensemble of simple spiking neural networks. Such approach utilizes an event-based nature and built-in ability to learn spatio-temporal patterns of spiking neurons, while ensemble provides additional robustness and scalability. This can help solve an active problem of limited time and processing resources in tasks of online machine learning, especially in very strict environments like IoT which also benefit in other ways from the use of spiking computations.
Objective. The aim of the work is the creation of an ensemble of simple spiking neural networks to act as a concept drift detector in the tasks of online data stream mining.
Method. The proposed approach is primary based on the accumulative nature of spiking neural networks, especially Leaky Integrate-and-Fire neurons can be viewed as gated memory units, where membrane time constant Гm is a balance constant between remembering and forgetting information. A training algorithm is implemented that utilizes a shallow two-layer SNN, which takes features and labels of the data as an input layer and the second layer consists of a single neuron. This neuron’s activation implies that an abrupt drift has occurred. In addition to that, such model is used as a base model within the ensemble to improve robustness, accuracy and scalability.
Results. An ensemble of shallow two-layer SNNs was implemented and trained to detect abrupt concept drift in the SEA data stream. The ensemble managed to improve accuracy significantly compared to a base model and achieved competitive results to modern state-of-the-art models.
Conclusions. Results showcased the viability of the proposed solution, which not only provides a cheap and competitive solution for resource-restricted environments, but also open doors for further research of SNN’s ability to learn spatio-temporal patters in the data streams and other fields.
References
Ramírez-Gallego Sergio [et al.] A survey on Data Preprocessing for Data Stream Mining: Current status and future directions [Electronic resource], Neurocomputing, 2017, Vol. 239. Mode of access: https://doi.org/10.1016/j.neucom.2017.01.078. Title from screen.
Gerstner Wulfram [et al.] Spiking Neuron Models: Single Neurons, Populations, Plasticity [Electronic resource] [S. l. : s. n.], 2002. Mode of access: https://doi.org/10.1017/CBO9780511815706. Title from screen.
Davidson S., Furber S. Comparison of Artificial and Spiking Neural Networks on Digital Hardware [Electronic resource], Frontiers in Neuroscience, 2021, Vol. 15, P. 651141. Mode of access: https://doi.org/10.3389/fnins.2021.651141. Title from screen.
Kasabov N. NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data [Electronic resource], Neural Networks, 2014, Vol. 52. Mode of access: https://doi.org/10.1016/j.neunet.2014.01.006. Title from screen.
Jesús López Lobo [et al.] Spiking Neural Networks and online learning: An overview and perspectives, 2020.
Jesús López Lobo [et al.] Drift Detection over Non-stationary Data Streams Using Evolving Spiking Neural Networks [Electronic resource], Studies in Computational Intelligence. [S. l.], 2018, pp. 82–94. Mode of access: https://doi.org/10.1007/9783-319-99626-4_8. Title from screen.
Hochreiter S., Schmidhuber J. {Long Short-Term Memory} [Electronic resource], Neural Computation, 1997, Vol. 9, No. 8, pp. 1735–1780. Mode of access: https://doi.org/10.1162/neco.1997.9.8.1735. Title from screen.
Cho Kyunghyun [et al.] Learning phrase representations using RNN encoder-decoder for statistical machine translation, arXiv preprint arXiv:1406.1078, 2014.
Voelker A., Kajic I., Eliasmith C. Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Networks, [S. l.], 2019.
Fang Wei [et al.] Incorporating learnable membrane time constant to enhance learning of spiking neural networks, Proceedings of the IEEE/CVF international conference on computer vision, [S. l.], 2021, pp. 2661–2671.
Samant Rucha [et al.] A Systematic Ensemble Approach for Concept Drift Detector Selection in Data Stream Classifiers [Electronic resource], International Journal of Engineering Trends and Technology, 2022, Vol. 70, pp. 119–130. Mode of access: https://doi.org/10.14445/22315381/IJETT-V70I9P212. Title from screen.
Priya S, Annie R. S P. Ensemble framework for concept drift detection and class imbalance in data streams [Electronic resource], Multimedia Tools and Applications, 2024. pp. 1–15. Mode of access: https://doi.org/10.1007/s11042-024-18349-y. Title from screen.
Wu Yafeng [et al.]. An Adaptive Ensemble Framework for Addressing Concept Drift in IoT Data Streams [Electronic resource], 2023. Mode of access: https://doi.org/10.36227/techrxiv.23304461. Title from screen.
Dakic Kosta [et al.]Spiking Neural Networks for Detecting Satellite-Based Internet-of-Things Signals, arXiv preprint arXiv:2304.03896, 2023.
Régis Antônio S. Albuquerque [et al.] A decision-based dynamic ensemble selection method for concept drift, 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI). [S. l.], 2019, pp. 1132–1139.
Yoonsik Shim [et al.] Unsupervised Learning in an Ensemble of Spiking Neural Networks Mediated by ITDP [Electronic resource], PLOS Computational Biology, 2016, Vol. 12, P. e1005137. Mode of access: https://doi.org/10.1371/journal.pcbi.1005137. Title from screen.
Deng L. The mnist database of handwritten digit images for machine learning research, IEEE Signal Processing Magazine, 2012, Vol. 29, No. 6, pp. 141–142.
Neculae G., Rhodes O., Brown G. Ensembles of Spiking Neural Networks, 2021.
Orchard Garrick [et al.] Converting static image datasets to spiking neuromorphic datasets using saccades, Frontiers in neuroscience, 2015, Vol. 9, P. 159859.
Amir Arnon [et al.]A Low Power, Fully Event-Based Gesture Recognition System [Electronic resource], 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 7388–7397. Mode of access: https://api.semanticscholar.org/CorpusID:19496174. Title from screen.
Fu Q., Dong H. An ensemble unsupervised spiking neural network for objective recognition [Electronic resource], Neurocomputing, 2021, Vol. 419, pp. 47–58. Mode of access: https://doi.org/10.1016/j.neucom.2020.07.109. Title from screen.
Krizhevsky A. Learning multiple layers of features from tiny images, [S. l. : s. n.], 2009.
Street W. N., Kim Y. S. A streaming ensemble algorithm (SEA) for large-scale classification [Electronic resource], Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY, USA, 2001, pp. 377–382. Mode of access: https://doi.org/10.1145/502512.502568. Title from screen.
Paszke Adam [et al.] Automatic differentiation in PyTorch, NIPS-W, [S. l.], 2017.
Fang Wei [et al.]SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence [Electronic resource], Science Advances, 2023, Vol. 9, No. 40, P. eadi1480. Mode of access: https://doi.org/10.1126/sciadv.adi1480. – Title from screen.
Angelopoulos Angelos [et al.] Impact of Classifiers to Drift Detection Method: A Comparison / // Proceedings of the 22nd Engineering Applications of Neural Networks Conference / ed. by L. Iliadis [et al.]. Cham, 2021, pp. 399–410.
Suryawanshi Shubhangi [et al.] Adaptive windowing based recurrent neural network for drift adaption in non-stationary environment [Electronic resource], Journal of Ambient Intelligence and Humanized Computing, 2022, Vol. 14, pp. 1–15. Mode of access: https://doi.org/10.1007/s12652-022-04116-0. Title from screen.
Tavanaei Amirhossein [et al.] Deep learning in spiking neural networks, Neural networks, 2019, Vol. 111, pp. 47–63.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Ye. V. Bodyanskiy, D. V. Savenkov
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Creative Commons Licensing Notifications in the Copyright Notices
The journal allows the authors to hold the copyright without restrictions and to retain publishing rights without restrictions.
The journal allows readers to read, download, copy, distribute, print, search, or link to the full texts of its articles.
The journal allows to reuse and remixing of its content, in accordance with a Creative Commons license СС BY -SA.
Authors who publish with this journal agree to the following terms:
-
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License CC BY-SA that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
-
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
-
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
