DETERMINATION AND COMPARISON METHODS OF BODY POSITIONS ON STREAM VIDEO
DOI:
https://doi.org/10.15588/1607-3274-2023-2-6Keywords:
computer vision, body position, keypoints, pose estimation, pose comparison, blazepose, mediapipe, tensorflowAbstract
Context. One of the tasks of computer vision is the task of determining the human body in the image. There are many methods to solve this problem, some are based on specific equipment (motion capture, kinect) and provide the highest accuracy, some give less accuracy but do not require additional equipment and use less computing power. But usually, such equipment has a high cost, so to ensure the low cost of developments designed to determine the body in the image, you should develop algorithms based on computer vision technology. These algorithms can then be applied to various fields to analyze and compare body positions for a variety of purposes.
Objective. The aim of the work is to study the effectiveness of existing libraries to determine the human body position in the image, as well as methods for comparing the obtained poses in terms of speed and accuracy of determination.
Methods. A set of libraries and pose comparison algorithms were analyzed for the purpose of developing a system for determining the correctness of exercise by the user in real time. OpenPose, PoseNet and BlazePose libraries were analyzed for their suitability in recognizing and tracking body parts and movements in real-time video streams. The advantages and disadvantages of each library were evaluated based on their performance, accuracy, and computational efficiency. Additionally, different pose comparison algorithms were analyzed. The effectiveness of each algorithm was evaluated based on their ability to accurately determine and compare body positions.
As a result, the combination of BlazePose and weighted distance method can achieve the best performance in pose recognition, with high accuracy and robustness across a range of challenging scenarios. The weighted distance method can be further enhanced with techniques such as L2 normalization and pose alignment to improve its accuracy and generalization. Overall, the combination of the BlazePose library and weighted distance methods offers a powerful and effective solution for pose recognition, with high F1 index.
Results. Existing models for determining poses have shown similar results in the quality of determination with a run-up of about 2%. When developing a cross-platform software product, the BlazePose library, which has an API for working directly in the browser and on mobile platforms, has a significant advantage in speed and accuracy. Also, as the library uses extended 33 keypoint topology it becomes applicable to a wider list of tasks. In the study of comparison methods, the greatest influence on the results was exerted by the quality of pose determination.
Conclusions. Among the methods of comparison, the method of weighted distances showed the best results. The speed of position determination is inversely proportional to the quality of determination and significantly exceeds the recommended value – 40ms.
References
Zhipeng Z., Dong X., Shijie H. A Survey of Body Pose Estimation: Recent Advances and Future Prospects, Journal of Imaging, 2021, Vol. 7, No. 3, pp. 1–31. DOI: 10.3390/jimaging7030045
Shcherbakova G. Y., Krylov V. N., Bilous N. V. Methods of automated classification based on wavelet-transform for automated medical diagnostics, 2015 Information Technologies in Innovation Business Conference (ITIB). Kharkiv, Ukraine, 7–9 October 2015, [S. l.], 2015. DOI: 10.1109/itib.2015.7355048
Rakova A. O., Bilous N. V. Research on Methods for Development of Software System for Face Orientation Vector Determining in the Image, Radio Electronics, Computer Science, Control, 2020, No. 3(54), pp. 121–129. DOI: 10.15588/1607-3274-2020-3-11
Shih-En W. Convolutional Pose Machines, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA, 27–30 June 2016. [S. l.], 2016. DOI: 10.1109/cvpr.2016.511
Kendall A. PoseNet: A Convolutional Network for RealTime 6-DOF Camera Relocalization [Electronic resource] / Alex Kendall, Matthew Grimes, Roberto Cipolla // 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015. – [S. l.], 2015. DOI: 10.1109/iccv.2015.336
Tsung-Yi L. Microsoft COCO: Common Objects in Context, Computer Vision – ECCV 2014. Cham, 2014, pp. 740–755. DOI: 10.1007/978-3-319-10602-1_48
Yang Y., Ramanan D. Articulated Human Detection with Flexible Mixtures of Parts, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, Vol. 35, No. 12, pp. 2878–2890. DOI: 10.1109/tpami.2012.261
Cao Z. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, P. 1. DOI: 10.1109/tpami.2019.2929257
Ge L. Real-Time 3D Hand Pose Estimation with 3D Convolutional Neural Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, Vol. 41, No. 4, pp. 956–970. DOI: 10.1109/tpami.2018.2827052
Liu Y. OpenPose-Based Yoga Pose Classification Using Convolutional Neural Network, Highlights in Science, Engineering and Technology, 2022, Vol. 23, pp. 72–76. DOI: 10.54097/hset.v23i.3130
Bilous N. V., Krasov A. I., Vlasenko V. P. Deletion method of image low-frequency components using fast median filter algorithm, Journal of Engineering Sciences, 2016, pp. 7–14. DOI: 10.21272/jes
Bazarevsky V., Grishchenko I., Raveendran K. BlazePose: On-device Real-time Body Pose tracking, Computer Vision and Pattern Recognition, 2020, P. 4. DOI: 10.48550/arXiv.2006.10204
Yu L., Gao Xiao-Shan Improve Robustness and Accuracy of Deep Neural Network with L2 Normalization, Journal of Systems Science and Complexity, 2022, pp. 1–26. DOI: 10.1007/s11424-022-1326-y
Friedhoff J. Move Mirror: An AI Experiment with Pose Estimation in the Browser using TensorFlow.js [Electronic resource]. Mode of access: https://blog.tensorflow.org/2018/07/move-mirror-aiexperiment-with-pose-estimation-tensorflow-js.html (date of access: 18.04.2023). Title from screen.
Borkar P. K., Pulinthitha M. M., Pansare A. Match Pose – A System for Comparing Poses, International journal of engineering research & technology, 2019, P. 3. DOI: 10.17577/IJERTV8IS100253
Dembczy´nski K., Waegeman W., Cheng W., H¨ullermeier E. An exact algorithm for F-measure maximization, Neural Information Processing Systems, 2011, P. 9.
Rutanen K. O-notation in algorithm analysis. Data Structures and Algorithms, 2022, P. 216. DOI: 10.48550/arXiv.1309.3210
Malmir B. Exploratory studies of Human Gait Changes using Depth Cameras and Sample Entropy [Electronic resource] : thesis. [S. l.], 2018. Mode of access: http://hdl.handle.net/2097/38949 (date of access: 18.04.2023). – Title from screen.
Bijelic M., Gruber T., Ritter W. A Benchmark for Lidar Sensors in Fog: Is Detection Breaking Down? 2018 IEEE Intelligent Vehicles Symposium (IV). Changshu, 26–30 June 2018, [S. l.], 2018. DOI: 10.1109/ivs.2018.8500543
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 N. V. Bilous, I. A. Ahekian, V. V. Kaluhin
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Creative Commons Licensing Notifications in the Copyright Notices
The journal allows the authors to hold the copyright without restrictions and to retain publishing rights without restrictions.
The journal allows readers to read, download, copy, distribute, print, search, or link to the full texts of its articles.
The journal allows to reuse and remixing of its content, in accordance with a Creative Commons license СС BY -SA.
Authors who publish with this journal agree to the following terms:
-
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License CC BY-SA that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
-
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
-
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
