RCF-ST: RICHER CONVOLUTIONAL FEATURES NETWORK WITH STRUCTURAL TUNING FOR THE EDGE DETECTION ON NATURAL IMAGES
DOI:
https://doi.org/10.15588/1607-3274-2023-4-12Keywords:
natural image, edge detection, convolutional network, richer convolutional features, structural tuning, batch normalizationAbstract
Context. The problem of automating of the edge detection on natural images in intelligent systems is considered. The subject of the research is the deep learning convolutional neural networks for edge detection on natural images.
Objective. The objective of the research is to improve the edge detection performance of natural images by structural tuning the richer convolutional features network architecture.
Method. In general, the edge detection performance is influenced by a neural network architecture. To automate the design of the network structure in the paper a structural tuning of a neural network is applied. Computational costs of a structural tuning are incomparably less compared with neural architecture search, but a higher qualification of the researcher is required, and the resulting solution will be suboptimal. In this research it is successively applied first a destructive approach and then a constructive approach to structural tuning of the based architecture of the RCF neural network. The constructive approach starts with a simple architecture network. Hidden layers, nodes, and connections are added to expand the network. The destructive approach starts with a complex architecture network. Hidden layers, nodes, and connections are then deleted to contract the network. The structural tuning of the richer convolutional features network includes: (1) reducing the number of convolutional layers; (2) reducing the number of convolutions in convolutional layers; (3) removing at each stage the sigmoid activation function with subsequent calculation of the loss function; (4) addition of the batch normalization layers after convolutional layers; (5) including the ReLU activation functions after the added batch normalization layers. The obtained neural network is named RCF-ST. The initial color images were scaled to the specified size and then inputted in the neural network. The advisability of each of the proposed stages of network structural tuning was reseached by estimating the edge detection performance using the confusion matrix elements and Figure of Merit. The advisability of a structural tuning of the neural network as a whole was estimated by comparing it with methods known from the literature using the Optimal Dataset Scale and Optimal Image Scale.
Results. The proposed convolutional neural network has been implemented in software and researched for solving the problem of edge detection on natural images. The structural tuning technique may be used for informed design of the neural network architectures for other artificial intelligence problems.
Conclusions. The obtained RCF-ST network allows to improve the performance of edge detection on natural images. RCF-ST network is characterized by a significantly fewer parameters compared to the RCF network, which makes it possible to reduce the resource consumption of the network. Besides, RCF-ST network ensures the enhancing of the robustness of edge detection on texture background.
References
Sun R., Lei T., Chen Q. et. al. Survey of image edge detection, Frontiers of Signal Processing, 2022, Vol. 2, article 826967. DOI: 10.3389/frsip.2022.826967
Leoshchenko S. D., Oliynyk A. O., Subbotin S. O., Hoffman E. O., Kornienko O. V. Method of structural adjustment of neural network models to ensure interpretability, Radio electronics, computer science, management, 2021, № 3, pp. 86–96. DOI: 10.15588/16073274-2021-3-8
D’souza R. N., Huang P. Y., Yeh F. C. Structural analysis and optimization of convolutional neural networks with a small sample size, Scientific Reports, 2020, № 10, pp. 1–13. DOI: 10.1038/s41598-020-57866-2
Nowakowski G., Dorogyy Y., Doroga-Ivaniuk O. Neural network structure optimization algorithm, Journal of Automation, Mobile Robotics and Intelligent Systems, 2018, № 12, pp. 5–13. DOI: 10.14313/JAMRIS_1-2018/1
Polyakova M. V. Image segmentation with a convolutional neural network without pooling layers in dermatological disease diagnostics systems, Radio Electronics, Computer Science, Control, 2023, № 1, pp. 51–61. DOI: http://doi.org/ 10.15588/1607-3274-2023-1-5
Matychenko A. D., Polyakova M. V. The structural tuning of the convolutional neural network for speaker identification in mel frequency cepstrum coefficients space, Herald of Advanced Information Technology, 2023, Vol. 6, № 2, pp. 115–127. DOI: 10.15276/hait.06.2023.7
Yang J., Price B., Cohen S., Lee H., Yang M.-H. Object сontour detection with a fully convolutional encoderdecoder network, Computer Vision and Pattern Recognition: IEEE Conference, CVPR, Las Vegas, NV, USA, 27–30 June 2016 : proceedings. IEEE, 2016, pp. 193– 202. DOI: 10.1109/cvpr.2016.28
Yu Z., Feng C., Liu M. Y., Ramalingam S. Casenet: Deep category-aware semantic edge detection, Computer Vision and Pattern Recognition: IEEE Conference, CVPR, Honolulu, HI, USA, 21–26 July 2017 : proceedings. IEEE, 2017, pp. 5964–5973. DOI: 10.1109/cvpr.2017.191
Pu M., Huang Y., Guan Q., Ling H. RINDNet: Edge detection for discontinuity in reflectance, illumination, normal and depth, Computer Vision: IEEE/CVF International Conference, ICCV, online, 11–17 October, 2021 : proceedings. IEEE, 2021, pp. 6879–6888. DOI: 10.1109/iccv48922.2021.00680
Maninis K.-K., Pont-Tuset J., Arbeláez P., Van Gool L. Convolutional oriented boundaries, Computer Vision – ECCV 2016. Lecture Notes in Computer Science / Leibe, B., Matas, J., Sebe, N., Welling, M. (eds). Springer, Cham, 2016, Vol. 9905, pp. 580–596. DOI: 10.1007/978-3-31946448-0_35
Arbeláez P., Maire M., Fowlkes C., Malik J. Contour detection and hierarchical image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 2010, Vol. 33, № 5, pp. 898– 916. DOI: 10.1109/TPAMI.2010.161
Pont-Tuset J., Arbelaez P., Barron J., Marques F., Malik J. Multiscale combinatorial grouping for image segmentation and object proposal generation, IEEE Trans. Pattern Anal. Mach. Intell., 2016, Vol. 39, № 1, pp. 128–140. DOI: 10.1109/TPAMI.2016.2537320
Xu D., Ouyang W., Alameda-Pineda X. et al. Learning deep structured multi-scale features using attention-gated CRFs for contour predict crisp boundaries, Neural Information Processing Systems: 31st International Conference, NIPS'17, Long Beach, California, USA, 4–9 December, 2017 : proceedings. Curran Associates Inc., 2017, pp. 3964– 3973.
Mnih V., Heess N., Graves A., Kavukcuoglu K. Recurrent models of visual attention, Neural Information Processing Systems: 27th International Conference, NIPS'14, Montreal, Canada, 8–13 December 2014 : proceedings. MIT Press, 2014, Vol. 2, pp. 2204–2212.
Xie S., Tu Z. Holistically-nested edge detection, International Journal of Computer Vision, 2017, Vol. 125, № 5, pp. 1–16. DOI: 10.1007/s11263-017-1004-z
Liu Y., Cheng M. M., Hu X., Wang K., Bai X. Richer convolutional features for edge detection, Computer Vision and Pattern Recognition: IEEE Conference, CVPR, Honolulu, HI, USA, 21–26 July 2017 : proceedings. IEEE, 2017, pp. 5872–5881. DOI: 10.1109/CVPR.2017.622
Deng R., Shen C., Liu S., Wang H., Liu X. Learning to predict crisp boundaries, Computer Vision: 15th European Conference, ECCV, Munich, Germany, 8–14 September, 2018 : proceedings. IEEE, 2018, part VI, pp. 562–578. DOI: 10.1007/978-3-030-01231-1_35
He J., Zhang S., Yang M., Chan Y., Huang T. BDCN: Bidirectional cascade network for all perceptual edge detection, IEEE Trans. Pattern Anal. Mach. Intell., 2022, Vol. 44, № 1, pp. 100–113. DOI: 10.1109/TPAMI.2020.3007074
Poma X. S., Riba E., Sappa A. Dense extreme inception network: towards a robust CNN model for edge detection, Applications of Computer Vision: IEEE/CVF Winter Conference, WACV, Snowmass Village, CO, USA, 1–5 March 2020 : proceedings. IEEE, 2020, pp. 1912–1921. DOI: 10.1109/WACV45572.2020.9093290
Deng R., Liu S. Deep structural contour detection, Multimedia: 28th ACM International Conference, MM'20, Virtual Event, Seattle, WA, USA, October 12–16, 2020 : proceedings. ACM, 2020, pp. 304–312. DOI: 10.1145/3394171.3413750
Su Z., Liu W., Yu Z. et al. Pixel difference networks for efficient edge detection, Computer Vision: IEEE/CVF International Conference, ICCV, online, 11–17 October, 2021: proceedings. IEEE, 2021, pp. 5117–5127. DOI: 10.1109/ICCV48922.2021.00507
Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition, Learning Representations: 3rd International Conference, ICLR 2015, San Diego, CA, USA, 7–9 May 2015 : proceedings [Electronic resource]. Access mode: https://arxiv.org/pdf/1409.1556. DOI: 10.48550/arXiv. 1409.1556
Pinheiro P. O., Lin T.-Y., Collobert R., Dollár P. Learning to refine object segments, Computer Vision – ECCV 2016. Lecture Notes in Computer Science / Leibe, B., Matas, J., Sebe, N., Welling, M. (eds). Springer, Cham, 2016, Vol. 9905, pp. 75–91. DOI: 10.1007/978-3-319- 46448-0_5
Chollet F. Xception: Deep learning with depthwise separable convolutions, Computer Vision and Pattern Recognition: IEEE Conference, CVPR, Honolulu, HI, USA, 21–26 July 2017 : proceedings. IEEE, 2017, pp. 1800– 1807. DOI: 10.1109/CVPR.2017.195
Dai J., Li Y., He K., Sun J. R-fcn: Object detection via region-based fully convolutional networks, Neural Information Processing Systems: 30th International Conference, NIPS'16, Barcelona, Spain, 4–8 December 2016 : proceedings. Red Hook, NY, ACM, 2016, pp. 379– 387.
Chen L. C., Papandreou G., Kokkinos I., Murphy K., Yuille A. L. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected Crfs, IEEE Trans. Pattern Anal. Mach. Intell., 2017, Vol. 40, № 4, pp. 834–848. DOI: 10.1109/TPAMI.2017.2699184
Elsken T., Metzen J. H., Hutter F. Neural architecture search: a survey, Journal of Machine Learning Research, 2019, Vol. 20, № 55, pp. 1–21.
Leung F. H. F., Lam H. K., Ling S. H., Tam P. K. S. Tuning of the structure and parameters of neural network using an improved genetic algorithm, IEEE Transactions on Neural Networks, 2003, Vol. 14, № 1, pp. 79–88. DOI: 10.1109/TNN.2002.804317
Xie W., Nagrani A., Chung J. S., Zisserman A. Utterancelevel aggregation for speaker recognition in the wild, Acoustics, Speech and Signal Processing: IEEE International Conference, ICASSP, Brighton, Great Britain, 12–17 May 2019 : proceedings. IEEE, 2019, pp. 5791– 5795. DOI: 10.1109/ICASSP.2019.8683120
Polyakova M. V., Krylov V. N. Data normalization methods to improve the quality of classification in the breast cancer diagnostic system, Applied Aspects of Information Technology, 2022, Vol. 5, № 1, pp. 55–63. DOI: 10.15276/aait.05.2022.5
The Berkeley Segmentation Dataset and Benchmark Web site (2019) [Electronic Resource]. Access mode: https://www.eecs.berkeley.edu/Research/Projects/CS/vision/ bsds.
The multi-cue boundary detection dataset. Video collection [Electronic Resource]. Access mode: http://serrelab.clps.brown.edu/resources-static/multicue-dataset.tar.bz2
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 M. V. Polyakova
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Creative Commons Licensing Notifications in the Copyright Notices
The journal allows the authors to hold the copyright without restrictions and to retain publishing rights without restrictions.
The journal allows readers to read, download, copy, distribute, print, search, or link to the full texts of its articles.
The journal allows to reuse and remixing of its content, in accordance with a Creative Commons license СС BY -SA.
Authors who publish with this journal agree to the following terms:
-
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License CC BY-SA that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
-
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
-
Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
