Artificial intelligence

Abstract: The article proposes an inverse architecture of the U-Net neural network, named InvU-Net, which differs from the traditional scheme by increasing the dimensionality of images during the initial stages of processing. A comparison was conducted between two approaches for increasing image resolution: UpSampling2D layers and transposed Conv2DTranspose convolutional layers. The latter demonstrated superior results due to its ability to learn weighting coefficients. As part of the study, several InvU-Net modifications were developed and tested: Small, Medium, and Large, differing in structural complexity, the number of layers, and parameters. To improve segmentation accuracy, the integration of attention mechanisms was proposed to enhance the relevance of feature processing. Experiments revealed that simplifying attention mechanisms, including reducing the number of parameters and optimizing integration points, achieves high performance with lower computational complexity. The best-performing model, which incorporated a simplified attention mechanism, achieved 95.6% accuracy, surpassing larger architectures. The results highlight the potential of InvU-Net for segmentation tasks and suggest further optimization directions, such as employing adaptive attention mechanisms and automating the selection of neural network parameters.

Keywords: image segmentation, neural networks, U-Net architecture, InvU-Net, convolutional layers, Conv2DTranspose, UpSampling2D, attention mechanism, parameter optimization

References:

1. J. Long, E. Shelhamer, and T. Darrell, “Fully Convolutional Networks for Semantic Segmentation,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, pp. 3431–3440. DOI: 10.1109/CVPR.2015.7298965.
2. V. Slyusar, M. Protsenko, A. Chernukha, V. Melkin, O. Petrova, M. Kravtsov, S. Velma, N. Kosenko, O. Sydorenko and M. Sobol, “Improving a neural network model for semantic segmentation of images of monitored objects in aerial photographs,” EasternEuropean Journal of Enterprise Technologies, 2021, vol. 2, no. 6 (114), pp. 86-95. DOI: 10.15587/1729-4061.2021.248390
3. Vadym Slyusar, Ihor Sliusar, Nataliia Bihun, and Volodymyr Piliuhin. Segmentation of analogue meter readings using neural networks. // 44th International Workshop on Modern Machine Learning Technologies and Data Science MOMLET&DS2022, Leiden - Lviv, The Netherlands - Ukraine, 23 -25 Nov. 2022, Pp. 165 - 175.
4. O. Ronneberger, P. Fischer, and T. Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv preprint arXiv:1505.04597, 2015. DOI: 10.48550/arXiv.1505.04597.
5. W. Jwaid, Z. Al-Husseini and A. Sabry, Development of brain tumor segmentation of magnetic resonance imaging (MRI) using U-Net deep learning, Eastern-European Journal of Enterprise Technologies, vol. 4, no. 9 (112), 2021, pp. 23-31. doi:10.15587/1729-4061.2021.238957.
6. N. Singh and K. Nongmeikapam, Semantic segmentation of satellite images using deep-UNet, Arabian Journal for Science and Engineering, 2022, pp. 1-13.
7. A. Soni, R. Koner, and V. Villuri, M-Unet: Modified U-Net segmentation framework with satellite imagery, in: Proceedings of the Global AI Congress 2019, Springer, 2020, pp. 47-59.
8. E. Irwansyah, Y. Heryadi, and A. Gunawan, Semantic image segmentation for building detection in urban area with aerial photograph image using U-Net models, in: Proceedings of the 2020 IEEE Asia-Pacific Conf. on Geoscience, Electronics and Remote Sensing Technology (AGERS), 2020, pp. 48-51.
9. Vadym Slyusar, Ihor Sliusar, Pavlenko Anatolii. Improved PSP and U-Net architectures for forest segmentation in remote sensing pictures.// IEEE 2nd Ukrainian Microwave Week (Virtual Event) IEEE UkrMW-2022, 14 – 18 November, 2022. - V. N. Karazin Kharkiv National University, Kharkiv, Ukraine. – 4 p. DOI: 10.1109/UkrMW58013.2022.10037105.
10. N. Singh and K. Nongmeikapam, “Semantic segmentation of satellite images using deep-UNet,” Arabian Journal for Science and Engineering, 2022, pp. 1-13.
11. A. Soni, R. Koner, and V. G. K. Villuri, “M-Unet: Modified U-Net segmentation framework with satellite imagery,” in Proceedings of the Global AI Congress 2019, Springer, 2020, pp. 47-59.
12. E. Irwansyah, Y. Heryadi, and A. Gunawan, “Semantic image segmentation for building detection in urban area with aerial photograph image using U-Net models,” in 2020 IEEE Asia-Pacific Conf. on Geoscience, Electronics and Remote Sensing Technology (AGERS), 2020, pp. 48-51.
13. V. Slyusar. The text segmentation by neural networks of image segmentation.//Artificial Intelligence, 2024, Vol. №98, No 1.- Pp. 46 - 55. DOI: 10.15407/jai2024.01.046.
14. A. Vaswani et al., “Attention Is All You Need,” in Advances in Neural Information Processing Systems, vol. 30, 2017, pp. 5998–6008. DOI: 10.48550/arXiv.1706.03762.
15. Vadym Slyusar. Application of neural network technologies for underwater munitions detection. // Radioelectronics and Communications Systems, Vol. 65, No. 12 (2022). - Pp. 654–664. DOI: 10.3103/S0735272723030020..
16. C. C. Aggarwal, Neural Networks and Deep Learning, 2nd ed. Cham, Switzerland: Springer, 2023. DOI: 10.1007/978-3-031-29642-0.
17. Slyusar V.I., Sliusar I.I. Analysis of the efficiency of attention mechanisms in the task of segmenting images of analog meter dials. // 21st International Scientific Conference "Neural Network Technologies and Applications NNTA-2022", December 7-8, 2022. Kramatorsk-Ternopil, Ukraine. - 7 p.
18. Slyusar, V. I. A Family of Face Products of Matrices and its Properties. // Cybernetics and Systems Analysis c/c of Kibernetika i Sistemnyi Analiz. – 1999, Vol 35; Part 3, pages 379-384. – Consultants Bureau (USA). - DOI: 10.1007/BF02733426.
19. Slyusar, V. I. Architectural and mathematical fundamentals of improvement neural networks for classification of images. // Artificial Intelligence, 2022, №1.- Pp. 245 - 258. DOI: 10.15407/jai2022.01.245.