Artificial intelligence

Abstract: The structural-parametric synthesis of neural networks of deep learning, in particular convolutional neural networks used in image processing, is considered. The classification of modern architectures of convolutional neural networks is given. It is shown that almost every convolutional neural network, depending on its topology, has unique blocks that determine its essential features (for example, Squeeze and Excitation Block, Convolutional Block of Attention Module (Channel attention module, Spatial attention module), Residual block, Inception module, ResNeXt block. It is stated the problem of structural-parametric synthesis of convolutional neural networks, for the solution of which it is proposed to use a genetic algorithm. The genetic algorithm is used to effectively overcome a large search space: on the one hand, to generate possible topologies of the convolutional neural network, namely the choice of specific blocks and their locations in the structure of the convolutional neural network, and on the other hand to solve the problem of structural-parametric synthesis of convolutional neural network of selected topology. The most significant parameters of the convolutional neural network are determined. An encoding method is proposed that allows to repre- sent each network structure in the form of a string of fixed length in binary format. After that, several standard genetic operations were identified, i.e. selection, mutation and crossover, which eliminate weak individuals of the previous generation and use them to generate competitive ones. An example of solving this problem is given, a database (ultrasound results) of patients with thyroid disease was used as a training sample.

Keywords: structural-parametric synthesis; convolutional neural network; multicriteria genetic algorithm; Squeeze and Excitation Block; Convolutional Block of Attention Module; Residual block; deep learning

References:

1. Al-Marzouqi H. (2009) Data Clustering Using a Modified Kuwahara Filter, Neural Networks. International Joint Conference. 128-132. doi.org/10.1109/IJCNN.2009.5178658
2. LeCun Y., Boser B., Denker J. S., Henderson D., Howard R. E., Hubbard W. and Jackel L. D. (1989) Backpropagation Applied to Handwritten Zip Code Recognition, 541-551. doi.org/10.1162/neco.1989.1.4.541
3. Lee H., Grosse R., Ranganath R., and Ng A. Y. (2009a). Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In ICML’2009. https://doi.org/10.1145/1553374.1553453
4. Synyehlazov V. M., Chumachenko O. I. (2017) Deep Learning Classifier Based on NEFCLASS and NEFPROX Neural Networks. Materialy mizhnarodnoyi naukovo-praktychnoyi konferentsiyi "Informatsiyni tekhnolohiyi ta kompʺyuterne modelyuvannya". – Ivano-Frankivsʹk – Yaremche, Ukrayina (15–20 travnya 2017), 278-281.
5. Chumachenko E. Y., Levytskyy O. YU. (2011) Razrabotka alhorytma obrabotky yzobrazhenyy dlya zadach dyahnostyky. Elektronika ta systemy upravlinnya. K.: NAU, 1 (27), 57-65.
6. Zgurovsky Michael Z., Sineglazov Victor M., Chumachenko Olena I. (2020). Artificial Intelligence Systems Based on Hybrid Neural Networks, Springer, p. 390. https://doi.org/10.1007/978-3-030-48453-8
7. Asifullah Khan, Anabia Sohail, Umme Zahoora, and Aqsa Saeed Qureshi. A Survey of the Recent Architectures of Deep Convolutional Neural Networks. Published in Artificial Intelligence Review, doi.org/10.1007/s10462-020-09825-6
8. Franocois Chollet. (2017). Xception: Deep Learning with Depthwise Separable Convolutions. arXiv:1610.02357v3 [cs.CV] 4 Apr 2017
9. Saining Xie, Ross Girshick, Piotr Dollaґr, Zhuowen Tu, Kaiming He. (2017). Aggregated Residual Transformations for Deep Neural Networks. arXiv:1611.05431v2 [cs.CV] 11 Apr 2017
10. Xingcheng Zhang, Zhizhong Li, and Chen Change Loy Dahua Lin. (2017). PolyNet: A Pursuit of Structural Diversity in Very Deep Networks. arXiv:1611.05725v2 [cs.CV] 17 Jul 2017
11. Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Cheng Li, Honggang Zhang, Xiaogang Wang, and Xiaoou Tang. (2017). Residual Attention Network for Image Classification. arXiv:1704.06904v1 [cs.CV] 23 Apr 2017
12. Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. (2018). CBAM: Convolutional Block Attention Module. arXiv:1807.06521v2 [cs.CV] 18 Jul 2018
13. Jie Hu1, Li Shen, and Gang Sun. (2018). Squeeze-and-Excitation Networks: https://openaccess.thecvf.com/content_cvpr_2018/papers/Hu_Squeeze-and-Excitation_Networks_CVPR_2018_paper.pdf