Artificial intelligence

Abstract: The paper identifies and describes promising architectural solutions for convolutional neural networks, along with key parameters for further structural and parametric synthesis. The advantages and disadvantages of different blocks are outlined, and their relevance in structural synthesis is substantiated. The use of a genetic evolutionary algorithm for structural-parametric synthesis is proposed, along with a review of contemporary approaches. The configuration process of the evolutionary algorithm is shown and described. Based on optimization criteria, the fitness function, selection, mutation, and crossover methods are defined. The results of the experimental evolutionary process are presented and analyzed. An example model created by evolutionary algorithms is considered, which is based on functional blocks aggregated from various architectural approaches in convolutional neural networks. For each model, performance criteria, including average reduction in training time, benefits, and architectural integration details, were calculated during the synthesis process. Experimental results demonstrate that using complex structural blocks instead of traditional layers with a flexible fitness function configuration according to quality and performance criteria yields significant improvements for the final model.

Keywords: structural parametric synthesis, evolutionary algorithm, convolutional neural network

References:

1. M. Meza-Sánchez, E. Clemente, M.C. Rodríguez-Liñán, G. Olague, "Synthetic-analytic behavior-based control framework: Constraining velocity in tracking for nonholonomic wheeled mobile robots" in Information Sciences, vol. 501, 2019, pp. 436-459, doi: 10.1016/j.ins.2019.06.025.
2. S. Jiang and S. Yang, "A Strength Pareto Evolutionary Algorithm Based on Reference Direction for Multiobjective and Many-Objective Optimization," in IEEE Transactions on Evolutionary Computation, vol. 21, no. 3, pp. 329-346, June 2017, doi: 10.1109/TEVC.2016.2592479.
3. J. Li, Y. Wen and L. He, "SCConv: Spatial and Channel Reconstruction Convolution for Feature Redundancy," 2023 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 2023, pp. 6153-6162, doi: 10.1109/CVPR52729.2023.00596.
4. Dürr, P., Mattiussi, C., Floreano, D., ”Neuroevolution with Analog Genetic Encoding” in Parallel Problem Solving from Nature, 2006, vol 4193. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11844297_68.
5. C. Nagpal and S. R. Dubey, "A Performance Evaluation of Convolutional Neural Networks for Face Anti Spoofing," in International Joint Conference on Neural Networks, Budapest, Hungary, 2019, pp. 1-8, doi: 10.1109/IJЗНМ.2019.8852422.
6. D. Kingma, J. Ba, “Adam: A Method for Stochastic Optimization. International Conference on Learning Representations”. 2015, 30 Jan 2017, pp. 1–15.
7. N. Hansen, D. V. Arnold and A. Auger, “Evolution Strategies”, 2018, doi: 10.1007/978-3-319-07124-4_13.
8. C. Cao, Y. Huang, Y. Yang, L. Wang, Z. Wang and T. Tan, "Feedback Convolutional Neural Network for Visual Localization and Segmentation," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 7, pp. 1627-1640, 1 July 2019, doi: 10.1109/TPAMI.2018.2843329.
9. M. K. Yadav and K. P. Sharma, "Intrusion Detection System using Machine Learning Algorithms: A Comparative Study," 2021 2nd International Conference on Secure Cyber Computing and Communications, Jalandhar, India, 2021, pp. 415-420, doi: 10.1109/ICSCCC51823.2021.9478086.
10. N. Shone, T. N. Ngoc, V. D. Phai and Q. Shi, "A Deep Learning Approach to Network Intrusion Detection," in IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 2, no. 1, pp. 41-50, Feb. 2018, doi: 10.1109/TETCI.2017.2772792.