Artificial intelligence

Abstract: The article explores approaches to improving the efficiency of intelligent diagnostic modeling systems in conditions of high dimensionality of the feature space and large amounts of data. It is shown that traditional inductive machine learning algorithms, in particular k-nearest neighbors, support vector machines, neural networks and ensemble models, are characterized by high computational complexity of searching for similar objects in multidimensional space, which is associated with the phenomenon of the “curse of dimensionality” and limits their application in operational diagnostic analysis tasks. To reduce computational costs, the paper explores the use of the Locality-Sensitive Hashing (LSH) method, which allows for effective search for nearest neighbors by mapping feature vectors into the hash value space. The combination of LSH with inductive learning algorithms reduces the number of comparison operations and increases the speed of models without significant loss of classification accuracy. The study conducted a comparative analysis of kNN, SVM, multilayer perceptron and ensemble methods in combination with locally sensitive hashing based on the Breast Cancer Wisconsin Diagnostic Dataset using the Python programming language and the Scikit-learn, NumPy, Pandas, Annoy and Matplotlib libraries. Unlike traditional approaches, the evaluation of the effectiveness of the models was carried out not only by the classical metrics Accuracy, Precision, Recall and F1-score and classification time characteristics, but also by the author's proposed system of multi-criteria indicators, which includes the hash index construction time, candidate search time, parametric complexity of the models, collision quality factor, local compactness factor of the hash space and the integral model efficiency indicator. The results obtained showed that the use of LSH allows to significantly increase the speed of inductive machine learning algorithms while maintaining high classification accuracy. It was found that the kNN + LSH model demonstrates the greatest increase in performance, while the Random Forest + LSH ensemble model provides the best generalized ratio between classification accuracy, performance and structural compactness of the formed hash space according to the integral efficiency criterion.

Keywords: inductive learning, locally sensitive hashing, diagnostic modeling, multi-criteria efficiency assessment, machine learning, kNN, SVM, neural networks, ensemble models

References:

1. Cover, T. M., & Hart, P. E. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21–27. https://doi.org/10.1109/TIT.1967.1053964
2. Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297. https://doi.org/10.1007/BF00994018
3. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324
4. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
5. Indyk, P., & Motwani, R. (1998). Approximate nearest neighbors: Toward removing the curse of dimensionality. In Proceedings of the 30th Annual ACM Symposium on Theory of Computing (STOC ’98) (pp. 604–613). ACM. https://doi.org/10.1145/276698.276876
6. Datar, M., Immorlica, N., Indyk, P., & Mirrokni, V. S. (2004). Locality-sensitive hashing scheme based on p-stable distributions. In Proceedings of the 20th Annual Symposium on Computational Geometry (SCG ’04) (pp. 253–262). ACM. https://doi.org/10.1145/997817.997857
7. Slaney, M., & Casey, M. (2008). Locality-sensitive hashing for finding nearest neighbors. IEEE Signal Processing Magazine, 25(2), 128–131. https://doi.org/10.1109/MSP.2007.914237
8. Paulevé, L., Jégou, H., & Amsaleg, L. (2010). Locality-sensitive hashing: A comparison of hash function types and querying mechanisms. Pattern Recognition Letters, 31(11), 1348–1358. https://doi.org/10.1016/j.patrec.2010.04.004
9. Chi, L., & Zhu, X. (2017). Hashing techniques: A survey and taxonomy. ACM Computing Surveys, 50(1), Article 11, 1–36. https://doi.org/10.1145/3047307
10. Wang, J., Zhang, T., Song, J., Sebe, N., & Shen, H. T. (2018). A survey on learning to hash. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 769–790. https://doi.org/10.1109/TPAMI.2017.2699960
11. Aumüller, M., Bernhardsson, E., & Faithfull, A. (2020). ANN-Benchmarks: A benchmarking tool for approximate nearest neighbor algorithms. Information Systems, 87, Article 101374. https://doi.org/10.1016/j.is.2019.02.006
12. Aumüller, M., & Ceccarello, M. (2021). The role of local dimensionality measures in benchmarking nearest neighbor search. Information Systems, 101, Article 101807. https://doi.org/10.1016/j.is.2021.101807
13. Subbotin, S. A. (2014). Methods and characteristics of locality-preserving transformations in the problems of computational intelligence. Radio Electronics, Computer Science, Control, (1), 120–128. https://doi.org/10.15588/1607-3274-2014-1-17
14. Naderan, M., & Zaychenko, Y. P. (2019). Methods for improving accuracy of the dementia diagnosis using feature dimension reduction. System Research and Information Technologies, (2), 25–30. https://doi.org/10.20535/SRIT.2308-8893.2019.2.03
15. Nastenko, I. A., Maksymenko, V. B., Potashev, S. V., Pavlov, V. A., Babenko, V. O., Rysin, S. V., Matviichuk, O. V., & Lazoryshynets, V. V. (2021). Random forest algorithm construction for the diagnosis of coronary heart disease based on echocardiography video data streams. Innovative Biosystems and Bioengineering, 5(1), 61–69. https://doi.org/10.20535/ibb.2021.5.1.225794
16. Petrunina, O., Shevaga, D., Babenko, V., Pavlov, V., Rysin, S., & Nastenko, I. (2021). Comparative analysis of classification algorithms in the analysis of medical images from speckle tracking echocardiography video data. Innovative Biosystems and Bioengineering, 5(3), 153–166. https://doi.org/10.20535/ibb.2021.5.3.234990
17. Tsiutsiura, M. I., Kovalenko, A. Yu. (2024). Otsinka alhorytmiv vyiavlennia anomalii za dopomohoiu metodiv mashynnoho navchannia (Evaluation of anomaly detection algorithms using machine learning methods). Upravlinnia rozvytkom skladnykh system, (58), 80–85. https://doi.org/10.32347/2412-9933.2024.58.80-85
18. Wolberg, W. H., Street, W. N., Mangasarian, O. L. (1995) Machine learning techniques to diagnose breast cancer from fine-needle aspirates. Cancer Letters, 77 (2–3), 163–171. doi: 10.1016/0304-3835(93)90194-N.