Штучний інтелект

Анотація: Sequential recommenders based on self-attention discard a rich supervised signal: per-interaction intensity (a 5-star rating, hours played, dwell time, purchase value). We hypothesised that re-introducing intensity into the attention computation should improve top-k accuracy in proportion to how informative the underlying signal is. We test the hypothesis with three drop-in modifications of SASRec (IA-SASRec-Add, IA-SASRec-Mul, IA-SASRec-Val), corresponding to the three algebraic positions where intensity can be threaded into the attention expression. Each variant adds a single learnable scalar λ per attention layer, designed so that λ=0 recovers vanilla SASRec exactly: the model is a strict super-set of the baseline. We evaluate across seven datasets (MovieLens×2, Amazon×2, Steam×3) with five seeds per configuration, under a dual-criterion significance protocol with a per-seed stability check (seed-level paired t and per-user paired bootstrap with B=2,000). Under this protocol, no variant achieves a robust improvement on NDCG@10, Recall@10, or MRR@10 anywhere. The pattern inverts the hypothesis: Steam (the dataset with the richest intensity signal) hosts the largest significant degradations (up to -9.5% NDCG@10, p<0.05). Mechanism analysis through the learned λ explains the failure: λ adapts freely on datasets where intensity is not informative (down to 0.07 on ML-1M) but stays at 0.55–0.99 on Steam, where the variants are at their worst. We argue that a single per-layer scalar gate is too limited a control for intensity injection, and outline three concrete designs that target this limitation.

Ключові слова: sequential recommendation, self-attention, SASRec, side information fusion, intensity-aware attention, reproducibility, negative result, mechanism analysis.

Посилання:

1. Kang, W.-C., & McAuley, J. (2018). Self-attentive sequential recommendation. Proceedings of the IEEE International Conference on Data Mining (ICDM), 197–206. https://doi.org/10.1109/ICDM.2018.00035
2. Sun, F., Liu, J., Wu, J., Pei, C., Lin, X., Ou, W., & Jiang, P. (2019). BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM), 1441–1450. https://doi.org/10.1145/3357384.3357895
3. Ferrari Dacrema, M., Cremonesi, P., & Jannach, D. (2019). Are we really making much progress? A worrying analysis of recent neural recommendation approaches. Proceedings of the 13th ACM Conference on Recommender Systems (RecSys), 101–109. https://doi.org/10.1145/3298689.3347058
4. Sun, Z., Yu, D., Fang, H., Yang, J., Qu, X., Zhang, J., & Geng, C. (2020). Are we evaluating rigorously? Benchmarking recommendation for reproducible evaluation and fair comparison. Proceedings of the 14th ACM Conference on Recommender Systems (RecSys), 23–32. https://doi.org/10.1145/3383313.3412489
5. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems (NeurIPS), 5998–6008. https://doi.org/10.48550/arXiv.1706.03762
6. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT 2019, 4171–4186. https://doi.org/10.18653/v1/N19-1423
7. Li, J., Wang, Y., & McAuley, J. (2020). Time interval aware self-attention for sequential recom-mendation. Proceedings of the 13th International Conference on Web Search and Data Mining (WSDM), 322–330. https://doi.org/10.1145/3336191.3371786
8. Hidasi, B., Karatzoglou, A., Baltrunas, L., & Tikk, D. (2016). Session-based recommendations with recurrent neural networks. International Conference on Learning Representations (ICLR). https://doi.org/10.48550/arXiv.1511.06939
9. Li, J., Ren, P., Chen, Z., Ren, Z., Lian, T., & Ma, J. (2017). Neural attentive session-based recommendation. Proceedings of the 2017 ACM Conference on Information and Knowledge Management (CIKM), 1419–1428. https://doi.org/10.1145/3132847.3132926
10. Rendle, S., Freudenthaler, C., & Schmidt-Thieme, L. (2010). Factorizing personalized Markov chains for next-basket recommendation. Proceedings of the 19th International Conference on World Wide Web (WWW), 811–820. https://doi.org/10.1145/1772690.1772773
11. Tang, J., & Wang, K. (2018). Personalized top-n sequential recommendation via convolutional sequence embedding. Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM), 565–573. https://doi.org/10.1145/3159652.3159656
12. Yuan, F., Karatzoglou, A., Arapakis, I., Jose, J. M., & He, X. (2019). A simple convolutional generative network for next item recommendation. Proceedings of the 12th ACM International Conference on Web Search and Data Mining (WSDM), 582–590. https://doi.org/10.1145/3289600.3290975
13. Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt-Thieme, L. (2009). BPR: Bayesian personalized ranking from implicit feedback. Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI), 452–461. https://doi.org/10.5555/1795114.1795167
14. Zhang, T., Zhao, P., Liu, Y., Sheng, V. S., Xu, J., Wang, D., Liu, G., & Zhou, X. (2019). Feature-level deeper self-attention network for sequential recommendation. Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI), 4320–4326. https://doi.org/10.24963/ijcai.2019/600
15. Zhou, K., Wang, H., Zhao, W. X., Zhu, Y., Wang, S., Zhang, F., Wang, Z., & Wen, J.-R. (2020). S^3-Rec: Self-supervised learning for sequential recommendation with mutual information maximization. Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM), 1893–1902. https://doi.org/10.1145/3340531.3411954
16. Hu, Y., Koren, Y., & Volinsky, C. (2008). Collaborative filtering for implicit feedback datasets. Proceedings of the 8th IEEE International Conference on Data Mining (ICDM), 263–272. https://doi.org/10.1109/ICDM.2008.22
17. Pan, R., Zhou, Y., Cao, B., Liu, N. N., Lukose, R., Scholz, M., & Yang, Q. (2008). One-class collaborative filtering. Proceedings of the 8th IEEE International Conference on Data Mining (ICDM), 502–511. https://doi.org/10.1109/ICDM.2008.16
18. Liu, C., Li, X., Cai, G., Dong, Z., Zhu, H., & Shang, L. (2021). Noninvasive self-attention for side information fusion in sequential recommendation. Proceedings of the 35th AAAI Conference on Artificial Intelligence, 4249–4256. https://doi.org/10.1609/aaai.v35i5.16549
19. Xie, Y., Zhou, P., & Kim, S. (2022). Decoupled side information fusion for sequential recommendation. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 1611–1621. https://doi.org/10.1145/3477495.3531963
20. Rashed, A., Elsayed, S., & Schmidt-Thieme, L. (2022). CARCA: Context and attribute-aware next-item recommendation via cross-attention. Proceedings of the 16th ACM Conference on Recommender Systems (RecSys), 71–80. https://doi.org/10.1145/3523227.3546777
21. Ma, C., Kang, P., & Liu, X. (2019). Hierarchical gating networks for sequential recommendation. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD), 825–833. https://doi.org/10.1145/3292500.3330984
22. Hou, Y., Mu, S., Zhao, W. X., Li, Y., Ding, B., & Wen, J.-R. (2022). Towards universal sequence representation learning for recommender systems. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD), 585–593. https://doi.org/10.1145/3534678.3539381
23. Wu, L., Li, S., Hsieh, C.-J., & Sharpnack, J. (2020). SSE-PT: Sequential recommendation via personalized transformer. Proceedings of the 14th ACM Conference on Recommender Systems (RecSys), 328–337. https://doi.org/10.1145/3383313.3412258
24. Cho, S. M., Park, E., & Yoo, S. (2020). MEANTIME: Mixture of attention mechanisms with multi-temporal embeddings for sequential recommendation. Proceedings of the 14th ACM Conference on Recommender Systems (RecSys), 515–520. https://doi.org/10.1145/3383313.3412216
25. Zhou, K., Yu, H., Zhao, W. X., & Wen, J.-R. (2022). Filter-enhanced MLP is all you need for sequential recommendation. Proceedings of the ACM Web Conference (WWW), 2388–2399. https://doi.org/10.1145/3485447.3512111
26. Fan, Z., Liu, Z., Wang, A., Nazari, Z., Zheng, L., Peng, H., & Yu, P. S. (2022). Sequential recommendation via stochastic self-attention. Proceedings of the ACM Web Conference (WWW), 2036–2047. https://doi.org/10.1145/3485447.3512077
27. Fan, X., Liu, Z., Lian, J., Zhao, W. X., Xie, X., & Wen, J.-R. (2021). Lighter and better: Low-rank decomposed self-attention networks for next-item recommendation. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 1733–1737. https://doi.org/10.1145/3404835.3462440
28. Hou, Y., Hu, B., Zhang, Z., & Zhao, W. X. (2022). CORE: Simple and effective session-based recommendation within consistent representation space. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 1796–1801. https://doi.org/10.1145/3477495.3531955
29. Xie, X., Sun, F., Liu, Z., Wu, S., Gao, J., Zhang, J., Ding, B., & Cui, B. (2022). Contrastive learning for sequential recommendation. Proceedings of the 38th IEEE International Conference on Data Engineering (ICDE), 1259–1273. https://doi.org/10.1109/ICDE53745.2022.00099
30. Qiu, R., Huang, Z., Yin, H., & Wang, Z. (2022). Contrastive learning for representation degeneration problem in sequential recommendation. Proceedings of the 15th ACM International Conference on Web Search and Data Mining (WSDM), 813–823. https://doi.org/10.1145/3488560.3498433
31. Chen, Y., Liu, Z., Li, J., McAuley, J., & Xiong, C. (2022). Intent contrastive learning for sequential recommendation. Proceedings of the ACM Web Conference (WWW), 2172–2182. https://doi.org/10.1145/3485447.3512090
32. Liu, Z., Fan, Z., Wang, Y., & Yu, P. S. (2021). Augmenting sequential recommendation with pseudo-prior items via reversely pre-training transformer. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 1608–1612. https://doi.org/10.1145/3404835.3463036
33. Rendle, S., Krichene, W., Zhang, L., & Anderson, J. (2020). Neural collaborative filtering vs. Matrix factorization revisited. Proceedings of the 14th ACM Conference on Recommender Systems (RecSys), 240–248. https://doi.org/10.1145/3383313.3412488
34. Krichene, W., & Rendle, S. (2020). On sampled metrics for item recommendation. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD), 1748–1757. https://doi.org/10.1145/3394486.3403226
35. Petrov, A., & Macdonald, C. (2022). A systematic review and replicability study of BERT4Rec for sequential recommendation. Proceedings of the 16th ACM Conference on Recommender Systems (RecSys), 436–447. https://doi.org/10.1145/3523227.3548487
36. Ba, J. L., Kiros, J. R., & Hinton, G. E. (2016). Layer normalization. arXiv preprint arXiv:1607.06450. https://doi.org/10.48550/arXiv.1607.06450
37. Akiba, T., Sano, S., Yanase, T., Ohta, T., & Koyama, M. (2019). Optuna: A next-generation hyperparameter optimization framework. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD), 2623–2631. https://doi.org/10.1145/3292500.3330701
38. Harper, F. M., & Konstan, J. A. (2015). The MovieLens datasets: History and context. ACM Transactions on Interactive Intelligent Systems, 5(4), 1–19. https://doi.org/10.1145/2827872
39. McAuley, J., Targett, C., Shi, Q., & Hengel, A. van den. (2015). Image-based recommendations on styles and substitutes. Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, 43–52. https://doi.org/10.1145/2766462.2767755
40. He, R., & McAuley, J. (2016). Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. Proceedings of the 25th International Conference on World Wide Web (WWW), 507–517. https://doi.org/10.1145/2872427.2883037
41. Pathak, A., Gupta, K., & McAuley, J. (2017). Generating and personalizing bundle recommendations on Steam. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, 1073–1076. https://doi.org/10.1145/3077136.3080724
42. Wan, M., & McAuley, J. (2018). Item recommendation on monotonic behavior chains. Proceedings of the 12th ACM Conference on Recommender Systems (RecSys), 86–94. https://doi.org/10.1145/3240323.3240369
43. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. Chapman & Hall/CRC. https://doi.org/10.1201/9780429246593 44. Zhao, W. X., Mu, S., Hou, Y., Lin, Z., Chen, Y., Pan, X., Li, K., Lu, Y., Wang, H., Tian, C., Min, Y., Feng, Z., Fan, X., Chen, X., Wang, P., Ji, W., Li, Y., Wang, X., & Wen, J.-R. (2021). RecBole: Towards a unified, comprehensive and efficient framework for recommendation algorithms. Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM), 4653–4664. https://doi.org/10.1145/3459637.3482016 45. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., … Chintala, S. (2019). PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems (NeurIPS), 8024–8035. https://doi.org/10.48550/arXiv.1912.01703