|
|
|
| Local Citation Recommendation Based on Contextual Semantics and Global Information |
| Zhang Xiaojuan, Ma Le |
| School of Public Administration, Sichuan University, Chengdu 610065 |
|
|
|
|
Abstract Local citation recommendations can help researchers obtain relevant references efficiently and quickly. Existing local citation recommendation methods primarily focus on identifying candidate citations based on limited contextual semantic information, which generally provides limited improvements in the accuracy of the recommendation results. To improve the accuracy of local citation recommendations, this study further integrated global information (i.e., global semantics and global relations) based on contextual semantics. First, the pre-trained SciBERT (scientific bidirectional encoder representations from transformers) model was fine-tuned using custom tasks to extract the semantic embeddings of the citation context. To take advantage of the highly summarized global information provided by titles and abstracts of articles, we used Sentence-BERT and fine-tuned SciBERT models to extract embedded vectors for titles and abstracts of articles, respectively. Subsequently, a heterogeneous graph including three types of nodes—authors, papers, and venues (conference or journals)—was constructed, and a relational graph convolutional network (R-GCN) was used to aggregate three different types of relationships (citation, authorship, and publication) for generating embedded vectors for papers and authors. Finally, the issue of citation recommendation was formulated as a multi-classification task. The embedded representations of the citation context, title, abstract, target paper, and author of the target paper were concatenated to construct the input for the recommendation model. This model was then trained using a feedforward neural network (FFNN) and softmax to generate a list of candidate citations for a given context of a target paper. Two optimization strategies (distributed data parallelism on multiple devices and model compression on a single device) were leveraged to further improve the computational efficiency of the proposed model and reduce its operation time cost. Experimental results demonstrated that the proposed method effectively improves the accuracy of local citation recommendations. Based on contextual semantics, the embedded representations of the target paper contributed the most to the performance of the proposed model among all the vectors of global information (i.e., title semantic, abstract, author, and paper vectors). The contribution of global relations to the recommendation performance of the proposed model was greater than that of global semantics. Among all global relationships (citation, authorship, and publication), the historical citation relation contributed the most to improving the overall performance of local citation recommendations. Both optimization strategies improved the operational efficiency of the model.
|
|
Received: 27 November 2024
|
|
|
|
1 张晓娟, 刘怡均, 刘杰, 等. 个性化学术论文推荐研究综述[J]. 情报学报, 2024, 43(1): 106-126.
2 马霄, 邓秋淼, 张红玉, 等. 融合异质图表示学习与注意力机制的可解释论文推荐[J]. 情报学报, 2024, 43(7): 802-817.
3 F?rber M, Jatowt A. Citation recommendation: approaches and datasets[J]. International Journal on Digital Libraries, 2020, 21(4): 375-405.
4 Jebari C, Herrera-Viedma E, Cobo M J. Context-aware citation recommendation of scientific papers: comparative study, gaps and trends[J]. Scientometrics, 2023, 128(8): 4243-4268.
5 陈海华, 孟睿, 陆伟. 学术文献引文推荐研究进展[J]. 图书情报工作, 2015, 59(15): 133-143, 147.
6 张金柱, 孙雯雯, 仇蒙蒙. 融合异构网络表示学习与注意力机制的引文推荐研究[J]. 数据分析与知识发现, 2024, 8(10): 14-27.
7 Kammari M, Bhavani S D. Citation recommendation using modified HITS algorithm[J]. Computing, 2024, 106(7): 2239-2259.
8 Pornprasit C, Liu X, Kiattipadungkul P, et al. Enhancing citation recommendation using citation network embedding[J]. Scientometrics, 2022, 127(1): 233-264.
9 熊回香, 黄晓捷, 陈子薇, 等. 基于篇章结构和引用动机的引文推荐研究[J]. 图书情报工作, 2023, 67(8): 115-128.
10 Blei D M, Ng A Y, Jordan M I. Latent Dirichlet allocation[J]. Journal of Machine Learning Research, 2003, 3: 993-1022.
11 Airoldi E M, Blei D M, Fienberg S E, et al. Mixed membership stochastic block models for relational data with application to protein-protein interactions[C]// Proceedings of the International Biometrics Society Annual Meeting, Hong Kong, 2006: 1-34.
12 He Q, Pei J, Kifer D, et al. Context-aware citation recommendation[C]// Proceedings of the 19th International Conference on World Wide Web. New York: ACM Press, 2010: 421-430.
13 Beltagy I, Lo K, Cohan A. SciBERT: a pretrained language model for scientific text[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg: Association of Computational Linguistics, 2019: 3613-3618.
14 Nallapati R M, Ahmed A, Xing E P, et al. Joint latent topic models for text and citations[C]// Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM Press, 2008: 542-550.
15 Huang W Y, Kataria S, Caragea C, et al. Recommending citations: translating papers into references[C]// Proceedings of the 21st ACM International Conference on Information and Knowledge Management. New York: ACM Press, 2012: 1910-1914.
16 Huang W Y, Wu Z H, Liang C, et al. A neural probabilistic model for context based citation recommendation[C]// Proceedings of the 29th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2015: 708-714.
17 Yang L B, Zhang Z Q, Cai X Y, et al. Attention-based personalized encoder-decoder model for local citation recommendation[J]. Computational Intelligence and Neuroscience, 2019, 2019(1): 1232581.
18 F?rber M, Sampath A. HybridCite: a hybrid model for context-aware citation recommendation[C]// Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020. New York: ACM Press, 2020: 117-126.
19 Han J L, Song Y, Zhao W X, et al. hyperdoc2vec: distributed representations of hypertext documents[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2018: 2384-2394.
20 Jeong C, Jang S, Park E, et al. A context-aware citation recommendation model with BERT and graph convolutional networks[J]. Scientometrics, 2020, 124(3): 1907-1922.
21 Dinh T N, Pham P, Nguyen G L, et al. Enhanced context-aware citation recommendation with auxiliary textual information based on an auto-encoding mechanism[J]. Applied Intelligence, 2023, 53(14): 17381-17390.
22 Dinh T N, Pham P, Nguyen G L, et al. Enhancing local citation recommendation with recurrent highway networks and SciBERT-based embedding[J]. Expert Systems with Applications, 2024, 243: 122911.
23 Yin M J, Wang B Y, Ling C. A fast local citation recommendation algorithm scalable to multi-topics[J]. Expert Systems with Applications, 2024, 238: 122031.
24 Reimers N, Gurevych I. Sentence-BERT: sentence embeddings using siamese BERT-networks[C]// Proceedings of the 9th International Joint Conference on Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2019: 3980-3990.
25 Schlichtkrull M, Kipf T N, Bloem P, et al. Modeling relational data with graph convolutional networks[C]// Proceedings of the 15th Extended Semantic Web Conference. Cham: Springer, 2018: 593-607.
26 陆佳丽. 基于BERT-TextCNN的开源威胁情报文本的多标签分类方法[J]. 信息安全研究, 2024, 10(8): 760-768.
27 Liu Y H, Ott M, Goyal N, et al. RoBERTa: a robustly optimized BERT pretraining approach[OL]. (2019-07-26). https://arxiv.org/pdf/1907.11692.
28 Kang D, Ammar W, Dalvi B, et al. A dataset of peer reviews (PeerRead): collection, insights and NLP applications[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2018: 1647-1661.
29 Raffel C, Shazeer N, Roberts A, et al. Exploring the limits of transfer learning with a unified text-to-text transformer[J]. Journal of Machine Learning Research, 2020, 21(140): 1-67.
30 Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks[C]// Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Lille: PMLR Press, 2010: 249-256.
31 Kingma D P, Ba J L. Adam: a method for stochastic optimization[OL]. (2017-01-30). https://arxiv.org/pdf/1412.6980.
32 Ebesu T, Fang Y. Neural citation network for context-aware citation recommendation[C]// Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM Press, 2017: 1093-1096.
33 Collins M. Statistical machine translation: IBM models 1 and 2[OL]. [2024-09-20]. https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1162/handouts/Collins_annotated.pdf. 责任编辑 王克平） |
|
|
|
