|
|
|
| Long-Text Relation Recognition Based on LLM-BERT Collaborative Framework |
| Wu Shuai1, He Lin1,2,3, Lyu Xingyue1, Lu Yingjie1, Wu Can1, Wang Xinzhe1 |
1.College of Information Management, Nanjing Agricultural University, Nanjing 211800
2.National Experimental Base for Intelligent Social Governance, Nanjing Agricultural University, Nanjing 211800
3.Library of Nanjing Agricultural University, Nanjing 211800 |
|
|
|
|
Abstract Long-Text Relation Recognition plays an important role in the fields of scientific and technological intelligence and digital humanities and is the key to realizing the transformation of knowledge reorganization to knowledge discovery. However, owing to the characteristics of long texts, such as large context span, scattered semantic clues, and complex entity references, the traditional Large Language Model (LLM) is prone to insufficient contextual understanding, semantic shifts, and illusions when dealing with this type of text. As a result, long texts do not yet better realize value-added content in the practical applications of scientific and technological intelligence, humanities computing, and other fields. To solve these problems, we first constructed an entity relationship system based on the clustering results of relationship trigger words. Second, for long-text features, a long-text relationship recognition algorithm based on the LLM-BERT synergistic framework was designed to improve semantic relevance. Third, the advantages of the pretrained model, deep learning network, and attention mechanism for processing text features are integrated to construct the BERT-CNN-BiLSTM-MHA (BCBM) model to deeply mine text semantics. Finally, a summary quality assessment mechanism was designed to mitigate the LLM illusion by combining model confidence and text similarity. The experimental results show that the measured effect of this algorithm is better than that of the traditional model, and to a certain extent, it alleviates the problems of insufficient contextual understanding, semantic shift, and hallucinations that are easily generated by LLM when processing long text.
|
|
Received: 04 June 2025
|
|
|
|
1 王超, 许海云, 齐砚翠, 等. 知识网络视角下科学、技术、产业间创新驱动关系识别方法研究[J]. 情报学报, 2024, 43(1): 10-24.
2 李月琳, 潘正源, 范斯诺, 等. 赋能新质生产力的情报学与情报工作: 2024年中国情报学年会综述[J]. 情报学报, 2025, 44(3): 369-380.
3 纪其顺, 王瑞琴, 黄熠旻, 等. 基于改进Transformer模型的超长序列处理方法[J]. 中文信息学报, 2025, 39(9): 32-42.
4 王人玉, 项威, 王邦, 等. 文档级事件抽取研究综述[J]. 中文信息学报, 2023, 37(6): 1-14.
5 刘晓明, 张兆晗, 杨晨阳, 等. 在线社交网络文本内容对抗技术[J]. 计算机学报, 2022, 45(8): 1571-1597.
6 Li Q, Li J X, Sheng J W, et al. A survey on deep learning event extraction: approaches and applications[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(5): 6301-6321.
7 武剑涛, 李俊达, 李佰文, 等. 面向新闻的长文本事件抽取方法[J]. 网络安全与数据治理, 2025, 44(5): 21-28.
8 曹碧薇, 曹玖新, 桂杰, 等. 面向中文文学作品的长文本人物关系抽取[J]. 中文信息学报, 2023, 37(5): 88-100.
9 楼雯, 赵星. 要素化大模型增强的科技成果评价理念前瞻[J]. 中国图书馆学报, 2025, 51(4): 50-65.
10 罗文, 王厚峰. 大语言模型评测综述[J]. 中文信息学报, 2024, 38(1): 1-23.
11 郑江龙, 陈锦秀. 基于混合树结构神经网络的隐式篇章关系识别[J]. 厦门大学学报(自然科学版), 2017, 56(4): 576-583.
12 罗计根, 杜建强, 聂斌, 等. 基于双向LSTM和GBDT的中医文本关系抽取模型[J]. 计算机应用研究, 2019, 36(12): 3744-3747.
13 谭咏梅, 刘姝雯, 吕学强. 基于CNN与双向LSTM的中文文本蕴含识别方法[J]. 中文信息学报, 2018, 32(7): 11-19.
14 Zhou W X, Huang K, Ma T Y, et al. Document-level relation extraction with adaptive thresholding and localized context pooling[C]// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2021: 14612-14620.
15 邓成汝, 凌捷. 融合预训练模型与神经网络的实体关系抽取[J]. 计算机工程与设计, 2023, 44(7): 2023-2029.
16 Sun Y, Wang S H, Li Y K, et al. ERNIE 2.0: a continual pre-training framework for language understanding[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020: 8968-8975.
17 Zhuang H L, Qin Z, Jagerman R, et al. RankT5: fine-tuning T5 for text ranking with ranking losses[C]// Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM Press, 2023: 2308-2313.
18 Floridi L, Chiriatti M. GPT-3: its nature, scope, limits, and consequences[J]. Minds and Machines, 2020, 30(4): 681-694.
19 Lewis M, Liu Y H, Goyal N, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 7871-7880.
20 文玉锋, 林伟杰, 夏翠娟, 等. 面向古籍文献智能处理的大语言模型效能测评[J]. 图书馆论坛, 2025, 45(8): 52-60.
21 赵志枭, 胡蝶, 刘畅, 等. 人文社科领域中文通用大模型性能评测[J]. 图书情报工作, 2024, 68(13): 132-143.
22 胡忠义, 税典程, 吴江. 基于大模型微调的生成式文献层次分类标引[J]. 情报学报, 2025, 44(4): 425-437.
23 潘磊, 袁鸿霄, 钟准, 等. 基于大模型构建图网络的事件因果关系识别[J/OL]. 西南交通大学学报, (2025-04-08) [2025-05-20]. https://link.cnki.net/urlid/51.1277.u.20250408.1541.004.
24 张强, 高颖, 任豆豆, 等. 融合DeepSeek-R1和RAG技术的先秦文化元典智能问答研究[J]. 现代情报, 2026, 46(1): 173-186.
25 汪禹, 吴彬. 情感分析视角下中文政策文本的倾向词典构建研究[J]. 情报资料工作, 2025, 46(5): 102-112.
26 杨伯峻. 春秋左传注·一[M]. 北京: 中华书局, 1990.
27 曹荣. 雨花台烈士传丛书·项英传[M]. 南京: 江苏人民出版社, 2016.
28 武帅, 何琳, 杨海龄, 等. 迁移学习视角下红色文献元数据表示体系构建探究[J]. 情报资料工作, 2024, 45(6): 84-92.
29 胡卓然. 雨花台烈士传丛书·陈子涛传[M]. 南京: 江苏人民出版社, 2021.
30 武帅, 何琳, 杨秀璋, 等. 基于BERT-RL的红色文献元数据标注研究[J]. 大学图书馆学报, 2025, 43(4): 76-89.
31 董恒峰. 雨花台烈士传丛书·袁国平传[M]. 南京: 江苏人民出版社, 2016.
32 本书编写组. 雨花台烈士传丛书·蒋云传[M]. 南京: 江苏人民出版社, 2017. |
|
|
|
