基于知识增强的文本语义匹配模型研究

doi:10.3772/j.issn.1000-0135.2024.04.004

情报学报

2024, Vol. 43

Issue (4): 416-429 DOI: 10.3772/j.issn.1000-0135.2024.04.004

情报理论与方法

本期目录 | 过刊浏览 | 高级检索

基于知识增强的文本语义匹配模型研究

张贞港, 余传明

中南财经政法大学信息与安全工程学院，武汉 430073

A Text Semantic Matching Model Based on Knowledge Enhancement

Zhang Zhengang, Yu Chuanming

School of Information and Safety Engineering, Zhongnan University of Economics and Law, Wuhan 430073

摘要
图/表
参考文献
相关文章 (15)

全文: PDF (1863 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要文本语义匹配模型在信息检索、文本挖掘等领域已经获得了广泛应用。为解决现有模型主要从文本自身角度判断文本之间的语义关系而忽略对外部知识有效利用的问题，本文提出一种新的基于知识增强的文本语义匹配模型，以知识图谱实体作为外部知识，有效建模文本的外部知识信息，并自适应地过滤外部知识中存在的噪声。针对自然语言推理和释义识别两个文本语义匹配任务，与基线方法相比，本文模型在大多数指标上取得了最优效果。研究结果表明，本文模型有助于揭示知识图谱在文本语义匹配任务中的作用，为将知识图谱应用到智能信息服务领域提供了参考。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	张贞港
	余传明

关键词 ：文本语义匹配, 信息检索, 知识图谱, 知识增强

收稿日期: 2023-03-30

基金资助:国家自然科学基金面上项目“基于知识增强的科技文献创新识别与评价模型研究”（72374219），“面向跨语言观点摘要的领域知识表示与融合模型研究”（71974202）；国家自然科学基金创新研究群体项目“信息资源管理”（71921002）。

作者简介: 张贞港，男，1997年生，博士研究生，主要研究领域为知识图谱、深度学习和技术管理；余传明，男，1978年生，博士，教授，博士生导师，主要研究领域为数据挖掘、信息检索和智能情报服务，E-mail：yucm@zuel.edu.cn；

引用本文:

张贞港, 余传明. 基于知识增强的文本语义匹配模型研究[J]. 情报学报, 2024, 43(4): 416-429.
Zhang Zhengang, Yu Chuanming. A Text Semantic Matching Model Based on Knowledge Enhancement. 情报学报, 2024, 43(4): 416-429.

链接本文:

https://qbxb.istic.ac.cn/CN/10.3772/j.issn.1000-0135.2024.04.004 或 https://qbxb.istic.ac.cn/CN/Y2024/V43/I4/416

1 Yang R Q, Zhang J H, Gao X, et al. Simple and effective text matching with richer alignment features[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 4699-4709.
2 Tiwari P, Jaiswal A K, Garg S, et al. SANTM: efficient self-attention-driven network for text matching[J]. ACM Transactions on Internet Technology, 2022, 22(3): Article No.55.
3 冯文政, 唐杰. 融合深度匹配特征的答案选择模型[J]. 中文信息学报, 2019, 33(1): 118-124.
4 梁柱, 沈思, 叶文豪, 等. 基于结构内容特征的裁判文书自动推荐研究[J]. 情报学报, 2022, 41(2): 167-175.
5 施元鹏, 单剑峰. 基于文本相似度的简历匹配推荐算法研究[J]. 计算机仿真, 2022, 39(4): 441-444, 491.
6 李纲, 余辉, 毛进. 基于多层语义相似的技术供需文本匹配模型研究[J]. 数据分析与知识发现, 2021, 5(12): 25-36.
7 Wang X, Yang H M. MGMSN: multi-granularity matching model based on Siamese neural network[J]. Frontiers in Bioengineering and Biotechnology, 2022, 10: 839586.
8 Wang H Z, Wang S, Han Y H. Detecting fake news on Chinese social media based on hybrid feature fusion method[J]. Expert Systems with Applications, 2022, 208: 118111.
9 张文慧, 汪美玲, 侯志荣. 融合语境语义差异特征的短文本匹配模型[J]. 北京大学学报(自然科学版), 2023, 59(1): 30-38.
10 魏明珠, 郑荣, 高志豪, 等. 融合知识图谱和深度神经网络的产业新兴技术预测模型研究[J]. 情报学报, 2022, 41(11): 1134-1148.
11 Liu J T, Zhang Z Q, Guo Z, et al. KEPT: knowledge enhanced prompt tuning for event causality identification[J]. Knowledge-Based Systems, 2023, 259: 110064.
12 林健成, 蔺晓川. 知识对话系统的零资源在线更新[J]. 中文信息学报, 2022, 36(5): 76-84, 93.
13 宋鹏程, 单丽莉, 孙承杰, 等. 基于查询路径排序的知识库问答系统[J]. 中文信息学报, 2021, 35(11): 109-117, 126.
14 丁亮, 何彦青. 融合领域知识与深度学习的机器翻译领域自适应研究[J]. 情报科学, 2017, 35(10): 125-132.
15 Chen T, Xu R F, He Y L, et al. A gloss composition and context clustering based distributed word sense representation model[J]. Entropy, 2015, 17(12): 6007-6024.
16 Lyu B E, Chen L, Zhu S, et al. LET: linguistic knowledge enhanced graph transformer for Chinese short text matching[C]// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2021: 13498-13506.
17 Zhou G Y, Liu Y, Liu F, et al. Improving question retrieval in community question answering using world knowledge[C]// Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2013: 2239-2245.
18 Chen Q, Zhu X D, Ling Z H, et al. Neural natural language inference models enhanced with external knowledge[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2018: 2406-2417.
19 Huang P S, He X D, Gao J F, et al. Learning deep structured semantic models for web search using clickthrough data[C]// Proceedings of the 22nd ACM International Conference on Information & Knowledge Management. New York: ACM Press, 2013: 2333-2338.
20 Hu B T, Lu Z D, Li H, et al. Convolutional neural network architectures for matching natural language sentences[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2014: 2042-2050.
21 Palangi H, Deng L, Shen Y L, et al. Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2016, 24(4): 694-707.
22 Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2017: 5998-6008.
23 Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2019: 4171-4186.
24 Liu Y H, Ott M, Goyal N, et al. RoBERTa: a robustly optimized BERT pretraining approach[OL]. (2019-07-26) [2022-06-11]. https://arxiv.org/pdf/1907.11692.pdf.
25 Lan Z Z, Chen M D, Goodman S, et al. ALBERT: a lite BERT for self-supervised learning of language representations[OL]. (2020-02-09) [2022-06-11]. https://arxiv.org/pdf/1909.11942.pdf.
26 Sun Y, Wang S H, Li Y K, et al. ERNIE 2.0: a continual pre-training framework for language understanding[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(5): 8968-8975.
27 Sun Y, Wang S H, Feng S K, et al. ERNIE 3.0: large-scale knowledge enhanced pre-training for language understanding and generation[OL]. (2021-07-05) [2022-06-11]. http://arxiv.org/pdf/2107.02137.pdf.
28 Wan S X, Lan Y Y, Guo J F, et al. A deep architecture for semantic matching with multiple positional sentence representations[C]// Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2016: 2835-2841.
29 Pang L, Lan Y Y, Guo J F, et al. Text matching as image recognition[C]// Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2016: 2793-2799.
30 Gong Y C, Luo H, Zhang J. Natural language inference over interaction space[OL]. (2018-05-26) [2020-04-02]. https://arxiv.org/pdf/1709.04348.pdf.
31 Huang G, Liu Z, van Der Maaten L, et al. Densely connected convolutional networks[C]// Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2261-2269.
32 Wang Z G, Hamza W, Florian R. Bilateral multi-perspective matching for natural language sentences[C]// Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2017: 4144-4150.
33 Chen Q, Zhu X D, Ling Z H, et al. Enhanced LSTM for natural language inference[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2017: 1657-1668.
34 Zhang K, Lyu G Y, Wu L, et al. LadRa-net: locally aware dynamic reread attention net for sentence semantic matching[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(2): 853-866.
35 Miller G A. WordNet: a lexical database for English[J]. Communications of the ACM, 1995, 38(11): 39-41.
36 Xia T Y, Wang Y, Tian Y, et al. Using prior knowledge to guide BERT’s attention in semantic textual matching tasks[C]// Proceedings of the Web Conference 2021. New York: ACM Press, 2021: 2466-2475.
37 Annervaz K M, Basu Roy Chowdhury S, Dukkipati A. Learning beyond datasets: knowledge graph augmented neural networks for natural language processing[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2018: 313-322.
38 Bordes A, Usunier N, Garcia-Duran A, et al. Translating embeddings for modeling multi-relational data[C]// Proceedings of the 27th Annual Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2013: 2799-2807.
39 Wang X Y, Kapanipathi P, Musa R, et al. Improving natural language inference using external knowledge in the science questions domain[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2019: 7208-7215.
40 Liu W J, Zhou P, Zhao Z, et al. K-BERT: enabling language representation with knowledge graph[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020: 2901-2908.
41 Zhang Z Y, Han X, Liu Z Y, et al. ERNIE: enhanced language representation with informative entities[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 1441-1451.
42 Yamada I, Asai A, Shindo H, et al. LUKE: deep contextualized entity representations with entity-aware self-attention[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2020: 6442-6454.
43 Song M N, Zhao W, HaiHong E. KGAnet: a knowledge graph attention network for enhancing natural language inference[J]. Neural Computing and Applications, 2020, 32(18): 14963-14973.
44 Huang W Y, Qu Q, Yang M. Interactive knowledge-enhanced attention network for answer selection[J]. Neural Computing and Applications, 2020, 32(15): 11343-11359.
45 Li H Z, Wang W H, Liu Z R, et al. A novel locality-sensitive hashing relational graph matching network for semantic textual similarity measurement[J]. Expert Systems with Applications, 2022, 207: 117832.
46 Xu C L, Wang H, Wu S L, et al. Tag-enhanced dynamic compositional neural network over arbitrary tree structure for sentence representation[J]. Expert Systems with Applications, 2021, 181: 115182.
47 Xu C L, Wang H, Wu S L, et al. TreeLSTM with tag-aware hypernetwork for sentence representation[J]. Neurocomputing, 2021, 434: 11-20.
48 Tai K S, Socher R, Manning C D. Improved semantic representations from tree-structured long short-term memory networks[C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2015: 1556-1566.
49 Zeb A, Saif S, Chen J D, et al. Complex graph convolutional network for link prediction in knowledge graphs[J]. Expert Systems with Applications, 2022, 200: 116796.
50 Ji S X, Pan S R, Cambria E, et al. A survey on knowledge graphs: representation, acquisition, and applications[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(2): 494-514.
51 Dettmers T, Minervini P, Stenetorp P, et al. Convolutional 2D knowledge graph embeddings[C]// Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2018: 1811-1818.
52 Jiang G Y, Liu S, Zhao Y, et al. Fake news detection via knowledgeable prompt learning[J]. Information Processing & Management, 2022, 59(5): 103029.
53 Wang X Z, Gao T Y, Zhu Z C, et al. KEPLER: a unified model for knowledge embedding and pre-trained language representation[J]. Transactions of the Association for Computational Linguistics, 2021, 9: 176-194.
54 Xu B, Xu Y, Liang J Q, et al. CN-DBpedia: a never-ending Chinese knowledge extraction system[C]// Proceedings of the 30th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. Cham: Springer, 2017: 428-438.
55 Song C G, Ning N W, Zhang Y L, et al. Knowledge augmented transformer for adversarial multidomain multiclassification multimodal fake news detection[J]. Neurocomputing, 2021, 462: 88-100.
56 Yu C M, Xue H D, Jiang Y F, et al. A simple and efficient text matching model based on deep interaction[J]. Information Processing & Management, 2021, 58(6): 102738.
57 Wehrmann J, Kolling C, Barros R C. Adaptive cross-modal embeddings for image-text alignment[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020: 12313-12320.
58 Khot T, Sabharwal A, Clark P. SciTail: a textual entailment dataset from science question answering[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2018: 5189-5197.
59 Marelli M, Menini S, Baroni M, et al. A SICK cure for the evaluation of compositional distributional semantic models[C]// Proceedings of the 9th International Conference on Language Resources and Evaluation. Stroudsburg: Association for Computational Linguistics, 2014: 216-223.
60 Dolan B W, Brockett C. Automatically constructing a corpus of sentential paraphrases[C]// Proceedings of the Third International Workshop on Paraphrasing. Stroudsburg: Association for Computational Linguistics, 2005: 9-16.
61 Liu X, Chen Q C, Deng C, et al. LCQMC: a large-scale Chinese question matching corpus[C]// Proceedings of the 27th International Conference on Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2018: 1952-1962.
62 Chen J, Chen Q C, Liu X, et al. The BQ corpus: a large-scale domain-specific Chinese corpus for sentence semantic equivalence identification[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2018: 4946-4951.
63 Xu L, Hu H, Zhang X W, et al. CLUE: a Chinese language understanding evaluation benchmark[C]// Proceedings of the 28th International Conference on Computational Linguistics. International Committee on Computational Linguistics, 2020: 4762-4772.
64 Hu H, Richardson K, Xu L, et al. OCNLI: original Chinese natural language inference[C]// Findings of the Association for Computational Linguistics: EMNLP 2020. Stroudsburg: Association for Computational Linguistics, 2020: 3512-3526.
65 Pennington J, Socher R, Manning C. GloVe: global vectors for word representation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2014: 1532-1543.
66 Yang B S, Yih S W T, He X D, et al. Embedding entities and relations for learning and inference in knowledge bases[C]// Proceedings of International Conference on Learning Representations. Washington: ICLR, 2015: 1-12.
67 Chen L, Zhao Y B, Lyu B E, et al. Neural graph matching networks for Chinese short text matching[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 6152-6158.
68 Cui Y M, Che W X, Liu T, et al. Pre-training with whole word masking for Chinese BERT[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 3504-3514.