|
|
|
| Patent-BARTKPG: A Contrastive Learning-Based Approach for Chinese Keyphrase Patent Generation |
| Ran Congjing1, Liu Xingshen1, Wang Haowei1, Liang Yulian2, Wang Fuxin1 |
1.School of Information Management, Wuhan University, Wuhan 430072
2.The College of Information Engineering, Wuchang Institute of Technology, Wuhan 430065 |
|
|
|
|
Abstract The traditional extraction methods used to generate keyphrases for patents are not sufficiently accurate. This is primarily manifested as excessive reliance on the literal content in the text, redundant information in the generated sequence of keyphrases, and inconsistency with the target keyphrases. To address these issues, this study combines the unique corpus characteristics of Chinese-patent texts to achieve a more accurate generation of keyphrases. A two-stage model is proposed for extracting, generating, and reordering keyphrases from patents. Additionally, a contrastive learning training strategy is introduced in both stages to further enhance the performance of the model. Finally, a Chinese-patent bidirectional auto-regressive transformer for keyphrase generation (BARTKPG), named Patent-BARTKPG, is constructed to accurately generate keyphrases for Chinese-patent texts. In preliminary studies, Patent-BARTKPG significantly outperformed other keyphrase extraction and generation models in generating high-quality keyphrases for the Chinese-patent dataset.
|
|
Received: 10 November 2024
|
|
|
|
1 俞琰, 尚明杰, 赵乃瑄. 权利要求特征驱动的专利关键词抽取方法[J]. 情报学报, 2021, 40(6): 610-620.
2 Meng R, Zhao S Q, Han S G, et al. Deep keyphrase generation[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2017: 582-592.
3 Luhn H P. A statistical approach to mechanized encoding and searching of literary information[J]. IBM Journal of Research and Development, 1957, 1(4): 309-317.
4 Sp?rck Jones K. A statistical interpretation of term specificity and its application in retrieval[J]. Journal of Documentation, 2004, 60(5): 493-502.
5 Mihalcea R, Tarau P. TextRank: bringing order into text[C]// Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2004: 404-411.
6 Florescu C, Caragea C. PositionRank: an unsupervised approach to keyphrase extraction from scholarly documents[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2017: 1105-1115.
7 Teneva N, Cheng W W. Salience Rank: efficient keyphrase extraction with topic modeling[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2017: 530-535.
8 Liu Z Y, Huang W Y, Zheng Y B, et al. Automatic keyphrase extraction via topic decomposition[C]// Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2010: 366-376.
9 Sterckx L, Demeester T, Deleu J, et al. Topical word importance for fast keyphrase extraction[C]// Proceedings of the 24th International Conference on World Wide Web. New York: ACM Press, 2015: 121-122.
10 Frank E, Paynter G W, Witten I H, et al. Domain-specific keyphrase extraction[C]// Proceedings of the 16th International Joint Conference on Artificial Intelligence. San Francisco: Morgan Kaufmann Publishers, 1999: 668-673.
11 Wang J B, Peng H. Keyphrases extraction from Web document by the least squares support vector machine[C]// Proceedings of the 2005 IEEE/WIC/ACM International Conference on Web Intelligence. Piscataway: IEEE, 2005: 293-296.
12 Ding Z Y, Zhang Q, Huang X J. Keyphrase extraction from online news using binary integer programming[C]// Proceedings of 5th International Joint Conference on Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2011: 165-173.
13 Sarkar K, Nasipuri M, Ghose S. Machine learning based keyphrase extraction: comparing decision trees, Na?ve Bayes, and artificial neural networks[J]. Journal of Information Processing Systems, 2012, 8(4): 693-712.
14 Zhang Q, Wang Y, Gong Y Y, et al. Keyphrase extraction using deep recurrent neural networks on Twitter[C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2016: 836-845.
15 王玉叶, 王玙. 基于图神经网络的专利关键词提取算法研究[J]. 情报理论与实践, 2023, 46(5): 202-208.
16 韦婷婷, 江涛, 郑舒玲, 等. 融合LSTM与逻辑回归的中文专利关键词抽取[J]. 数据分析与知识发现, 2022, 6(2/3): 308-317.
17 Chen J, Zhang X M, Wu Y, et al. Keyphrase generation with correlation constraints[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2018: 4057-4066.
18 Zhang Y, Xiao W D. Keyphrase generation based on deep Seq2Seq model[J]. IEEE Access, 2018, 6: 46047-46057.
19 Lewis M, Liu Y H, Goyal N, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 7871-7880.
20 Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2019: 4171-4186.
21 Brown T B, Mann B, Ryder N, et al. Language models are few-shot learners[C]// Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2020: 1877-1901.
22 Liu X, Zhang F J, Hou Z Y, et al. Self-supervised learning: generative or contrastive[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 35(1): 857-876.
23 Chen C, Yin Y C, Shang L F, et al. BERT2BERT: towards reusable pretrained language models[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2022: 2134-2148.
24 Dong L, Yang N, Wang W, et al. Unified language model pre-training for natural language understanding and generation[C]// Proceedings of the 32nd Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2020: 13019-13031.
25 Raffel C, Shazeer N, Roberts A, et al. Exploring the limits of transfer learning with a unified text-to-text transformer[J]. Journal of Machine Learning Research, 2020, 21(1): 5485-5551.
26 van der Maaten L, Hinton G E. Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9(11): 2579-2605. |
|
|
|
