|
|
|
| Research on the Extraction and Application of Ancient Books' Restricted Domain Relation Based on Large Language Model Technology |
| Liu Chang1, Zhang Qi2, Wang Dongbo1, Shen Si3, Wu Mengcheng1, Liu Liu1, Su Yushi1 |
1.College of Information Management, Nanjing Agricultural University, Nanjing 211800
2.School of Economics and Management, Shanxi University, Taiyuan 030006
3.School of Economics & Management, Nanjing University of Science & Technology, Nanjing 210094 |
|
|
|
|
Abstract Automatic extraction and structuring of fine-grained knowledge units in ancient books can provide a database for digital humanities research on ancient books, such as group biographies and historical maps. The extraction method based on the discriminative model is severely restricted by the semantic complexity of ancient Chinese and missing training samples, which limit the extraction and domain transfer effects. Related research is required to develop generative artificial intelligence technology. This study explores methods for restricted domain relation extraction in ancient texts based on large language models and the automatic generation of high-quality training corpora. After comparing the impact of different prompt templates on model extraction performance and proving the significance of fine-tuning methods in improving model performance, we utilize the ChatGPT4 application programming interface (API) service, combined with self-instruction, thought chains, and human feedback, to create a domain-specific relation extraction dataset for ancient texts. After data augmentation, F1 scores of 56.07% and 30.50% are achieved on two ancient text relation extraction datasets, exhibiting a significant improvement in transferability compared with models trained on the entire dataset. The study also explores the collaborative use of self-instruction and automatic evaluation models to synthesize training corpora, evaluation information, and trained models based on synthetic data, effectively alleviating the problem of insufficient training data. The findings indicate that using large language models to extract relational triplets and synthesize training data can significantly reduce the labor costs previously associated with domain-specific relation extraction and improve the efficiency of constructing knowledge graphs in the field of ancient texts.
|
|
Received: 22 February 2024
|
|
|
|
1 陆伟, 刘家伟, 马永强, 等. ChatGPT为代表的大模型对信息资源管理的影响[J]. 图书情报知识, 2023, 40(2): 6-9, 70.
2 Wei J, Bosma M, Zhao V Y, et al. Finetuned language models are zero-shot learners[OL]. (2022-02-08). https://arxiv.org/pdf/2109.01652.
3 Ouyang L, Wu J, Jiang X, et al. Training language models to follow instructions with human feedback[C]// Proceedings of the 36th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2022: 27730-27744.
4 Bai J Z, Bai S, Chu Y F, et al. Qwen technical report[OL]. (2023-09-28). https://arxiv.org/pdf/2309.16609.
5 Zhao W X, Zhou K, Li J Y, et al. A survey of large language models[OL]. (2024-10-13). https://arxiv.org/pdf/2303.18223.
6 Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2017: 6000-6010.
7 Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2019: 4171-4186.
8 Lewis M, Liu Y H, Goyal N, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 7871-7880.
9 Raffel C, Shazeer N, Roberts A, et al. Exploring the limits of transfer learning with a unified text-to-text transformer[J]. Journal of Machine Learning Research, 2020, 21(1): 5485-5551.
10 Radford A, Narasimhan K, Salimans T, et al. Improving language understanding by generative pre-training[OL]. (2018-06-09). https://cdn.openai.com/research-covers/language-unsupervised/language_ understanding_paper.pdf.
11 Radford A, Wu J, Child R, et al. Language models are unsupervised multitask learners[OL]. (2019-02-15). https://cdn.openai.com/ better-language-models/language_models_are_unsupervised_multitask_learners.pdf.
12 Han X, Zhang Z Y, Ding N, et al. Pre-trained models: past, present and future[J]. AI Open, 2021, 2: 225-250.
13 Kaplan J, McCandlish S, Henighan T, et al. Scaling laws for neural language models[OL]. (2020-01-23). https://arxiv.org/pdf/2001.08361.
14 Niklaus C, Cetto M, Freitas A, et al. A survey on open information extraction[C]// Proceedings of the 27th International Conference on Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2018: 3866-3878.
15 侯景, 邓晓梅, 汉鹏武. 限定域关系抽取技术研究综述[J]. 计算机科学, 2024, 51(1): 252-265.
16 Zeng D, Liu K, Lai S, et al. Relation classification via convolutional deep neural network[C]// Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers. Stroudsburg: Association for Computational Linguistics, 2014: 2335-2344.
17 Xu Y, Mou L L, Li G, et al. Classifying relations via long short term memory networks along shortest dependency paths[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2015: 1785-1794.
18 Shi P, Lin J. Simple BERT models for relation extraction and semantic role labeling[OL]. (2019-04-10). https://arxiv.org/pdf/1904.05255.
19 Xie T B, Wu C H, Shi P, et al. UnifiedSKG: unifying and multi-tasking structured knowledge grounding with text-to-text language models[C]// Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2022: 602-631.
20 Lu Y J, Liu Q, Dai D, et al. Unified structure generation for universal information extraction[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2022: 5755-5772.
21 张颖怡, 章成志, 周毅, 等. 基于ChatGPT的多视角学术论文实体识别: 性能测评与可用性研究[J]. 数据分析与知识发现, 2023, 7(9): 12-24.
22 鲍彤, 章成志. ChatGPT中文信息抽取能力测评——以三种典型的抽取任务为例[J]. 数据分析与知识发现, 2023, 7(9): 1-11.
23 Gui H H, Qiao S F, Zhang J T, et al. InstructIE: a bilingual instruction-based information extraction dataset[C]// Proceedings of the 23rd International Semantic Web Conference. Cham: Springer, 2024: 59-79.
24 Wei X, Cui X Y, Cheng N, et al. ChatIE: zero-shot information extraction via chatting with ChatGPT[OL]. (2024-05-24). https://arxiv.org/pdf/2302.10205.
25 中共中央办公厅 国务院办公厅印发《关于推进新时代古籍工作的意见》[EB/OL]. (2022-04-11) [2023-12-27]. https://www.gov.cn/gongbao/content/2022/content_5687500.htm.
26 王军. 从人文计算到可视化——数字人文的发展脉络梳理[J]. 文艺理论与批评, 2020(2): 18-23.
27 柳润杰. 面向纪传体史书的知识图谱构建与检索的研究[D]. 太原: 中北大学, 2020.
28 张琪, 王东波, 黄水清, 等. 史书多维知识重组与可视化研究——以《史记》为对象[J]. 情报学报, 2022, 41(2): 130-141.
29 张琪. 《史记》多维知识组织与可视化研究[D]. 南京: 南京农业大学, 2020.
30 刘欢, 刘浏, 王东波. 数字人文视角下的领域知识图谱自动问答研究[J]. 科技情报研究, 2022, 4(1): 46-59.
31 吴梦成, 林立涛, 齐月, 等. 数字人文视域下先秦典籍植物知识挖掘与组织研究[J]. 图书情报工作, 2023, 67(12): 103-113.
32 刘佳, 张心祺, 张承坤. 基于人文计算的藏医古籍服务平台知识服务功能设计研究[J]. 现代情报, 2023, 43(11): 47-57.
33 张卫东, 张晓晓. 中医古籍数字资源知识组织与可视化研究——以《金匮要略》为例[J]. 情报科学, 2022, 40(8): 107-117.
34 周莉娜, 洪亮, 高子阳. 唐诗知识图谱的构建及其智能知识服务设计[J]. 图书情报工作, 2019, 63(2): 24-33.
35 龙从军, 安波, 张圣彦. 吐蕃藏文金石铭刻知识图谱构建研究[J]. 图书情报工作, 2023, 67(8): 141-150.
36 基于众包标注系统的文言文语言理解测评基准及数据集[DS/OL]. [2023-12-29]. http://data.openkg.cn/dataset/c-clue.
37 Taori R, Gulrajani I, Zhang T Y, et al. Alpaca: a strong, replicable instruction-following model[EB/OL]. (2023-03-13) [2023-12-30]. https://crfm.stanford.edu/2023/03/13/alpaca.html.
38 Wang Y Z, Kordi Y, Mishra S, et al. Self-instruct: aligning language models with self-generated instructions[C]// Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2023: 13484-13508.
39 Touvron H, Lavril T, Izacard G, et al. LLaMA: open and efficient foundation language models[OL]. (2023-02-27). https://arxiv.org/pdf/2302.13971.
40 Wei J, Wang X Z, Schuurmans D, et al. Chain-of-thought prompting elicits reasoning in large language models[C]// Proceedings of the 36th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2024: 24824-24837.
41 Lee H, Phatale S, Mansoor H, et al. RLAIF: scaling reinforcement learning from human feedback with AI feedback[OL]. (2023-09-01). https://arxiv.org/pdf/2309.00267v1.
42 文心一言[EB/OL]. [2024-05-17]. https://yiyan.baidu.com/welcome.
43 Xunzi-LLM-of-Chinese-classics/XunziALLM[DS/OL]. [2024-01- 03]. https://github.com/Xunzi-LLM-of-Chinese-classics/XunziALLM.
44 Shang Y M, Huang H Y, Mao X L. OneRel: joint entity and relation extraction with one module in one step[OL]. (2022-03-17). https://arxiv.org/pdf/2203.05412.
45 Wei Z P, Su J L, Wang Y, et al. A novel cascade binary tagging framework for relational triple extraction[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2020: 1476-1488.
46 Wang D B, Liu C, Zhao Z X, et al. GujiBERT and GujiGPT: construction of intelligent information processing foundation language models for ancient texts[OL]. (2023-07-11). https://arxiv.org/pdf/2307.05354.
47 Hu E J, Shen Y L, Wallis P, et al. LoRA: low-rank adaptation of large language models[OL]. (2021-10-16). https://arxiv.org/pdf/2106.09685.
48 Kwon W, Li Z H, Zhuang S Y, et al. Efficient memory management for large language model serving with PagedAttention[C]// Proceedings of the 29th Symposium on Operating Systems Principles. New York: ACM Press, 2023: 611-626.
49 徐晨飞. 数字人文视域下方志物产知识库构建研究——以《方志物产》云南卷为例[D]. 南京: 南京农业大学, 2020.
50 宋雪雁, 张伟民, 张祥青. 基于RDF的语义知识超图存储研究[J]. 情报学报, 2023, 42(8): 967-979. 责任编辑 王克平） |
|
|
|
