|
|
|
| Fine-Grained Sentiment Analysis of Online Government- Public Interaction Texts Using Large Language Models: Multistage Optimization Approach |
| Teng Jie1,2, He Huanglan1,2, Hu Guangwei1,2 |
1.School of Information Management, Nanjing University, Nanjing 210023
2.Government Data Resources Institution of Nanjing University, Nanjing 210023 |
|
|
|
|
Abstract Online government-public interaction texts contain a rich emotional information that reflects public opinion, making the fine-grained sentiment analysis valuable for enhanced government governance capabilities. However, traditional methods struggle to accurately capture complex emotional features in these texts. In this study, we propose a multistage optimization framework for a fine-grained sentiment analysis of online government-public interaction texts based on large language models. First, we establish a sentiment classification system with eight emotional zones and 56 emotional labels based on the arousal-valence theory of emotion, and employ GPT-4 with BROKE framework-designed prompting strategies for an initial sentiment annotation, effectively addressing complex contextual understanding challenges. Second, we innovatively propose a “large model annotation-expert evaluation-judgment model training-multistage optimization” data quality control mechanism, using the Claude 3.5 Sonnet model for correction and judgment model for filtering, solving the “hallucination” problem of large models in domain applications. Finally, we achieve emotional category balancing through Claude 3.5 Sonnet’s data generation capabilities, reducing the imbalance Gini coefficient from 0.866 to 0.181, significantly enhancing the model generalization ability. Experimental results show that this framework achieves an accuracy of 89.70% in fine-grained emotional analysis tasks, with an F1 score improvement of 21.65% on average compared to existing methods, providing a new technical paradigm for an enhanced government-public interaction effectiveness.
|
|
Received: 28 November 2024
|
|
|
|
1 Akhtar M S, Gupta D, Ekbal A, et al. Feature selection and ensemble construction: a two-step method for aspect based sentiment analysis[J]. Knowledge-Based Systems, 2017, 125: 116-135.
2 Li S S, Wang Z Q, Zhou G D, et al. Semi-supervised learning for imbalanced sentiment classification[C]// Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2011: 1826-1831.
3 Zhao J Q, Gui X L, Zhang X J. Deep convolution neural networks for Twitter sentiment analysis[J]. IEEE Access, 2018, 6: 23253-23260.
4 Sun S L, Luo C, Chen J Y. A review of natural language processing techniques for opinion mining systems[J]. Information Fusion, 2017, 36: 10-25.
5 Mao R, Ge M S, Han S, et al. A survey on pragmatic processing techniques[J]. Information Fusion, 2025, 114: 102712.
6 颜学明, 黄翰, 金耀初, 等. 面向不平衡短文本情感多分类的三阶语义图数据增广方法[J]. 计算机学报, 2024, 47(12): 2742-2759.
7 李诗晨, 王中卿, 周国栋. 大语言模型驱动的跨领域属性级情感分析[J]. 软件学报, 2025, 36(2): 644-659.
8 Mao R, Chen G Y, Zhang X L, et al. GPTEval: a survey on assessments of ChatGPT and GPT-4[C]// Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation. Stroudsburg: Association for Computational Linguistics, 2024: 7844-7866.
9 Russell J A. A circumplex model of affect[J]. Journal of Personality and Social Psychology, 1980, 39(6): 1161-1178.
10 Bradley M M, Lang P J. Measuring emotion: the self-assessment manikin and the semantic differential[J]. Journal of Behavior Therapy and Experimental Psychiatry, 1994, 25(1): 49-59.
11 Barrett L F, Russell J A. The structure of current affect: controversies and emerging consensus[J]. Current Directions in Psychological Science, 1999, 8(1): 10-14.
12 Preo?iuc-Pietro D, Schwartz H A, Park G, et al. Modelling valence and arousal in Facebook posts[C]// Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. Stroudsburg: Association for Computational Linguistics, 2016: 9-15.
13 Mohammad S M, Salameh M, Kiritchenko S. How translation alters sentiment[J]. Journal of Artificial Intelligence Research, 2016, 55: 95-130.
14 陶林娟, 华庚兴, 李波. 基于位置增强词向量和GRU-CNN的方面级情感分析模型研究[J]. 计算机工程与应用, 2024, 60(9): 212-218.
15 Li X Y, Raga R C, Shi X M. GloVe-CNN-BiLSTM model for sentiment analysis on text reviews[J]. Journal of Sensors, 2022, 2022: 7212366.
16 Zhang L K, Lei Y X, Wang Z Y. Long-text sentiment analysis based on semantic graph[C]// Proceedings of the 2020 IEEE International Conference on Embedded Software and Systems. Piscataway: IEEE, 2020: 1-6.
17 高玮军, 孙子博, 刘书君. 基于多视角的图像文本情感分析[J]. 计算机科学, 2024, 51(S2): 138-145.
18 Buechel S, Hahn U. Emobank: studying the impact of annotation perspective and representation format on dimensional emotion analysis[C]// Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2017: 578-585.
19 蒋昊达, 赵春蕾, 陈瀚, 等. 基于改进TF-IDF与BERT的领域情感词典构建方法[J]. 计算机科学, 2024, 51(S1): 162-170.
20 李大宇, 李旸, 王素格. 大规模中文金融情感分析数据集自动构建[J]. 山西大学学报(自然科学版), 2024, 47(4): 776-785.
21 公确多杰, 索南才让. 基于卷积神经网络的藏语情感词典自动构建方法[J]. 高原科学研究, 2024, 8(3): 117-124.
22 Wu Y L, Wang K C, Ma K, et al. Werkzeug at SemEval-2024 task 8: LLM-generated text detection via gated mixture-of-experts fine-tuning[C]// Proceedings of the 18th International Workshop on Semantic Evaluation. Stroudsburg: Association for Computational Linguistics, 2024: 547-552.
23 Qu X Y, Meng X F. TM-TREK at SemEval-2024 task 8: towards LLM-based automatic boundary detection for human-machine mixed text[C]// Proceedings of the 18th International Workshop on Semantic Evaluation. Stroudsburg: Association for Computational Linguistics, 2024: 710-715.
24 Kyriakou T, Maslaris I, Arampatzis A. DUTh at SemEval 2024 task 8: comparing classic machine learning algorithms and LLM based methods for multigenerator, multidomain and multilingual machine-generated text detection[C]// Proceedings of the 18th International Workshop on Semantic Evaluation. Stroudsburg: Association for Computational Linguistics, 2024: 1080-1086.
25 刘华玲, 张子龙, 彭宏帅. 面向闭源大语言模型的增强研究综述[J]. 计算机科学与探索, 2025, 19(5): 1141-1156.
26 袁毓林. ChatGPT等语言大模型到底做对了什么?——语言大模型的世界模型和感觉知识的来源探秘[J]. 当代修辞学, 2025(1): 22-39.
27 Callan D, Foster J. How interesting and coherent are the stories generated by a large-scale neural language model? Comparing human and automatic evaluations of machine-generated text[J]. Expert Systems, 2023, 40(6): e13292.
28 马灿, 黄瑞章, 任丽娜, 等. 基于大语言模型的多输入中文拼写纠错方法[J]. 计算机应用, 2025, 45(3): 849-855.
29 李居昊, 石磊, 丁锰, 等. 基于大语言模型的社交媒体文本立场检测[J]. 计算机科学与探索, 2025, 19(5): 1302-1312.
30 关威, 曹健, 赵海燕, 等. 基于大语言模型的业务流程语义异常检测方法[J/OL]. 计算机集成制造系统, (2025-01-14). https://link.cnki.net/doi/10.13196/j.cims.2025.BPM03.
31 Wei X. The use of large language models for translating Buddhist texts from classical Chinese to modern English: an analysis and evaluation with ChatGPT 4, ERNIE bot 4, and gemini advanced[J]. Religions, 2024, 15(12): 1559.
32 Shin D, Kim H, Lee S, et al. Using large language models to detect depression from user-generated diary text data as a novel approach in digital mental health screening: instrument validation study[J]. Journal of Medical Internet Research, 2024, 26: e54617.
33 林佳瑞, 陈柯吟, 郑哲, 等. 建筑工程标准规范智能解译关键技术及应用[J]. 工程力学, 2025, 42(2): 1-14.
34 Scherer K R. What are emotions? And how can they be measured?[J]. Social Science Information, 2005, 44(4): 695-729.
35 Bradley M M, Lang P J. International affective picture system[M]// Encyclopedia of Personality and Individual Differences. Cham: Springer, 2017. DOI: 10.1007/978-3-319-28099-8_42-1.
36 Ekman P. An argument for basic emotions[J]. Cognition and Emotion, 1992, 6(3/4): 169-200.
37 Plutchik R. A general psychoevolutionary theory of emotion[M]// Theories of Emotion. New York: Academic Press, 1980: 3-33.
38 Watson D, Clark L A, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales[J]. Journal of Personality and Social Psychology, 1988, 54(6): 1063-1070.
39 Tracy J L, Robins R W. The psychological structure of pride: a tale of two facets[J]. Journal of Personality and Social Psychology, 2007, 92(3): 506-525.
40 Fredrickson B L. What good are positive emotions?[J]. Review of General Psychology, 1998, 2(3): 300-319.
41 Bandura A. Self-efficacy: the exercise of control[M]. New York: Freeman, 1997.
42 McCullough M E, Emmons R A, Tsang J A. The grateful disposition: a conceptual and empirical topography[J]. Journal of Personality and Social Psychology, 2002, 82(1): 112-127.
43 Spielberger C D. Anxiety: current trends in theory and research[M]. New York; Academic Press, 1972.
44 Beck A T. Depression: clinical, experimental, and theoretical aspects[M]. Pennsylvania: University of Pennsylvania Press, 1967: 103-113.
45 Buss A H, Durkee A. An inventory for assessing different kinds of hostility[J]. Journal of Consulting Psychology, 1957, 21(4): 343-349.
46 Zeelenberg M, van Dijk W W, Manstead A S R, et al. On bad decisions and disconfirmed expectancies: the psychology of regret and disappointment[J]. Cognition and Emotion, 2000, 14(4): 521-541.
47 Borkovec T D, Robinson E, Pruzinsky T, et al. Preliminary exploration of worry: some characteristics and processes[J]. Behaviour Research and Therapy, 1983, 21(1): 9-16.
48 Peplau L A, Perlman D. Loneliness: a sourcebook of current theory, research and therapy[M]. New York: Wiley, 1982.
49 Surowiecki J. The wisdom of crowds[M]. New York: Anchor, 2005.
50 Blumer A, Ehrenfeucht A, Haussler D, et al. Occam’s razor[J]. Information Processing Letters, 1987, 24(6): 377-380.
51 Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2019: 4171-4186.
52 Liu Y H, Ott M, Goyal N, et al. RoBERTa: a robustly optimized BERT pretraining approach[OL]. (2019-07-26). https://arxiv.org/pdf/1907.11692.
53 Liao W X, Zeng B, Yin X W, et al. An improved aspect-category sentiment analysis model for text sentiment analysis based on RoBERTa[J]. Applied Intelligence, 2021, 51(6): 3522-3533.
54 Kim Y. Convolutional neural networks for sentence classification[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2014: 1746-1751.
55 Liu X Z, Zhang L F, Li T, et al. Dual attention guided multi-scale CNN for fine-grained image classification[J]. Information Sciences, 2021, 573: 37-45.
56 Liu G, Guo J B. Bidirectional LSTM with attention mechanism and convolutional layer for text classification[J]. Neurocomputing, 2019, 337: 325-338.
57 Cheng Y Y, Chen Y M, Yeh W C, et al. Valence and arousal-infused bi-directional LSTM for sentiment analysis of government social media management[J]. Applied Sciences, 2021, 11(2): 880.
58 Lin E, Sun J, Chen H, et al. Data quality matters: suicide intention detection on social media posts using a RoBERTa-CNN[C]// Proceedings of the 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Piscataway: IEEE: 2024: 1-5.
59 Cai Y Y, Zhao Y C, Luo H B, et al. Evaluation model of learner’s cognitive level based on RoBERTa fused with CNN[C]// Proceedings of the 2023 3rd International Conference on Neural Networks, Information and Communication Engineering. Piscataway: IEEE, 2023: 66-69.
60 Rahman M M, Shiplu A I, Watanobe Y, et al. RoBERTa-BiLSTM: a context-aware hybrid model for sentiment analysis[J/OL]. IEEE Transactions on Emerging Topics in Computational Intelligence, (2025-06-02). http://doi.org/10.1109/TETCI.2025.3572150.
61 Jahin M A, Shovon M S H, Mridha M F, et al. RoBERTa and attention-based BiLSTM for interpretable sentiment analysis of tweets[OL]. (2024-09-09). https://arxiv.org/pdf/2404.00297v3. |
|
|
|
