基于多模态联合注意力机制的网民情感分析研究

doi:10.3772/j.issn.1000-0135.2021.06.010

情报学报

2021, Vol. 40

Issue (6): 656-665 DOI: 10.3772/j.issn.1000-0135.2021.06.010

情报用户与行为研究

本期目录 | 过刊浏览 | 高级检索

基于多模态联合注意力机制的网民情感分析研究

范涛¹, 吴鹏², 王昊¹, 凌晨²

1.南京大学信息管理学院，南京 210023
2.南京理工大学经济管理学院，南京 210094

Sentiment Analysis of Online Users Based on Multimodal Co-attention

Fan Tao¹, Wu Peng², Wang Hao¹, Ling Chen²

1.School of Information Management, Nanjing University, Nanjing 210023
2.School of Economics and Management, Nanjing University of Science and Technology, Nanjing 210094

摘要
图/表
参考文献
相关文章 (15)

全文: PDF (1670 KB) HTML (195 KB)
输出: BibTeX | EndNote (RIS)

摘要网民情感对网络舆情的演化和发展具有重要的推动作用，如何精准分析网民情感对于网络舆情管理有着重大的现实意义。现有的网民情感研究主要是基于文本进行分析，鲜有研究从多模态视角出发，结合文本和图片对网民情感进行分析。在多模态情感分析中，现有研究多结合不同模态的全面特征、进行高维融合，容易造成信息冗余、引入噪声，忽略了不同模态之间的交互性和相关性。本文提出基于多模态联合注意力机制的情感分析模型，通过联合词引导的注意力机制和图引导的注意力机制，结合文本和图片分析网民情感，并且在“新冠肺炎疫情”等网络舆情数据中进行实证研究，与相关优异的基线模型进行了对比分析。实验结果表明，本文提出的基于多模态联合注意力机制模型具有一定的优越性，能够有效捕捉不同模态间的交互和相关关系。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	范涛
	吴鹏
	王昊
	凌晨

关键词 ：网络舆情, 情感分析, 多模态融合, 神经网络, 注意力机制

收稿日期: 2020-05-06

基金资助:国家自然科学基金项目“突发事件网民负面情感的模型检测研究”（71774084），“社会化影响下个体信息认知处理中的扭曲与偏见机制研究”（71471089）。

作者简介: 范涛，男，1995年生，博士研究生，主要研究领域为自然语言处理；吴鹏，男，1976年生，博士，教授，博士生导师，主要研究领域为网络舆情，E-mail：wupeng@njust.edu.cn；王昊，男，1981年生，博士，教授，博士生导师，主要研究领域为自然语言处理；凌晨，男，1983年生，博士，副研究员，主要研究领域为网络舆；

引用本文:

范涛, 吴鹏, 王昊, 凌晨. 基于多模态联合注意力机制的网民情感分析研究[J]. 情报学报, 2021, 40(6): 656-665.
Fan Tao, Wu Peng, Wang Hao, Ling Chen. Sentiment Analysis of Online Users Based on Multimodal Co-attention. 情报学报, 2021, 40(6): 656-665.

链接本文:

https://qbxb.istic.ac.cn/CN/10.3772/j.issn.1000-0135.2021.06.010 或 https://qbxb.istic.ac.cn/CN/Y2021/V40/I6/656

1 吴鹏, 强韶华, 高庆宁. 基于SOAR模型的网民群体负面情感建模研究[J]. 中国管理科学, 2018, 26(3): 126-138.
2 Wu P, Li X T, Shen S, et al. Social media opinion summarization using emotion cognition and convolutional neural networks[J]. International Journal of Information Management, 2020, 51: 101978.
3 Zhang W, Wang M, Zhu Y C. Does government information release really matter in regulating contagion-evolution of negative emotion during public emergencies? From the perspective of cognitive big data analytics[J]. International Journal of Information Management, 2020, 50: 498-514.
4 范涛, 吴鹏, 曹琪. 基于深度学习的多模态融合网民情感识别研究[J]. 信息资源管理学报, 2020, 10(1): 39-48.
5 Majumder N, Hazarika D, Gelbukh A, et al. Multimodal sentiment analysis using hierarchical fusion with context modeling[J]. Knowledge-Based Systems, 2018, 161: 124-133.
6 Huang F R, Zhang X M, Zhao Z H, et al. Image-text sentiment analysis via deep multimodal attentive fusion[J]. Knowledge-Based Systems, 2019, 167: 26-37.
7 Zhang Q, Fu J L, Liu X Y, et al. Adaptive co-attention network for named entity recognition in tweets[C]// Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2018: 5674-5681.
8 吴鹏, 应杨, 沈思. 基于双向长短期记忆模型的网民负面情感分类研究[J]. 情报学报, 2018, 37(8): 845-853.
9 辜丽琼, 夏志杰, 宋祖康, 等. 基于在线网民评论情感追踪分析的企业危机舆情应对研究[J]. 情报理论与实践, 2019, 42(12): 67-73.
10 夏一雪, 王娟, 何巍, 等. 基于舆情大数据的突发事件负面情感引导“时度效”研究[J]. 图书与情报, 2019(5): 120-126.
11 朱晓霞, 宋嘉欣, 孟建芳. 基于动态主题——情感演化模型的网络舆情信息分析[J]. 情报科学, 2019, 37(7): 72-78.
12 张鹏, 崔彦琛, 兰月新, 等. 基于扎根理论与词典构建的微博突发事件情感分析与舆情引导策略[J]. 现代情报, 2019, 39(3): 122-131, 143.
13 Balazs J A, Velásquez J D. Opinion mining and information fusion: a survey[J]. Information Fusion, 2016, 27: 95-110.
14 Kamps J, Marx M, Mokken R J, et al. Words with attitude[M]. Institute for Logic, Language and Computation, University of Amsterdam, 2001.
15 黄伟, 范磊. 基于多分类器投票集成的半监督情感分类方法研究[J]. 中文信息学报, 2016, 30(2): 41-49, 106.
16 唐慧丰, 谭松波, 程学旗. 基于监督学习的中文情感分类技术比较研究[J]. 中文信息学报, 2007, 21(6): 88-94, 108.
17 Kim Y. Convolutional neural networks for sentence classification[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2014: 1746-1751.
18 Ma Y K, Peng H Y, Cambria E. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM[C]// Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2018: 5876-5883.
19 Yanulevskaya V, Uijlings J, Bruni E, et al. In the eye of the beholder: employing statistical analysis and eye tracking for analyzing abstract paintings[C]// Proceedings of the 20th ACM International Conference on Multimedia. New York: ACM Press, 2012: 349-358.
20 Lu X, Suryanarayan P, Adams R B, et al. On shape and the computability of emotions[C]// Proceedings of the 20th ACM International Conference on Multimedia. New York: ACM Press, 2012: 229-238.
21 Machajdik J, Hanbury A. Affective image classification using features inspired by psychology and art theory[C]// Proceedings of the 18th ACM International Conference on Multimedia. New York: ACM Press, 2010: 83-92.
22 Peng X S, Zhang X M, Li Y P, et al. Research on image feature extraction and retrieval algorithms based on convolutional neural network[J]. Journal of Visual Communication and Image Representation, 2020, 69: 102705.
23 Chen T, Borth D, Darrell T, et al. DeepSentiBank: visual sentiment concept classification with deep convolutional neural networks[OL]. [2014-10-30]. https://arxiv.org/pdf/1410.8586v1.pdf.
24 Yang J F, She D Y, Sun M, et al. Visual sentiment prediction based on automatic discovery of affective regions[J]. IEEE Transactions on Multimedia, 2018, 20(9): 2513-2525.
25 Campos V, Jou B, Giró-I-Nieto X. From pixels to sentiment: fine-tuning CNNs for visual sentiment prediction[J]. Image and Vision Computing, 2017, 65: 15-22.
26 Poria S, Cambria E, Bajpai R, et al. A review of affective computing: from unimodal analysis to multimodal fusion[J]. Information Fusion, 2017, 37: 98-125.
27 Poria S, Chaturvedi I, Cambria E, et al. Convolutional MKL based multimodal emotion recognition and sentiment analysis[C]//Proceedings of the 2016 IEEE 16th International Conference on Data Mining. IEEE, 2016: 439-448.
28 Pérez Rosas V, Mihalcea R, Morency L P. Multimodal sentiment analysis of Spanish online videos[J]. IEEE Intelligent Systems, 2013, 28(3): 38-45.
29 Zadeh A, Chen M H, Poria S, et al. Tensor fusion network for multimodal sentiment analysis[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2017: 1103-1114.
30 Williams J, Comanescu R, Radu O, et al. Dnn multimodal fusion techniques for predicting video sentiment[C]// Proceedings of Grand Challenge and Workshop on Human Multimodal Language. Stroudsburg: Association for Computational Linguistics, 2018: 64-72.
31 Song K S, Nho Y H, Seo J H, et al. Decision-level fusion method for emotion recognition using multimodal emotion recognition information[C]// Proceedings of the 2018 15th International Conference on Ubiquitous Robots. IEEE, 2018: 472-476.
32 Deng J, Dong W, Socher R, et al. ImageNet: a large-scale hierarchical image database[C]// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2009: 248-255.
33 Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[OL]. [2014-12-19]. https://arxiv.org/pdf/1409.1556v4.pdf.
34 Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2013: 3111-3119.
35 Greff K, Srivastava R K, Koutník J, et al. LSTM: a search space odyssey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(10): 2222-2232.
36 吴鹏, 刘恒旺, 沈思. 基于深度学习和OCC情感规则的网络舆情情感识别研究[J]. 情报学报, 2017, 36(9): 972-980.
37 金占勇, 田亚鹏, 白莽. 基于长短时记忆网络的突发灾害事件网络舆情情感识别研究[J]. 情报科学, 2019, 37(5): 142-147, 154.
38 Rhanoui M, Mikram M, Yousfi S, et al. A CNN-BiLSTM model for document-level sentiment analysis[J]. Machine Learning and Knowledge Extraction, 2019, 1(3): 832-847.