|
|
|
| Emotional Recognition of Visual-perception Oriented Images and Its Application in the Recommendation System |
| Chen Fen, He Yuan, Tang Liping |
| School of Economics and Management, Nanjing University of Science and Technology, Nanjing 210094 |
|
|
|
|
Abstract Visual information is one of the most important sources of external information. Images are a major form of visual information. In this paper, the authors first optimize the algorithm of color histogram based on image segmentation, introduce the Itti visual attention model, extract the image saliency map according to the principle that different image regions arouse different degrees of attention, and calculate the weighted histogram based on the saliency map. Secondly, different types of visual emotional features are extracted, combining low-level color, texture, and shape, as well as high-level facial emotion features to generate a composite image sentiment feature description vector. Finally, the emotion recognition model is used to make emotion-based film recommendations, combined with movie posters and synopsis texts, and based on the combination of graphic and textual emotion recognition, to meet users’ emotional needs. In conclusion, this paper proposes a framework of image emotion recognition that combines visual-perception oriented features and facial emotions. This framework is efficient and exhibits good performance. To a certain extent, the framework proposed in this paper narrows the “semantic gap” in this area of research.
|
|
Received: 17 August 2018
|
|
|
|
1 EkmanP. An argument for basic emotions[J]. Cognition and Emotion, 1992, 6(3-4): 169-200.
2 MikelsJ A, FredricksonB L, LarkinG R, et al. Emotional category data on images from the international affective picture system[J]. Behavior Research Methods, 2005, 37(4): 626-630.
3 黄崑, 赖茂生. 图像情感特征的分类与提取[J]. 计算机应用, 2008, 28(3): 659-661.
4 李娉婷, 石跃祥, 戴皇冠. 基于颜色特征的家居设计图分类[J]. 计算机工程, 2011, 37(16): 224-226.
5 曹建芳, 陈俊杰, 李海芳. 基于改进的OCC情感模型的自然风景图像分类研究[J]. 计算机应用与软件, 2014, 31(6): 181-184.
6 张海波, 张春艳, 商书元, 等. 男T 恤图像情感语义空间研究[J]. 纺织科技进展, 2016(6): 53-55.
7 SiersdorferS, MinackE, DengF, et al. Analyzing and predicting sentiment of images on the social web[C]// Proceedings of the 18th ACM International Conference on Multimedia. New York: ACM Press, 2010: 715-718.
8 ShinY, KimE Y, SungT E. Affect-based retrieval of landscape images using probabilistic affective model[C]// Proceedings of the International Conference on Human-Computer Interaction. Heidelberg: Springer, 2013: 362-371.
9 SeoS, KangD. Study on predicting sentiment from images using categorical and sentimental keyword-based image retrieval[J]. Journal of Supercomputing, 2016, 72(9): 3478-3488.
10 ParkM W, KoD, HwangH, et al. Image classification using color and spatial frequency in terms of human emotion[C]// Proceedings of the International Conference on Future Information Technology. Singapore: Springer, 2017, 448: 91-96.
11 LeeG, KwonM, KavuriS, et al. Action-perception cycle learning for incremental emotion recognition in a movie clip using 3D fuzzy GIST based on visual and EEG signals[J]. Integrated Computer-Aided Engineering, 2014, 21(3): 295-310.
12 LiZ H, FanY Y, LiuW H, et al. Image sentiment prediction based on textual descriptions with adjective noun pairs[J]. Multimedia Tools and Applications, 2018, 77(1): 1115-1132.
13 LuX, SuryanarayanP, R B JrAdams, et al. On shape and the computability of emotions[C]// Proceedings of the 20th ACM International Conference on Multimedia. New York: ACM Press, 2012: 229-238.
14 JoshiD, DattaR, FedorovskayaE, et al. Aesthetics and emotions in images[J]. IEEE Signal Processing Magazine, 2011, 28(5): 94-115.
15 WangX H, JiaJ, YinJ M, et al. Interpretable aesthetic features for affective image classification[C]// Proceedings of the IEEE International Conference on Image Processing. IEEE, 2014: 3230-3234.
16 曹建芳, 陈俊杰, 李海芳. 基于Adaboost-BP神经网络的图像情感分类方法研究[J]. 山西大学学报(自然科学版), 2013, 36(3): 331-337.
17 刘颍霜. 基于LFCM-SVM的版画情感映射改进算法研究[D]. 云南: 云南大学, 2015.
18 庄千料. 基于概率神经网络的图像细粒度情感分类[D]. 上海: 东华大学, 2017.
19 KoE, KimE Y. Recognizing the sentiments of web images using hand-designed features[C]// Proceedings of the IEEE 14th International Conference on Cognitive Informatics & Cognitive Computing. IEEE, 2015: 156-161.
20 ZhuoY. Research on perception-oriented image scene and emotion categorization[J]. The Open Automation and Control Systems Journal, 2015, 7: 2070-2073.
21 IttiL, KochC. Feature combination strategies for saliency-based visual attention systems[J]. Journal of Electronic Imaging, 2001, 10(1): 161-169.
22 LoweD G. Distinctive image features from scale-invariant keypoints[M]. Kluwer Academic Publishers, 2004.
23 GeusebroekJ M, BurghoutsG J, SmeuldersA W M. The Amsterdam library of object images[J]. International Journal of Computer Vision, 2005, 61(1): 103-112.
24 PengX J, WangL M, WangX X, et al. Bag of visual words and fusion methods for action recognition: Comprehensive study and good practice[J]. Computer Vision and Image Understanding, 2016, 150: 109-125.
25 Microsoft cognitive services[EB/OL]. [2018-04-15]. https://www.microsoft.com/cognitive-services/en-us/apis.
26 Schr?derM. Evidence for a three-factor theory of emotions[C]// Proceedings of the Conference on the European Cetacean Society. 2017: 716-721.
27 ValdezP, MehrabianA. Effects of color on emotions[J]. Journal of Experimental Psychology: General, 1994, 123(4): 394-409.
28 HaralickR M, ShanmugamK, DinsteinI H. Textural features for image classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1973, SMC-3(6): 610-621.
29 AnwerR M, VázquezD, LópezA M. Opponent colors for human detection[C]// Proceedings of the 5th Iberian conference on Pattern recognition and image analysis. Heidelberg: Springer, 2011: 363-370.
30 LangP J, BradleyM M, CuthbertB N. International affective picture system (IAPS): Affective ratings of pictures and instruction manual[R]. University of Florida, 2008: No. A-8.
31 吕微微. 基于情感的图像分类算法研究与实现[D]. 北京: 北京交通大学, 2012.
32 刘硕研. 面向感知的图像场景及情感分类算法研究[D]. 北京: 北京交通大学, 2011.
33 宋璇. 图像情感分类算法研究[D]. 秦皇岛: 燕山大学, 2017.
34 吕鹏霄. 图像情感分类研究[D]. 秦皇岛: 燕山大学, 2014. |
|
|
|
