|
|
|
| Factors Influencing Relevance Judgment in Video Retrieval: An Empirical Study Based on PLS Path Analysis |
| Wang Zhihong, Cao Shujin |
| School of Information Management, Sun Yat-sen University, Guangzhou 510006 |
|
|
|
|
Abstract Relevance evaluation is a key aspect of video information seeking retrieval. Thus, to design and develop a better support system, it is crucial to investigate the factors influencing humans' relevance judgment in video retrieval . Based on existing theories, this paper constructed a multiple-factor model of relevance judgment in video retrieval. Subsequently, structured data were collected from 56 subjects who completed three simulated tasks and answered pre-task and post-task questionnaires. PLS path analysis was used to analyze the collected data and test the model. The results showed that the measurement model had confirmed reliability and validity. The structural model test revealed that topicality, scope, and authority, respectively, were significant factors, whereas understandability, availability, and video characteristics were found to have no statistically significant impact on relevance judgment in video retrieval. Scope also had a significant influence on topicality, thus influencing relevance judgment indirectly. Group analysis indicated that moderating factors including gender, information search ability, and topic familiarity adjusted the impact of the other influencing factors on relevance judgment. This study demonstrates that the factors influencing relevance judgment have some stability and commonality across different contexts and types of information. These results suggest that video system design and development should consider factors beyond topicality to help users effectively and efficiently access video.
|
|
Received: 24 July 2019
|
|
|
|
1 中国互联网络信息中心. 第43次《中国互联网络发展状况统计报告》全文[R/OL]. (2019-02-28) [2019-05-19]. http://www.cac.gov.cn/2019-02/28/c_1124175677.htm.
2 中国互联网络信息中心. 2013年中国网民网络视频应用研究报告[R/OL]. (2014-06-01) [2019-05-19]. http://www.cac.gov.cn/files/pdf/wangluoshipin.pdf.
3 中国互联网络信息中心. 2016年中国网民搜索行为调查报告[R/OL]. (2017-07-01) [2019-05-19]. http://202.116.81.74/cache/8/03/www.cnnic.net.cn/32ddac133cc3d5fb5b79a684994105f6/P020 180109484661983457.pdf.
4 Darvish S, Chin A. Dealing with the video tidal wave: The relevance of expertise for video tagging[C]// Proceedings of the 21st ACM Conference on Hypertext and Hypermedia. New York: ACM Press, 2010: 289-290.
5 Halvey M, Vallet D, Hannah D, et al. Supporting exploratory video retrieval tasks with grouping and recommendation[J]. Information Processing & Management, 2014, 50(6): 876-898.
6 Albertson D. Influences of users' familiarity with visual search topics on interactive video digital libraries[J]. Journal of the American Society for Information Science and Technology, 2010, 61(12): 2490-2502.
7 Google Cloud. Video AI[EB/OL]. [2019-05-19]. https://cloud.google.com/video-intelligence/#header.
8 Saracevic T. Relevance: A review of and a framework for the thinking on the notion in information science[J]. Journal of the American Society for Information Science, 1975, 26(6): 321-343.
9 Saracevic T. Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance[J]. Journal of the American Society for information Science and Technology, 2007, 58(13): 2126-2144.
10 Xu Y C, Chen Z W. Relevance judgment: What do information users consider beyond topicality?[J]. Journal of the American Society for Information Science and Technology, 2006, 57(7): 961-973.
11 Zhang Y L, Zhang J, Lease M, et al. Multidimensional relevance modeling via psychometrics and crowdsourcing[C]// Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval. New York: ACM Press, 2014: 435-444.
12 Wang P L, Soergel D. A cognitive model of document use during a research project. Study I. Document selection[J]. Journal of the American Society for Information Science, 1998, 49(2): 115-133.
13 Cool C, Belkin N J, Frieder O, et al. Characteristics of text affecting relevance judgments[C]// Proceedings of the 14th National Online Meeting. New York: Learned Information, 1993, 14: 77-84.
14 Park T K. The nature of relevance in information retrieval: An empirical study[J]. The Library Quarterly, 1993, 63(3): 318-351.
15 Schamber L. Relevance and information behavior[J]. Annual Review of Information Science and Technology, 1994, 29: 3-48.
16 Barry C L. User‐defined relevance criteria: An exploratory study[J]. Journal of the American Society for Information Science, 1994, 45(3): 149-159.
17 Maglaughlin K L, Sonnenwald D H. User perspectives on relevance criteria: A comparison among relevant, partially relevant, and not‐relevant judgments[J]. Journal of the American Society for Information Science and Technology, 2002, 53(5): 327-342.
18 Hamid R A, Thom J A. Criteria that have an effect on users while making image relevance judgments[C]// Proceedings of the 15th Australasian Document Computing Symposium, Melbourne, Australia, 2010: 76-83.
19 Redi M, Merialdo B. A multimedia retrieval framework based on automatic graded relevance judgments[C]// Proceedings of the International Conference on Multimedia Modeling. Heidelberg: Springer, 2012: 300-311.
20 Choi Y, Rasmussen E M. Users' relevance criteria in image retrieval in American history[J]. Information Processing & Management, 2002, 38(5): 695-726.
21 Sedghi S, Sanderson M, Clough P. A study on the relevance criteria for medical images[J]. Pattern Recognition Letters, 2008, 29(15): 2046-2057.
22 Laplante A. Users' relevance criteria in music retrieval in everyday life: An exploratory study[C]// Proceedings of the 11th International Society for Music Information Retrieval Conference, Utrecht, Netherlands, 2010: 601-606.
23 Inskip C, MacFarlane A, Rafferty P. Creative professional users' musical relevance criteria[J]. Journal of Information Science, 2010, 36(4): 517-529.
24 Savolainen R, Kari J. User-defined relevance criteria in web searching[J]. Journal of Documentation, 2006, 62(6): 685-707.
25 Crystal A, Greenberg J. Relevance criteria identified by health information users during Web searches[J]. Journal of the American Society for Information Science and Technology, 2006, 57(10): 1368-1382.
26 Chen Y C, Shang R A, Li M J. The effects of perceived relevance of travel blogs' content on the behavioral intention to visit a tourist destination[J]. Computers in Human Behavior, 2014, 30: 787-799.
27 Wang X L, Hong Z J, Xu Y, et al. Relevance judgments of mobile commercial information[J]. Journal of the Association for Information Science and Technology, 2014, 65(7): 1335-1348.
28 Rosenthal S. Motivations to seek science videos on YouTube: Free-choice learning in a connected society[J]. International Journal of Science Education, Part B, 2018, 8(1): 22-39.
29 孙建军, 裴雷, 刘虹. 基于期望确认模型的视频网站持续使用模型构建[J]. 图书情报知识, 2013(5): 82-88, 45.
30 侯德林. 网络视频服务用户行为意愿实证研究[D]. 武汉: 华中科技大学, 2012.
31 梁霞. 基于UTAUT模型的用户接受网络付费视频的影响因素研究[D]. 哈尔滨: 哈尔滨工业大学, 2017.
32 王永昂. 大学生对哔哩哔哩网的使用行为及认同建构研究[D]. 广州: 广州大学, 2017.
33 Jung I, Lee Y. YouTube acceptance by university educators and students: A cross-cultural perspective[J]. Innovations in Education and Teaching International, 2015, 52(3): 243-253.
34 卢俊. 大学生微博短视频使用状况研究[D]. 呼和浩特: 内蒙古大学, 2017.
35 谭金波. 网络视频检索的用户信息行为研究[J]. 图书情报工作, 2013, 57(8): 125-129.
36 Halvey M J, Keane M T. Analysis of online video search and sharing[C]// Proceedings of the 18th Conference on Hypertext and Hypermedia. New York: ACM Press, 2007: 217-226.
37 Kim H H. Toward video semantic search based on a structured folksonomy[J]. Journal of the American Society for Information Science and Technology, 2011, 62(3): 478-492.
38 Albertson D. Analyzing user interaction with the ViewFinder video retrieval system[J]. Journal of the American Society for Information Science and Technology, 2010, 61(2): 238-252.
39 Albertson D, Meadows III C. Situated topic complexity in interactive video retrieval[J]. Journal of the American Society for Information Science and Technology, 2011, 62(9): 1676-1695.
40 Yang M. An exploration of users' video relevance criteria[D]. Chapel Hill: The University of North Carolina at Chapel Hill, 2005.
41 Albassam S A A, Ruthven I. Users' relevance criteria for video in leisure contexts[J]. Journal of Documentation, 2018, 74(1): 62-79.
42 Schamber L, Eisenberg M B, Nilan M S. A re-examination of relevance: Toward a dynamic, situational definition[J]. Information Processing & Management, 1990, 26(6): 755-776.
43 Kek?l?inen J, J?rvelin K. Evaluating information retrieval systems under the challenges of interaction and multidimensional dynamic relevance[C]// Proceedings of the 4th CoLIS conference, Seattle, USA, 2002: 253-270.
44 Borlund P. The concept of relevance in IR[J]. Journal of the American Society for information Science and Technology, 2003, 54(10): 913-925.
45 Hj?rland B. The foundation of the concept of relevance[J]. Journal of the American Society for Information Science and Technology, 2010, 61(2): 217-237.
46 Saracevic T. Relevance reconsidered[C]// Proceedings of the Second Conference on Conceptions of Library and Information Science. New York: ACM Press, 1996: 201-218.
47 Greisdorf H. Relevance thresholds: A multi-stage predictive model of how users evaluate information[J]. Information Processing & Management, 2003, 39(3): 403-423.
48 Cunningham S J, Nichols D M. How people find videos[C]// Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries. New York: ACM Press, 2008: 201-210.
49 林怡君. 网络影音分享网站之使用者检索行为研究[D]. 台湾: 台湾师范大学, 2010.
50 Xu Y J. Relevance judgment in epistemic and hedonic information searches[J]. Journal of the American Society for Information Science and Technology, 2007, 58(2): 179-189.
51 Taylor A. User relevance criteria choices and the information search process[J]. Information Processing & Management, 2012, 48(1): 136-153.
52 童迎, 李鹏, 夏慧. 合作信息查寻与检索相关性判据的探索性因子分析[J]. 图书情报工作, 2013, 57(19): 30-36.
53 成颖. 信息检索相关性判据及应用研究[D]. 南京: 南京大学, 2011.
54 Bales S, Wang P L. Consolidating user relevance criteria: A meta‐ethnography of empirical studies[J]. Proceedings of the American Society for Information Science and Technology, 2005, 42(1): 1-7.
55 Chin W W, Marcolin B L, Newsted P R. A partial least squares latent variable modeling approach for measuring interaction effects: Results from a Monte Carlo simulation study and an electronic-mail emotion/adoption study[J]. Information Systems Research, 2003, 14(2): 189-217.
56 Kahai S S, Cooper R B. Exploring the core concepts of media richness theory: The impact of cue multiplicity and feedback immediacy on decision quality[J]. Journal of Management Information Systems, 2003, 20(1): 263-299.
57 Wong K K K. Partial least squares structural equation modeling (PLS-SEM) techniques using SmartPLS[J]. Marketing Bulletin, 2013, 24(1): 1-32.
58 Hair Jr J F, Hult G T M, Ringle C, et al. A primer on partial least squares structural equation modeling (PLS-SEM)[M]. SAGE Publications, 2016: 24.
59 Eberl M. An application of PLS in multi-group analysis: The need for differentiated corporate-level marketing in the mobile communications industry[M]// Handbook of Partial Least Squares. Heidelberg: Springer, 2009: 487-514.
60 Matthews L. Applying multigroup analysis in PLS-SEM: A step-by-step process[M]// Partial Least Squares Path Modeling. Cham: Springer, 2017: 219-243.
61 Henseler J, Fassott G. Testing moderating effects in PLS path models: An illustration of available procedures[M]// Handbook of Partial Least Squares. Heidelberg: Springer, 2010: 713-735.
62 Sarstedt M, Henseler J, Ringle C M. Multigroup analysis in partial least squares (PLS) path modeling: Alternative methods and empirical results[M]// Measurement and Research Methods in International Marketing. Emerald Group Publishing Limited, 2011: 195-218.
63 Hair Jr J F, Sarstedt M, Hopkins L, et al. Partial least squares structural equation modeling (PLS-SEM): An emerging tool in business research[J]. European Business Review, 2014, 26(2): 106-121.
64 Henseler J, Ringle C M, Sinkovics R R. The use of partial least squares path modeling in international marketing[M]// New Challenges to International Marketing. Emerald Group Publishing Limited, 2009: 277-319.
65 Henseler J, Hubona G, Ray P A. Using PLS path modeling in new technology research: Updated guidelines[J]. Industrial Management & Data Systems, 2016, 116(1): 2-20.
66 Fornell C, Larcker D F. Structural equation models with unobservable variables and measurement error: Algebra and statistics[J]. Journal of Marketing Research, 1981, 18(3): 382-388.
67 SmartPLS. Model Fit[EB/OL]. [2019-05-19]. https://www.smartpls.com/documentation/algorithms-and-techniques/model-fit.
68 邱钰淇. 從服務主導導向觀點探討協同消費[D]. 台湾: 屏东大学, 2017.
69 Grice H P. Logic and conversation[M]// Syntax and Semantic, Vol 3: Speech Arts. 1975: 41-58.
70 Salim F A, Haider F, Conlan O, et al. An approach for exploring a video via multimodal feature extraction and user interactions[J]. Journal on Multimodal User Interfaces, 2018, 12(4): 285-296.
71 Nakamoto S. Bitcoin: A peer-to-peer electronic cash system[OL]. https://bitcoin.org/bitcoin.pdf.
72 Barry C L, Schamber L. Users' criteria for relevance evaluation: a cross-situational comparison[J]. Information Processing & Management, 1998, 34(2-3): 219-236.
73 曹树金, 王志红. 虚拟社区知识共享意愿与行为的影响因素及其调节变量: 元分析研究[J]. 图书情报工作, 2018, 62(8): 74-83.
74 Schamber L, Bateman J. User criteria in relevance evaluation: Toward development of a measurement scale[C]// Proceedings of the ASIS Annual Meeting, 1996, 33: 218-225.
75 Eisenberg M B. Measuring relevance judgments[J]. Information Processing & Management, 1988, 24(4): 373-389.
76 Zhu X L, Gauch S. Incorporating quality metrics in centralized/distributed information retrieval on the World Wide Web[C]// Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM Press, 2000: 288-295.
77 Cosijn E, Ingwersen P. Dimensions of relevance[J]. Information Processing & Management, 2000, 36(4): 533-550. |
|
|
|
