|
|
|
| Societal Impact Indicators of Research Outcomes in Humanities Based on REF Case Studies |
| Zeng Yueliang1, Zheng Han2, Yang Siluo3 |
1.School of Information Management, Central China Normal University, Wuhan 430079
2.Wee Kim Wee School of Communication and Information, Nanyang Technological University, Singapore 637718
3.School of Information Management, Wuhan University, Wuhan 430072 |
|
|
|
|
Abstract While it is critical for scholars to demonstrate the academic impact of scientific research outputs, the academic community and industry have recently become increasingly concerned with the societal impact of scientific research. As a key academic discipline, the societal impact of humanities research is usually deemed as intangible and indirect. Consequently, selecting appropriate indicators for research evaluation is challenging and requires further exploration in this area. Accordingly, this study aims to explore the societal impact indicators for humanities research. Based on the Research Contribution Framework, this study used an abductive approach for analyzing 1,617 impact case studies submitted to the Arts and Humanities Panel of the Research Excellence Framework (REF) in the United Kingdom. In particular, the impact indicators embedded in the case studies were extracted through topic modeling; then, they were further mapped with the six indicators in the Research Contribution Framework (RCF). The indicators observed underlined that three emerging stages had not yet been recognized in the RCF, namely, changes in spread scope, current benefits, and forward-looking benefits. The indicator framework is more comprehensive and more in line with the characteristics of humanities research. In addition, this framework also provides a reference for the assessment mechanism optimization of societal impact of research outcomes in humanities of China.
|
|
Received: 29 March 2022
|
|
|
|
1 Chi P S, Gl?nzel W. Comparison of citation and usage indicators in research assessment in scientific disciplines and journals[J]. Scientometrics, 2018, 116(1): 537-554.
2 刘兴凯, 左小娟. 科研卓越框架(REF): 英国高校科研评估的改革创新及其价值取向[J]. 学术论坛, 2015, 38(8): 85-90.
3 de Jong S, Barker K, Cox D, et al. Understanding societal impact through productive interactions: ICT research as a case[J]. Research Evaluation, 2014, 23(2): 89-102.
4 谢冉, 肖建. 英国科研卓越框架的科研影响力评估[J]. 高教探索, 2018(6): 55-61.
5 王茜, 谭宗颖, 钱力. 科学研究社会影响力评价综述[J]. 图书情报工作, 2015, 59(14): 143-148.
6 REF 2014. Decisions on assessing research impact[R]. Bristol: REF, 2011.
7 教育部 科技部印发《关于规范高等学校SCI论文相关指标使用 树立正确评价导向的若干意见》的通知[EB/OL]. (2020-02-18) [2022-05-30]. http://www.gov.cn/zhengce/zhengceku/2020-03/03/content_5486229.htm.
8 国家市场监督管理总局, 国家标准化管理委员会. 人文社会科学期刊评价(GB/T 40108—2021)[EB/OL]. (2021-05-21) [2022-05-30]. http://c.gb688.cn/bzgk/gb/showGb?type=online&hcno=E6324994D8E56285C1A347553E6CD454.
9 国务院办公厅关于完善科技成果评价机制的指导意见[EB/OL]. (2021-08-02) [2022-05-30]. http://www.gov.cn/zhengce/content/2021-08/02/content_5628987.htm.
10 苏娜. 科学研究的社会影响力评价: 研究与实践进展[J]. 情报学报, 2020, 39(10): 1114-1119.
11 Pan S L, Pee L G. Usable, in-use, and useful research: a 3U framework for demonstrating practice impact[J]. Information Systems Journal, 2020, 30(2): 403-426.
12 陈芳, 杨建林. 科研成果社会影响力评价研究——评价方法视角下的解读[J]. 现代情报, 2021, 41(11): 111-119.
13 Wiek A, Talwar S, O’Shea M, et al. Toward a methodological scheme for capturing societal effects of participatory sustainability research[J]. Research Evaluation, 2014, 23(2): 117-132.
14 Gl?nzel W, Chi P S. The big challenge of Scientometrics 2.0: exploring the broader impact of scientific research in public health[J]. Scientometrics, 2020, 125(2): 1011-1031.
15 Asveld L, van Dam-Mieras R. Introduction: responsible research and innovation for sustainability[M]// Responsible Innovation 3. Cham: Springer, 2017: 1-6.
16 刘志. 论人文科学研究的价值与方法[J]. 青海社会科学, 2007(4): 134-137.
17 陈先达. 寻求科学与价值之间的和谐——关于人文科学性质与创新问题[J]. 中国社会科学, 2003(6): 14-24, 205.
18 郑文涛. 人文社会科学若干概念辨析[J]. 首都师范大学学报(社会科学版), 2008(3): 141-148.
19 林坚. 更加重视人文社科成果转化[N]. 人民日报, 2015-12-25(7).
20 庞学铨. 论人文科学的价值与功能[J]. 中共浙江省委党校学报, 2009, 25(2): 11-17.
21 徐芳, 刘文斌, 李晓轩. 英国REF科研影响力评价的方法及启示[J]. 科学学与科学技术管理, 2014, 35(7): 9-15.
22 Bornmann L. What is societal impact of research and how can it be assessed? A literature survey[J]. Journal of the American Society for Information Science and Technology, 2013, 64(2): 217-233.
23 王悠然, 陈禹同. 加强科研项目的社会影响力研究[N]. 中国社会科学报, 2020-11-27(2).
24 Given L M, Kelly W, Willson R. Bracing for impact: the role of information science in supporting societal research impact[J]. Proceedings of the Association for Information Science and Technology, 2015, 52(1): 1-10.
25 Moed H F, Halevi G. Multidimensional assessment of scholarly research impact[J]. Journal of the Association for Information Science and Technology, 2015, 66(10): 1988-2002.
26 Morrow E M, Goreham H, Ross F. Exploring research impact in the assessment of leadership, governance and management research[J]. Evaluation, 2017, 23(4): 407-431.
27 Morton S. Progressing research impact assessment: a ‘contributions’ approach[J]. Research Evaluation, 2015, 24(4): 405-419.
28 Ozanne J L, Davis B, Murray J B, et al. Assessing the societal impact of research: the relational engagement approach[J]. Journal of Public Policy & Marketing, 2017, 36(1): 1-14.
29 Kuruvilla S, Mays N, Pleasant A, et al. Describing the impact of health research: a research impact framework[J]. BMC Health Services Research, 2006, 6(1): Article No.134.
30 Landry R, Amara N, Lamari M. Utilization of social science research knowledge in Canada[J]. Research Policy, 2001, 30(2): 333-349.
31 Bornmann L, Marx W. How should the societal impact of research be generated and measured? A proposal for a simple and practicable approach to allow interdisciplinary comparisons[J]. Scientometrics, 2014, 98(1): 211-219.
32 Sousa S B, Brennan J L. The UK Research Excellence Framework and the transformation of research production[M]// Reforming Higher Education. Dordrecht: Springer, 2014: 65-80.
33 Sigurearson E S. Capacities, capabilities, and the societal impact of the humanities[J]. Research Evaluation, 2020, 29(1): 71-76.
34 Benneworth P. Between certainty and comprehensiveness in evaluating the societal impact of humanities research[R/OL]. (2015-02-01) [2022-05-30]. https://ris.utwente.nl/ws/files/14233610/chepswpseries201502.pdf.
35 Levitt R, Celia C, Diepeveen S, et al. Assessing the impact of arts and humanities research at the University of Cambridge[R/OL]. Santa Monica: RAND, 2010. [2022-05-30]. http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=00502C1A2C522C76B7BFAC7126B54274?doi=10.1.1.231.2618&rep=rep1&type=pdf.
36 Pedersen D B, Gr?nvad J F, Hvidtfeldt R. Methods for mapping the impact of social sciences and humanities—a literature review[J]. Research Evaluation, 2020, 29(1): 4-21.
37 Knott J, Wildavsky A. If dissemination is the solution, what is the problem?[J]. Science Communication, 1980, 1: 537-578.
38 Morton S C. Exploring and assessing social research impact: a case study of a research partnership's impacts on policy and practice[D]. Edinburgh: University of Edinburgh, 2012. Morton S C. Exploring and assessing social research impact: a case study of a research partnership’s impacts on policy and practice[D]. Edinburgh: The University of Edinburgh, 2012.
39 Morton S, Flemming J. Assessing research impact: a case study of participatory research[R/OL]. (2013-02-01) [2022-05-30]. https://era.ed.ac.uk/bitstream/handle/1842/6562/Briefing;jsessionid=EB441671BD299EBFA0C35BC5C77F607A?sequence=1.
40 Ter?m? E, Smallman M, Lock S J, et al. Beyond academia - interrogating research impact in the research excellence framework[J]. PLoS One, 2016, 11(12): e0168533.
41 Grant J, Hinrichs S. The nature, scale and beneficiaries of research impact: an initial analysis of Research Excellence Framework (REF) 2014 impact case studies[R/OL]. (2015-03-01) [2022-05-30]. https://webarchive.nationalarchives.gov.uk/20180319114338/http://www.hefce.ac.uk/pubs/rereports/year/2015/analysisREFimpact/.
42 Marcella R, Lockerbie H, Bloice L. Beyond REF 2014: the impact of impact assessment on the future of information research[J]. Journal of Information Science, 2016, 42(3): 369-385.
43 Kelly D, Kent B, McMahon A, et al. Impact case studies submitted to REF 2014: the hidden impact of nursing research[J]. Journal of Research in Nursing, 2016, 21: 256-268.
44 Greenhalgh T, Fahy N. Research impact in the community-based health sciences: an analysis of 162 case studies from the 2014 UK Research Excellence Framework[J]. BMC Medicine, 2015, 13: 232.
45 Gill A J, Hinrichs-Krapels S, Blanke T, et al. Insight workflow: systematically combining human and computational methods to explore textual data[J]. Journal of the Association for Information Science and Technology, 2017, 68(7): 1671-1686.
46 刘莉. 英国大学科研评价改革: 从RAE到REF[J]. 科学学与科学技术管理, 2014, 35(2): 39-45.
47 刘兴凯, 左小娟. 英国大学科研影响力评估机制及其启示[J]. 中国高教研究, 2015(8): 67-71, 75.
48 邓荔萍. 基于科研绩效框架(REF)的三种文献计量模式[J]. 新世纪图书馆, 2011(11): 46-49.
49 REF 2014. Assessment framework and guidance on submissions[EB/OL]. (2011-07-01) [2022-05-30]. https://www.ref.ac.uk/2014/pubs/2011-02/.
50 Blei D M. Topic modeling and digital humanities[J/OL]. Journal of Digital Humanities, 2012, 2(1). https://journalofdigitalhumanities.org/2-1/topic-modeling-and-digital-humanities-by-david-m-blei/.
51 Dubois A, Gadde L E. Systematic combining: an abductive approach to case research[J]. Journal of Business Research, 2002, 55(7): 553-560.
52 Lindberg A. Developing theory through integrating human and machine pattern recognition[J]. Journal of the Association for Information Systems, 2020, 21(1): 90-116.
53 Chuang J, Gupta S, Manning C D, et al. Topic model diagnostics: assessing domain relevance via topical alignment[C]// Proceedings of the 30th International Conference on Machine Learning. JMLR.org, 2013: III-612-III-620.
54 Piepenbrink A, Gaur A S. Topic models as a novel approach to identify themes in content analysis[C/OL]// Proceedings of the Academy of Management Annual Meeting. Briarcliff Manor: Academy of Management, 2017. [2017-10-30]. https://doi.org/10.5465/ambpp.2017.141.
55 Gerstle A. Bringing Kabuki prints of the 18th and 19th centuries to modern audiences and modern art markets[R/OL]. [2022-05-30]. https://impact.ref.ac.uk/casestudies/CaseStudy.aspx?Id=42788.
56 Clark C. The rise and downfall of Prussia 1600-1947[R/OL]. [2022-05-30]. https://impact.ref.ac.uk/casestudies/CaseStudy.aspx?Id=23341.
57 Whitty M T, Buchanan T. Confronting online dating scams: working with police and the industry[R/OL]. [2022-05-30]. https://impact.ref.ac.uk/casestudies/CaseStudy.aspx?Id=40560.
58 Wilkinson L. Henry III Fine Rolls Project 1216-1272[R/OL]. [2022-05-30]. https://impact.ref.ac.uk/casestudies/CaseStudy.aspx?Id=40360.
59 Hardy J. Media regulation and communications policy reform[R/OL]. [2022-05-30]. https://impact.ref.ac.uk/casestudies/CaseStudy.aspx?Id=42767.
60 Newcastle University. Informing and enhancing the public understanding of the classical world[R/OL]. [2022-05-30]. https://impact.ref.ac.uk/casestudies/CaseStudy.aspx?Id=21731.
61 Dovey L. Bringing African film to international audiences[R/OL]. [2022-05-30]. https://impact.ref.ac.uk/casestudies/CaseStudy.aspx?Id=42779.
62 Toulmin V. Working class entertainment: economic and cultural impact on blackpool[R/OL]. [2022-05-30]. https://impact.ref.ac.uk/casestudies/CaseStudy.aspx?Id=12213.
63 Hancox G. Singing, health and wellbeing[R/OL]. [2022-05-30]. https://impact.ref.ac.uk/casestudies/CaseStudy.aspx?Id=40398.
64 University of Liverpool. The application of embedded analytics to hyper-scale and distributed data archives[R/OL]. [2022-05-30]. https://impact.ref.ac.uk/casestudies/CaseStudy.aspx?Id=27427.
65 黄水清. 不断完善人文社科期刊评价体系[N]. 中国社会科学报, 2022-01-25(6).
66 杨思洛. 新时期我国人文社科成果国际影响力评价: 问题与趋势[J]. 情报资料工作, 2020, 41(3): 20-25. |
|
|
|
