|
|
|
| Measurement of Paper Innovation Quality by Integrating Novelty and Academic Influence Characteristics |
| Li Jing, Qiu Xinpeng |
| School of Information Management, Sun Yat-sen University, Guangzhou 510006 |
|
|
|
|
Abstract With an emphasis on the high-quality development of science and technology and breaking the “five principles” of science and technology evaluation, accurately measuring the innovation quality of academic papers will provide a methodological basis for improving the evaluation system of scientific and technological achievements. Based on the AHP (analytic hierarchy process) - entropy weight - TOPSIS (technique for order preference by similarity to an ideal solution) method, this study proposes a measurement framework for paper innovation quality considering the characteristics of paper novelty and academic influence. The novelty score of the papers is obtained by calculating the frequency of question words, method words, and problem-method combination words based on the principle of word frequency. The academic influence score is obtained by calculating the number and quality of citations. The empirical results show that the measurement method of paper innovation quality constructed in this study can, to some extent, address the limitations of existing paper evaluation methods, such as the use of a single evaluation index, low discrimination of results, and emphasis on the number of citations but neglecting the quality of citations, and can effectively identify papers with high innovation quality from journals with weak impact factors.
|
|
Received: 17 April 2024
|
|
|
|
1 新质生产力的内涵特征和发展重点[EB/OL]. (2024-03-01) [2024-03-27]. http://www.xinhuanet.com/politics/20240301/ec9c1f 5a8ab748fab5f74c59d3fe5862/c.html.
2 为建设世界科技强国而奋斗——在全国科技创新大会、两院院士大会、中国科协第九次全国代表大会上的讲话[N]. 人民日报, 2016-06-01(2).
3 教育部科技部印发意见要求规范高等学校SCI论文相关指标使用[EB/OL]. (2020-02-23) [2024-03-27]. http://www.moe.gov.cn/jyb_xwfb/gzdt_gzdt/s5987/202002/t20200223_423329.html.
4 约瑟夫·熊彼. 特经济发展理论[M]. 郭武军, 吕阳, 译. 北京: 华夏出版社, 2015.
5 杨幽红. 创新质量理论框架: 概念、内涵和特点[J]. 科研管理, 2013, 34(S1): 320-325.
6 Haner U E. Innovation quality—a conceptual framework[J]. International Journal of Production Economics, 2002, 80(1): 31-37.
7 Chu J S G, Evans J A. Slowed canonical progress in large fields of science[J]. Proceedings of the National Academy of Sciences of the United States of America, 2021, 118(41): e2021636118.
8 索传军, 于淼, 牌艳欣, 等. 数据驱动的学术评价理论框架研究[J]. 图书情报工作, 2024, 68(1): 5-12.
9 叶继元. 学术“全评价”分析框架与创新质量评价的难点及其对策[J]. 河南大学学报(社会科学版), 2016, 56(5): 151-156.
10 汪雪锋, 于慧妍, 郑思佳, 等. 学术论文创新质量评价研究——以多能干细胞技术为例[J]. 数据分析与知识发现, 2024, 8(5): 127-138.
11 李晶, 邱昕鹏. 基于大语言模型的论文创新质量测度研究[J/OL]. 情报理论与实践, (2024-10-17) [2024-12-19]. https://kns.cnki.net/kcms/detail/11.1762.G3.20241016.1745.002.html.
12 李晶, 杨雪, 苏秋丹, 等. 基于知识单元理论的科技成果创新性测度研究述评[J]. 现代情报, 2023, 43(8): 161-177.
13 Popadiuk S, Choo C W. Innovation and knowledge creation: how are these concepts related?[J]. International Journal of Information Management, 2006, 26(4): 302-312.
14 杨杰, 邓三鸿, 王昊. 科学研究的颠覆性创新测度——相对颠覆性指数[J]. 情报学报, 2023, 42(9): 1052-1064.
15 逯万辉, 谭宗颖. 学术成果主题新颖性测度方法研究——基于Doc2Vec和HMM算法[J]. 数据分析与知识发现, 2018, 2(3): 22-29.
16 Lee P C, Su H N. Investigating the structure of regional innovation system research through keyword co-occurrence and social network analysis[J]. Innovation, 2010, 12(1): 26-40.
17 Uddin S, Khan A. The impact of author-selected keywords on citation counts[J]. Journal of Informetrics, 2016, 10(4): 1166-1177.
18 Luo Z R, Lu W, He J G, et al. Combination of research questions and methods: a new measurement of scientific novelty[J]. Journal of Informetrics, 2022, 16(2): 101282.
19 钱佳佳, 罗卓然, 陆伟. 基于问题-方法组合的科技论文新颖性度量与创新类型识别[J]. 图书情报工作, 2021, 65(14): 82-89.
20 van Houten B A, Phelps J, Barnes M, et al. Evaluating scientific impact[J]. Environmental Health Perspectives, 2000, 108(9): A392-A393.
21 Wu L F, Wang D S, Evans J A. Large teams develop and small teams disrupt science and technology[J]. Nature, 2019, 566(7744): 378-382.
22 何春建. 单篇论文学术影响力评价指标构建[J]. 图书情报工作, 2017, 61(4): 98-107.
23 刘运梅, 李长玲, 冯志刚, 等. 改进的p指数测度单篇论文学术质量的探讨[J]. 图书情报工作, 2017, 61(21): 106-113.
24 罗伯特·K.默顿. 科学社会学: 理论与经验研究[M]. 鲁旭东, 林聚任, 译. 北京: 华夏出版社, 1989.
25 Bornmann L, Mutz R. Further steps towards an ideal method of measuring citation performance: the avoidance of citation (ratio) averages in field-normalization[J]. Journal of Informetrics, 2011, 5(1): 228-230.
26 刘雪立, 魏雅慧, 盛丽娜, 等. 期刊PR8指数: 一个新的跨学科期刊评价指标及其实证研究[J]. 图书情报工作, 2017, 61(11): 116-123.
27 邱均平, 胡博, 徐中阳, 等. 学者学术话语权评价指标体系的构建与应用研究——以有机化学领域为例[J]. 情报理论与实践, 2024, 47(2): 43-52.
28 李娟, 李保安, 方晗, 等. 基于AHP-熵权法的发明专利价值评估——以丰田开放专利为例[J]. 情报杂志, 2020, 39(5): 59-63.
29 Bornmann L, Tekles A. Disruption index depends on length of citation window[J]. El Profesional de la Información, 2019, 28(2): e280207.
30 和晋飞, 房俊民. 基于Google Scholar引文数据的引文速度指标探析——以“图灵奖文献”为例[J]. 情报理论与实践, 2015, 38(4): 86-91, 56.
31 Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 4171-4186.
32 Wang J, Veugelers R, Stephan P. Bias against novelty in science: a cautionary tale for users of bibliometric indicators[J]. Research Policy, 2017, 46(8): 1416-1436.
33 Taylor J M. Kendall’s and Spearman’s correlation coefficients in the presence of a blocking variable[J]. Biometrics, 1987, 43(2): 409-416.
34 Ruan X M, Lyu D Q, Gong K L, et al. Rethinking the disruption index as a measure of scientific and technological advances[J]. Technological Forecasting and Social Change, 2021, 172: 121071.
35 张林, 陈梓慕. 中国区域创新质量空间演化与影响机制[J]. 开发研究, 2024(3): 10-19. |
|
|
|
