|
|
|
| Research on High-Level Paper Selection from Citation Review Evidence Perspective |
| Ma Ruimin1,2, Feng Yumei1, Song Guoqing1 |
1.School of Economics and Management, Shanxi University, Taiyuan 030006
2.Research Center for Science Evaluation, Shanxi University, Taiyuan 030006 |
|
|
|
|
Abstract A paper is an important expression of scientific research results. It is crucial to scientifically select high-level papers under the current background of “breaking the five-only” approach, which is the basic starting point to ensure the implementation of representative work evaluation. Citation comments by the academic community are the most direct form of recognition of a paper’s value and serve as substantive and critical evidence of academic evaluation. Analyzing citation content more deeply helps to discover high-level papers more scientifically. First, this paper explains the basic idea of model construction based on the theory of academic identity and the characteristics of evidence. Second, it redefines and classifies citing emotions and citing functions based on content semantics, and fully considers the “citing author’s credibility.” This leads to the construction of a comprehensive weighted selection model for high-level academic papers. Third, relevant papers from Angewandte Chemie-International Edition, a top journal in the field of chemistry, are selected for empirical research. The results show that the model proposed in this paper accurately identifies “Very Important Papers (VIPs).” Finally, compared with other mainstream evaluation indexes, the model demonstrates a high degree of differentiation and discrimination, significantly improving the evaluation of the academic level of a paper.
|
|
Received: 06 October 2023
|
|
|
|
1 宋艳辉, 朱李, 邱均平. “破五唯”背景下我国科研评价体系构建的几点思考[J]. 情报杂志, 2022, 41(2): 190-197.
2 邱均平, 张裕晨, 周子番. 新时代我国科研评价体系重构中必须处理好八大关系[J]. 中国图书馆学报, 2021, 47(1): 47-60.
3 何春建. 单篇论文学术影响力评价指标构建[J]. 图书情报工作, 2017, 61(4): 98-107.
4 Garfield E. Citation indexes for science: a new dimension in documentation through association of ideas[J]. Science, 1955, 122(3159): 108-111.
5 祝清松, 冷伏海. 基于引文内容分析的高被引论文主题识别研究[J]. 中国图书馆学报, 2014, 40(1): 39-49.
6 彭秋茹, 阎素兰, 黄水清. 基于全文本分析的引文指标研究——以F1000推荐论文为例[J]. 信息资源管理学报, 2019, 9(4): 82-88.
7 Schubert A. Using the h-index for assessing single publications[J]. Scientometrics, 2009, 78(3): 559-565.
8 Ye F Y, Leydesdorff L. The “academic trace” of the performance matrix: a mathematical synthesis of the h-index and the integrated impact indicator (I3)[J]. Journal of the Association for Information Science and Technology, 2014, 65(4): 742-750.
9 王凌峰, 张泽玺. 基于引文网络的单篇论文新评价指标: Hi指标[J]. 情报理论与实践, 2012, 35(11): 52-55.
10 Nykl M, Je?ek K, Fiala D, et al. PageRank variants in the evaluation of citation networks[J]. Journal of Informetrics, 2014, 8(3): 683-692.
11 Wu L F, Wang D S, Evans J A. Large teams develop and small teams disrupt science and technology[J]. Nature, 2019, 566(7744): 378-382.
12 Piwowar H. Value all research products[J]. Nature, 2013, 493(7431): 159.
13 Bornmann L, Haunschild R. Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data[J]. PLoS One, 2018, 13(5): e0197133.
14 Zhu X D, Turney P, Lemire D, et al. Measuring academic influence: not all citations are equal[J]. Journal of the Association for Information Science and Technology, 2015, 66(2): 408-427.
15 胡志刚. 全文引文分析方法与应用[D]. 大连: 大连理工大学, 2014.
16 Mari?i? S, Spaventi J, Pavi?i? L, et al. Citation context versus the frequency counts of citation histories[J]. Journal of the American Society for Information Science, 1998, 49(6): 530-540.
17 徐琳宏, 丁堃, 孙晓玲, 等. 施引文献视角下正面引用论文的影响力及其影响因素的研究——以自然语言处理领域为例[J]. 情报学报, 2021, 40(4): 354-363.
18 王佳敏, 李信, 刘齐进. 全文本文献计量分析学术沙龙综述[J]. 信息资源管理学报, 2018, 8(4): 119-125.
19 Hernández-Alvarez M, Gómez J M. Citation impact categorization: for scientific literature[C]// Proceedings of the 2015 IEEE 18th International Conference on Computational Science and Engineering. Piscataway: IEEE, 2015: 307-313.
20 Chubin D E, Moitra S D. Content analysis of references: adjunct or alternative to citation counting?[J]. Social Studies of Science, 1975, 5(4): 423-441.
21 耿树青, 杨建林. 基于引用情感的论文学术影响力评价方法研究[J]. 情报理论与实践, 2018, 41(12): 93-98.
22 Dong C L, Sch?fer U. Ensemble-style self-training on citation classification[C]// Proceedings of the 5th International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, 2011: 623-631.
23 尹莉, 郭璐, 李旭芬. 基于引用功能和引用极性的一个引用分类模型研究[J]. 情报杂志, 2018, 37(7): 139-145.
24 Tuarob S, Kang S W, Wettayakorn P, et al. Automatic classification of algorithm citation functions in scientific literature[J]. IEEE Transactions on Knowledge and Data Engineering, 2020, 32(10): 1881-1896.
25 陆伟, 孟睿, 刘兴帮. 面向引用关系的引文内容标注框架研究[J]. 中国图书馆学报, 2014, 40(6): 93-104.
26 魏杨烨, 孙玉琦, 汪琦凡. 基于引用频次和引用内容的引文评价分析[J]. 情报探索, 2016(8): 116-120.
27 杨思洛, 聂颖. 结合全文本分析的论文影响力评价模型研究[J]. 现代情报, 2022, 42(3): 133-146.
28 冯长根. 一种自然而然的科技成果评价方法值得国家推广[J]. 中国人大, 2017(7): 33-35.
29 尚海茹, 冯长根, 孙良. 用学术影响力评价学术论文——兼论关于学术传承效应和长期引用的两个新指标[J]. 科学通报, 2016, 61(26): 2853-2860.
30 Le X Q, Chu J D, Deng S Y, et al. CiteOpinion: evidence-based evaluation tool for academic contributions of research papers based on citing sentences[J]. Journal of Data and Information Science, 2019, 4(4): 26-41.
31 马瑞敏, 刘志芳, 吕宇涵, 等. 基于引用评论加权的论文代表作遴选研究[J]. 情报学报, 2023, 42(3): 279-288.
32 刘文山. 学术期刊、学术共同体与学术评价[J]. 岭南学刊, 2016(2): 5-8.
33 刘珺珺. 科学社会学[M]. 上海: 上海科技教育出版社, 2009.
34 R. K. 默顿. 科学社会学: 理论与经验研究[M]. 鲁旭东, 林聚任, 译. 北京: 商务印书馆, 2009.
35 陈光中. 证据法学[M]. 4版. 北京: 法律出版社, 2019.
36 Ding Y, Zhang G, Chambers T, et al. Content-based citation analysis: the next generation of citation analysis[J]. Journal of the Association for Information Science and Technology, 2014, 65(9): 1820-1833.
37 刘盛博, 王博, 唐德龙, 等. 基于引用内容的论文影响力研究——以诺贝尔奖获得者论文为例[J]. 图书情报工作, 2015, 59(24): 109-114.
38 Ikram M T, Afzal M T. Aspect based citation sentiment analysis using linguistic patterns for better comprehension of scientific knowledge[J]. Scientometrics, 2019, 119(1): 73-95.
39 姜霖, 张麒麟. 基于引文细粒度情感量化的学术评价研究[J]. 数据分析与知识发现, 2020, 4(6): 129-138.
40 乔纳森·科尔, 斯蒂芬·科尔. 科学界的社会分层[M]. 赵佳苓, 顾昕, 黄绍林, 译. 北京: 华夏出版社, 1989.
41 马瑞敏, 张慧, 杨雨华. 加权引用视角下的作者学术影响力评价研究[J]. 情报学报, 2017, 36(8): 790-797.
42 Tseng S, Fogg B J. Credibility and computing technology[J]. Communications of the ACM, 1999, 42(5): 39-44.
43 徐琳宏, 丁堃, 陈娜, 等. 中文文献引文情感语料库构建[J]. 情报学报, 2020, 39(1): 25-37.
44 Xu L H, Ding K, Lin Y. Do negative citations reduce the impact of cited papers?[J]. Scientometrics, 2022, 127(2): 1161-1186.
45 Garfield E. Can citation indexing be automated?[C]// Statistical Association Methods for Mechanized Documentation, Symposium Proceedings. Washington: National Bureau of Standards, 1964: 182-192.
46 Weinstock M. Citation index[M]. Philadephia: Institute for Scientific Information, 1971.
47 Petri? B, Harwood N. Task requirements, task representation, and self-reported citation functions: an exploratory study of a successful L2 student's writing[J]. Journal of English for Academic Purposes, 2013, 12(2): 110-124.
48 Lyu D Q, Ruan X M, Xie J, et al. The classification of citing motivations: a meta-synthesis[J]. Scientometrics, 2021, 126(4): 3243-3264.
49 White H D. Authors as citers over time[J]. Journal of the American Society for Information Science and Technology, 2001, 52(2): 87-108.
50 Hirsch J E. An index to quantify an individual’s scientific research output[J]. Proceedings of the National Academy of Sciences of the United States of America, 2005, 102(46): 16569-16572.
51 易平涛, 李伟伟, 郭亚军. 综合评价理论与方法[M]. 2版. 北京: 经济管理出版社, 2019.
52 Mutz R, Wolbring T, Daniel H D. The effect of the “very important paper” (VIP) designation in Angewandte Chemie-International Edition on citation impact: a propensity score matching analysis[J]. Journal of the Association for Information Science and Technology, 2017, 68(9): 2139-2153.
53 Rao G. What is an ROC curve?[J]. Journal of Family Practice, 2003, 52(9): 695. |
|
|
|
