融合文献计量与同行评议共识的期刊评价方法研究

doi:10.3772/j.issn.1000-0135.2022.05.005

情报学报

2022, Vol. 41

Issue (5): 486-496 DOI: 10.3772/j.issn.1000-0135.2022.05.005

情报理论与方法

本期目录 | 过刊浏览 | 高级检索

融合文献计量与同行评议共识的期刊评价方法研究

张旭¹, 吕冬晴¹, 阮选敏¹, 成颖¹, 鞠秀芳²

1.南京大学信息管理学院，南京 210023
2.南京大学中国社会科学研究评价中心，南京 210093

Journal Evaluation Method Based on the Integration of Peer-Review Consensus and Bibliometric Indicators

Zhang Xu¹, Lyu Dongqing¹, Ruan Xuanmin¹, Cheng Ying¹, Ju Xiufang²

1.School of Information Management, Nanjing University, Nanjing 210023
2.Chinese Social Sciences Research Evaluation Center, Nanjing University, Nanjing 210093

摘要
图/表
参考文献
相关文章 (1)

全文: PDF (1601 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要针对当前期刊评价研究与实践中存在的定性与定量融合不足的问题，本研究将学科/领域同行对一流期刊的共识作为评价出发点，借助TrustRank算法，提出了融合同行评议共识与文献计量方法的期刊权威性因子JAF（journal authoritativeness factor），并进一步整合JAF与JIF（journal impact factor）提出了融合指标JAII（journal authoritativeness and impact index），用于进行期刊双维度评价。为验证JAF与JAII的有效性，本研究选取FMS（Federation of Management Societies of China）管理科学领域推荐的期刊作为评价对象，采用CSSCI（Chinese Social Sciences Citation Index）数据库2000—2018年的引文数据进行实证分析。结果表明，JAF对引文时间窗口具有稳健性；与两种基线指标相比，JAF与FMS的结果更加一致；JAF与JIF的融合指标JAII与FMS的相关性高达0.716。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	张旭
	吕冬晴
	阮选敏
	成颖
	鞠秀芳

关键词 ： TrustRank, PageRank, 期刊影响因子, 指标融合

收稿日期: 2021-07-13

基金资助:国家社会科学基金项目“学术文献颠覆性创新评价的理论及实证研究”（20BTQ086）。

作者简介: 张旭，女，1996年生，硕士研究生，主要研究方向为信息检索、自然语言处理；吕冬晴，女，1994年生，博士研究生，主要研究方向为科学计量；阮选敏，女，1994年生，博士研究生，主要研究方向为科学计量，E-mail：534795593@qq.com；成颖，男，1971年生，博士，教授，博士生导师，主要研究方向为信息检索、信息行为；鞠秀芳，女，1976年生，博士，研究馆员，主要研究方向为期刊评价、信息计；

引用本文:

张旭, 吕冬晴, 阮选敏, 成颖, 鞠秀芳. 融合文献计量与同行评议共识的期刊评价方法研究[J]. 情报学报, 2022, 41(5): 486-496.
Zhang Xu, Lyu Dongqing, Ruan Xuanmin, Cheng Ying, Ju Xiufang. Journal Evaluation Method Based on the Integration of Peer-Review Consensus and Bibliometric Indicators. 情报学报, 2022, 41(5): 486-496.

链接本文:

https://qbxb.istic.ac.cn/CN/10.3772/j.issn.1000-0135.2022.05.005 或 https://qbxb.istic.ac.cn/CN/Y2022/V41/I5/486

1 Serenko A, Bontis N. First in, best dressed: the presence of order-effect bias in journal ranking surveys[J]. Journal of Informetrics, 2013, 7(1): 138-144.
2 Cameron B D. Trends in the usage of ISI bibliometric data: uses, abuses, and implications[J]. portal: Libraries and the Academy, 2005, 5(1): 105-125.
3 Truex D, University G S, Cuellar M, et al. Assessing scholarly influence: using the hirsch indices to reframe the discourse[J]. Journal of the Association for Information Systems, 2009, 10(7): 560-594.
4 苏金燕. 人文社会科学期刊评价中同行评议与影响因子相关性分析[J]. 图书情报知识, 2020(3): 128-137.
5 Serenko A, Bontis N. What’s familiar is excellent: the impact of exposure effect on perceived journal quality[J]. Journal of Informetrics, 2011, 5(1): 219-223.
6 Page L, Brin S, Motwani R, et al. The PageRank citation ranking: bringing order to the Web[R]. Stanford InfoLab, 1998.
7 Maslov S, Redner S. Promise and pitfalls of extending Google’s PageRank algorithm to citation networks[J]. Journal of Neuroscience, 2008, 28(44): 11103-11105.
8 Ma N, Guan J C, Zhao Y. Bringing PageRank to the citation analysis[J]. Information Processing & Management, 2008, 44(2): 800-810.
9 SCImago journal & country rank[EB/OL]. [2020-12-22]. http://www.scimagojr.com/index.php.
10 Carl B. Eigenfactor: measuring the value and prestige of scholarly journals[J]. College & Research Libraries News, 2007, 68(5): 314-316.
11 Allen L, Jones C, Dolby K, et al. Looking for landmarks: the role of expert review and bibliometric analysis in evaluating scientific publication outputs[J]. PLoS One, 2009, 4(6): e5910.
12 Lowry P B, Moody G D, Gaskin J, et al. Evaluating journal quality and the association for information systems senior scholars’ journal basket via bibliometric measures: do expert journal assessments add value?[J]. MIS Quarterly, 2013, 37(4): 993-1012.
13 俞立平, 潘云涛, 武夷山. 学术期刊评价中不同利益主体关系研究[J]. 科学学与科学技术管理, 2009, 30(12): 43-47.
14 Gy?ngyi Z, Garcia-Molina H, Pedersen J. Combating Web spam with TrustRank[C]// Proceedings of the Thirtieth International Conference on Very Large Data Bases. VLDB Endowment, 2004: 576-587.
15 Garfield E. Citation indexes for science: a new dimension in documentation through association of ideas[J]. Science, 1955, 122(3159): 108-111.
16 Sharma O P. Journal impact factor: an essential primary quality indicator[J]. Current Science, 2007, 93(1): 5.
17 Rousseau R. Sir, why am I not cited or why are multi-authored papers more cited than others?[J]. Journal of Documentation, 1992, 48(1): 79-80.
18 Pinski G, Narin F. Citation influence for journal aggregates of scientific publications: theory, with application to the literature of physics[J]. Information Processing & Management, 1976, 12(5): 297-312.
19 Chen P, Xie H, Maslov S, et al. Finding scientific gems with Google’s PageRank algorithm[J]. Journal of Informetrics, 2007, 1(1): 8-15.
20 李江, 孙建军. 链接分析对引文分析的启示: 从PageRank到Paperank[J]. 情报学报, 2009, 28(4): 618-625.
21 Ding Y, Yan E J, Frazho A, et al. PageRank for ranking authors in co-citation networks[J]. Journal of the American Society for Information Science and Technology, 2009, 60(11): 2229-2243.
22 Yan E J, Ding Y. Discovering author impact: a PageRank perspective[J]. Information Processing & Management, 2011, 47(1): 125-134.
23 Bollen J, Rodriquez M A, van de Sompel H. Journal status[J]. Scientometrics, 2006, 69(3): 669-687.
24 苏成, 潘云涛, 袁军鹏, 等. 基于PageRank的期刊评价研究[J]. 中国科技期刊研究, 2009, 20(4): 614-617.
25 Cheang B, Chu S K W, Li C S, et al. A multidimensional approach to evaluating management journals: refining PageRank via the differentiation of citation types and identifying the roles that management journals play[J]. Journal of the Association for Information Science and Technology, 2014, 65(12): 2581-2591.
26 van Raan A F J. Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises[J]. Scientometrics, 1996, 36(3): 397-420.
27 Lewison G, Cottrell R, Dixon D. Bibliometric indicators to assist the peer review process in grant decisions[J]. Research Evaluation, 1999, 8(1): 47-52.
28 Gunashekar S, Wooding S, Guthrie S. How do NIHR peer review panels use bibliometric information to support their decisions?[J]. Scientometrics, 2017, 112(3): 1813-1835.
29 Goldstein H, Maier G. The use and valuation of journals in planning scholarship: peer assessment versus impact factors[J]. Journal of Planning Education and Research, 2010, 30(1): 66-75.
30 Maier G. Impact factors and peer judgment: the case of regional science journals[J]. Scientometrics, 2006, 69(3): 651-667.
31 Tsai C F, Hu Y H, George Ke S W. A Borda count approach to combine subjective and objective based MIS journal rankings[J]. Online Information Review, 2014, 38(4): 469-483.
32 Serenko A, Bontis N. Global ranking of knowledge management and intellectual capital academic journals[J]. Journal of Knowledge Management, 2009, 13(1): 4-15.
33 Serenko A, Bontis N. Global ranking of knowledge management and intellectual capital academic journals: 2013 update[J]. Journal of Knowledge Management, 2013, 17(2): 307-326.
34 Serenko A, Bontis N. Global ranking of knowledge management and intellectual capital academic journals: 2017 update[J]. Journal of Knowledge Management, 2017, 21(3): 675-692.
35 Chen Y L, Chen X H. An evolutionary PageRank approach for journal ranking with expert judgements[J]. Journal of Information Science, 2011, 37(3): 254-272.
36 宋晓晨, 李梦豪, 周良. 一种新型期刊评价方法——基于论文作者简介的分析[J]. 情报学报, 2018, 37(9): 874-881.
37 Opatrny T. Playing the system to give low-impact journal more clout[J]. Nature, 2008, 455(7210): 167.
38 Franck G. Scientific communication—A vanity fair?[J]. Science, 1999, 286(5437): 53-55.
39 van Noorden R. Brazilian citation scheme outed[J]. Nature, 2013, 500(7464): 510-511.
40 Cronin B. The citation process: the role and significance of citation in scientific communication[M]. London: Taylor Graham, 1984.
41 Harter S P. Psychological relevance and information science[J]. Journal of the American Society for Information Science, 1992, 43(9): 602-615.
42 谢娟, 成颖, 孙建军, 等. 基于信息使用环境理论的引用行为研究: 参考文献分析的视角[J]. 中国图书馆学报, 2018, 44(5): 59-75.
43 González-Pereira B, Guerrero-Bote V P, Moya-Anegón F. A new approach to the metric of journals’ scientific prestige: the SJR indicator[J]. Journal of Informetrics, 2010, 4(3): 379-391.
44 Franceschet M. The difference between popularity and prestige in the sciences and in the social sciences: a bibliometric analysis[J]. Journal of Informetrics, 2010, 4(1): 55-63.
45 Gyongyi Z, Garcia-Molina H. Seed selection in TrustRank[R]. Stanford University, 2004.
46 Sombatsompop N, Kositchaiyong A, Markpin T, et al. Scientific evaluations of citation quality of international research articles in the SCI database: Thailand case study[J]. Scientometrics, 2006, 66(3): 521-535.
47 李超. “HIF指数”评价科技期刊学术影响的机理与实践[J]. 情报理论与实践, 2011, 34(7): 44-48.
48 王建军, 杨德礼. 管理学CSSCI检索期刊学术影响力分析[J]. 情报学报, 2010, 29(1): 142-150.
49 王映. 加权TOPSIS与RSR法在学术期刊影响力综合评价中的应用研究[J]. 图书情报工作, 2013, 57(2): 92-96.
50 FMS管理科学高质量期刊(中文期刊)[EB/OL]. [2021-03-15]. http://www.fms-journal.net/journals_cn.
51 南京大学. 文科一流期刊目录(2017年修订)[EB/OL]. [2020-12-10]. https://skch.nju.edu.cn//glgz/cgxg/cgrd/20160719/i43082.html.
52 张琳, SivertsenGunnar. 科学计量与同行评议相结合的科研评价——国际经验与启示[J]. 情报学报, 2020, 39(8): 806-816.
53 Association of Universities in the Netherlands. Standard evaluation protocol 2015-2021[R/OL]. (2014-03-21). https://www.rug.nl/about-ug/organization/quality-assurance/research/sep2015-2021. pdf.
54 Ancaiani A, Anfossi A F, Barbara A, et al. Evaluating scientific research in Italy: the 2004-10 research evaluation exercise[J]. Research Evaluation, 2015, 24(3): 242-255.
55 李江, 姜明利, 李玥婷. 引文曲线的分析框架研究——以诺贝尔奖得主的引文曲线为例[J]. 中国图书馆学报, 2014, 40(2): 41-49.
56 Jacsó P. Five-year impact factor data in the Journal Citation Reports[J]. Online Information Review, 2009, 33(3): 603-614.
57 俞立平, 王作功, 孙建红. 时间窗口对学术期刊评价的影响研究[J]. 情报杂志, 2017, 36(10): 137-140.
58 刘海燕, 李琳, 李雪蓉. 《系统工程理论与实践》1981—2015年文献计量分析[J]. 系统工程理论与实践, 2017, 37(3): 805-816.
59 Ayres I, Vars F E. Determinants of citations to articles in elite law reviews[J]. The Journal of Legal Studies, 2000, 29(S1): 427-450.
60 Stremersch S, Verniers I, Verhoef P C. The quest for citations: drivers of article impact[J]. Journal of Marketing, 2007, 71(3): 171-193.
61 李明, 柯青, 石进, 等. 学术期刊作者机构指数AAI排名实证探索——以中文高等教育学期刊为例[J]. 西南民族大学学报(人文社科版), 2019, 40(9): 236-240.