|
|
|
| Evaluating the Scholarly Impact of Institutes Based on the Journal Editorship Index: Empirical Analysis in Geology |
| Lu Xiaoli1,2,3, Li Jing1,4, Wu Dengsheng1 |
1. Institutes of Science and Development, Chinese Academy of Sciences, Beijing 100190;
2. School of Public Policy and Management, University of Chinese Academy of Science, Beijing 100049;
3. National Geological Library of China, Beijing 100083;
4. School of Management, Anhui University, Hefei 230039 |
|
|
|
|
Abstract To solve the difficulty in obtaining multidimensional data and the insufficiency of the quantitative and qualitative combination, this paper proposes a Journal Editorship Index (JEI) to assess the scholarly impact of academic institutes based on the journal editorial board data. The editorial board members are responsible in combining an assessment of both quantitative approaches and peer review. The JEI takes into account the influence of the journal rating, editor title, and board size. Experiments are conducted to assess the scholarly impact of geology institutions based on the editorial boards of 111 geology journals (which is comprised of 3811 members) in the Excellence in Research for Australia journal list. The results show that the JEI can well characterize the academic influence of the scientific research institutions. Moreover, the JEI score and editorial board member of the academic institutions in the field of geology show an unequal distribution. The JEI scores have a significant correlation with the number of journal papers, including the total cited times and H indices. The robustness analysis of the evaluation results shows that considering the influence of the factors, such as editorial level, journal grade, and editorial scale, will change the evaluation results. The results also indicate that the different weights of the influence of the factors will generate the same evaluation results. By calculating the correlation between the JEI index and the derived indices, such as JEI/JT, JEI/JR, and JEI/JE, the JEI score of the academic institutes has the greatest impact when the journal rating is considered.
|
|
Received: 14 June 2017
|
|
|
|
[1] Meho L I, Sugimoto C R.Assessing the scholarly impact of information studies: A tale of two citation databases—Scopus and Web of Science[J]. Journal of the American Society for Information Science and Technology, 2009, 60(12): 2499-2508.
[2] Kaur J, Radicchi F, Menczer F.Universality of scholarly impact metrics[J]. Journal of Informetrics, 2013, 7(4): 924-932.
[3] 王菲菲, 杨辰毓妍. 多维信息计量学视角下的学术影响力综合评价研究现状及展望[J]. 情报资料工作, 2016(2): 23-28.
[4] 周建中, 徐芳. 国立科研机构同行评议方法的模式比较研究[J]. 科学学研究, 2013, 31(11): 1642-1648.
[5] 俞立平, 潘云涛, 武夷山. 科技评价中同行评议与指标体系关系的研究——以《泰晤士报》世界大学排名为例[J]. 科学学研究, 2008, 26(5): 927-931, 912.
[6] 龚旭. 同行评议公正性的影响因素分析[J]. 科学学研究, 2004, 22(6): 613-618.
[7] Torres-Salinas D, Moreno-Torres J G, Delgado-López-Cózar E, et al. A methodology for Institution-Field ranking based on a bidimensional analysis: the IFQ2 A index[J]. Scientometrics, 2011, 88(3): 771-786.
[8] Beaulier S, Elder R, Han C, et al.An ordinal ranking of economic institutions[J]. Applied Economics, 2016, 48(26): 2482-2490.
[9] Wu D S, Li M L, Zhu X Q, et al.Ranking the research productivity of business and management institutions in Asia-Pacific region: empirical research in leading ABS journals[J]. Scientometrics, 2015, 105(2): 1253-1272.
[10] Wu D S, Xie Y J, Dai Q Z, et al.A systematic overview of operations research/management science research in Mainland China: Bibliometric analysis of the period 2001-2013[J]. Asia-Pacific Journal of Operational Research, 2016, 33(6): 1650044.
[11] Kuan C H, Huang M H, Chen D Z.Cross-field evaluation of publications of research institutes using their contributions to the fields’ MVPs determined by h-index[J]. Journal of Informetrics, 2013, 7(2): 455-468.
[12] Moed H F, Halevi G.Multidimensional assessment of scholarly research impact[J]. Journal of the Association for Information Science and Technology, 2015, 66(10): 1988-2002.
[13] 蒋世银, 李春旺. 基于关联数据科研机构综合评价模型研究[J]. 情报理论与实践, 2015, 38(11): 71-75, 114.
[14] Bornmann L.Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics[J]. Journal of Informetrics, 2014, 8(4): 895-903.
[15] 朱卫东, 刘芳, 王东鹏, 等. 科学基金项目立项评估:综合评价信息可靠性的多指标证据推理规则研究[J]. 中国管理科学, 2016, 24(10): 141-148.
[16] Brinn T, Jones M J.The composition of editorial boards in accounting: A UK perspective[J]. Accounting, Auditing & Accountability Journal, 2008, 21(1): 5-35.
[17] Kaufman G G.Rankings of finance departments by faculty representation on editorial boards of professional journals: A note[J]. The Journal of Finance, 1984, 39(4): 1189-1197.
[18] Kurtz D L, Boone L E.Rating marketing faculties on the basis of editorial review board memberships[J]. Journal of Marketing Education, 1988, 10(1): 64-68.
[19] Gibbons J D, Fish M.Rankings of economics faculties and representation on editorial boards of top journals[J]. The Journal of Economic Education, 1991, 22(4): 361-372.
[20] Chan K C, Fung H G, Lai P.Membership of editorial boards and rankings of schools with international business orientation[J]. Journal of International Business Studies, 2005, 36(4): 452-469.
[21] Braun T, Dióspatonyi I, Zsindely S, et al.Gatekeeper index versus impact factor of science journals[J]. Scientometrics, 2007, 71(3): 541-543.
[22] Lahiri S, Kumar V.Ranking international business institutions and faculty members using research publication as the measure[J]. Management International Review, 2012, 52(3): 317-340.
[23] Frey B S, Rost K.Do rankings reflect research quality?[J]. Journal of Applied Economics, 2010, 13(1): 1-38.
[24] Haslam N, Koval P.Possible research area bias in the Excellence in Research for Australia (ERA) draft journal rankings[J]. Australian Journal of Psychology, 2010, 62(2): 112-114.
[25] Franceschini F, Maisano D, Mastrogiacomo L, et al.Empirical analysis and classification of database errors in Scopus and Web of Science[J]. Journal of Informetrics, 2016, 10(4): 933-953.
[26] Dorta-González P, Dorta-González M I. Comparing journals from different fields of science and social science through a JCR subject categories normalized impact factor[J]. Scientometrics, 2013, 95(2): 645-672.
[27] Moed H F.Comprehensive indicator comparisons intelligible to non-experts: the case of two SNIP versions[J]. Scientometrics, 2015, 106(1): 51-65.
[28] 吴登生, 李若筠. 中国管理科学领域机构合作的网络结构与演化规律研究[J]. 中国管理科学, 2017, 25(9): 168-177.
[29] Chan K C, Fok R C W. Membership on editorial boards and finance department rankings[J]. The Journal of Financial Research, 2003, 26(3): 405-420.
[30] Vanclay J K.An evaluation of the Australian Research Council’s journal ranking[J]. Journal of Informetrics, 2011, 5(2): 265-274.
[31] Morris H, Harvey C, Kelly A.Journal rankings and the ABS journal quality guide[J]. Management Decision, 2009, 47(9): 1441-1451.
[32] Bedeian A G, Van Fleet D D, Hyman III H H. Scientific achievement and editorial board membership[J]. Organizational Research Methods, 2008, 12(2): 211-238.
[33] 李万伦, 段怡春, 王海华, 等. 从文献计量看中国科学院、高等院校和中国地质调查局三大系统地质学科的分布现状[J]. 地质通报, 2009, 28(2-3): 397-402.
[34] 史静, 肖仙桃, 王鑫, 等. 2000~2014年国外部分地质调查机构科研态势分析——基于文献计量学方法研究[J]. 地质学报, 2015, 89(12): 2433-2442.
[35] Hardin III W G, Liano K, Chan K C, et al. Finance editorial board membership and research productivity[J]. Review of Quantitative Finance and Accounting, 2008, 31(3): 225-240. |
|
|
|
