|
|
|
| Measure of Disruptive Innovation in Science: Relative Disruptive Index (RDI) |
| Yang Alex J.1,2, Deng Sanhong1,2, Wang Hao1,2 |
1.School of Information Management, Nanjing University, Nanjing 210023
2.Jiangsu Key Laboratory of Data Engineering and Knowledge Service, Nanjing 210023 |
|
|
|
|
Abstract Innovation serves as a propulsive impetus behind scientific advancement. Assessing the innovative merit of scientific research holds particular significance. Papers endowed with elevated levels of innovation often challenge and upend existing disciplinary paradigms within research fields. A newly introduced metric called disruptive index has emerged as a means of directly appraising the innovation encapsulated within papers. By leveraging the profound interplay of paper citations and the network of cited relationships, this index effectively surmounts the limitations associated with unidimensional evaluations. As a result, it has garnered considerable attention within the scientific metrology community. This study employed a comprehensive synthesis and expansion of the latest research pertaining to the disruptive index. It provides a profound analysis of the limitations and influencing factors associated with this index. Additionally, it proposes a novel metric called relative disruptive index (RDI), which builds upon the foundation laid by the disruptive index. By encompassing the deep quotation proportions derived from the citation network, the RDI strives to mitigate the issues stemming from the non-uniform distribution and inconsistent evaluation encountered within the realm of the disruptive index. Consequently, it engenders a more precise and objective means of gauging the innovation inherent within scientific research endeavors. Empirical results clearly demonstrate the inconsistency in parameter magnitudes across the disruptive index. Conversely, the relative disruptive index emerges as a more efficacious gauge of research innovation compared to the original disruptive index and DI5. It is confirmed that the RDI exhibits superior consistency in evaluation, yielding a lower relative subversion index for papers that build upon previous work and higher relative subversion index for papers that embody groundbreaking innovation.
|
|
Received: 16 September 2022
|
|
|
|
1 Acemoglu D, Akcigit U, Kerr W R. Innovation network[J]. Proceedings of the National Academy of Sciences of the United States of America, 2016, 113(41): 11483-11488.
2 Park M, Leahey E, Funk R J. Papers and patents are becoming less disruptive over time[J]. Nature, 2023, 613(7942): 138-144.
3 Hicks D, Wouters P, Waltman L, et al. The Leiden Manifesto for research metrics[J]. Nature, 2015, 520(7548): 429-431.
4 Cagan R. The San Francisco declaration on research assessment[J]. Disease Models & Mechanisms, 2013, 6(4): 869-870.
5 教育部, 科技部. 教育部 科技部印发《关于规范高等学校SCI论文相关指标使用 树立正确评价导向的若干意见》的通知[J]. 中华人民共和国教育部公报, 2020(3): 24-26.
6 Fortunato S, Bergstrom C T, B?rner K, et al. Science of science[J]. Science, 2018, 359(6379): eaao0185.
7 Battiston F, Musciotto F, Wang D S, et al. Taking census of physics[J]. Nature Reviews Physics, 2019, 1(1): 89-97.
8 Min C, Bu Y, Sun J J. Predicting scientific breakthroughs based on knowledge structure variations[J]. Technological Forecasting and Social Change, 2021, 164: 120502.
9 Funk R J, Owen-Smith J. A dynamic network measure of technological change[J]. Management Science, 2017, 63(3): 791-817.
10 Wu L F, Wang D S, Evans J A. Large teams develop and small teams disrupt science and technology[J]. Nature, 2019, 566(7744): 378-382.
11 Lyu D Q, Gong K L, Ruan X M, et al. Does research collaboration influence the “disruption” of articles? Evidence from neurosciences[J]. Scientometrics, 2021, 126(1): 287-303.
12 Wang S Y, Ma Y X, Mao J, et al. Quantifying scientific breakthroughs by a novel disruption indicator based on knowledge entities[J]. Journal of the Association for Information Science and Technology, 2023, 74(2): 150-167.
13 Yang A J, Deng S H, Wang H, et al. Disruptive coefficient and 2-step disruptive coefficient: novel measures for identifying vital nodes in complex networks[J]. Journal of Informetrics, 2023, 17(3): 101411.
14 Ruan X M, Lyu D Q, Gong K L, et al. Rethinking the disruption index as a measure of scientific and technological advances[J]. Technological Forecasting and Social Change, 2021, 172: 121071.
15 Liang G Q, Lou Y, Hou H Y. Revisiting the disruptive index: evidence from the Nobel Prize-winning articles[J]. Scientometrics, 2022, 127(10): 5721-5730.
16 Leydesdorff L, Bornmann L. Disruption indices and their calculation using web-of-science data: indicators of historical developments or evolutionary dynamics?[J]. Journal of Informetrics, 2021, 15(4): 101219.
17 Rodríguez-Navarro A. Research assessment based on infrequent achievements: a comparison of the United States and Europe in terms of highly cited papers and Nobel Prizes[J]. Journal of the Association for Information Science and Technology, 2016, 67(3): 731-740.
18 Koestler A. The act of creation[M]. New York: The MacMillan Company, 1964.
19 Fleming L. Recombinant uncertainty in technological search[J]. Management Science, 2001, 47(1): 117-132.
20 Boudreau K J, Guinan E C, Lakhani K R, et al. Looking across and looking beyond the knowledge frontier: intellectual distance, novelty, and resource allocation in science[J]. Management Science, 2016, 62(10): 2765-2783.
21 Packalen M, Bhattacharya J. Neophilia ranking of scientific journals[J]. Scientometrics, 2017, 110(1): 43-64.
22 Uzzi B, Mukherjee S, Stringer M, et al. Atypical combinations and scientific impact[J]. Science, 2013, 342(6157): 468-472.
23 Boyack K W, Klavans R. Atypical combinations are confounded by disciplinary effects[C]// Proceedings of the 19th International Conference on Science and Technology Indicators, 2014.
24 Wang J, Veugelers R, Stephan P. Bias against novelty in science: a cautionary tale for users of bibliometric indicators[J]. Research Policy, 2017, 46(8): 1416-1436.
25 Uddin S, Khan A. The impact of author-selected keywords on citation counts[J]. Journal of Informetrics, 2016, 10(4): 1166-1177.
26 Fontana M, Iori M, Montobbio F, et al. New and atypical combinations: an assessment of novelty and interdisciplinarity[J]. Research Policy, 2020, 49(7): 104063.
27 Azoulay P, Zivin J S G, Manso G. Incentives and creativity: evidence from the academic life sciences[J]. The RAND Journal of Economics, 2011, 42(3): 527-554.
28 罗卓然, 陆伟, 蔡乐, 等. 学术文本词汇功能识别——在论文新颖性度量上的应用[J]. 情报学报, 2022, 41(7): 720-732.
29 Shi F, Evans J. Surprising combinations of research contents and contexts are related to impact and emerge with scientific outsiders from distant disciplines[J]. Nature Communications, 2023, 14: 1641.
30 Waltman L. A review of the literature on citation impact indicators[J]. Journal of Informetrics, 2016, 10(2): 365-391.
31 Wu L F, Kittur A, Youn H, et al. Metrics and mechanisms: measuring the unmeasurable in the science of science[J]. Journal of Informetrics, 2022, 16(2): 101290.
32 Min C, Bu Y, Wu D, et al. Identifying citation patterns of scientific breakthroughs: a perspective of dynamic citation process[J]. Information Processing & Management, 2021, 58(1): 102428.
33 Belter C W. A relevance ranking method for citation-based search results[J]. Scientometrics, 2017, 112(2): 731-746.
34 Popper K. The logic of scientific discovery[M]. New York: Routledge, 2002.
35 Kuhn T S. Historical structure of scientific discovery[J]. Science, 1962, 136(3518): 760-764.
36 Zeng A, Shen Z S, Zhou J L, et al. The science of science: from the perspective of complex systems[J]. Physics Reports, 2017, 714/715: 1-73.
37 Lin Y L, Evans J A, Wu L F. New directions in science emerge from disconnection and discord[J]. Journal of Informetrics, 2022, 16(1): 101234.
38 Azoulay P. Small research teams ‘disrupt’ science more radically than large ones[J]. Nature, 2019, 566(7744): 330-332.
39 Bornmann L, Tekles A. Convergent validity of several indicators measuring disruptiveness with milestone assignments to physics papers by experts[J]. Journal of Informetrics, 2021, 15(3): 101159.
40 Zhou Y B, Xu X L, Yang X H, et al. The influence of disruption on evaluating the scientific significance of papers[J]. Scientometrics, 2022, 127(10): 5931-5945.
41 Liu X W, Wang X Z, Lyu L C, et al. Identifying disruptive technologies by integrating multi-source data[J]. Scientometrics, 2022, 127(9): 5325-5351.
42 Davies M, Calma A. Getting cited: a reconsideration of purpose[J]. Australian Universities’ Review, 2017, 59(1): 69-75.
43 Bu Y, Waltman L, Huang Y. A multidimensional framework for characterizing the citation impact of scientific publications[J]. Quantitative Science Studies, 2021, 2(1): 155-183.
44 Bornmann L, Tekles A. Disruptive papers published in Scientometrics[J]. Scientometrics, 2019, 120(1): 331-336.
45 Bornmann L, Tekles A. Disruption index depends on length of citation window[J]. El Profesional de la Información, 2019, 28(2): e280207.
46 van Eck N J, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping[J]. Scientometrics, 2010, 84(2): 523-538.
47 Larsen P O, von Ins M. The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index[J]. Scientometrics, 2010, 84(3): 575-603.
48 Narin F, Stevens K, Whitlow E S. Scientific cooperation in Europe and the citation of multinationally authored papers[J]. Scientometrics, 1991, 21(3): 313-323.
49 Moed H F, de Bruin R E, van Leeuwen T N. New bibliometric tools for the assessment of national research performance: database description, overview of indicators and first applications[J]. Scientometrics, 1995, 33(3): 381-422.
50 Schubert A, Braun T. International collaboration in the sciences 1981-1985[J]. Scientometrics, 1990, 19(1): 3-10.
51 Weingart P. Impact of bibliometrics upon the science system: inadvertent consequences?[J]. Scientometrics, 2005, 62(1): 117-131.
52 Gl?nzel W, Leta J, Thijs B. Science in Brazil. Part 1: a macro-level comparative study[J]. Scientometrics, 2006, 67(1): 67-86.
53 Gl?nzel W, Schubert A. A new classification scheme of science fields and subfields designed for scientometric evaluation purposes[J]. Scientometrics, 2003, 56(3): 357-367.
54 van Raan A F J. Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises[J]. Scientometrics, 1996, 36(3): 397-420.
55 Katz J S, Hicks D. How much is a collaboration worth? A calibrated bibliometric model[J]. Scientometrics, 1997, 40(3): 541-554.
56 van Raan A F J. Fatal attraction: conceptual and methodological problems in the ranking of universities by bibliometric methods[J]. Scientometrics, 2005, 62(1): 133-143.
57 Egghe L. Theory and practise of the g-index[J]. Scientometrics, 2006, 69(1): 131-152.
58 van Raan A F J. Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups[J]. Scientometrics, 2006, 67(3): 491-502.
59 Batista P D, Campiteli M G, Kinouchi O. Is it possible to compare researchers with different scientific interests?[J]. Scientometrics, 2006, 68(1): 179-189.
60 Gl?nzel W. On the h-index - a mathematical approach to a new measure of publication activity and citation impact[J]. Scientometrics, 2006, 67(2): 315-321.
61 Egghe L, Rousseau R. An informetric model for the hirsch-index[J]. Scientometrics, 2006, 69(1): 121-129.
62 Iglesias J E, Pecharromán C. Scaling the h-index for different scientific ISI fields[J]. Scientometrics, 2007, 73(3): 303-320.
63 Braun T, Gl?nzel W, Schubert A. A Hirsch-type index for journals[J]. Scientometrics, 2006, 69(1): 169-173.
64 Sidiropoulos A, Katsaros D, Manolopoulos Y. Generalized Hirsch h-index for disclosing latent facts in citation networks[J]. Scientometrics, 2007, 72(2): 253-280.
65 Beaver D D. Reflections on scientific collaboration (and its study): past, present, and future[J]. Scientometrics, 2001, 52(3): 365-377.
66 Katz J S, Martin B R. What is research collaboration?[J]. Research Policy, 1997, 26(1): 1-18.
67 Meyer M, Persson O. Nanotechnology-interdisciplinarity, patterns of collaboration and differences in application[J]. Scientometrics, 1998, 42(2): 195-205.
68 Braun T, Schubert A, Zsindely S. Nanoscience and nanotecnology on the balance[J]. Scientometrics, 1997, 38(2): 321-325.
69 邱均平, 张裕晨, 周子番. 新时代我国科研评价体系重构中必须处理好八大关系[J]. 中国图书馆学报, 2021, 47(1): 47-60.
70 van Raan A F J. Sleeping beauties in science[J]. Scientometrics, 2004, 59(3): 467-472. |
|
|
|
