|
|
|
| The Challenges of Webometrics and Altmetrics and the Evaluation of Robustmetric and Non-robustmetric in Societal Impact |
| Liu Tingyuan, Liu Shuman |
| Library of Southwest Petroleum University, Chengdu 610500 |
|
|
|
|
Abstract In the wake of increasingly extensive evidence of the impact of scientific achievements, the challenges of webometrics and altmetrics and their societal impact evaluation are increasing. Due to the widespread prevalence of high zero values (left), multiple outliers (right), and extremely right-skewed distribution as web-altmetric data with societal impacts, the authenticity and rationality of the data set and the resistance of bias-error, reliability, and stability of their informetric methods and results are facing many unique challenges. In this study, in the face of high zero value, the quartile zero value scaling-down method is used for verification, and the proposed accurate calculation formula has good consistency and resistance of bias-error, which is an important basis for the reasonable correction of outliers and their robustmetric. The quartile zero-value rate is defined and derived based on the inter-quartile range，and the actual risk rate of its maximum scaling-down is low, which belongs to the ideal position parameter estimation point. For multiple outliers, the robustify winsorizing method is used for modification, and compared with the non-robust method, the corrected data set has more tolerance and reliability. For extremely right-skewed distribution, the linear proportional method based on the tailed mean is adopted for dimensionless, so that the results of mapping and transformation are more stable and consistent compared with the linear proportional method based on the mean. The solution of weight coefficient is based on the organic integration of subjective weight into objective weight method, and the weight set of G1 method (subjective), objective G1 method, and semi-objective G1 method is regarded as a triangular fuzzy number for defuzzification so that the weight values have a subjective-objective dual realization mechanism, thereby improving the reliability and stability of the comprehensive evaluation results. Compared with methods of non-robustmetric evaluation, the stability, reliability, and resistance of bias-error of robustmetric evaluation is greatly improved, which is conducive to promoting the development of informetrics and evaluation science towards complexity precision science.
|
|
Received: 18 September 2021
|
|
|
|
1 Priem J, Costello K L. How and why scholars cite on Twitter[J]. Proceedings of the American Society for Information Science and Technology, 2010, 47(1): 1-4.
2 Priem J, Groth P, Taraborelli D. The altmetrics collection[J]. PLoS One, 2012, 7(11): e48753.
3 Priem J. Altmetrics[M]// Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact. Cambridge: The MIT Press, 2014: 263-287.
4 Penfield T, Baker M J, Scoble R, et al. Assessment, evaluations, and definitions of research impact: a review[J]. Research Evaluation, 2014, 23(1): 21-32.
5 邱均平, 余厚强. 论推动替代计量学发展的若干基本问题[J]. 中国图书馆学报, 2015, 41(1): 4-15.
6 Melero R. Altmetrics - a complement to conventional metrics[J]. Biochemia Medica, 2015, 25(2): 152-160.
7 Erfanmanesh M, Noorhidawati A, Abrizah A. What can Bookmetrix tell us about the impact of Springer Nature’s books[J]. Scientometrics, 2019, 121(1): 521-536.
8 谢娟, 龚凯乐, 成颖, 等. 使用数据与引用数据间的补充或替代关系探讨[J]. 情报学报, 2018, 37(5): 486-494.
9 赵蓉英, 王旭. 引入Altmetrics指标的学术期刊影响力评价研究——以国际图书情报学期刊为例[J]. 图书与情报, 2018(5): 1-10.
10 Bornmann L, Haunschild R. Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact?[J]. Scientometrics, 2017, 110(2): 937-943.
11 Adie E, Roe W. Altmetric: enriching scholarly content with article-level discussion and metrics[J]. Learned Publishing, 2013, 26(1): 11-17.
12 Ingwersen P. The calculation of web impact factors[J]. Journal of Documentation, 1998, 54(2): 236-243.
13 Bj?rneborn L, Ingwersen P. Toward a basic framework for webometrics[J]. Journal of the American Society for Information Science and Technology, 2004, 55(14): 1216-1227.
14 Thelwall M. The pros and cons of the use of altmetrics in research assessment[J]. Scholarly Assessment Reports, 2020, 2(1): Article 2.
15 Thoma B, Sanders J L, Lin M, et al. The social media index: measuring the impact of emergency medicine and critical care websites[J]. Western Journal of Emergency Medicine, 2015, 16(2): 242-249.
16 Zarei A, Khodadadi Z, Maleki M, et al. Robust mixture regression modeling based on two-piece scale mixtures of normal distributions[J/OL]. Advances in Data Analysis and Classification, (2022-03-23). https://doi.org/10.1007/s11634-022-00495-6.
17 Bookstein A. Informetric distributions. III. Ambiguity and randomness[J]. Journal of the American Society for Information Science, 1997, 48(1): 2-10.
18 Nane T, Kooijman K. A bootstrap analysis for finite populations[OL]. (2018-04-14). https://arxiv.org/ftp/arxiv/papers/1804/1804.05274.pdf.
19 Haustein S, Costas R, Larivière V. Characterizing social media metrics of scholarly papers: the effect of document properties and collaboration patterns[J]. PLoS One, 2015, 10(3): e0120495.
20 Thelwall M. Using altmetrics to support research evaluation[C]// Proceedings of the International Workshop on Altmetrics for Research Outputs Measurements and Scholarly Information Management. Singapore: Springer, 2018: 11-28.
21 Mas-Bleda A, Thelwall M. Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research[J]. Scientometrics, 2016, 109(3): 2007-2030.
22 Chi P S, Gorraiz J, Gl?nzel W. Comparing capture, usage and citation indicators: an altmetric analysis of journal papers in chemistry disciplines[J]. Scientometrics, 2019, 120(3): 1461-1473.
23 Sugimoto C R, Work S, Larivière V, et al. Scholarly use of social media and altmetrics: a review of the literature[J]. Journal of the Association for Information Science and Technology, 2017, 68(9): 2037-2062.
24 Gl?nzel W, Chi P S. The big challenge of Scientometrics 2.0: exploring the broader impact of scientific research in public health[J]. Scientometrics, 2020, 125(2): 1011-1031.
25 Farooqui T, Mustafa I, Christie T. Outliers in educational achievement data: their potential for the improvement of performance[J]. Pakistan Journal of Statistics, 2014, 30(1): 71-82.
26 Templ M, Gussenbauer J, Filzmoser P. Evaluation of robust outlier detection methods for zero-inflated complex data[J]. Journal of Applied Statistics, 2020, 47(7): 1144-1167.
27 Tague-Sutcliffe J. In memoriam: B.C. (Bertie) Brookes, 1910-1991[J]. Journal of the American Society for Information Science, 1992, 43(4): 273-275.
28 刘春丽. 基于PLOS API的论文影响力选择性计量指标研究[J]. 图书情报工作, 2013, 57(7): 89-95.
29 Costas R, Zahedi Z, Wouters P. Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective[J]. Journal of the Association for Information Science and Technology, 2015, 66(10): 2003-2019.
30 罗付岩, 赵佳星. 基于零膨胀模型的我国银企关系规模影响因素研究[J]. 数理统计与管理, 2017, 36(2): 351-360.
31 Thelwall M, Maflahi N. Guideline references and academic citations as evidence of the clinical value of health research[J]. Journal of the Association for Information Science and Technology, 2016, 67(4): 960-966.
32 Bornmann L. Scientific revolution in scientometrics: the broadening of impact from citation to societal[M]// Theories of Informetrics and Scholarly Communication. Boston: De Gruyter Saur, 2016: 347-359.
33 许勤, 魏嶷. 稳健高效的切尾均值乘数估值技术[J]. 同济大学学报(自然科学版), 2007, 35(2): 271-275.
34 Huber P J. Robust statistics[M]// International Encyclopedia of Statistical Science. Heidelberg: Springer, 2011: 1248-1251.
35 Piwowar H. Introduction altmetrics: what, why and where?[J]. Bulletin of the American Society for Information Science and Technology, 2013, 39(4): 8-9.
36 肖其师, 吴春莹, 孙志庆. 基于Altmetrics的科技项目立项社会影响力评价框架研究及实证分析[J]. 图书馆杂志, 2019, 38(5): 79-87.
37 李伟伟, 易平涛, 李玲玉. 综合评价中异常值的识别及无量纲化处理方法[J]. 运筹与管理, 2018, 27(4): 173-178.
38 张发明, 刘志平. 组合评价方法研究综述[J]. 系统工程学报, 2017, 32(4): 557-569.
39 郭亚军. 综合评价理论、方法及应用[M]. 北京: 科学出版社, 2007.
40 李刚, 王斌, 周立斌, 等. 基于标准差修正G1组合赋权的人的全面发展评价模型及实证[J]. 系统工程理论与实践, 2012, 32(11): 2473-2485.
41 李刚, 李建平, 孙晓蕾, 等. 主客观权重的组合方式及其合理性研究[J]. 管理评论, 2017, 29(12): 17-26, 61.
42 易平涛, 李伟伟, 郭亚军. 基于指标特征分析的区域创新能力评价及实证[J]. 科研管理, 2016, 37(S1): 371-378.
43 Cronin B. Metrics à la mode[J]. Journal of the American Society for Information Science and Technology, 2013, 64(6): 1091. |
|
|
|
