|
|
|
| A “Multiple Indicator” View of Preprint Impact Evaluation in Information and Library Science |
| Chen Yue, Wang Zhiqi, Liu Zeyuan, Song Cha |
| Institution of Science of Science and S&T Management & WISE Lab, Dalian University of Technology, Dalian 116023 |
|
|
|
|
Abstract This paper focuses on those preprints in arXiv that are also published in three major journals in Library and Information Science (LIS) and further explores the border impact of the preprints from several perspectives. In particular, the following four indicators are used to examine the 550 arXiv and 5782 non-arXiv papers: citations from the Web of Science Core Collection (CF-WoS), Scopus, and Google Scholar; usage counts in WoS (UC); Mendeley readers (MR); and Tweets (TM), which are considered proxies for social attention. The results show different citation trends for the two sets of papers, wherein preprints have an obvious citation advantage compared with the other documents. The development of altmetrics in research evaluation promotes the open access process of scientific resources. The impact advantage of arXiv papers can also be observed in MR but is hardly reflected by UC or TM. A linear regression analysis substantiates that MR and UC do strongly correlate with CF-WoS, which also holds for the relationship with Scopus and Google Scholar citations, but MR is more suitable to assist in the evaluation of the impact of preprints. The strong correlation between readers/usage and citations may be interpreted in the sense that arXiv papers gain broader attention than non-arXiv papers, not only from subscribers of the WoS. This study helps to reveal the role of preprints in LIS and provides inspiration to build a more complete evaluation index system currently suitable for different methods of scientific communication from the “multiple indicator” view.
|
|
Received: 08 October 2018
|
|
|
|
1 英)J.D.贝尔纳. 科学的社会功能[M]. 陈体芳, 译. 北京: 商务印书馆, 1982.
2 PiwowarH, PriemJ, LarivièreV, et al. The state of OA: A large-scale analysis of the prevalence and impact of Open Access articles[J]. PeerJ, 2018, 6: e4375.
3 Gl?nzelW, SchoepflinU. A bibliometric study of reference literature in the sciences and social sciences[J]. Information Processing & Management, 1999, 35(1): 31-44.
4 LawrenceS. Online or invisible[J]. Nature, 2001, 411: 521.
5 CraigI D, PlumeA M, McVeighM E, et al. Do open access articles have greater citation impact?: A critical review of the literature[J]. Journal of Informetrics, 2007, 1(3): 239-248.
6 DavisP M. Open access, readership, citations: a randomized controlled trial of scientific journal publishing[J]. The FASEB Journal, 2011, 25(7): 2129-2134.
7 GargouriY, HajjemC, LarivièreV, et al. Self-selected or mandated, open access increases citation impact for higher quality research[J]. PLoS ONE, 2010, 5(10): e13636.
8 MoedH F. The effect of “open access” on citation impact: An analysis of ArXiv’s condensed matter section[J]. Journal of the American Society for Information Science and Technology, 2014, 58(13): 2047-2054.
9 LarivièreV, SugimotoC R, MacalusoB, et al. arXiv e-prints and the journal of record: An analysis of roles and relationships[J]. Journal of the American Society for Information Science and Technology, 2014, 65(6): 1157-1169.
10 MetcalfeT S. The citation impact of digital preprint archives for solar physics papers[J]. Solar Physics, 2006, 239(1-2): 549-553.
11 SchwarzG J, R C JrKennicutt. Demographic and citation trends in astrophysical journal papers and preprints[OL]. https://arxiv.org/abs/astro-ph/0411275.
12 DavisP M, FromerthM J. Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles?[J]. Scientometrics, 2006, 71(2): 203-215.
13 ChenY, WangZ, TanJ, et al. The position of preprint in scholarly communication: A biliometric and empirical study of arXiv[C]// Proceedings of the 16th Conference on International Society of Scientometrics and Informetrics, 2017: 799-809.
14 ThelwallM, KoushaK. ResearchGate versus Google Scholar: Which finds more early citations?[J]. Scientometrics, 2017, 112(1): 1-7.
15 ThelwallM. Dimensions: A competitor to Scopus and the Web of Science?[J]. Journal of Informetrics, 2018, 12(2): 430-435.
16 HarzingA W, AlakangasS. Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison[J]. Scientometrics, 2016, 106(2): 787-804.
17 Martín-MartínA, Orduna-MaleaE, López-CózarE D. Coverage of highly-cited documents in Google Scholar, Web of Science, and Scopus: A multidisciplinary comparison[J]. Scientometrics, 2018, 116(3): 2175-2188.
18 Gl?nzelW, ThijsB, DebackereK. Productivity, performance, efficiency, impact—What do we measure anyway?[J]. Journal of Informetrics, 2016, 10(2): 658-660.
19 Gl?nzelW, GorraizJ. Usage metrics versus altmetrics: Confusing terminology?[J]. Scientometrics, 2015, 102(3): 2161-2164.
20 王贤文, 方志超, 胡志刚. 科学论文的科学计量分析: 数据、方法与用途的整合框架[J]. 图书情报工作, 2015, 59(16): 74-82.
21 HausteinS, LarivièreV, ThelwallM, et al. Tweets vs. Mendeley readers: How do these two social media metrics differ?[J]. Information Technology, 2014, 56(5): 207-215.
22 CostasR, ZahediZ, WoutersP. Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective[J]. Journal of the Association for Information Science and Technology, 2015, 66(10): 2003-2019.
23 LiX, ThelwallM, GiustiniD. Validating online reference managers for scholarly impact measurement[J]. Scientometrics, 2012, 91(2): 461-471.
24 ThelwallM, WilsonP. Mendeley readership altmetrics for medical articles: An analysis of 45 fields[J]. Journal of the Association for Information Science and Technology, 2016, 67(8): 1962-1972..
25 MohammadiE, ThelwallM. Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows[J]. Journal of the Association for Information Science and Technology, 2014, 65(8): 1627-1638.
26 ThelwallM, KoushaK. ResearchGate articles: Age, discipline, audience size, and impact[J]. Journal of the Association for Information Science and Technology, 2017, 68(2): 468-479.
27 ThelwallM. Are Mendeley reader counts useful impact indicators in all fields?[J]. Scientometrics, 2017, 113(3): 1721-1731.
28 MaflahiN, ThelwallM. When are readership counts as useful as citation counts? Scopus versus Mendeley for LIS journals[J]. Journal of the Association for Information Science and Technology, 2016, 67(1): 191-199.
29 ThelwallM. Can Microsoft Academic be used for citation analysis of preprint archives? The case of the Social Science Research Network[J]. Scientometrics, 2018, 115(2): 913-928.
30 WangX W, LiuC, MaoW L, et al. The open access advantage considering citation, article usage and social media attention[J]. Scientometrics, 2015, 103(2): 555-564.
31 ShuaiX, PepeA, BollenJ. How the scientific community reacts to newly submitted preprints: Article downloads, Twitter mentions, and citations[J]. PLoS ONE, 2012, 7(11): e47523.
32 BrodyT, HarnadS, CarrL. Earlier web usage statistics as predictors of later citation impact[J]. Journal of the Association for Information Science and Technology, 2010, 57(8): 1060-1072.
33 ChiP S, Gl?nzelW. An empirical investigation of the associations among usage, scientific collaboration and citation impact[J]. Scientometrics, 2017, 112(1): 403-412.
34 WangX, MaoW, XuS, et al. Usage history of scientific literature: Nature metrics, and metrics of Nature, publications[J]. Scientometrics, 2014, 98(3): 1923-1933.
35 WangX, FangZ, SunX. Usage patterns of scholarly articles on Web of Science: A study on Web of Science usage count[J]. Scientometrics, 2016, 109(2): 917-926.
36 Gl?nzelW, MoedH F. Opinion paper: Thoughts and facts on bibliometric indicators[J]. Scientometrics, 2013, 96: 381-394.
37 Gl?nzelW, SchoepflinU. A bibliometric study on ageing and reception process of scientific literature[J]. Journal of Information Science, 1995, 21(1): 37-53.
38 Gl?nzelW, ThijsB, ChiP S. The challenges to expand bibliometric studies from periodical literature to monographic literature with a new data source: The book citation index[J]. Scientometrics, 2016, 109(3): 2165-2179.
39 SudP, ThelwallM. Evaluating altmetrics[J]. Scientometrics, 2014, 98(2): 1131-1143.
40 ThelwallM, FaircloughR. The influence of time and discipline on the magnitude of correlations between citation counts and quality scores[J]. Journal of Informetrics, 2015, 9(3): 529-541. |
|
|
|
