Scientometrics for research assessment

Presentation at the International Conference on Scientometrics and Bibliometrics. Kiev, Ukraine, June 4, 2019.

1. Scientometrics for research assessment Ludo Waltman Centre for Science and Technology Studies (CWTS), Leiden University International Conference on Scientometrics and Bibliometrics Kiev, Ukraine June 4, 2019

2. Centre for Science and Technology Studies (CWTS) Research center at Leiden University in the Netherlands Science and technology studies, with a strong emphasis on scientometrics About 50 staff members Our mission: Making the science system better! 1

3. Outline Research assessment and scientometrics Scientometric data sources Scientometric indicators and tools Scientometric analysis of Ukrainian science Recent developments 2

4. Research assessment and scientometrics 3

5. Research evaluation: Why, what, and how? Why evaluate? Advocacy Allocation Analysis Accountability What to evaluate? Countries Research institutions Research groups Individual researchers How to evaluate? Expert judgment Scientometrics 4 Adam et al. (2018)

6. Leiden Manifesto Leiden Manifesto provides 10 principles for responsible use of scientometrics in research assessments Used by research institutions to develop internal policies for use of scientometrics 5

7. Scientometric data sources 6

8. Scientometric data sources Web of Science (Clarivate Analytics) “ With the Web of Science platform, you can access an unrivalled breadth of world-class research literature linked to a rigorously selected core of journals ” Consists of multiple citation indices (SCIE, SSCI, AHCI, ESCI, CPCI, BCI, …) Content selection by Editorial Development team Scopus (Elsevier) “ Scopus is the largest abstract and citation database of peer-reviewed literature ” Content selection by Content Selection and Advisory Board Dimensions (Digital Science) “ Re-imagining discovery and access to research: grants, publications, citations, clinical trials, patents and policy documents in one place ” Based on Crossref and PubMed data, enriched with data from publishers Content selection by end user Basic web interface is freely accessible; data is freely available for scientometric research 7

9. Scientometric data sources Google Scholar Most comprehensive scientometric data source Low data quality and easy to manipulate No possibilities for large-scale data access Microsoft Academic Graph Data can be accessed through API and Microsoft Azure ODC-BY license (free to share, create, and adapt as long as you attribute) 8

10. Comparison of data sources 9 Overlap of publications in different data sources (1996–2017); WoS includes only SCIE, SSCI, AHCI, and CPCI

11. Scientometric indicators and tools 10

12. Journal impact factor 11

13. h-index 12 h-index is arbitrary and inconsistent

14. Field-normalized citation impact indicators 13

15. Beyond citation impact Scientific collaboration Collaboration with industry Open access publishing Gender balance Mobility 14

16. University rankings 15 www.researchresearch.com/news/article/?articleId=1368350

17. VOSviewer 16

18. Leiden algorithm 17

19. Micro fields of science 18 Social sciences Biomed. & health sciences Math. & comp. science Physical sciences & eng. Life & earth sciences

20. Scientometric analysis of Ukrainian science 19

21. Scientometric analysis of Ukrainian science Based on Web of Science (SCIE, SSCI, AHCI) data Only international literature; no local literature Time period 2014–2017 20

22. Publication output and citation impact 21

23. Micro fields of science 22 Social sciences Biomed. & health sciences Math. & comp. science Physical sciences & eng. Life & earth sciences

24. Micro fields of science: Contribution from Ukraine 23 Social sciences Biomed. & health sciences Math. & comp. science Physical sciences & eng. Life & earth sciences Share of publications from Ukraine Nuclear accidents

25. Micro fields of science: International comparison 24 Ukraine Poland Germany Russian

26. Nuclear accidents micro field 25

27. Nuclear accidents micro field: Contribution from Ukraine 26 Share of publications from Ukraine

28. Nuclear accidents micro field: Contribution from Ukraine 27 Share of publications from Ukraine

29. Recent developments 28

30. Open citations Initiative for Open Citations (I4OC) advocates open availability of citation data More than half of all citations in Crossref have been opened 29

31. New journal: Quantitative Science Studies New high-quality journal established in January 2019 as a result of Journal of Informetrics flip Owned by the International Society for Scientometrics and Informetrics and published by MIT Press 30

32. Full-text data Increasing availability of full text of scientific publications for scientometric research 31

33. More diverse journal indicators Call for more diverse journal indicators (not just citation impact indicators) covering the different functions of journals Coordination by a governing body that brings together different stakeholders 32

34. CWTS Scientometrics Summer School Summer school for PhD students and researchers in scientometrics Previous editions in Leiden (Summer 2018) and Haining (Spring 2019) Next edition in Leiden (Summer 2020) 33

35. Thank you for your attention! 34