FSCI 2021

FSCI courses are now closed. Course attendees must go to the course OSF page do see course materials and Zoom links.

--------

In this course, participants will learn about recent developments in the world of research performance evaluation. Together with the instructors, attendees will also practice how to bridge the principles of open science with research intelligence methods and tools to provide actionable knowledge about open science performance in research institutions. This course will provide the means to let participants explore research intelligence, a growing field of interest for professionals in scholarly communication. By learning and using open science evaluation practices, participants will be able to show research intelligence outcomes to policy makers to foster change in their institution. This course is specifically designed for data stewards, librarians, and policy makers who want to discover new approaches to advance the open science agenda in a data-driven way.

In three sessions, the course will cover the landscape of open science evaluation, show how we can gather research information, apply open science evaluation techniques on analyse data related to research institutions, and discuss the outcomes of such analyzes.

In the first module, Antonio will guide the audience through a general overview of open science with a focus on institutional and funding policies on recognition and rewards and societal impact (particularly in Europe and the Netherlands). He will then review and critically examine some of the evaluation criteria typically used to rank institutions as well as individual researchers and their publications, highlighting the inability of such metrics to reflect the amount of transparency, accountability, and reusability of the scholarly output. Afterwards, he will introduce alternative evaluation frameworks that allow a more comprehensive analysis of the content of research rather than quantitative (publication) metrics. The audience will have the opportunity to engage in live discussions and reflect on how research is evaluated in their own institutions. At the end of this session, participants will have contextualized old and new evaluation criteria and be able to choose appropriate metrics that better map onto desirable principles of transparency, accountability, and reusability of intellectual products.

In the second module, Tung Tung will provide an introduction to research intelligence applications and its recent developments in the evaluation of scholarly outputs. The goal here is to introduce participants to a variety of data sources, present a set of standard and alternative metrics through use cases, and define strategic questions that guide research intelligence efforts. During live sessions, the course participants are expected to work on strategic questions that are relevant to their context, and operationalize performance evaluation using both standard and alternative metrics, open science metrics, as well as reflect on the comparison between outcomes from standard approaches and alternative performance assessments.

The course ends with a guided assignment specifically aimed at retrieving and presenting research intelligence outcomes, and therefore contributing to the implementation of responsible research evaluation for advancing open science. The guided assignment will consist of a recorded step by step example as well as two live sessions led by Armel for discussions and presentations with the course participants. Here, participants will introduce an open science analysis on their own organization, using the techniques and tools presented by Tung Tung in the second part of the course.

At the end of this course, participants will feel at ease with the major developments in research intelligence reporting for open science by learning about the concepts of open science, apply (novel) evaluation techniques and practice with open access research information sources.


Note: This course follows a different time schedule than other FSCI courses, with first sessions at 12AM Pacific and repeat sessions at 8AM Pacific.)

LIVE ZOOM SESSION SCHEDULE
(All times Pacific)
Wednesday, July. 28
12-1:30AM: Session 1
8-9:30AM: REPEAT Session 1
Monday, Aug. 2
12-1:30AM: Session 2
8-9:30AM: REPEAT Session 2
Wednesday, Aug. 4
12-1:30AM: Session 3
8-9:30AM: REPEAT: Session 3