Monitoring

Last updated 4.8.2022

The objective of the monitoring of open science and research is to support the development of open science and research in organisations, support and verify the achievement of the objectives agreed in the Declaration and policies, and form an overall view of the state of openness in Finnish science and research. The monitoring will use as much available, comparable data collected in the National Research Information Hub as possible. The monitoring also involves collecting data using a survey. The role of the survey as a source of monitoring data will decrease as more data accumulate in the National Research Information Hub. Initially, the survey plays a key role.

Key documents:

Monitoring schedule

The monitoring will be carried out for the first time in 2022. In 2022, the monitoring will be light and largely based on existing and well-known indicators and the needs of European monitoring. The monitoring carried out in 2024 will be significantly broader. Piloting the indicators used in this context will be a key task in the coordination of Open Science and Research (OScaR) in 2022. The entire research community will be invited to participate in the piloting. The decision to add new indicators in the implementation of the 2024 monitoring will be made by the National Open Science and Research Steering Group in spring 2023.

The results of the monitoring will be published on the research.fi website and the results of the survey will be openly available.

Indicators and scoring

The indicators used in the monitoring will be used as the basis for determining organisations’ openness profiles and their level of open science and research. Limited indicators will be used in 2022. In the 2024 monitoring, the indicators will be complemented to cover all areas. The principles of scoring are transparent and accessible to all.

Each indicator has been scored. The points scored are either “base points” or “additional points”. The base points apply to practices that are central to the implementation of policies for all organisations. The additional points apply to practices that are still new, under development or create added value.

The scores are used to determine the degree of openness in the organisation for each area. The degree of openness (1–5) in a given area is determined for each organisation based on the percentage given to them of the maximum number of the base points on one hand and of the maximum number of the total points on the other. The percentage-based model makes it possible to add new indicators to monitoring in the future and it can be scaled according to the development of open science and research.

Together, the degrees of openness in the various areas comprise an openness profile for the organisations. The profile makes an organisation’s degree of openness visible in different areas (publications, materials and infrastructure, learning and operating culture), and, if necessary, the degree of openness can also be studied from the perspective of operating methods (policy documents, services, cooperation and output).

The profile is also used as the basis for an overall assessment of openness in the organisation. In addition to the baseline, there are five levels in the overall assessment. The levels are defined in such a way that climbing to the next level requires either a one-step increase in two areas or a two-step increase in one area (i.e. the average score for the areas increases by half a step). Climbing to the higher overall assessment levels also requires increasing the degrees given to all profile areas to some extent.

Background

In 2020, the National Open Science and Research Steering Group appointed a working group to develop a next-generation model for monitoring open science and research in Finland. In addition to the working group, the expert members of Open Science and Research Coordination in Finland participated in the development of the monitoring model through co-creation carried out in May 2020, and the model was out for an open round of comments at the national level between 27 September and 1 November 2021. 54 organisations commented on the monitoring model.