Why monitor Open Research Information?
Across the research ecosystem, data about research outputs, researchers, affiliations and funding flows through many systems, including institutional CRIS platforms, repositories, and open infrastructures such as OpenAlex, OpenAIRE and Crossref. These data are used for reporting, evaluation, analysis and dissemination.
But how complete and accurate are each of these sources? And where should we prioritize improvements?
The ORI Monitoring Framework report (published at https://doi.org/10.5281/zenodo.18667698) addresses these questions. The report is part of the PID-to-Portal project, which aims to strengthen the open research information chain from persistent identifiers to public discovery platforms, so that Dutch research becomes more visible and reusable.
The framework introduces a monitoring approach that focuses on metadata quality, PID adoption and metadata openness, allowing the community to identify gaps and target interventions where they matter most.
Starting with real use cases
A key design principle of the framework is that improvements in metadata should be driven by real use cases.
To determine priorities, the project worked with the Dutch signatories of the Barcelona Declaration on Open Research Information. These organizations, including universities, funders and national infrastructures, shared their most important use cases for research information.
In total, 14 use cases from 10 organizations were collected and analyzed.
Most of these use cases fall into three main categories:
- Reporting, such as open access monitoring and institutional reporting
- Evaluation and assessment, including research performance analysis
- Analysis, for example understanding collaboration networks or research trends
These use cases demonstrate the value of reliable open research information. Without them, there would be little reason to improve the completeness and accuracy of metadata in upstream systems.
What information matters most?
The analysis revealed clear priorities for the research information landscape.
Key entities
The most important entities for monitoring include:
- Research outputs, such as journal articles
- Researchers
- Affiliations and institutions
- Journals and funding information
Affiliation data (including information on corresponding authors) in particular emerged as critical for many workflows, yet it is also one of the areas where quality and consistency problems often occur.
Core metadata elements and PIDs
Across the use cases, a small set of identifiers proved essential:
- DOIs or research outputs (as well as other PIDs, such as handles)
- ROR identifiers for institutions
- ORCID IDs for researchers
These identifiers appeared in half or more of the use cases, making them the core metadata elements for monitoring.
Additional metadata elements relevant for some use cases include:
- Corresponding author information
- Open access status
- ISSN for journal analysis
- Grant identifiers
Together these identifiers form the backbone of a connected and interoperable research information ecosystem.
What do we mean by metadata quality?
The report also explores what “quality” means when working with research metadata.
Across the use cases, four dimensions of metadata quality were identified:
- Completeness, is the metadata present for all records and entities?
- Accuracy, is the information correct, e.g. does it match authoritative sources?
- Up-to-date-ness, is the information current?
- Openness, is the metadata openly available with reusable licenses and APIs?
Among these, completeness and accuracy were considered the most critical. Openness also plays an important role, and is mostly a characteristic of the respective database, rather than individual metadata fields.
Comparing sources to reveal gaps
Based on these priorities, the report proposes a monitoring framework that compares metadata across multiple data sources.
The framework initially focuses on open infrastructures such as:
- OpenAlex
- OpenAIRE
- Crossref
These are complemented by institutional CRIS systems, which remain key authoritative sources within universities.
By comparing metadata across these systems, the framework can reveal gaps such as:
- missing affiliation metadata, for example absent ROR identifiers
- incomplete ORCID coverage
- differences in open access status
- missing or incorrect PIDs
Identifying these gaps is only the first step. The goal is to enable targeted interventions, for example improving publisher metadata deposits, fixing affiliation matching in open infrastructures, or enriching institutional CRIS records.
Experimenting with a new ORI data lake
To support this work, the PID-to-Portal project is also experimenting with a new ORI data lake, fully open-source!
This experimental infrastructure aims to:
- aggregate open research information sources
- enable cross-source comparison of metadata
- power dashboards that monitor progress in open research information
Read more about this experiment here:
https://communities.surf.nl/en/open-research-information/article/behind-the-scenes-experimenting-with-a-next-gen-ori-data-0
Part of the PID-to-Portal project
The ORI Monitoring Framework is one of the first results of the PID-to-Portal project, which aims to strengthen the open research information chain from persistent identifiers to discovery portals.
You can read more about the project here:
https://communities.surf.nl/en/open-research-information/article/from-pid-to-portal-strengthening-the-open-research-information
What comes next?
This framework report is only the first step.
A follow-up report will define specific metrics for monitoring metadata quality and PID adoption, enabling dashboards and regular monitoring of the open research information landscape.
In the meantime, we invite you to:
📄 Read the report
https://doi.org/10.5281/zenodo.18667698
📅 Join the workshop at the SURF Research Day – May 19th 2026
The agenda of the workshop will be announced here:
https://communities.surf.nl/open-research-information/agenda
In this workshop we will discuss possible interventions to close gaps in the open research information landscape (based on initial monitoring results) and how the community can contribute.
Improving open research information is a shared effort. By working together on the quality, completeness and openness of research metadata, we can strengthen the entire research information ecosystem.
0 Praat mee