In an era where research evaluation methods are evolving, the Research Contribution Claim Network makes trustworthy tracking of non-traditional research output easy!

Introduction 

Globally, research assessment initiatives are advocating for the inclusion of a wider range of scholarly contributions in evaluation reports — beyond traditional outputs such as journal articles and books. Examples are the DORA and COARA collectives advocating for the inclusion of research data, software, presentations, lecture notes, media appearances and performances (i.e. non-traditional research outputs) in assessment of researcher excellence. This broader view is echoed in the 2020 Dutch position paper “Room for Everyone’s Talent” which calls for a modernized recognition system across Research, Education, Impact, and Leadership (REIL).

These developments are applauded from the perspective of research funders and evaluators, requiring that these non-traditional outputs be accounted for in a trustworthy manner. Traditionally, the research-support office is tasked with keeping university research information systems and repositories up to date with relevant research outputs. In practice, however, identifying non-traditional research contributions presents significant challenges:

• The contributions are scattered across the Web at large; and unlike their journal-article and book counterparts, they often lack persistent identifiers and researcher identifiers such as DOIs and ORCIDs, respectively. This makes systematic discovery within current academic-recognition frameworks very difficult.
• Not every podcast, blog or newspaper article may be a claim-worthy scholarly milestone that needs to be counted as part of the REIL-picture.

For the research-support office, keeping track of all blog posts, podcasts and interviews produced by the institution’s researchers, and knowing whether these outputs are relevant, or not, is nearly impossible. Therefore, the research-support office may ask the researchers themselves to keep track of all their research outputs -- traditional and non-traditional, and to make updates in the central research information systems. As a result, researchers often spend hours collecting such information from emails, printouts, and other fragmented channels in order to update their universities’ repository, or don’t get around to updating at all because it is simply an additional task on top of an already overburdened agenda. 

In practice, identifying non-traditional research contributions presents significant challenges

So overall, we agree that research evaluation based on REIL is a worthy cause, but we do think that a successful implementation should be based on an approach that tracks non-traditional research outputs in a manner that is trustworthy and puts a minimal burden on the researcher. To the best of our knowledge, currently, no tools or processes exist that help the researcher or research support expedite collecting information about non-traditional research outputs.

Informal scholarly communication and the development of a research contribution claim network

The widespread adoption of social networks has significantly reshaped scholarly communication, as these platforms offer researchers an almost instantaneous way to connect, not only with fellow researchers but also with the broader public.

Researchers could use social network communications to distinguish between the research output that needs to be added to their cv/profile and the output that doesn't need to be claimed

In 2023, SURF launched a scholarly Mastodon pilot at https://social.edu.nl/ as a first step towards providing Dutch researchers an alternative to centralized social media platforms. Anyone with an institutional account linked to SURFconext can log into this academic instance and use it as a collaborative environment to connect and exchange ideas with peers and public alike, making it straightforward to communicate about research. But we speculated that researchers could also use social network communications to distinguish between the research output that needs to be added to their cv/profile and the output that doesn't need to be claimed.

To verify this idea, in 2024-2025, SURF and IMEC-IDLab at Ghent University created, as part of their innovation agendas, a prototype implementation of a Research Contribution Claim Network (Figure 1), a process based on the idea that informal communication of researchers on social media platforms could serve as triggers to identify traditional and non-traditional REIL contributions and add this information to the researchers personal bibliography.

We’ll explain how…

Toot to claim research output!

On Mastodon, the user communicates via short messages (default maximum of 500 characters) that are shared across the distributed network. Each message on Mastodon is called a ‘toot’. Using 'toots’, researchers can share insights into their academic activities, including updates and reflections on their scientific work and community engagements such as attending a conference or doing a podcast. These messages are inherently valuable from a social perspective, but they can also be used to automatically track, catalogue, and archive research outputs (or the digital traces such outputs leave on the web), thereby reducing some of the administrative burdens faced by researchers.

In the toot of Figure 2, the researcher announces a new contribution while referencing the ‘@claimbot’ account. This claimbot is a smart agent that uses the URL of the contribution in the ‘toot’ to extract metadata from the referenced page. This metadata is then used to update the personal contribution list of the researcher.

Indeed, a few seconds after the researcher posted the toot of Figure 2 to claim a new REIL contribution, they receive a message from the claimbot, as shown in Figure 3.

In a prototype version of this work, SURF used a Wiki service to create personal contribution lists. The claimbot points to the updated Wiki entry of the researcher:

https://wiki.mycontributions.info/researcher/orcid/0000-0001-8390-6171

On the personal Wiki page, in the ‘Contributions’ section, the new entry is visible, as shown in Figure 4.

From the researcher's standpoint, informing their peers, claiming a REIL contribution, automatically generating metadata pertaining to the contribution, and updating their personal record is a single, integrated workflow (see Figure 5), which only requires mentioning the claimbot in a ‘toot’.

From the standpoint of research evaluation, because the researcher uses a SURFconext login and that this SURFconext association can be verified by the Claimbot by means of the researcher's public Mastodon profile, toots can be unambiguously attributed to a specific researcher, adding a degree of trust to the process.

We tested the feasibility of the prototype, by processing a total of 900 ‘toots’ by researchers that are active on Mastodon. The focus of this experiment was to quantify the metadata generation capabilities of the claim logging process. The results are published elsewhere, but in summary, we found that the prototype was able to update personal contribution lists for 91% of the toots, demonstrating that the research contribution claim network can be used to automatically collect information on both traditional and non-traditional research contributions and save the researchers a lot of precious time.

The claim network is portable and digitally sovereign by design

Based on lessons learned from the prototype, the blueprint at https://claimnetwork.info was created to allow other communities/consortia to deploy their own research contribution claim network workflow (Figure 5). The blueprint details core components of the setup, focuses on aspects related to identity and trust, and highlights areas in which interoperability may be beneficial. For a comprehensive overview, please review the Overview Table that lists the core ingredients of a research contribution claim network and indicates how those were implemented in the prototype and how they are conceived in the blueprint: https://claimnetwork.info/whitepaper

We devised the prototype and blueprint with ‘digital sovereignty’ in mind. ‘Digital sovereignty’ refers to the way individuals and organizations control the collection, storage, processing, and transfer of data in accordance with local laws and public values; it also emphasizes the ability to manage one’s own digital infrastructure, thereby reducing reliance on technologies and platforms developed and controlled by others (e.g. social media firms, cloud platforms, and AI models).

Adoption of social media has reshaped scholarly communication and, over time, services such as Twitter (now X) and LinkedIn became prominent channels for academic engagement. However, these centralized platforms, largely owned by major U.S.-based corporations, have faced growing public scrutiny in recent years. In the shifting geopolitical context, it is increasingly important to protect the autonomy of academic digital infrastructures and to decrease reliance on corporate‑owned platforms, thereby re‑emphasizing the need for decentralized, locally managed social‑media services such as Mastodon.

To overcome the gravitational pull from centralized social media platforms, researchers need incentives to use decentralized alternatives

But, to overcome the gravitational pull from centralized social media platforms, researchers need incentives to use decentralized alternatives. The research contribution claim network approach, which we have extensively prototyped, provides such incentive by reducing the administrative burden involved in collecting information about a wide range of research contributions as required for new research assessment approaches.

Summary and next steps…

With our project we showed that the toots researchers post on Mastodon can be used to identify traditional and non-traditional REIL contributions. A research contribution claim network can generate summaries for URLs of contributions mentioned in toots. These summaries can be used to update personal contribution lists and serve as input for research assessment workflows.

We believe the research contribution claim network approach benefits the stakeholders in the recognition and rewards workflow:

• Researchers get a decentralized and digital sovereign communication system for scholarly exchange, coupled with an integrated workflow to create a “personal memory” of research contributions. This removes the burden to collect this information manually.
• Institutions can have an active role in scholarly communication by supporting the infrastructure and can automatically keep track of contributions made by their researchers.
• The general public gets transparency in the research process and access to the latest research contributions.
• Funders are provided with an overview of non-traditional research outputs that can be used for evaluation purposes.

Going forward, SURF will continue to explore how the research contribution claim network can strengthen Open Science initiatives, including MORIS, and the OSNL funded DURF, BROCCOLI and Publish-Review-Curate (PRC) projects.

We hope that our SURF-IDLab experiments inspire others to explore the potential of decentralized web approaches for scholarly communication. And we would be happy to collaborate with parties interested in decentralized scholarly communication and enhanced evaluation to fine-tune the research contribution claim network architecture as described in the Blueprint.