Can simple interventions improve the reproducibility of research?

That question sits at the back of Work Package 4 (WP4), which focuses on developing and evaluating practical tools for two influential stakeholder groups in the research ecosystem: publishers and research funders.

In this edition of the Catch Up with the OSIRIS Work Package Leads series, Mariska Leeflang, the OSIRIS WP4 Lead, shares the latest progress from Work Package 4 and explains how the team is testing reproducibility-focused interventions in real-world research settings.

Watch the Video:

Supporting Better Decisions in Publishing and Funding

Research reproducibility is influenced by many actors across the research lifecycle. While researchers play a central role, publishers and funders also have significant opportunities to encourage transparency and robust research practices. Work Package 4 focuses specifically on these stakeholder groups through the development of practical checklists designed to support decision-making.

The project is evaluating:

  • A checklist for peer reviewers used by publishers
  • A checklist for reviewers assessing research funding proposals

Both interventions aim to encourage greater attention to Open Science and reproducibility practices during the evaluation process.

Testing a Peer Review Checklist

One of the flagship activities within Work Package 4 is a randomised controlled trial conducted in collaboration with the BMJ Publishing Group. The study evaluates whether providing peer reviewers with a structured checklist focused on Open Science practices can improve the quality and reproducibility of submitted research. More than 400 manuscripts have already been included in the study, making it one of the largest intervention evaluations undertaken within the OSIRIS project.

The trial provides a unique opportunity to assess whether practical guidance can influence peer review outcomes and encourage stronger reproducibility practices.

Measuring Reproducibility in Practice

Evaluating reproducibility requires more than reviewing policies or intentions; it requires testing whether research findings can actually be reproduced. To achieve this, the Work Package 4 team is undertaking the demanding task of reproducing research findings from participating manuscripts. Ayu Devi, the PhD researcher leading much of this work, is systematically examining manuscripts after peer review and attempting to reproduce their reported results.

This approach allows the team to directly evaluate whether the intervention influences the reproducibility of published research.

“She is actually reproducing the results from every manuscript that comes back after peer review.” – Mariska Leeflang

Exploring Reproducibility in Funded Research

Alongside the publisher study, Work Package 4 is investigating whether reproducibility indicators can be identified earlier in the research process. The team is analysing research proposals and linking them to the resulting scientific publications. The goal is to determine whether checklist-based assessments conducted during funding evaluations can help predict the future reproducibility of research outputs.

Although this study could not be implemented as a randomised controlled trial, it provides valuable insights into how funders might support more reproducible research from the earliest stages of project development.

Awaiting the Results

The studies are currently ongoing, and the team is eagerly awaiting the first complete results. Some initial findings are expected to be presented at upcoming scientific conferences, providing the research community with early insights into the effectiveness of these interventions. The final analyses will help answer important questions about whether structured checklists can contribute to more transparent, reproducible, and trustworthy research.

Why This Matters

Many discussions about reproducibility focus on identifying challenges and recommending improvements. Work Package 4 takes the next step by rigorously testing whether proposed solutions actually work. By evaluating interventions through real-world studies and large-scale datasets, the team is generating evidence that can inform future policies and practices for publishers, peer reviewers, and research funders.

The findings could help shape future approaches to research evaluation and support broader adoption of Open Science principles.

About the Catch Up with the OSIRIS Work Package Leads Series

Recorded during the OSIRIS General Assembly in London, this interview series highlights the achievements, outputs, and future plans of the teams driving the project. As OSIRIS moves toward its final symposium, The Legacy of OSIRIS: Open Science for Reproducible Research, taking place on 5–6 October 2026 in Prague, Czech Republic, the series offers an opportunity to reflect on progress and explore how project results are contributing to a more transparent and reproducible research system.

Stay Tuned

With data collection well underway and analyses progressing, Work Package 4 is approaching one of its most exciting phases. As results become available, the OSIRIS project will continue to share updates through its website, publications, and events.

Follow OSIRIS to learn how these interventions perform and what they may mean for the future of reproducible research.

OSIRIS “Creating Trust in Open Science & Reproducibility through Accessibility and Transparency!”