Skip to main content
Menu

Improving reproducibility

Scientific progress relies on the validation of studies by other researchers. In 2017, a working group of publishers, editors, and academics formed to address how scientific journals could improve the reproducibility of life science research by applying the Transparency and Openness Promotion (TOP) Guidelines. As motivated in the statement below, the group developed a framework—Materials Design Analysis Reporting (MDAR)—and in 2019, launched a pilot study that used a checklist to test its implementation. The framework and checklist have been revised, and the development process is described in a paper published in the Proceedings of the National Academy of Sciences USA. In line with supporting the TOP Guidelines, Science adopted the draft checklist in 2019 and plans to apply the revised framework by using the updated version of the checklist.

Valda Vinson is the Editor of biological science research at Science.

The MDAR Framework – a new tool for life sciences reporting

(Materials, Design, Analysis, Reporting)

Incomplete or imprecise reporting of life sciences research contributes to challenges with reproducibility, replicability, and biomedical applications. For the last three years we – a group of journal editors and researchers – have been working together to develop a new framework for transparent reporting of life sciences research. This framework has just been published in PNAS.

The MDAR Framework establishes the four domains – research Materials, Design, Analysis, and Reporting – in which we define both a set of basic minimum requirements, and best practice recommendations.

We were motivated to develop the MDAR Framework as part of our own and others’ attempts to improve reporting to drive research improvement and ultimately greater trust in science. Existing tools, such as the ARRIVE guidelines, guidance from FAIRSharing, and the EQUATOR Network, speak to important sub-elements of biomedical research. This new MDAR Framework aims to be  more general and less deep, and therefore complements these important specialist guidelines. 

Previous approaches have led to improved reporting, but often at considerable cost to both authors’ and editors’ time. A recent period of experimentation has resulted in a thorough but fragmented landscape of reporting guidelines for life science journals. A drive for efficiency  inspired us to learn from each other’s experiences and to harmonize the most effective practices.

The MDAR Framework provides flexibility along with broad applicability. The standard articulation of expectations across different journals will make it easier for: (i) authors to better understand what is expected of them, and (ii) for more journals to adopt an established approach rather than develop it from scratch. Journals can choose a level of implementation appropriate to their needs, enabling greater adoption potential.

We also hope that the MDAR Framework will be helpful for other organizations such as funders, who can signal reporting expectations early and therefore have an effect at the time the studies are designed, and tool/software developers, who can devise means of facilitating compliance for authors and journals.

Alongside the framework, the project provides a checklist (for authors, journals or reviewers) as an optional implementation tool, and an explanation and elaboration document. The checklist was piloted on over 289 manuscript submissions across 13 journals, seeking feedback from authors and editors actually using the checklist. Our team analysed agreement between observers, sought feedback from outside experts, and revised the framework in the light of this experience.

The full set of MDAR resources will be maintained and updated as a community resource, in a Collection on the Open Science Framework.

We are sharing this update on the MDAR Framework through coordinated posts on working group member platforms. Working group members have been free to add any additional context as appropriate.

The MDAR working group:

  • Andy Collings (eLife)
  • Chris Graf (Wiley)
  • Veronique Kiermer (PLOS)
  • David Mellor (Center for Open Science)
  • Malcolm Macleod (University of Edinburgh)
  • Sowmya Swaminathan (Nature Portfolio/Springer Nature)
  • Deborah Sweet (Cell Press/Elsevier)
  • Valda Vinson (Science/AAAS)

This work is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. To view a copy of this license, visit https://creativecommons.org/licenses/by/4.0/.