Reproducibility in the scientific literature has been a big issue for some time now, and it’s not going away any time soon. There are arguments and counterarguments about how much of the literature is not reproducible, how reproducible the attempts to reproduce it are, what the standards should be for such efforts, and how much the problem might vary by scientific discipline. I’m not going to attempt to link to all the relevant articles (and all my past blog posts) on this topic right now – art is long and life is short (see, I managed to quote that in English rather than Latin).
But what I do want to do is call attention to a proposed solution from Michael Rosenblatt, who is Chief Medical Officer at Merck. See what you think about this one:
Here is the essence of the proposal: What if universities stand behind the research data that lead to collaborative agreements with industry, and what if industry provides a financial incentive for data that can be replicated? Currently, industry expends and universities collect funding, even when the original data cannot be reproduced. In the instance of failure, collaborations dissolve, with resulting opportunity loss for both academia and industry. But what if universities offered some form of full or partial money-back guarantee? With such assurance, companies could proceed with a project more rapidly and more frequently. They would also be likely to pay a premium over current rates for data backed by such assurance over “nonguaranteed” data, even from the same university. This approach places the incentive squarely with the investigator (including his or her laboratory) and the institution—precisely the leverage points for change. The premium would provide universities with the financial wherewithal to cover the cost of affirming their data if they choose to replicate it before entering into a collaboration.
That might work, but I can certainly see universities offering some resistance to the idea, since it places the onus right on them. The expense will be real, while the promise of revenue to make up for it remains to be seen. This recalls the 2011 discussions about venture capital firms and academic science. That turned into a “Don’t you trust me?” sort of argument, with some VCs being pretty hard-nosed about testing things, and others finding the whole idea to be against the deal-making spirit of collaboration. I lined up at the time (and still do) more towards the hard-nosed end. Biopharma deals are about data, and nullius in verba, folks, take no one’s word for anything and check it again.
I’d like to see Rosenblatt’s proposal tried, but if you’re going to wait for the universities to try it, you’re going to have a long wait. So the way to do it is probably from the industrial end. You could try putting in that money-back-guarantee language and see how what the tech transfer offices think of it, but I’m pretty sure I can guess. If it’s structured as “this better work or we take back the cash”, they’ll probably balk immediately. For psychological reasons, you’re probably better off making a low offer for the data and the idea as they come, with a much more generous one if it’s been reproduced first. Let them seek the carrot rather than fear the stick. Reproducible results are already worth a lot more in reality; we should adjust our pricing to reflect that.
Addendum: lest anyone think that I’m just bashing academic science here, I’m all for adding conditions like this to all-industrial deals as well. Some of them already come close to this, but why not put it right out there on the table? How would, say, the GSK-Sirtris deal have gone under such conditions, eh?