Skip to main content

Clinical Trials

The Real Numbers on Tamiflu

I’ve been meaning to cover this controversy about Tamiflu (oseltamivir). The Cochrane group has reviewed all the clinical data obtainable on the drug’s efficacy, and has concluded that it doesn’t have much. That’s in contrast to an earlier review they’d conducted in 2008, which said that, overall, the evidence was slightly positive.
But as Ben Goldacre details in that Guardian piece, a comment left on the Cochrane paper pointed out that the positive conclusions were almost entirely due to one paper. That one summarized ten clinical studies, but only two of the ten had ever appeared in the literature. And this sent the Cochrane Collaboration on a hunt to find the rest of the data, which turned out to be no simple matter:

First, the Cochrane researchers wrote to the authors of the Kaiser paper. By reply, they were told that this team no longer had the files: they should contact Roche. Here the problems began. Roche said it would hand over some information, but the Cochrane reviewers would need to sign a confidentiality agreement. This was tricky: Cochrane reviews are built around showing their working, but Roche’s proposed contract would require them to keep the information behind their reasoning secret from readers. More than this, the contract said they were not allowed to discuss the terms of their secrecy agreement, or publicly acknowledge that it even existed. . .Then, in October 2009, the company changed tack. It would like to hand over the data, it explained, but another academic review on Tamiflu was being conducted elsewhere. Roche had given this other group the study reports, so Cochrane couldn’t have them.

And so on and very much so on. Roche’s conduct here appears shameful, and just the sort of thing that has lowered the public opinion of the entire pharma industry. And not just the public opinion: it’s lowered the industry in the eyes of legislators and regulators, who have even more direct power to change the way pharma does business. Over the years, we’ve been seeing a particularly nasty Tragedy of the Commons – each individual company, when they engage in tactics like this to product an individual drug, lowers the general standing of the industry a bit more, but no one company has the incentive to worry about that common problem. They have more immediate concerns.
So what about Tamiflu? After years of wrangling, the data finally emerged, and they’re not all that impressive:

So does Tamiflu work? From the Cochrane analysis – fully public – Tamiflu does not reduce the number of hospitalisations. There wasn’t enough data to see if it reduces the number of deaths. It does reduce the number of self-reported, unverified cases of pneumonia, but when you look at the five trials with a detailed diagnostic form for pneumonia, there is no significant benefit. It might help prevent flu symptoms, but not asymptomatic spread, and the evidence here is mixed. It will take a few hours off the duration of your flu symptoms.

I’ve never considered it much of a drug, personally, and that’s without any access to all this hard-to-get data. One of the biggest raps on oseltamivir is that it has always appeared to be most effective if it could be taken after you’ve been infected, but before you know you’re sick. That’s not a very useful situation for the real world, since a person can come down with the flu any time at all during the winter. Goldacre again:

Roche has issued a press release saying it contests these conclusions, but giving no reasons: so now we can finally let science begin. It can shoot down the details of the Cochrane review – I hope it will – and we will edge towards the truth. This is what science looks like. Roche also denies being dragged to transparency, and says it simply didn’t know how to respond to Cochrane. This, again, speaks to the pace of change. I have no idea why it was withholding information: but I rather suspect it was simply because that’s what people have always done, and sharing it was a hassle, requiring new norms to be developed. That’s reassuring and depressing at the same time.

That sounds quite likely. No one wants to be the person who sets a new precedent in dealing with clinical data, especially not at a company the size of Roche, so what we might have here is yet another tragedy of the commons: it would have been in the company’s best interest to have not gone through this whole affair, but there may have been no one person there who felt as if they were in any position to do something about it. When in doubt, go with the status quo: that’s the unwritten rule, and the larger the organization, the stronger it holds. After all, if it’s a huge, profitable company, the status quo clearly has a lot going for it, right? It’s worked so far – who are you, or that guy over there, to think about rearranging it?

12 comments on “The Real Numbers on Tamiflu”

  1. Anonymous says:

    anybody want to take bets on whether the vertex CF data coming 5/20 will be good enough to save the company? i hope so, derek.

  2. Anonymous says:

    If you read Ben Goldacre’s book it seems to me that the ‘D’ department in ‘R & D’ have lost the plot altogether. Being engaged in dishonest practices that would get you shown the door in ‘R’.

  3. dearieme says:

    “This is what science looks like.” That’s a bit rich coming from Goldacre: after sermonising for years he tried doing a bit of science himself and, in no time at all, had to beat a retreat from his own conclusions.

  4. Gyges says:

    One of the things that sticks out for me is that (brace yerself uppercase shouting) the BRITISH GOV BOUGHT A LEMON!!!!
    They bought something without looking at the information; just like you and me going to buy a car without taking a trained mechanic and insisting that the mechanic looks under the bonnet (hood) before buying it.
    Stupid, shameful and incompetent behaviour by the UK gov. Wankers all!

  5. Industryfan says:

    You are assuming the Cochrane group is unbiased and their conclusions are relevant. The Cochrane team have started a campaign against Tamiflu stockpiling, they can’t say “sorry we were wrong” now and they want to demonstrate that we need better access to clinical data. And what would that do to the sales of Goldacre’s book?
    We all know statistics can made to show many or no effects and Cochrane are understating the effectiveness of Tamiflu and overstating the side effects. They also ignore data from outside randomised trials even though observational studies are the only thing we’ve got from the 2011 pandemic. This limits the statistical power of their analysis since seasonal flu is such a mild disease that hospitalisations are extremely rare, with or without the drug.
    Yes, Tamiflu is not very effective in seasonal flu. But it does have a (small) benefit, enough to allow approval. But most importantly, we stockpile Tamiflu for pandemics, not seasonal flu. Eliminating stockpiles because of this review could be disastrous and many flu experts have publicly criticised it.
    Please also see this comment in nature:

  6. For another perspective (from an infectious disease specialist):
    “Like evolution, there are multiple lines of evidence to demonstrate oseltamivir efficacy against influenza. When it should be used to get the most bang for the buck depends on the strain of influenza and the patient being treated. And when (not if) we get a pandemic influenza that is both highly infections and highly virulent, I hope we will have a drug like olsetamivir. It will not be a panacea, like penicillin for syphilis. But it will keep some people alive who would otherwise die. ”

  7. Joe says:

    Is Roche unbiased? They only published their best trials and public health officials made decisions based on those. If the public health officials had access to all the Roche trials at the time, then they may have made a different decision.
    For 2009 H1N1, WHO recommended “the use of oseltamivir and zanamivir to prevent severe illness and deaths, reduce the need for hospitalization, and reduce the duration of hospital stays” ( Would the WHO, or the countries following their guidelines, follow the plan to stockpile if they knew that the randomized trials did not show convincing evidence for any of those claims?
    My understanding of what Cochrane was trying to do was to show the importance of having access to all the clinical data – not show that Tamiflu is useless.
    Comparing the evidence for oseltamivir to the evidence supporting evolution is laughable, especially when it comes from someone who makes the unsubstantiated claim that influenza will evolve to be highly virulent and infectious. Each pandemic since 1918 has been less pathogenic and many of the deaths from 1918 may have been more due to treating people with loads of aspirin or co-infection with bacteria (
    It seems that Tamiflu is modestly effective at reducing symptoms by a number of hours and individuals may decide that it is worth the money. This does not mean that it makes sense to stockpile tons of it in anticipation for a public health disaster.
    At the end of the day, the Cochrane Collaboration accomplished a good thing by pushing for the release of the clinical trial data. Feel free to ignore their analysis and look at the trials yourself (thanks to them) if you don’t trust them. It is very unfortunate that Roche’s actions/inaction may further damage public confidence in science and medicine.

  8. Jose says:

    The Cochrane method, while imperfect, is transparent and rigorous, with different types of studies getting weighed based upon the quality of the clinical/statistical evidence.
    While all of epidemiology has inherent limitations,, to argue with a Cochrane report makes you look truly clueless (and/or a paid shrill)

  9. Industryfan says:

    Roche is certainly not unbiased, but I find it quite annoying when the self-proclaimed “independent scientists” imply that all industry sponsored research is dubious even though they themselves have an agenda as well. For science to work we have to assume everyone is trying to find the truth and that fraud is the exception. We need peace between academia and industry.
    Randomised trials are the gold standard, but they are not adequate in this case. Estimating the performance of Tamiflu in a pandemic from seasonal flu randomised trials is impossible as the trials are hopelessly underpowered and seasonal flu does not equal pandemic flu. For example: in a trial with 10,000 people with influenza like illness the flu might just cause 10 hospitalisations. To get a measurable effect on hospitalisations you need extremely large trials. But in pandemic flu 1,000 of 10,000 may be hospitalised and then the effect of Tamiflu can be seen. (These numbers are of course made up.) Therefore it is not surprising that the reduction of hospitalisations and pneumonia were not statistically significant in this very conservative analysis. On the other hand, the next pandemic may simply be Tamiflu resistant.
    btw, doesn’t the (in some cases very rapid) emergence of Tamiflu resistance tell us that the drug decreases the evolutionary fitness of susceptible virus particles?

  10. MAtthew Herper says:

    I saw an even better post on this, but this is the one I can find:
    I’m not sure I believe Tamiflu is useless just because of the meta-analysis. It’s certainly not useful for garden variety flu, but it might be something we’d need in a pandemic.

  11. Handles says:

    I was annoyed by the poorly worded picture caption in the Guardian article: “Star anise provides the principal component of Tamiflu.” I immediately imagined non-chemists being ripped off, and google quickly confirmed this has been happening:

  12. Joe says:

    Tamiflu resistance also shows-up in the absence of Tamiflu selection (I forget if it was a pre-Tamiflu strain that was already resistant or if it was just a dominant strain in a population without Tamiflu), so that does not necessarily mean that the drug is a great treatment. Either way, Tamiflu definitely works in vitro and seems to work in people (to some extent).
    I understand what you mean about the serious cases. This brings up a great point about how hard it is to make good decisions about public health preparedness. How many doses of a potentially efficacious antiviral should be stockpiled for a potentially dangerous strain that may or may not arise in the near future (IIRC pandemics are usually once every thirty or so years)? How do you decide what is cost-effective? Drifting a little off topic, but these problems also hold true for many high-containment viruses that are only really a public health threat if you are trying to get some anti-bioterror money.

Comments are closed.