Skip to Content

There Are Failures, You Know

This exact point came up around here when we last discussed FDA reform, so it’s good to see it made at length in the New England Journal of Medicine. Remember solanezumab? That was the amyloid-targeting antibody that Eli Lilly kept on investigating in trial after trial, looking for some effect on Alzheimer’s. Last November, the final, final word finally came down that it really, truly, does not work. To recap, mouse model results with a similar antibody were published in 2001. Phase I results of solanezumab itself were published in 2010, and Phase II results were published in 2012.

The authors of the NEJM paper would like to point out that under the current system, the cost of investigating all this was largely borne by the drug’s developers, not the patients and not the taxpayers:

But the failure of solanezumab also offers a window into the U.S. drug regulatory system, particularly in the con- text of the recently signed 21st Century Cures Act and the ongoing national debate about the role of the Food and Drug Ad- ministration (FDA). The solanezumab story is an important case of a regulatory system that worked.

Indeed it is. Under a system designed to speed up drug approvals, people might have started taking it back in 2010-2012, when the Phase I and II results showed no adverse effects. I was actually worried about this at the time – not so much that people would get early access, but that a trial might show eventually show wispy possible little hints of efficacy, just barely enough to get the drug approved, but to no beneficial effect in the real world.

But there really is such a huge demand for something, anything, with any hint of hope. People would line up to buy anything that got FDA approval, no matter how tenuous the evidence was. And that puts the agency in a very tough position. . .You can argue that one of the main purposes of the agency is to make sure that medicines that people can be prescribed in this country will actually do some good, rather than raise hopes for nothing. You could also argue that responsible adults – and their physicians, and their insurance companies – should be able to make such choices for themselves, and should be able to spend their time and money in the ways that they best see fit. You could argue that companies with marginally effective (or ineffective) therapies face a huge moral hazard, in that their incentives are to get such treatments onto the market whether they do anyone else any good or not. None of these are foolish positions, but they are also, in places, mutually incompatible. Alzheimer’s disease might well turn into the next place in which we thrash them out.

I think all those points stand, and this latest article is basically a sigh of relief that we didn’t get to that point:

Lilly’s decision to undertake Expedition3, though logical, was not inevitable. The trial was undertaken after those reported discussions with regulators because the law requires manufacturers to provide the FDA with substantial evidence of a product’s efficacy before bringing it to market, and prospective, randomized trials are generally the best way to do so. In recent years, however, this standard has increasingly been criticized as overly stringent; some commentators have even suggested that the FDA should simply approve all drugs that are determined not to be toxic, letting “the marketplace” determine which ones work best.

Normally, as a free-market capitalist guy, I would take exception to those scare quotes around “the marketplace”, but unfortunately, in health care they’re warranted. We have a very tightly regulated and opaque market indeed in this country for prescription drugs and every other form of health care, and it’s not a very good place to discover prices or utilities. You could imagine a system where these things could be done better than we’re doing them, but such a system would be pretty far from what we have going now.

And even in an idealized situation like that, I’m not sure how well “efficacy discovery” could ever work. How would one collect all the patient data? How could that be done in a way where the data themselves could be compared? Issues of patient selection, diagnosis, compliance, measurement of outcomes, and other issues would all get much more difficult and complicated. Dealing with all those variables and getting them out of the way is exactly why we run controlled trials in the first place, so tossing all that aside seems odd, to say the least. You might imagine a world in which you could say “Well, come on now, does this drug work or doesn’t it? Simple!”, but that’s not the world we’re living in. It really isn’t.

The NEJM paper estimates, pretty conservatively, that had solanezumab been given conditional approval back in 2012 or so, that we – meaning Medicare, for the most part, which is to say all taxpayers, but also insurance companies and patients – would have spent at least ten billion dollars injecting Alzheimer’s patients with an expensive placebo. No one would have gotten the tiniest bit better. False hope all around, with no benefit, and billions of dollars down the tubes.

This would be a good time to remember Bastiat’s broken window fallacy. Actually, it’s always a good time to remember Bastiat; the man was a fountain of good sense. You hear this one trotted out after hurricanes and the like to the effect that at least all that rebuilding will stimulate the local economy. But Bastiat draws attention to “what is not seen” – the fact that the money spent doing this would have been doing something else, had all those windows not been broken and all those buildings not destroyed. Then we would have had the benefits of that activity, on top of keeping all the buildings intact. The same goes for the billions that would have been spent on solanezumab. One might say “Oh well, no harm done”, since the drug at least met the primum non nocere cutoff. But those ten billion dollars could have helped those Alzheimer’s patients (and their families) in other, more meaningful ways, and been used for other things entirely as well. Pouring money and effort (and human hopes) down ratholes is never a good thing to rationalize. Let’s not.

43 comments on “There Are Failures, You Know”

  1. SirWired says:

    In a weak, half-hearted, defense of “the marketplace”, one could guess that in the absence of the FDA requiring efficacy, drug insurers could perform much the same function through control of their formulary. (Similar to what they do today, but to a greater degree.)

    We actually see something similar with electrical products. The CPSC does not require safety testing (self-certified or otherwise) before an electrical product can be sold. But, in practice, few retailers in the US will sell anything that plugs into a wall outlet unless it’s received a UL Listing, which does require actual independent testing at a UL facility.

    I think insurance-driven efficacy testing would be a poor substitute for FDA pre-market efficacy approval, and the political pressure on insurers to pay for an expensive forlorn hope would be immense, but it would act as something of a filter.

    1. Isidore says:

      A very interesting comparison. I wonder if it is possible to find a way to have both, FDA pre-market efficacy approval and insurance-driven efficacy and whether under certain circumstances a drug developer may be able to opt for the latter (again, only for efficacy, safety approval will remain with the FDA). I do not have an answer, but it is an intriguing idea.

      By the way, Derek, in as much as I agree with your assessment I will quibble with the statement that “the cost of investigating [solanezumab] was largely borne by the drug’s developers, not the patients and not the taxpayers”. This is certainly accurate in a narrow sense, but surely it will be factored in Lilly’s overall costs of doing business and so in the cost of other Lilly medications that make it to market. In the end we all bear the costs of drug development. I am not arguing that this should not be the case, just that that’s how it is.

      1. Derek Lowe says:

        Good point – the customers do get costs passed on to them eventually.

        1. Anon says:

          No, not really. In a free working market, people, payers and providers only pay what they are willing and able to pay for the direct benefits they are getting. Of course Pharma would *like* to pass on the cost of all its failures (and then some), but that is not what people will actually pay for. When a business makes losses that can’t be covered by the profits of its successes, nobody owes them anything, and it simply goes out of business.

          I wish people would stop saying that people pay for the cost of Pharma’s failures. Not true, they only pay more than the cost of its successes depending on the value created. And the risk of development is borne entirely by Pharma’s investors.

          1. Isidore says:

            Think about it: If, say, one out of ten of a company’s development candidates makes it to market aren’t some of the proceeds from its sales going to be used to fund all of a company’s activities and not just those that resulted in the successful launch of the new drug? Where did the money that funded the very expensive solanezumab R&D come from? From Lilly’s other products that did make it. If a company had 100% success, i.e. if every drug they developed were approved don’t you think that it would be sold for less than now, when only a small percentage make it through.
            There’s nothing wrong with this, by the way, I mean someone has to pay for drug development. And this is not unique to the drug industry; the computer industry, the automobile industry, the potato chip industry, all work the same way. But in those industries the percentage of failures is nowhere near as high as in the drugs business, so the issue of “passing the costs of all its failures” (an unfortunate way of putting it, I agree) to consumers does not really come up.

          2. CR says:

            The costs of failures are written into every successful drug launch, or the failures of any product in any business model. To think otherwise is naive, at best, or simply Pharma defending.

          3. Anon says:

            The point is that Pharma does not charge more to cover the cost of all its failures. It charges as much as it can (based on what people are willing and able to pay) in any case, regardless of all its failures.

            Does anyone really think that Pharma would start lowering its prices if it did not have so many failures? No, it would just charge the same and make more profit, because that is what their shareholders expect of them to do.

            So no, I’m not naive, and I’m not defending Pharma. I’m actually saying that the cost of failures is not factored at all into the price.

            I mean can you imagine, trying to convince a payer/provider: “please can we charge you more for our me-too survive-just-an-extra 2-weeks cancer drug because our Alzheimer trial just failed and cost us a bomb?”

            You guys need to get out the lab and figure out how the world really works!

          4. Design Monkey says:

            Your are not thinking, anon (and pretty much seems, that you not capable to do it at all). If drug discovery had a significantly higher success rate, then there would be much more new drugs. More various drugs = more competition, lower prices.

          5. Anon says:

            If competition affected drug prices then pharma companies wouldn’t all be following each other and chasing the same targets like sheep.

          6. Anon says:

            … and have you ever seen drug prices come down as more me-too competitors hit the market? No, branded drug prices are *still* going up by >10% per year, even after all the Schkreli schit. Competition does nothing to drug prices until they become truly generic.

            So I’m afraid you still need to get out the lab to see how the real world works in practice. And look at the actual data and not your economics theory textbooks.

    2. Me says:

      In Europe, where govt pays most healthcare, the govt health insurers usually perform efficacy assessments on products to decide whether they should reimburse them or not. So it goes on to a certain extent. Not sure how that would play out in the US market though.

    3. Shalon Wood says:

      Except that they wouldn’t.

      Insurers don’t _care_ about whether or not something is efficacious; they care about whether or not it _costs_ too much.

      I will guarantee you, in the sense that if you can prove me wrong I’ll give you $100, that an insurance company would _much_ rather pay for a drug that costs it $10/month and kills the patient after 10 months 50% of the time than a drug that costs $10,000 a month and cures the patient 50% of the time after 1 month and 100% of the time after two months.

      At this point, I am personally convinced that the problem with the US health system has essentially nothing to do with the pharmecutical companies, despite the averaged costs of bringing a successful drug to market, and everything to do with insurance companies.

      I’m sorry, what? You want a solution? Sorry. I got nothing. Maybe… uh… ask that guy there. He should know. I sure don’t.

      1. Shalon Wood says:

        er. The last paragraph there should have been:

        I’m sorry, what? You want a solution? Sorry. I got nothing. Maybe… uh… [points in a random direction] ask that guy there. He should know. I sure don’t.

      2. Morten G says:

        True. In fact some insurers could probably save money by re-running efficacy trials for sketchy drugs. I just don’t think there are any single insurer big enough. And insurers can probably weasel out of the worst bills anyway. It only seems like it would be economical for a single payer system or collaboration of single payers.

        1. Colintd says:

          This is like the analysis that shows stopping smoking improves fit life length, but pushes up total health spending because there are less sudden deaths, and more high dependency deaths extended over a couple of years.

        2. Ken says:

          And what insurer would foot the bill? Any results from a trial would have to be kept a trade secret or else another insurer would take the results and run with it (so insurer reimbursements have to be secret? My head is hurting from the implications.) Either an industry trade-group funded by all insurers on some per-capita ratio or as you said, single payer group would put up $ for a study.

  2. John Thacker says:

    However, surely “what is not seen” also applies even more to drugs that are never developed or released, many because their target market is too small, among other reasons. I’d even argue that those are even less seen than the problems with this case, for here we have a specific identified high profile failure to point to as a waste of money. Anything not pursued because of regulation costs will not have an obvious specific target to blame, and is even more “not seen.” Surely there are both types of errors, and people (on all sides) would do well to remember it. However, while you make a good point for the other type of error, Bastiat’s argument would point overall towards bias favoring more regulation, since the errors of too much regulation preventing targets from even being developed are even less seen.

    I can certainly understand an argument that statins should not have been approved until actual long term survival rate improvements were demonstrated, instead of just cholesterol effects.

    There are often multiple sorts of failures. Does the recent Gates Foundation and Pfizer study of tranexamic acid mean that lack of FDA approval for childbirth and surgery has cost more lives than the money wasted on solanezumab could have saved? (That is a case where it’s difficult to completely rule out no effect, but the condition is common enough that small positive effect could save a lot of live.) Is that “more” the fault of the drug companies for not funding the appropriate research especially when the drug was on patent, and is it the responsibility of public policy to just tell drug companies to “research smarter, and only research the good ones” or to craft the policy that steers best between two types of errors?

    1. Derek Lowe says:

      Your point about drugs that don’t get developed is a good one, but I don’t think that the regulatory environment is as big a factor there as is pricing/market size and the underlying science (lack of targets or understanding of disease biology).

  3. mallan says:

    For many at the top of drug companies, along with a majority of biotechs, getting return on the investment that has taken many years and cost hundreds of millions or over a billion dollars, becomes a huge driver. Their view is that money already lost is gone but the investment needs to produce a reward, eg make money back, so are willing to put more into the growing deep dark hole of cost. Their view of lost opportunities don’t seem to come into equation, since anything earlier in the pipeline will be even further away from bolster stock price, and hence their bonuses. This quarter, and the one after that and the next too is their horizon. When the a company’s drivers ultimately are individual metrics and bottom line profits, then altruistic vision flies out of the building.

    1. Shalon Wood says:

      That sounds… disturbingly like they’ve completely misunderstood the sunk cost fallacy, and have in fact taken its opposite as a sacrosanct law.

      I really want to believe that either I’ve misunderstood, or you’re wrong if I haven’t. Sadly, I’m feeling incredibly cynical tonight….

  4. HTSguy says:

    Since nearly all drugs (even quite benign ones) have some risk profile, how is one to assess “safety” if you don’t know the efficacy and thus the risk/reward ratio? Passing Phase 1 indicates a drug is safe enough for further testing in a larger population. It doesn’t indicate that it is “safe” in absolute terms. If the latter (absolute safety) were required for the “safety only” FDA approval, then very few new drugs would meet this bar.

    1. Chris says:

      You are absolutely correct. I have worked on cytotoxic cancer drugs (which are still the mainstay of cancer therapy despite some advancements in immunotherapy) and these are not “safe”! Who would voluntarily take (and pay for) a drug that makes you lose your hair, causes you to vomit and have diarrhea, may lead to bad infections or even long term cardiac toxicity??? If there is no PROVEN benefit, there would be few if any, cancer drugs.

    2. tangent says:

      This is a very valid point that many “safety only” campaigners seem to miss. But the response from those who have gotten this is something like: the testing is not to pass a specific safety threshold, but to characterize the risks and the remaining uncertainty about the risks. Then adults and their doctors can make their own decisions about exposing themselves to those risks. (And presumably drugs that are more risky than their efficacy could possibly justify are not given approval.)

      Which just kicks the same problem over to the patient, of course. They inherently can’t make a well-informed decision to balance these risks against unknown benefits.

  5. Imaging guy says:

    Link to the NEJM article?

    1. Derek Lowe says:

      Knew I’d forgotten something – it’s in there now. Thanks!

  6. steve says:

    I think you may be forgetting the role of reimbursement. Even if FDA had approved solanezumab it probably would not have gotten reimbursement without efficacy data. However, it would have been available to those who could afford it. The question then becomes whether FDA should be the arbiter of what’s on the market or if CMS and insurance payers should decide. That’s the level at which the debate should be had. I can see validity to the argument that FDA should opine on safety and CMS/insurance payers should opine on efficacy.

    1. Imaging guy says:

      FDA approval is often linked to insurance coverage. Medicaid must cover essentially all FDA-approved drugs, and Medicare similarly has limited ability to decline to cover FDA-approved drugs. Even private insurers are generally required to cover at least some prescription drugs, although in some cases this may be on a more limited basis.
      http://blogs.harvard.edu/billofhealth/2016/11/30/your-weekly-reminder-that-fda-approval-and-insurance-coverage-are-often-linked/

    2. Derek Lowe says:

      I agree with you about the place where the real arguing begins. We’re used to a safety/efficacy world where FDA approval is the gatekeeper, and insurance payers mostly fall into line. Approve-on-safety would presumably throw that responsibility onto the insurance companies, and I doubt if they’re enthusiastic about that idea.

    3. Janex says:

      With our legal system, that would soon devolve into the courts decide as patients denied coverage for a safe but ineffective medicine would flood the court system with lawsuits.

  7. Chemistry guy says:

    Forgive me if I’ve missed the point here but why has this not been raised?:

    Even if safety can be demonstrated in a product, isn’t letting the marketplace decide essentially equal to saying “let the guys with the best marketing department decide”?

    Especially in the world of CNS disease where the placebo effect can be colossal.

    1. Home says:

      Agreed. and on top of that, there will be an even stronger link that the consumer will have to try to disassociate between Naturopathic drugs from actual drugs.

    2. Noni Mausa says:

      Indeed. “Natural,” “organic,”and naturopathic medicines get a free pass in the marketplace due to their presumed LACK of a proven effect, whether for good or ill. Interesting video here about the lack of proof of efficacy for such products in Canada. http://www.cbc.ca/news/health/health-canada-licensing-of-natural-remedies-a-joke-doctor-says-1.2992414

      To be fair, I think (though I’m not sure) that the agency has its hands tied due to the legislation governing such licensing. As the article says, “Pharmaceutical drugs require years of clinical research before they are approved for sale. But Health Canada allows natural health product manufacturers to make similar health claims based on traditional medicine or homeopathic use, instead of scientific evidence.”

  8. Peter S. Shenkin says:

    Maybe this has been raised, but what if the FDA approved for both safety and for efficacy, but separately? Medicare and insurance companies could then be expressly permitted to cover only drugs which the FDA had certified as both safe and efficacious, but individuals would be free to directly purchase drugs that had been certified only as safe.

  9. emjeff says:

    So you’re going to hold this example up as a paradigm of FDA virtue, but ignore the approval of eteplirsen based on almost no evidence?? Come on Derek, your bias is showing…

    1. Jorden says:

      It’s not like he hasn’t expressed his (negative) opinion on etplirsen before

      http://blogs.sciencemag.org/pipeline/archives/2016/09/20/sarepta-gets-an-approval-unfortunately

  10. Earl Boebert says:

    I cannot visualize running an R&D operation on any basis other than opportunity cost. Why is this even an issue?

  11. David says:

    A world where only “safe” drugs on the market doesn’t exactly enthuse. I would assume it only lays the burden on doctors. If my doctor can prescribe me medicine A or B, and prescribes A, but A doesn’t work and B does, and I lose a foot, or my child is born with HIV because my wife was given medicine A by the doctor to prevent this…I’m going to be suing that doctor the same as if that doctor had amputated my foot to cure my poor eyesight.

  12. Chris says:

    Derek- many people, myself included, are becoming doubtful that amyloid-targeting therapies are ever going to be of much use in patients with established degeneration. However, there is some interest in giving existing therapies, such as solanezumab, to patients who don’t yet have obvious symptoms (but can be identified through CSF or PET or some other metric) as a preventative measure. What do you think of this approach, as opposed to jumping ship entirely- and what do you think the companies that make these drugs think of it?

  13. Yami says:

    A problem with letting insurance companies do the drug approvals is that it’s not in their interest for sick patients to survive and cost them money. If it is this “marketplace” that were to decide which drugs are approved I would not expect drugs that slow down a terminal disease to do very well (ie a large part of oncology).

  14. Little Buddy says:

    Ah, the age old question of solanezumab. As with most things, I’d ask Micah…..

  15. DrOcto says:

    ”Pouring money and effort (and human hopes) down ratholes is never a good thing to rationalize”

    Comparison of Eli-Lilly to a rat-hole was perhaps a little strong……

    1. Mark Thorson says:

      Quite unfair to ratholes. Rats know when to stop digging.

  16. Druid says:

    Thirty years ago, I was told that in Japan, a drug could only be approved if it was very safe while efficacy was hardly important. The result was a setback for Japanese pharma from which they are still trying to recover (imports $33 billion, exports $1.2 billion), because in spite of the difficulties of registering in Japan, patients need drugs that work.

Leave a Reply to Noni Mausa Cancel reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.