Skip to main content

Clinical Trials

I Do Hate To Tell You This, But. . .

The timing of this report from the FDA is surely no accident, but it’s always a good time to think about this: the great majority of all drugs that enter clinical trials fail. They fail because they don’t do anyone any good, or because what good they might do is outweighed by some serious and unexpected harm. Around 90% of all compounds that start in the clinic never make it out. Even by the time you get to Phase III – and these are drugs that have apparently already worked in sick patients by that point – the failure rate is still nearly 40%. Drug projects fail constantly.

It’s hard, sometimes, for people who’ve worked in other industries to appreciate this. Drug development is a unique combination of very high regulatory burden and very high failure rates, so it’s temping to say that the regulations cause the failures. But that isn’t true. Biology causes the high failure rates – specifically, our lack of understanding of biology. And that’s the essential part that it’s hard for people outside the field to grasp – just how little we know. We know a lot more about how some new code will run, how a new car will corner, how a new recipe will taste, how a new building will look, or how a new movie will do on its first weekend. A lot more. Those all have their uncertainties, of course – the code has bugs, the recipe needs less salt, the new movie actually sort of tanked in the end. But those uncertainties are, believe me, tiny compared to the uncertainty of taking a drug into the clinic and giving it to people. I don’t think that there’s anything else quite like it in the modern industrial world. It’s terrifying. Once you grasp the situation, you won’t believe that we do it this way, but we do, because we don’t have any other way to do it.

That how you end up with the examples in that FDA report. You have things like bitopertin, which was a glycine reuptake inhibitor developed for schizophrenia. No one had ever tried that mechanism before – this was new science. It made it through Phase I, and in Phase II, where real schizophrenic patients first took the drug, an eight-week controlled double-blind study in 320 patients showed a real effect. On to Phase III, 1800 patients for a year, and guess what? Nothing. This is also how you end up with darapladib, which was developed for atherosclerosis as an inhibitor of liproprotein-associated phospholipase A2. That was another promising new mechanism that made a lot of biological sense, and the compound’s Phase II trials moved blood chemistry in just the right ways, as predicted. People forecast that it was going to be a blockbuster. But for atherosclerosis, you need a lot of patients in your Phase III trial, and you need to follow them for a long time. 15,000 patients over a median of almost four years of treatment later, and guess what? Nothing. No benefit. Another Phase III trial looked at 13,000 patients who had just had heart attacks, and followed them over the next two or three years looking for a benefit. Nothing. The results weren’t statistically significant, but if the trend had been just a bit stronger, then the drug-treatment group would actually have been worse than the placebo.

These aren’t musty cases from the files; both these failures took place in 2013. The FDA report linked to above has plenty more, and if you want to go further back, I (and anyone else with experience) can send you to a long and expensive list of drugs, drug classes, and whole research approaches that have been complete wipeouts, despite looking really exciting and promising at the time. These things really do happen constantly, and it’s not because we’re a little slow on the uptake around here. We in the industry would, in fact, rather not blow hundreds of millions of dollars at a time on trials of drug candidates that don’t do anything. We are (and have been) frantically trying to find ways to keep from doing that sort of thing, but what you see is the actual state of the art in 2017.

So, to sum up, most investigational drugs don’t work. “Freeing up” the system in order to give them to more people will merely allow more people (and more insurance companies) to pay for drug candidates that don’t do anything, or actually make people worse. That’s what we’ve got for you at the moment. That’s why we run those big, expensive clinical trials, the ones that just look like stupid delays imposed by bureaucrats – turns out that those trials are not only a good idea, scientifically, but it’s also ethically unacceptable to do otherwise. If you think the current system is expensive, wait until you try tearing it down.

Update: see this by Matthew Herper for more – an excellent history lesson.

56 comments on “I Do Hate To Tell You This, But. . .”

  1. David says:

    “Sounds like the drug industry needs more ALTERNATIVE FACTS!”

    – brought to you by people who don’t believe you because they want reality to be different.

  2. Isidore says:

    Thank you, this and the Matthew Harper article are very informative. Derek, I’d like to take you up on your offer, if it was indeed an offer, to send me the list of expensive drug failures, if you already have it and don’t need to spend time putting it together (you have my email). I will be giving a general talk at a college in a few months and it would be quite helpful.

    To the main issue: Like all other federal agencies the efficiency with which the FDA does business can certainly be improved. People can take the approach that efficiency improvement is a pretext for gutting the FDA, because Trump is evil and hell-bent to make all Americans sick and so there’s no point in trying to reason with him or his appointees and we should just hurl insults instead. Or they can take the approach that he does not wish to see us all dead from disease or drug poisoning and he and his appointees are willing to consider expert opinion and entertain options for improvement that do not affect the core function of the FDA. I prefer the latter approach.

    1. Emjeff says:

      Isidore; you present an approach I agree with completely. Derek is quite right, drugs are not failing because of the FDA. But, if there is anyone in the Agency that thinks things are just peachy, with both them and their processes, then that is also nonsense.

      The principal thing I’ve noticed about the FDA over the past few years, is how little effort they seem to spend in reading the briefing documents we so carefully prepare for the. There is no understanding of how long it takes to create these documents , and very frequently, during an FDA meeting it becomes quite clear that they have barely read anything. Then the meeting becomes an educational one, rather than what it should be, which is a discussion of the data at hand. From a sponsor stand-point this is extremely frustrating.

      It is also becoming clear that designations like “Breakthrough Therapy” do not translate into any meaningful improvement in FDA interactions. This is also a source of frustration for the industry.

      1. Isidore says:

        Thank you all for your suggestions.

        Emjeff, I have noticed thisalso. Over the years I have contributed portions of CMC sections and in the beginning it had been perplexing when regulatory would hand me questions from the FDA, the answers to which could be found in the document that had been submitted, if anyone had read it carefully, that is. I learned quickly not to point this out however, but write a response paraphrasing what had been there all along.

    2. Me says:

      Hey Isidore

      My gold medal for an expensive 2016 failure would be solanezumab – which would probably be a strong contender in the other years where it ran in a failed Phase III study. Also the other guys in that arena, like bapineuzumab.

      Other notable is Torcetrapib (and a few others with the same mechanism of action – Forget what it is).

      Sepsis has had a lot of failures.

      1. Anon2 says:

        Torcetrapib was a CTEP (cholesterylester transfer protein) inhibitor from Pfizer. Merck and Roche also had them in the clinic with disappointing results.

    3. worried says:

      I don’t think that anyone who has interacted with the FDA would disagree that there is room for improvement. But I disagree that everyone has the best intentions in trying to make these improvements. The current administration and friends in Congress, including unfortunately Tom Price, fundamentally do not believe in regulations and do not believe in deferring to experts such as, for example, statisticians. Really, I would like to give people the benefit of the doubt, but every sentence I read and hear tells me that “reforms” will reflect ideology, not best practice, not facts, not the best interests of patients. I am sorry to bring politics here, it should be a forum for science, but this is today’s reality.

      1. dearieme says:

        Are you inviting me to believe that the status quo is free from ideological beliefs and politics? If so, it’s a rare bird.

      2. Emjeff says:

        When someone bemoans the injection of “politics” into any issue, what they mean is “politics I don’t agree with…”

        If you seriously believe that 1)Trump is going to dismantle the FD&C Act, and 2) government employees are not doing their best to scare people away from any kind of meaningful reform within the FDA, you’re not thinking straight. We can all agree that drugs fail in Phase 3 – that does NOT mean there shouldn’t be changes in the FDA to make them more efficient and responsive. Businesses constantly evaluate themselves in this way, why does the FDA get a pass?

      3. Design Monkey says:

        There is hard to translate, wordplay saying in Russian, attributed to Murphy’s laws of soviet army:
        Schas my razberjomsja, kak sleduet, i nakazhem kogo popalo.
        meaning approximately:
        Now we will investigate this accident thoroughly, and then we will punish random people.

        This pretty much describes how revolutionary improvers propose to improve FDA

    4. loupgarous says:

      Good point – Trump can be approached. The press, in trying to shape the outcome of the last presidential election, has given us a picture of the President that’s simplistic and perhaps a purposeful self-fulfilling prophecy. It’s not like Trump’s predecessor didn’t react badly to criticism – he did, and it resulted in his last six years in office being characterized by gridlock.

      It’s imperative to educate the President that Big Pharma’s different than any other business. The CEO of a car company may think he makes life-and-death decisions – but a Big Pharma executive really does making life-and-death decisions, every day. The pharmaceutical business is fraught with risks that would appall any other businessman, risks to his company’s existence as well as the well-being of his customer. FDA’s regulations are there to manage those risks – they make the development of drugs in today’s litigious environment possible.

      It’s possible to reduce the cost of drug development and lower the cost of drugs. About half the cost of rolling out a new drug is marketing. Eliminate direct-to-patient advertising and control gratuities to prescribers, and you’ve cut a much bigger slice of the cost of new drug development than Phase III trials,

      Another upward driver of drug costs is litigation. If attorneys in medical malpractice cases had to deal with “British rules,” where the loser in (say) a poorly-founded lawsuit had to pay court costs and both sides’ legal fees, there’d be less poorly-founded lawsuits (I hate to say “frivolous,” because when someone’s injured, it’s a big deal, but the TV ads I see soliciting patients often paint a picture of easy money to would-be patients). Changing this would be the hardest sell to Congress, for the US Senate in particular is infested with attorneys, many rich from practicing medical liability law. The press would have ask some of their favorite politicians very uncomfortable questions and make sure their viewers and readers called their people in Congress to ask the same questions.

      These are much bigger slices of the cost “pie” than the cost of adequate safety trials, And our President ought to think about that.

      1. zero says:

        Trump can be approached only if you stroke his ego. His own Twitter feed demonstrates what happens if you don’t. No ‘mainstream media’ involved.

  3. Tuck says:

    It sounds like, from an investment standpoint, that we’d almost be better off going back to the serendipity model.

    Drug discovery as a moon-shot program (lots of money produces an outcome) seems to be failing…

    1. loupgarous says:

      The actual concept of Project Apollo was simple compared to developing new drugs. Build a big-ass rocket, put someone on it, and land on the Moon. Take back off. Land back on Earth. Each of those was a problem with known solutions. Costly, but in almost every case, Apollo was done because we knew how to take every step involved, and even Apollo 13 was case where we knew how to manage the failure modes.

      New drugs are new science. We literally don’t know if they’ll work the way we think they should till we try them in the wild, dimly-understood wilderness of the human body. We don’t know if off-target interactions will give the patient a heart attack (or to Pfizer’s amazing luck, let older men have erections for the first time in years). Drug development’s a thicket of risks and uncertain outcomes.

      So when anyone (not just Joe Biden, but Mark Zuckerberg and his wife the physician) talks about “moon shots” to cure cancer or (in the Zuckerbergs’ case, disease in general), they’re talking not about merely being ambitious in accomplishing large tasks in which the science is known and settled, and the problems are the engineering, but talking about discovering new concepts which we don’t now have, THEN making it so with engineering.

      Actual interplanetary travel’s for pikers compared to finding new cures for diseases that work in people and won’t make them worse off than before.

  4. Chad Irby says:

    Even the ones that do work have that scary risk of having something show up when they start doing the big trials. Vioxx was one of those. It’s a good painkiller, but the increased risk of heart attacks took it off the market (the part about how Merck hid that is left to another post). What people don’t realize is that it seems that most of the other COX-2 inhibitors and NSAIDS have similar problems, but they didn’t catch it in those old clinical trials.

    The sad part is that a LOT of the people who were taking Vioxx loved it, even to the point of wanting to sign off on the heart attack risk so they could keep taking the drug, due to its primary effectiveness and its lower gastrointestinal side effects.

    1. Me says:

      A former colleague’s auntie was one of those ‘gimme Vioxx and I’ll sign anything you put in front of me’ people. She said the NSAIDs they gave her to replace Vioxx might as well be M&Ms for all the good they do.

      1. Eric says:

        My dad was a small town internal medicine/GI doc. He said many of his patients would (despite doc’s orders) just take aspirin instead once Vioxx was off the market. His general impression was that the number of serious GI bleeds because of this was probably far greater than the number of heart attacks associated with Vioxx.

    2. tnr says:

      We need to remember that it was Merck that pulled Vioxx off the market as opposed to the FDA. I think that was a mistake. A black box warning would have been appropriate. Of course, with such a warning, Merck probably felt that it was a waste of time to market,

      I am willing to wait and see how much a new FDA head affects the agency. In my years of dealing with FDA, I never saw much difference in FDA between Dem and Rep administrations. I do think that the criteria for demonstrated evidence of efficacy should not always be statistical significance at a two-tailed 0.05 level in a randomized trial.

      1. Mark Thorson says:

        Maybe Merck should have spun Vioxx off to another company. Even with a black box warning, there should be enough customers to support a good business.

        1. Old enough to remember says:

          Same for Benoxaprofen. There was a rush among elderly patients to buy up the remaining stashes before pharmacies shipped them back/trashed them.

      2. RM says:

        To some extent that could be a failing of the “blockbuster” drug model that’s been forced on us by the high failure rate: most drugs fail and drug development is costly, therefore, the drugs that do succeed need to bring in extraordinarily large amounts of cash. – Being used clinically isn’t enough, you need to be phenomenal success in doing so, otherwise it’s not worth the effort.

        Vioxx was supposed to be a “blockbuster” for Merck. When it turned out it couldn’t be one due to the side effects, it became an albatross. Better to cut your losses and move on to the next “blockbuster”, rather than trying to eke out a middling return. Management is used to dealing with total failures and run-away successes. It doesn’t really know what to do with the “okay but not great” drugs in the middle.

        1. ab says:

          This is not quite right. It’s true that drugs that aren’t PREDICTED to be blockbusters aren’t prosecuted from the get go in big pharma (by and large). But once a drug has been approved, all of the R&D, clinical trials, manufacturing infrastructure, etc., have already been sunk, and the cost to make the drug is (practically) trivial. Therefore, even a drug that ekes out a couple hundred million a year is adding real dollars to the bottom line at that point. A drug has to be a collassal sales failure to be removed from the marked due to lack of profitability post-approval. Exubera, for example, pulled in less than $20m/yr, if I remember correctly. Vioxx, even with a black box warning, would likely have pulled in an order of magnitude more. Perhaps two orders of magnitude more.

          Vioxx was pulled from the market mostly as a result of the focused actions of one idividual at Merck, initials P.K., who thought it was ‘the right thing to do.’ I happen to think this was an extremely poor decision, as do many others. But it was not the FDA’s decision. It was Merck’s.

        2. Eric Nuxoll says:

          Merck already had factories churning out huge quantities of an approved drug with a large, established market share. I was under the impression that the cost per pill for manufacture and distribution (sans marketing blitz) was a miniscule fraction of the pill’s price. Even if market share drops by an order of magnitude, if you’re getting a 98% return on shipping pills to patients that still want them, how could you lose money (ignoring sunk costs that you won’t get back either way) on it at that point? Did they remove it voluntarily because someone from FDA told them it would be removed involuntarily otherwise (hence reference to it on this blog post)? Fear of additional lawsuit payouts if it remained on market with black box? General brand erosion fears from having a black box product remain in the public eye?

    3. Tony Fox says:

      Rofecoxib (Vioxx) was removed from a market that was general. The drug was being used as an all-purpose NSAID after dentistry, minor surgery, headache, etc., etc. The cardiovascular risk versus the general benefit was adjudged unfavourable by Merck, and the product was withdrawn (doubtless also with a view to defending even more lawsuits had they not done so). However, had the drug been marketed to those with serious, chronic and painful diseases such as arthritis, then the benefit-risk assessment could have been positive, maybe at the financial cost of not being a ‘blockbuster’, which Merck seemed to want to concentrate on at the time. There are now doubtless people being treated with steroids and opioids, or who have needed to be started earlier on those drugs than if Vioxx were still available, and the costs and dangers of those alternative treatments is usually never included in risk-benefit assessments of prodicts like Vioxx.

  5. Anon says:

    No wonder the fail rate is so high when we keep trying the same thing over and over. Look at all the failed antibodies directed against amyloid beta for Alzheimer’s disease. We have to start considering new mechanisms of actions for drugs targeting Alzheimer’s, cancer, HIV.

    1. zero says:

      That will require basic research into the mechanisms of these diseases which leads to advances in our understanding of the pathologies. This isn’t a manageable project with milestones, deadlines and a final report. It’s science; worse, it’s biology. We could throw ten billion dollars a year at these projects and still make no meaningful progress in a decade.

      Until we have sustained long-term funding for expanding our understanding of human biology, the field of possibilities for drug discovery will remain sparse and full of landmines.

  6. Andy II says:

    I do agree with some of the comments that FDA lacks of expertise inhouse to help support and make “right” judgement on the clinical data for approval/rejection. But, this report reminds me of the John Jenkins’ interview with BioCentury that this blog talked about ( recently. How many of those sponsors did seriously take the suggestions when they had an end of phase 2 study meeting with FDA?

  7. Jonathan says:

    Just to find a little bit of hope, what are some recent success stories of pharmaceuticals that started out as investigational drugs?

    1. Vaudaux says:

      Cystic fibrosis drugs from Vertex, targeting the defective CFTR molecules.

      Hepatitis C drugs from several companies.

    2. loupgarous says:

      Viagra was investigational for primary pulmonary hypertension, wasn’t it? Just had this interesting side effect….

      1. Fenichel says:

        Actually sildenafil (Viagra®) was originally developed for vanilla hypertension. It was developed for pulmonary hypertension only after it had been approved for erectile dysfunction.

  8. Anon says:

    “Around 90% of all compounds that start in the clinic never make it out…”

    And at least 75% of the 10% that *do* make it out, fail to pay for their own development, never mind all the drugs that fail on the way.

  9. Peter Kenny says:

    There is what might be termed a ‘risk asymmetry’ for FDA decisions in that the cost of letting something through that should have been stopped is much greater than the cost of stopping something that should have been approved.

    1. Barry says:

      Peter Kenney gets to a big problem; science is conducted by humans. Humans have egos and career concerns that color what gets done. The FDA would get widely blamed if a dangerous drug (e.g. thalidomide) went to market; its reputation barely suffers for blocking a drug that might helped some.
      Likewise, inside the Drug Industry, we reward the scientist who drives a drug candidate from the mouse into the Clinic, and from Phase II to Phase III. Who ever got promoted for killing a compound?

      1. Anon says:

        “Who ever got promoted for killing a compound?”

        I don’t know anyone, but I know plenty that got promoted after multiple late-stage failures that were completely foreseeable and avoidable.

        Is it any wonder then why we are where we are?

        1. Peter Kenny says:

          It’s not just the promotions, it’s also the language. The management will usually talk about “failure” when a project is put out of its (and the project team’s) misery as swiftly and mercifully as possible. Something I observed repeatedly during my pharma career was how perceptions of “compound quality” differed between those handing over a project and those charged with taking it forward. I have linked a blog post from 5 years ago as the URL for this comment.

          1. Me says:

            Yeah – I know a guy in med chem whose career was derailed for (IMO) rightly terminating a project.

          2. Anon says:

            Agreed. Instead of encouraging and rewarding success vs failure, we should be encouraging and rewarding true vs false results. After all, the ultimate fate (success or failure) of a drug is largely built into the molecule itself before it even reaches the clinic, yet largely unpredictable and out of our control. Ambigous results should be penalised.

        2. Cellbio says:

          I was! I thought we finally had a system where rewards went to clarifying the rationale for further investment into a program rather than going non-stop once a review board OK’ed a target as validated from data such as mouse KO’s.

          Silly me though, I was just a pawn in the politics of a new order ousting the old guard. Later, when tanking a program for clear failure of mechanism (a proposed antagonist was functioning as an agonist producing a chemokine response as TeGenero had seen), the new guard fiercely protected the program for reasons of career advancement. Interestingly, the highest level management clearly saw the science and rationale and killed the program in less than 5 minutes of review, as their careers were tied to any one of the myriad of programs that could advance, while the project champion and her direct line of low/mid level managers were riding this horse. The problem was not at the top but at the therapeutic area level where the ‘leaders’ were motivated to populate the pipeline. The dominate measure of success was number of development candidates rather than operational excellence for executing an overarching, well defined strategy that fits within an agreed upon timeframe for returning value. Success is hard, rational approaches not so much, and gamed systems totally predictable based upon human behavior. Serious shortage of leadership in the scientific ranks, imo.

  10. Alexander_Borodin says:

    Hints at potential regulatory reform for the FDA may come from changes introduced in Russia in recent years. I’ve come across a review on “Drug Registration in Russia and the New Law”, describing changes enacted in 2010. Timelines are described with a goal of registering new drugs in no more than 210 days. Does anyone have any knowledge or opinion of how well these changes have worked in practice, and if there may be useful lessons that could be applied back in the USA?

  11. AndyM says:

    There’s a good companion article currently trending on Forbes that’s worth a read, “Keep Politics Out of Science, Says Departing FDA Chief”:

    It’s too bad Califf only had about a year at the helm of the FDA. It’s hard to think of anyone more qualified for the position.

  12. mallam says:

    A broad change in allowing access to compounds not approved by the traditional mechanism(s) is indeed dangerous. There’s no guarantee insurance companies will guarantee such use, so the cost is likely to fall to patients. In a few cases, such as the use of novel anticancer drugs not yet in a primary cause yet has failed all other approved meds would, in my mind, should be allowed, and should be required to be paid for by the drug company, insurance, or split between the two. Of course, the drug company could use the information for their own benefit, and the patient could see longer / better life if it would work.

    1. tangent says:

      Right, that’s how this is going to shake out, unless it’s regulated otherwise. Insurance will tier into less-expensive-premium Old Reliable Medicine plans and more-expensive-premium Your Body Your Choice plans, depending on whether they charge 50% copay for drugs of poorly-known value, or 10% copay.

      That looks to me like it could be a death spiral for the expensive-premium plan, since people who are willing to pay 10% for one questionable drug are probably willing to do it for a dozen other. Every all-you-can-eat expensive healthcare plan has some of this, but more so here — there will be a questionable-Cadillac drug positioned for every indication, and why wouldn’t you choose it? (If you’re this type of person who likes to dream on new drugs in the first place.)

  13. An Old Chemist says:

    Derek or anyone else, does any such data exist for the biologic drugs including peptide drugs? I mean what percentage of biologic drugs and peptides make it to market after entering phase-I? I guess that the chances of success should be much higher than for the small molecule drugs.

    1. Eric says:

      It does exist for antibodies, Tufts did a review a few years back. If I recall correctly it’s about a 1 in 5 chance of success for antibodies. So the attrition is worse for small molecules but it’s not as though antibodies are easy either.

  14. Design Monkey says:

    >Update: see this by Matthew Herper for more – an excellent history lesson.

    What the fukk? Censorship like in North Korea already? Link source nonexistant.

  15. Dan T. says:

    Areas where it might be possible to reduce the regulatory burden in ways that produce more good than harm might include expediting approval of drugs already approved and in use in other countries, and in allowing additional generic manufacturers of already-approved drugs (the high regulation level on this is what has caused some profiteers to be able to jack up the price greatly on some drugs that aren’t even under patent).

  16. Paul says:

    Well, if we start on the expensiveness of the system…
    You actually are paying clinicians mostly, who stuff their pocket for participating in the trials. And in the end, taxpayers and patients have to pay for these expenses.

  17. Mike Rea says:

    One thing to bear in mind: those pIII failures cited were where they took a scientific proof of principle (proof of mechanism/ scientific markers) and then did something different in pIII… Failure is sometimes unpredictable, but it helps when you don’t make it easier…

Comments are closed.