Skip to Content

Now This Is A Drug That Does Not Work

I’ve already written about how Eli Lilly’s inhibitor of cholesteryl ester transfer protein (CETP) did not work in the clinic. Now that the data from their failed trial have been published in the NEJM, though, it’s worth taking a look at a few graphs (first pointed out to me on Twitter by Sek Kathiresan. Shown are the effects of evacetrapib on HDL and LDL, and he’s right: if you’d seen these graphs ten years ago, you’d have said that you’re looking at a cardiovascular wonder drug for sure. LDL down 30%, HDL up 125% – what else do you want?

Well, you want actual cardiovascular outcomes. And that’s where this graph comes in. 12,000 total patients for over two years, and this is what you get: nothing. You lose – good day, sir! Absolutely no change in CV events whatsoever. There is something that we don’t understand here, about the various effects of CETP inhibition, or the various types of HDL and LDL and their downstream effects, or about the influence of these two on cardiovascular health in general. CETP inhibitors have been a total wipeout for every single company that has tried to develop a drug in this area. Merck has not yet released data on their own compound in this area, anacetrapib, but let’s put it this way: if it works, it’ll be one of the major medical mysteries of our time.

And I think I’ll add this every time we have a big drug trial wipe out: keep in mind, that under a safety-only regulatory regime, that people would have been taking this drug (and paying for it, one way or another) for the last few years now, and it would have done them no good at all. When they could have been actually taking something that might help them. We do not know enough about investigational drugs to approve them without efficacy data.

76 comments on “Now This Is A Drug That Does Not Work”

  1. David says:

    Pardon my ignorance, but why would Lipitor/Atorvastatin be effective vis a vis CV outcomes, and this not?

    Or is that the trillion-dollar-question?

    1. NHR_GUY says:

      Some suggest it is the pleiotropic effects of the drug. This is likely true though it is not well understood if it is associated with CRP lowering or an overall anti-inflammatory effect of the drug. Pharmacologically, the LDL lowering seems to be just a bonus.

  2. Peter S. Shenkin says:

    Bear in mind that high LDL is a marker, not a cause, of arteriosclerosis. And though statins lower LDL, that might not be their main mode of action; even heart attack victims who are immediately given statins for the first time in the hospital show a significantly greater rate of survival than those who are not given statins. (According to my cardiologist, the hypothesis is that statins modify the surface of the plaque quickly in a manner that increases blood flow.) Calcium score, which is believed to measure, rather than merely correlate with, actual plaque buildup, does not decrease even with long-term statin administration; yet those who get statins long term have a lower incidence of heart attacks than those who do not, even if they have high calcium scores.

    So, given all this, I’m not surprised to find that raising HDL doesn’t help. I’m guessing that HDL is just an out-of-band predictor of heart-attack health, as LDL is an out-of-band predictor of heart-attack risk.

    The reason that intrinsically high HDL correlates with health remains unknown, just as the reason that intrinsically high LDL correlates with risk remains unknown. The latter doesn’t imply the former; but maybe the result shouldn’t be a complete surprise.

    1. GladToMoveToProcess says:

      My vascular guy agrees, saying that statins somehow stabilize the existing plaque.

    2. Hal says:

      Very interesting comments. Along similar lines, I’d be curious to know if you’d say the same about C-Reactive Protein (that it’s indicator not cause)? I always assumed it was viewed as a downstream indicator of inflammation and thus risk of unstable plaque, but then I heard that some companies are making it a drug development target.

  3. Mike says:

    I’d love to see someone add up all the money spent on CETP’s (Merck, Lily, Pfizer, others?). I remember hearing that Pfizer built an entire spray-dry dispersion facility in preparation for torcetrapib’s launch. Well into the $100’s of millions of dollars. All for naught when the safety signals killed that drug (maybe they repurposed?).

    It might give the public a sense as to how risky and expensive drug development is.

    1. Nobody in particular says:

      Hindsight is always 20/20, but it would be interesting if one could somehow compare the decision making on this target between companies that felt they had a lipid/CV “franchise” to feed/defend (e.g. Merck and Pfizer) vs those who did not.

  4. SteveM says:

    I’ve read the updated theory that CV event probability is actually a function of concentrations of specific sub-types of LDL. Large, fluffy particles, (LDL-A) are essentially benign, while it is the smaller sub-types that induce atherosclerosis.

    I see in this study that total LDL was the measured statistic, not the sub-type concentrations. If evacetrapib selectively reduces LDL-A leaving LDL-B concentrations relatively unaffected AND the influence of higher HDL concentration has little effect on CHD in the presence of high LDL-B then the outcome results would make sense.

    So the interesting supplementary analysis to see would be plots of LDL-A and LDL-B rather than just total LDL.

    Incidentally, another theory being advanced is that dietary saturated fat is converted primarily into LDL-A. While it is dietary carbs that are selectively converted into LDL-B. Which if true, means the nutrition advice space could be turned upside down.

    1. Derek Lowe says:

      That’s just the sort of thing – if these are viable theories (and why not?), it shows how little we understand human lipid handling.

  5. Yvar says:

    Asking as a medical consumer and not as a professional, does this mean that there is no point in measuring LDL and HDL levels for patients? Those graphs could not be clearer in showing no predictive effect of total LDL and HDL unless I’m missing something.

    1. Eric says:

      No, it doesn’t mean that. An individual’s baseline LDL and HDL are predictive of CV risk. This has been demonstrated many times. So measuring baseline levels does help a physician assess your relative risk.

      It does not mean, however, that if you give a drug and alter LDL or HDL that you know for sure what the outcome will be (as was nicely demonstrated in this study!). Long term CV outcomes trials are needed for any new therapy. Statins have shown to be beneficial in these type of studies. CETP inhibitors – not so much.

      1. SteveM says:

        From what I’ve read the recent studies in alternative diets makes a lipid baseline a moving target. Low Carb/High Fat diet proponents point to studies in which the standard lipid response to the diet is increased HDL, decreased Triglycerides with LDL levels remaining relatively constant. As I noted above, if the LDL-A to LDL-B ratio increases even with a constant LDL-C, that would be considered a positive response. So the diet would provide significant positive changes in a lipid panel without medication.

        LCHF proponents do note that a small percentage of subjects (1-2%) are genetically disposed to hypercholesterolemia so an LCHF diet would not be appropriate for them. However they also note that those exceptions should not negate the general guidance.

        The above begs the question about what should be the target population for statin intervention? And by implication, lipid interventions targeting alternative mechanisms like CETP. If diet modification works, why use statins on asymptomatic patients? Moreover, could current CV pharmaceutical research be upended by the updated insights of the effects of diet yielding a much smaller suggested target population?

        A March 2017 entry in the Lancet (“Lessons from the controversy over statins”) suggests that this controversy is far from over. I imagine the politics and the money are even messier than the science.

        BTW, the anecdotal reports of obese Diabetes Type 2 response to an LCHF diet are pretty remarkable. I.e., along with significant weight loss, a high reduction or elimination entirely of the need for insulin with lipid panels converging to normal ranges within weeks.

  6. Erebus says:

    Many if not most of the drugs in use today, from aspirin to pennicilin to corticosteroids, were developed well before the institution of efficacy testing.

    If your view is that it’s better to let 1000 patients die of neglect than have one die of an adverse drug reaction, then spending 10 years and a few billion dollars on efficacy testing makes sense. If you feel that it’s better to let 1000 patients die of neglect than have a few waste their money on useless new drugs, then the modern efficacy testing paradigm makes sense. Otherwise… Well…

    I’d say that if we’re actually interested in fighting a war on cancer and heart disease, we’ll need to act like we’re in a war: We’ll need to develop our weapons in a responsive way, with the tightest possible feedback loops, and we may need to engage in some polypharmacy. We can’t do this under the thumb of the excessively timid FDA. Only with sufficient institutional courage — the courage to let scientists do their damn jobs, so that they can wage a real war on cancer and other diseases — can things be done faster, cheaper, and [much] more efficiently.

    There are very few good arguments against a safety-testing-only protocol.

    Adverse drug reactions? Post-marketing surveillance. Besides, many, if not most, of these adverse effects will be discovered during initial safety testing.

    Lack of efficacy? Post-marketing surveillance. It can easily do a better job than the FDA’s excessive, indeed onerous, process.

    1. Hap says:

      The problem is, that the few likely to buy drugs that don’t work are more likely on the order of the lives cost by the available drugs found that do work, maybe more. At that point, a significant fraction of drug spending would be going to things that don’t work, which costs lives (because they aren’t using something that works) and money (which of course can’t be spent on finding something that does work). Add that trials will likely be much more difficult and expensive (to sort out the lots of patient subtypes under a free-for-all) and you are likely to get even fewer effective drugs. Fewer effective drugs, more difficulty for everyone in finding them, and the large moral hazard involved [why do you think with a reduced barrier to entry that you will have fewer pump-and-dump schemes masquerading as small pharmas, with the added available cash flow of selling drugs that are unlikely to work (because they weren’t chosen to work, but to sell)?)] make safety-only a bad idea.

      Also note: early drugs were likely going after indications with no treatments and had enough effects that they could be distinguished from ineffective ones without needing computational power that didn’t exist, and we don’t see the ones that failed (and the people that died taking them).

      1. Erebus says:

        >”At that point, a significant fraction of drug spending would be going to things that don’t work, which costs lives (because they aren’t using something that works) and money (which of course can’t be spent on finding something that does work). Add that trials will likely be much more difficult and expensive (to sort out the lots of patient subtypes under a free-for-all) and you are likely to get even fewer effective drugs.”

        I couldn’t parse your first sentence, but the quoted section above seems like a trivially, self-evidently wrong argument.

        Let’s count the ways:

        1. Insurance companies don’t need to reimburse for, or purchase, drugs that don’t meet their own efficacy requirements, or their cost:efficacy ratio requirements, or whatever other requirement metrics they choose to evaluate the utility of drugs…

        1.1 Most prescription drug spending in the USA is via insurance companies or Medicare/Medicaid.

        1.2 Even mid-sized insurance companies are perfectly capable of performing their own post-marketing surveillance, and, on long enough timescales, they are capable of evaluating the efficacy of drugs.

        2. Nobody is forcing doctors to prescribe drugs that are new and unproven.

        2.1 Most new drugs are only available with a prescription.

        2.2 Doctors, when prescribing drugs, are very frequently informed by guidelines and “best practices.” In oncology, much is standardized. Drugs of no efficacy are going to be a hard sell to national medical groups like the AMA.

        3. There was an era before mandated efficacy testing. Oddly enough, it was called “The Golden Age of Drug Development.”

        3.1 The low-hanging fruit argument is overplayed. Acetaminophen, for instance, wouldn’t clear pre-clinical these days, to say nothing of efficacy tests vs. the “standard of care”!

        3.2 The period from the 1940s through the mid 1960s was, assuredly, not characterized by useless drugs. If you don’t believe me, how about a wager: For every useless drug you can name, I’ll come up with five useful ones. (“Useless” here meaning zero or negative efficacy.)

        3.3 The period from the 1940s through the mid 1960s was not characterized by expensive clinical testing. Hah. Very much to the contrary.

        4. Nothing in your argument supports the assertion that we’d get fewer effective drugs on a safety-only testing regimen. To the contrary, if development is made hundreds of millions of dollars (if not billions) cheaper and many years faster, we’re certain to get more drugs, better drugs, and better polypharmacy. This should go without saying.

        Of course, there’ll be useless drugs, and there’ll be harmful drugs, and there’ll certainly be harmful interactions. But all things being equal, I’ll take the risk of side effects. As a society, we err far to heavily on the side of caution, to the extent that we are becoming ridiculous in our timidity and aversion to risk.

        1. David Antonini says:

          I doubt it, you’re more likely, in my opinion, to get better marketing of ineffective drugs. Y’know, like Thalidomide.

          1. Erebus says:

            “But… but… Thalidomide!” is the kneejerk reaction to every suggestion that the FDA roll back its over-regulation of the industry.

            Nevermind that Thalidomide was not approved for use in the USA in 1960, due to the fact that there was insufficient safety data. In other words, the pre-1968 regulations, which did not require efficacy testing, worked just fine.

            That aside, thalidomide is, emphatically, not “ineffective.”

      2. Erebus says:

        Another thing:

        >”which costs lives (because they aren’t using something that works) and money (which of course can’t be spent on finding something that does work).”

        Both of these arguments are incorrect for reasons not touched on above.

        For one thing, for many indications there isn’t anything that works. Hence the heartache — and the institutional hypocrisy — associated with Sarepta’s recent Duchenne muscular dystrophy drug approval case. If drug development is made faster and cheaper by orders of magnitude, surely that would help those people suffering from presently-untreatable diseases? That it would do a better job than the current paradigm, where those patients are essentially dying of neglect, should go without saying.

        For indications where there are drugs that work, your point is almost entirely moot.

        And your second argument makes no sense. Think about it for a moment. Let’s say I have a child with Duchenne muscular dystrophy. I pay Drug Company A a sum of money for a drug of questionable efficacy. How does that imply that my money “can’t [or won’t] be spent on finding something that does work?” If anything, it stands to reason that the sum I paid Drug Company A will fund further drug development, to at least some extent. What’s the alternative? Doing nothing, and spending that money on a fancy coffin? Who wins in that scenario?

        1. Hap says:

          You keep saying “postmarket analysis” like it’s some sort of magic wand that takes crap and turns it into analyzable results, Without being able to standardize on any of the many variable involved, how will people be able to tell what works and what doesn’t? You don’t seem to like the “low-hanging fruit” argument, but that doesn’t make it go away. It’s a whole lot easier to find drugs with effects big enough to notice above the noise when you have your choice of diseases and drugs, and the effect is big enough not to require much statistics to render.

          The problem on spending money without trials is that you don’ t know that company A is developing a drug for something – they have something but no one that goes after you will have any idea if it actually works. You aren’t developing a drug in that case, and given that everyone under your system will be in that spot, you won’t have drugs for anyone, except you will be spending lots of money on…something. If the effect of the drug is large enough to notice, then the trials will not need to be sufficiently large to be onerous. Most of the drugs for which “right-to-try” has been an issue have had effects small enough to be difficult to find, and those are unlikely to be curative.

          All of those insurance post-market studies, and funding for efficacy-only “drugs”…where does the money come from to do this stuff? Insurance companies aren’t going to give up profit for it – they will charge someone more for insurance or pay less for drugs or both. The former means some people won’t be able to get any drugs, most likely, while the latter means that people will develop (or produce) fewer drugs because they aren’t likely to get paid as much when they do find one. (The latter would be worse because you might actually know they work and either not get them or have the “T%^&%g” effect take over.) If individuals pay for them, then that money comes from somewhere, too, and doesn’t do other things.

          If you can’t tell that drugs work, then you don’t have drugs. Post-market analysis as a sole determinant of efficacy is a recipe to get lots of crap that doesn’t work – instead of having the chance of finding things that work, you have lots of things that you don’t know work (and, hence have no idea for which the risks are tolerable – if you don’t know the expected reward, any risk is bad. Dying more quickly is still a loss, even if you’re going to die anyway.). At that point, you have…the supplement market. That’ll work out well.

    2. AC says:

      Do you have references to support your claims of:
      “Many if not most of the drugs in use today” (I’m ignoring your weasel-word of “many”)
      “let 1000 patients die of neglect than have one die of an adverse drug reaction” (do these numbers come from somewhere that isn’t your ass?)
      “can things be done faster, cheaper, and [much] more efficiently”
      “Besides, many, if not most, of these adverse effects will be discovered during initial safety testing.” (ignoring weaseling again)
      “Post-marketing surveillance. It can easily do a better job”

      Also, have you ever heard of opportunity cost?

      1. Erebus says:

        Do you actually have an argument, or are you just sniping from the sidelines? In any case, I’m not about to waste my time composing a detailed response to somebody so obviously lacking in common courtesy.

        1. AC says:

          The burden of proof lies with the person making the claim. I try not to waste my time arguing against points that don’t have basis in reality.

          On the other hand, I am willing to listen to well-reasoned arguments that actually provide sources for the claims they make. In fact, I would be happy to be informed about examples that support your points (e.g. countries that only require safety for approval develop effective drugs faster than those that don’t).

          1. Erebus says:

            Here’s a tip: Rhetorical devices like “let 1000 patients die of neglect than have one die of an adverse drug reaction” typically don’t need citations.

            And another one: The USA of 1955 developed safe, effective drugs a hell of a lot faster and cheaper than the USA of 2015. The tools weren’t better back then, nor were the people any smarter, and the “low hanging fruit” argument is partially if not entirely spurious. Yet it seems to be the case that The Golden Age ended when mandated efficacy testing began.

            Drug development has gone the way of NASA. What was possible 60 years ago isn’t possible today. We’re clearly doing something wrong, wouldn’t you say?

          2. David says:

            Yes, Erebus, we’re doing something different. Armstrong rated his chances of returning from the moon as about 50/50. Are you suggesting we accept survival risks on this magnitude for everything? Because I’m sure you wouldn’t for a car, 50/50 chance you don’t make it home tonight. Got a cold? I can stop your nose from dripping, but it’s 50/50 it falls off.
            That tetraethyl lead that fixes your engine, boosts its performance? I’ve got a cupful of it for you. Don’t worry if you spill it on your hands, it’s fine, and it works!

          3. AC says:

            Erebus:
            First, thanks for replying.

            I called out your use of rhetorical devices and what I called “weasel words” because there is often no point in arguing against them. They are used to be persuasive without actually forming a specific argument. I’ll apologize for treating your use of them as disingenuous because I know that you meant to raise the point that there is a cost to being cautious and that wasting money on ineffective drugs isn’t necessarily a good enough reason to be cautious.

            These are all questions that we should be considering for this issue:
            How many patients are suffering due to the obstacles of expensive efficacy trials? How much of an obstacle would ineffective drugs that benefit from the first-mover advantage be? How much of an effect would marketing (to doctors, patients, and insurance companies/policy makers) have on which drugs are used? How are ineffective drugs removed from the market? How much money is a person’s life worth?

            There are a lot of differences from the USA of 1955 and the present day. If you could show that the requirement of efficacy testing actually caused the end of “The Golden Age”, then that would be a very strong point in favor of removing the requirement. I will not assume it is true without evidence, just as you do not with the “low hanging fruit” explanation.

    3. kyosce says:

      Erebus, I think you miss the point of drug testing. Our current regulatory framework is such that a seller may not make a health benefit claim for their drug without specific evidence the claim is true, validated by FDA approval. The value to the consumer of a drug is based in this health benefit.

      In a “safety testing only” approval process, would you give sellers the right to make a claim for any health benefit they choose? Or legislate they could make no benefit claim at all? Either way the transactional value is destroyed.

      I also consider “Many if not most of the drugs in use today, from aspirin to penicillin to corticosteroids, were developed well before the institution of efficacy testing.” a specious argument. These drugs were efficacy tested by trial and error. Institutions don’t use this method today because it has higher risks and is more inefficient than clinical trials.

      1. Erebus says:

        So penicillin couldn’t be sold as an antibiotic in 1960, because the FDA didn’t spend ten years measuring its efficacy as an antibiotic? Surely you realize how wrong-headed that is. And, moreover, how many millions such a policy would have condemned to death. (And in such ways is institutional inertia taken to grotesque depths…)

        Surely you realize that there was no such thing as mandated efficacy testing until 1968?

        As for this:

        >”These drugs were efficacy tested by trial and error. Institutions don’t use this method today because it has higher risks and is more inefficient than clinical trials.”

        No. They don’t use that method today because it is illegal. It is orders of magnitude more efficient than the way the FDA mandates clinical trials be run. On this point the evidence is certain. Compare drug development 60 years ago to drug development today, in both time to completion and cost. Again, drug development has become NASA: What was possible 60 years ago is no longer possible today.

        1. Derek Lowe says:

          What was possible 60 years ago is, in part, low-hanging fruit that has already been picked. We will not suddenly come up with penicillin and tetracycline again, or their equivalent, just by lowering efficacy requirements in the clinic.

        2. kyosce says:

          “So penicillin couldn’t be sold as an antibiotic in 1960, because the FDA didn’t spend ten years measuring its efficacy as an antibiotic? Surely you realize how wrong-headed that is. And, moreover, how many millions such a policy would have condemned to death. (And in such ways is institutional inertia taken to grotesque depths…)”

          Penicillin was made widely available to the general public in 1945, due a heroic cooperative effort by the government and pharma to solve the production challenge. The 1945 Penicillin Amendment required FDA testing and certification of safety and effectiveness of all penicillin products.

          “Surely you realize that there was no such thing as mandated efficacy testing until 1968?”

          Efficacy testing of pharmaceuticals generally was mandated by the Kefauver-Harris Drug Amendments in 1962. Perhaps you are referring to the 1968 formation of the Drug Efficacy Study Implementation (DESI) by the FDA to investigate the effectiveness of drugs first marketed between 1938 and 1962 and approved on a safety only basis?

          “As for this:
          >”These drugs were efficacy tested by trial and error. Institutions don’t use this method today because it has higher risks and is more inefficient than clinical trials.”
          No. They don’t use that method today because it is illegal. It is orders of magnitude more efficient than the way the FDA mandates clinical trials be run. On this point the evidence is certain. Compare drug development 60 years ago to drug development today, in both time to completion and cost. Again, drug development has become NASA: What was possible 60 years ago is no longer possible today.”

          Trial and error has higher risk and is less efficient if both the seller and the patient population are included in the assessment. Yes, it is less so from the sellers side, which is why it is illegal.

          1. Erebus says:

            >”Efficacy testing of pharmaceuticals generally was mandated by the Kefauver-Harris Drug Amendments in 1962. Perhaps you are referring to the 1968 formation of the Drug Efficacy Study Implementation (DESI) by the FDA to investigate the effectiveness of drugs first marketed between 1938 and 1962 and approved on a safety only basis?”

            Sorry, you’re right. 1962, not 1968. My mistake.

            However, that does nothing to counter my point. Efficacy testing is a product of the 60s, and many of the drugs on the market today predate it. Many of those, it is often said, wouldn’t have a chance in hell of reaching the market under the current protocol.

            >”Trial and error has higher risk and is less efficient if both the seller and the patient population are included in the assessment. Yes, it is less so from the sellers side, which is why it is illegal.”

            This is complete nonsense. “Trial and error” is how everything is done. The current method is “trial and error” — it’s just that the trials are super slow and ultra expensive. Generally, the faster the trial-and-error cycle, the faster the rate of technological progress. Look at the engineering advances we’ve made over the course of WWII. Now THAT was a war. We’re not fighting the “War on Cancer” like it’s a war.

            There are, assuredly, risks associated with moving faster in drug development — but only in certain cases. There are a huge subset of cases where the increased risk is totally negligible. There’s an smaller subset of cases where the current paradigm is clearly more harmful than a safety-only expedited approval process would be. (For e.g., terminal diseases with no good treatments.)

    4. tangent says:

      Would you set up your post-marketing surveillance programs for the existing safety-only drug market in the U.S. so we can see how it operates? I mean the ‘supplements’ industry of course.

      Some of them work, to some extent, probably. Let’s see how effective it is to tell which. And then the acid test: let’s see whether that affects sales.

  7. mb says:

    Do insurance companies pay for every approved drug? How about medicaid, medicare, VA, etc.? Of course not. Rendering your closing argument a red herring at best. Add in off label use, yes all of the above approve and disapprove off label use, and it really is a bad argument.

  8. James says:

    It’s semantics, but I’d argue from the top graphs that the drug works great – hits the target and delivers the predicted PD biomarkers exactly as advertised. Where the problem comes in is that the underlying hypothesis (lower LDL+raise HDL=reduced CV risk) was what crashed and burned in a giant pile of cash…

    1. Mol Biologist says:

      Yes, James. It is a sad true. Interconnected chemical reactions are teamwork, where one reaction gives an appreciable yield of desired product only under conditions when another reaction takes place. Such a mutual chemical interaction is called chemical induction. Lipitor is capable for mutual effect and induction but the effects of evacetrapib on HDL and LDL I has different scenario, the induction of a second chemical reaction does not occur, so a therapeutic effect is not possible. https://en.wikipedia.org/wiki/Metabolic_pathway

  9. Kelvin Stott says:

    I think it’s worth saying: Congrats to the development team for getting such a clear and conclusive result. Good science indeed, much better than an ambiguous result that raises false hope and leads to throwing more good money after bad.

    If only more trials were run this well, R&D productivity would be much higher.

  10. flem says:

    what might make sense is to allow on the market drugs with demonstrated safety but uncertain hard outcomes but at a low price point (to reimburse manufacturing costs only). If the positive outcomes materialize in the future allow market pricing (to reward R&D). I don’t necessarily think CETP would qualify given availability of statins, but in other high unmet need areas this arrangement could be win-win

  11. bacillus says:

    If the stakes were sufficiently high, I’d argue for approval based on efficacy with post-marketing for safety. To repeat, if the stakes were sufficiently high. An historical example is the smallpox vaccine which completely eradicated an extremely lethal infectious disease from the planet. It never underwent a formal large-scale clinical trial or regulatory approval, and sometimes induced severe, sometimes lethal, adverse reactions. Nevertheless, the benefit far outweighed the risk. Not sure that’s the case with many other medicines.

    1. Anon says:

      On that basis you may as well approve sand for marketing as as an oral drug for cancer, as it’s perfectly safe to eat in small quantities and completely ineffective. Except that you’d only be encouraging people to take that instead of something else that might actually be effective.

      In other words, what a completely stupid and dangerously unethical idea.

      1. Erebus says:

        You’re dangerously ignorant of history, or are simply being disingenuous, or both. Why do you think sand wasn’t approved as an oral drug for cancer back in 1955 — when there was absolutely no efficacy testing requirement?

        1. Anon says:

          Don’t be stupid.

  12. Peter S. Shenkin says:

    @erebus I don’t see what your arguments (which have good counterarguments, for the most part) have to do with this particular blog posting.

    You might believe that the efficacy testing for CETP inhibitors was a waste of time; on the other hand, now that the testing has been done, hopefully you don’t think that anyone in his right mind would pay good money (or even bad) to take a drug that has shown to be INeffective? (Or maybe you do think they would….)

    I will say that aside from the other questions involving efficacy testing as policy, a failed efficacy test such as this one gives demonstrates a likely research dead end. A failed efficacy test falsifies an etiological hypothesis. A successful efficacy test merely supports an etiological hypothesis, but does not prove it, as the efficacy of statins demostrates.

    1. Erebus says:

      The counterarguments I’ve seen have not been persuasive. Perhaps you could enlighten me?

      Besides, I’m not even sure that we can say that this drug is ineffective. You (and Derek, and the FDA,) are asserting here that the failure to prove a positive is equivalent to proving a negative, which is a fallacy.

      We know that this CETP inhibitor lowers LDL and raises HDL. We know that, taken on its own, it doesn’t improve outcomes. (Measured in terms of mortality.) We don’t know anything else at the present time. What if, in combination with a beta-blocker, or high dose niacin, or something else, it improved outcomes very significantly? What if the combination of this CETP inhibitor with, for e.g., niacin, outperforms statins? We’ll never know, now will we?

      There are, I think, a number of very strong arguments for the broader use of investigational drugs. If they’re “proven” safe — that is, if they’re determined to be reasonably non-toxic — there’s very little harm in it, and there’s much we can potentially gain. And this is especially true if it pushes drug development to become faster, cheaper, and less constrained.

      1. Derek Lowe says:

        The clinical trial, though, was specifically designed to prove that positive, and it did not – thus the statement that the trial failed. You’re correct that we don’t know what this CETP inhibitor might do when combined with other agents, but if you’re going to go that far afield, how do you know where to stop? Lilly’s Alzheimer’s antibody also failed completely, but how do we know that it wouldn’t work if we gave it with. . .(name a drug). There is not enough time and money in the world to investigate all combinations of everything, and if you’re going to have to prune back your clinical investigations from that level (which has to be done), the logical place to begin is to cross off things that have shown no trace of benefit at the beginning. There has to be a better rationale for going into the clinic than “Well, no one has shown that it won’t work”, which is what a CETP/niacin trial (for example) would come down to.

        I’m still puzzled that the idea of trying all these things out will make drug development faster and cheaper. . .

        1. Erebus says:

          My position is essentially that the current method of drug development and regulatory approval has proven itself to be a failure. Moreover, that the old method was objectively better.

          Eli Lilly has two patents which describe Evacetrapib: US7786108 and US8299060. The first dates to 2004, the second to 2009. It’s safe to say, therefore, that the drug is around 10 years old, if not older than that.

          Evacetrapib passed its safety checks many years ago. It has been in Phase II since at least April 2011. See, for instance:
          http://web.archive.org/web/20110513122549/http://lilly.com/pdf/Pipeline_Slide.pdf
          (I believe, but am not certain, that it entered phase II in mid-2010.)

          So here’s a drug that is a decade old, which has been stuck in regulatory limbo for most of its life, and which we know almost nothing about. It is a mystery.

          Let’s assume things are done differently. Let’s assume that it was introduced to the market in 2010. By now it would have been used by hundreds of thousands if not millions of people, and we’d have an enormous trove of past-market data that could be examined by [a handful of] epidemiologists. (Cheaply, I might add, especially compared to the current method of performing clinical trials.) Most plausible interactions — like, for e.g., the one with niacin — could have been determined to P < 0.05 a long time ago. Also, the drug's standalone efficacy — or lack thereof, as the case may be — would be clearly demonstrated.

          This point becomes far more forceful where intractable diseases, like certain forms of cancer, and diseases for which there are no good treatments, like Duchenne, are concerned. Do you feel that we're doing a good job for those particular patients? To me it feels as though we are failing them, utterly. We're literally leaving them to die of neglect, when we aren't killing them with double-blinded placebos. And this is because of a regulatory attitude which can be summed up: "Better 1000 deaths from disease than one death from bad medicine!"

          In any case, it can certainly be said that one of the many failings of the current paradigm is that experiments in polypharmacy are excessively difficult to run, when they're not illegal outright.

          Of course, if John Doe has terminal cancer, takes 12 investigational drugs, and gets better, then it's impossible to know which of the drugs worked, or whether the result is due to some sort of combination effect. Surely there will be some drugs that didn't help, and we won't be able to determine what those are. But we will still know far more than we did before Mr. Doe started taking those drugs, which is that something (!!) worked. That's a start. Here's a question for you: Is that less ethical than having Mr. Doe enroll in a clinical trial, where he may be given a placebo, or the inferior "standard of care" of years past? Note that both of those are sure to kill him.

          We need tight feedback loops, faster drug development, and cheaper drug development. There's one way to get there, and that's to get rid of the efficacy testing requirements. It has worked for us before. We can go back to clinical judgement, case reports, observational studies, and exhaustive post-market analysis. This is not unethical — it is ethically superior. It is not unsafe — it would clearly benefit patients like Mr. Doe, just as it would benefit future generations who have access to more and better drugs.

          1. Skerdi Haviari says:

            “and we’d have an enormous trove of past-market data that could be examined by [a handful of] epidemiologists. (Cheaply, I might add, especially compared to the current method of performing clinical trials.)”

            This is the axiom of your argument, but as an MD with experience in both epidemiology and clinical trials, I would beg to differ massively.

            If you speak from society’s perspective, someone will have to pay for someone doing the clinical and epidemiological work, whether it is a trial or real-world data. Trials are more expensive because you pay a lot of smart people to get very high quality data ; in exchange, you get much faster and clearer answers. You do have some friction linked to financing and corporate dick-stuck-in-fan, but I’d rather have the FDA run all phase III trials based on a grant system to correct this than remove this requirement.

            In addition, when talking about penicillin and the rest you speak in hindsight about drugs with stellar efficacy on previously fatal diseases. If you have a drug with stellar efficacy (say, that cures cancer instead of prolonging survival by 6 weeks…), you will blast through current efficacy clinical trials (often not even needing a phase III), especially with interim analyses. Shouldn’t it make you wonder that the only ones whining about efficacy trials are developers of drugs with really mild clinical improvements, and that’s saying it politely ?

            The problem of hindsight here is also that it tends to ignore the impact of a market filled with snake oils, and to forget how many of them there were. The very funny, if sad, history of snake oil (and where the expression comes from) gives you an idea about, to put it bluntly, how stupid markets are with respect to health.

            Finally, as someone with training in clinical methodology and biostatistics, I could have a blast gaming a system based on epidemiological post-marketing efficacy analysis (basically lining my pockets litterally scamming people to death), chiefly by carefully hidden selection bias. With such a system you either expose yourself to a continuous litany of scandals, courtesy of the Shkrelis of the world, or you will have to pay an army of epidemiologists and criminal investigators that will end up being more expensive than running trials.

            This being said, let me finish on a constructive note. I do agree that the current system may slow down experimentation, but in my experience this is more due to pharmaceutical companies’ obsessive compulsive behavior with their pipeline. In response, while I would not advocate having real-world data replace trials, I can envision a system where trials replace real-world. By this I mean that any institution running a phase II or III trial should pre-publish their protocol and be obligated to accept any patient that their (possibly certified) physician wants to include, and more importantly, that companies with experimental products should be obligated to provide their molecules to clinical investigators that want to start a trial, provided the regulator clears the design of said trial. Having had to navigate the ridiculous, 8-different-committees process of Big Pharma for investigator-initiated trials, I can guarantee you that this would massively speed up progress especially when it comes to combinations.

      2. Anon says:

        Here’s a good counter-argument:

        If I was a shareholder or senior executive of your company, I’d fire you on the spot for suggesting such stupid unethical ideas that would burn money and most likely harm patients. And also for being an argumentative twat.

        1. Erebus says:

          What’s unethical about my position?

          I can give you a long list of things that are unethical about the way clinical trials are now run, from the possibility that participants are harmed by receiving a placebo instead of an active treatment. Also the deception involved. Many other things.

          I suppose you think that unethical people will take advantage of a safety-only drug approval protocol. I would suggest that you go back and evaluate levels of public trust in the pharmaceutical industry in the 40s and 50s. Protip: It was far higher back then. George W. Merck — who famously uttered the words “medicine is for the people, not the profits” — was one of the most admired men in America. Today, pharma execs are reviled, not admired.

          1. Anon says:

            It’s not unethical to enrol people in a trial when you and they both know there are risks, but it IS unethical to approve (and sell) a drug as though it is effective, when you know it is not effective, even if it is safe. That’s because they would be buying something thinking it will do them good when it isn’t, instead of buying something that really is doing them good.

            I’m actually so concerned that you just don’t “get it”, but thankfully we have laws in place to protect people from guys like you.

          2. Erebus says:

            Anon,

            >”When you know it is not effective”

            ???

            Am I correct to assume you mean, “when you don’t know whether or not it is effective”? (The alternative, that you mean what you wrote, is simply too asinine. Moreover, the historical record totally negates that point. The pre-1962 era was no less ethical than our own, and on this the evidence is certain.)

            Then the question becomes a one of logic: Your position is that the failure to prove a positive is equivalent to proving a negative. In other words, your definition of “proof” is backwards. It is, effectively, impossible to prove efficacy; it is possible, if one takes very great pains, to prove a p-value that indicates efficacy under the circumstances associated with the clinical trials performed.

            And, again, is it worth tremendous sums of money, 10 years, and in many cases the lives of patients, to prove a p-value that we could gain via other means, such as via epidemiology?

            Do we really need those p-values so badly, especially for illnesses like cancer? Are we obtaining them in an efficient way?

            >”It’s not unethical to enrol people in a trial when you and they both know there are risks, but it IS unethical to approve a drug as though it is effective,”

            Your grasp of ethics is as poor as your grasp of logic.

            First of all, nobody would be approving drugs “as though they are effective.” The question of efficacy, and the task of responding to patient feedback, would be left to clinicians and other professionals.

            Second, somebody who purchases an experimental or new drug [(a) on the advice of his or her doctor, and (b) which has been proven reasonably safe,] may lose money, but somebody who has a terminal disease and is given a placebo is sure to die. There is no possibility that, in the first case, the experimental or new treatment will have killed the patient. In the second case, the false treatment may directly be responsible for expediting or causing the death of the patient, particularly if the drug under study shows efficacy in the treatment arm.

            Try to think outside the box, just a little bit. Is the current drug approval protocol truly optimal? And if you feel so strongly about the need to prove efficacy, why are you not UTTERLY FURIOUS about the fact that the FDA has repeatedly bowed to political pressure and approved drugs that, emphatically, have not demonstrated efficacy?

          3. anaon says:

            Erebus, in the ideal world, with everyone in the business honest (!), it might work. So just in this example, the CETP inhibitor, the company will issue the advert: “This is a new experimental substance, we have no idea if it will actually help you apart from changing some biomarkers but is not likely to kill you outright. Only $100,000 per year !”. But no, it will be “This is a new Miracle Drug Which Rebalance Your Bad an Good Cholesterol and Cure Your Heart !! FDA certified it is safe and might save your life for only $100,000!!”
            So why discover the real drugs if very safe sugar pills can earn you $$$ with a proper advertising and a pressure on Patient Group like in a recent DMD disgrace ?

      3. tnr says:

        While it is true that failure to reject the hypothesis of no drug effect does not imply strong evidence that the hypothesis is true, in this study, the confidence limit for the hazard ratio is 0.91 – 1.11. Thus, if there is any effect, it is very small. It is just not worth it to the company to pursue any further. As a consumer, I would not be interested in taking the drug even if it were free.

      4. passionlessDrone says:

        > I’m not even sure that we can say that this drug is ineffective.

        Sweet Jesus, look at the graph of CV results. What more do you need?

        > What if, in combination with a beta-blocker, or high dose niacin, or something else, it improved outcomes very significantly?

        How on Earth do you propose figuring that out with post market analysis, where everyone takes it, and twelve other things simultaneously because it is a ‘more efficient’ system? How do you tease out results in a completely uncontrolled experiment? Did this person take niacin? Did they exercise? Did they have a heart attack previously? Did they take niacin and beta blockers? You have no mechanism to identify any of these things in the ‘faster more efficient’ model you prescribe. (?)

        > We’ll never know, now will we?

        You’d never know in an free wheeling, take whatever you feel like taking and tell us how things went model *either*, except you’d have a country full of people taking handfuls of pills all day long because the market.

        And I still have no idea who is going to pay for all of the post market analysis. Why would anyone? Just say, ‘It’s roasted’ and keep on selling.

  13. Chrispy says:

    Erebrus, the critical point you are missing is that clinical trials are randomized and blinded. There is simply no other way to convincingly demonstrate efficacy. If you let people take drugs that are only proven safe you may never know the efficacy because the people who end up taking them will have all kinds of confounding factors that are not controlled for — like being wealthy enough to afford insurance that lets them do it. Even the placebo effect can have a strong effect on outcomes if a trial is not blinded. In your world people would benefit from the placebo effect, but it would be at the expense of true efficacy.

    1. Erebus says:

      Efficacy does not need to be convincingly demonstrated in a clinical setting.

      First of all, it’s proving almost impossible. It costs something like $2B to get a drug approved these days. It also takes, what, 10-12 years on average? Even so, the data is often equivocal, and drugs like Addyi and Sarepta’s Duchenne treatment — which have not demonstrated convincing efficacy — are nevertheless approved.

      Second, the notion that drugs need to be proved efficacious before they’re sold is a relatively new one. As I mentioned earlier, drugs from aspirin to cortisone were not proved efficacious prior to their introductions. They still work, don’t they? Ask a cancer sufferer is a p-value is worth dying for.

      Third, every one of today’s new drugs owes something of its success to luck. A hundred screw-ups — from errors in trial design, to corporate failures, to, of all things, a stronger than expected placebo response — could have killed them. These are often not the fault of the drugs. A faster, safety-only testing regimen would give all drug candidates the best chance to succeed. And isn’t that what we want?

      Fourth, you totally underestimate what can be done after a drug is brought to market. Consider troglitazone. It was approved as a new drug and sold broadly. Reports started coming out about liver failure, so the FDA ordered a reevaluation. No expensive clinical trials, no blinded studies, no multiple phases. Instead, just two people — David Graham and Lanh Green — evaluated post-marketing data. It took them two months.

      …Some will argue that this is an extreme example. Sure. But post-marketing surveillance is vastly faster and cheaper than clinical trials in every possible case. For far less than the average cost of one new drug approval under the current protocol, the FDA could keep 1000 epidemiologists on staff, each at $100k per annum, for ten years.

      Think of what they could do with Big Data™!

      1. HTSguy says:

        I’m afraid this that you are willfully blind to the fact that very few drugs are absolutely “safe”. Even aspirin at anti-inflammatory doses is not “safe” by that standard – there are rare, but serious, side effects. So safety only makes sense in the context of clinical efficacy and they both need to be considered in judging the clinical benefit of a drug. As for Big Data, you might want to consider the relative evidentiary power of blinded randomized clinical trials relative to epidemiology. The latter is better than nothing, but often proves to be misleading when the hypothesis is rigorously tested in an adequately powered, blinded, randomized clinical trial.

        1. Erebus says:

          Of course safety is relative, the dose makes the poison, and so on… Come on, I’m not that stupid.

          As I’ve said before, I believe that drugs should undergo preclinical and Phase I safety studies. I also don’t think that medicine can ever be completely safe — in fact I believe that any attempt to make medicine completely safe is a fool’s errand, to put it politely — and I’m all in favor of making experimental new drugs available to those who need them.

          Having said all that, the “safety makes no sense without efficacy” argument is spurious.

          As a starting point, I’d note the trivially self-evident fact that it’s a hell of a lot easier to measure safety than it is to measure efficacy. We’re talking many orders of magnitude here. It doesn’t take double-blinded, 12,000-patient, multi-year studies to tell you that if you take Drug X, you’ve got a 3-5% chance of vomiting.

          The rest can, generally, be left to clinical judgement and medical associations — in short, let’s have the doctors on the front-lines make the calls. Some drugs might be deemed too unsafe for certain indications, whereas some other indications, such as late-stage cancer, might tolerate (or indeed require) drugs whose side-effects include death.

          Aside: It would be simple to institute a points-based system to determine whether or not a drug is safe for a given indication. Almost laughably easy, actually. To scale it with dosing would also be very easy.

          1. MrRogers says:

            Is doxorubicin safe?
            What about cisplatin?
            methotrexate?
            dexamethasone?

          2. Erebus says:

            Mr. Rogers: Wouldn’t you say that this depends on how they’re used and what they’re used for? (Questions that, emphatically, should not require double-blind 12,000-patient studies for their answers!) As I said in my previous post, “It would be simple to institute a points-based system to determine whether or not a drug is safe for a given indication.

            Besides, half of the drugs on your short list were developed prior to the era of mandated efficacy testing — a fact which indicates that, practically speaking, there’s no basis for whatever argument you’re trying to make here.

          3. MrRogers says:

            Precisely. If efficacy data are not required, you can change your indication to fit your toxicity profile. In fact, it is to your advantage to include at least one rare indication (Marburg virus, necrotizing fasciitis, angiosarcoma) so that the FDA can’t collect sufficient data to demonstrate that your drug is ineffective. Once that’s taken care of, the rest is marketing.

  14. Barry says:

    well, there’s increasing circulating [HDL] by making more of it, and there’s increasing circulating [HDL] by blocking its function of carrying cholesterol away. Pfizer was $billion into torcetrapib without ever trying to falsify the hypothesis that circulating [HDL] was a good enough surrogate for delta mortality. Then Merck jumped onto a still-unvalidated drug target, and drove the same unproven surrogate marker down, and was also unable to affect actual human disease.
    It costs no less time/effort/money to do the med. chem. and chem. dev. on a bad target than for a good target. What’s lacking is the determination to kill an invalid drug target early. That experiment might cost $million, but save (in this case) $billion. But the lead CETP biologist knows that he’ll get promoted for heading a project that yields a profitable drug, and won’t for falsifying the hypothesis, although that would have served Pfizer better.

  15. Dieter Weber says:

    The general goal is to bring a benefit to as many people as possible as quickly as possible at a reasonable cost. First of all, efficacy should be easy to prove for drugs with immediate, pronounced benefits for a wide population. There we don’t need change in any case because there’s no problem — the drug will hit the market quickly. Only drugs with supposed long-term benefits such as slowing the progression of a chronic disease should be up to debate. Here the challenge are long test cycles and high cost of trials.

    The input variable here is how many drugs that do work are stuck for too long in the testing pipeline vs. how many drugs that don’t work are eliminated before they generate extra cost and divert money from investments that actually benefit the patients. As an example, money would be better spent on a long vacation and a comfortable hospice than on a cancer therapy that doesn’t work.

    The way it seems the number of therapies that don’t work or have no benefits compared to existing ones is MUCH larger than the number of therapies that do work better.

    It should be obvious that controlled trials generate much better data at a much higher rate than having the general population take a drug. If that was to change, it would mean much tighter monitoring, resulting in the same setup as a trial, just with patients and insurance companies paying for it and not drug makers. Therefore eliminating efficacy testing before approval is nonsense. The only point to make would be to help companies distribute the risk of failed trials better because the trials get so large that a string of “bad luck” is enough to ruin a business.

    In the end, I think this cry for “compassionate use” is just an elaborate attempt of drug makers to shift the cost of trials from their own pockets to patients, doctors and medical insurance, while keeping all the income if the drug turns out to work. Nice try…

  16. Mark Burns says:

    Bravo Erebus! Well done, your arguments to change a broken system are compelling and seem to have withstood the counter-arguments by the smartest in the business. Biases against change in this industry are strong indeed. Bias based on money are easy to understand…but controlling markets are a recipe for disaster (re: Adam Smith). IMO, I agree, we are obviously not servicing our customers under the current regime. You made the point that Drs. prescribing the drugs would have a clear view on efficacy. They certainly would add their data on patient benefits, I believe willingly, and with an ethical obligation to prescribe effective drugs. Insurance companies also have an interest in paying for effective medicines. Let them contribute to the post-analysis. There’s two interests that can help serve Mr. Doe better.

    1. tangent says:

      Ahem. The point is made every time this stuff comes up, here by HTSGuy: “As for Big Data, you might want to consider the relative evidentiary power of blinded randomized clinical trials relative to epidemiology. The latter is better than nothing, but often proves to be misleading when the hypothesis is rigorously tested in an adequately powered, blinded, randomized clinical trial.”

      1. Erebus says:

        In the short term, yes. Granted. On longer time scales, it is debatable.

        Yet it doesn’t follow, and it is by no means apparent, that blinded randomized clinical trials are the right tool for the job. In fact, it’s almost entirely certain that, very much to the contrary, they are inferior to other possible approaches.

        Is medicine for p-values, or is it for patients?

        I believe that a safety-only approval system — coupled with epidemiology, properly applied — would be better for patients & better for the drug industry. The inescapable conclusion is that it would be better for society at large.

        It would also be better for doctors, who would be less constrained in the treatments they’re able to offer their patients. Doctors are not that stupid, either. The cancer chemotherapy regimens still in use today were developed not by medicinal chemists, but by doctors in the trenches. Frei, Freireich, Farber, and the rest, weren’t performing double-blinded studies in 12,000 patients. This account, from Freireich, is worth a quick look:

        “Johnson had done work [on vincristine] in mice, and he showed me the results. I said, ‘Gee whiz, I’ve got ten kids on the ward dying. I’ll give it to them tomorrow.’ So I went to Zubrod. He said, ‘I don’t think it’s a good idea.’ But I said, ‘These kids are dying. What’s the difference?’ He said, ‘O.K., I’ll let you do a few children.’ The response rate was fifty-five per cent. The kids jumped out of bed.”

        Gerald Bodey, a specialist in infectious disease who worked with Frei and Freireich, also wrote a bit about that non-randomized, non-blinded trial: “Giving these kids four drugs all at once! As a Christian, I thought it was immoral because if they relapsed we would have no fallback. I thought Freireich was crazy.” The results stunned Bodey. “We had sixteen patients, and eleven were cured. It was astonishing. I kept track of one of the children for decades until she died of breast cancer.” Frei was awed, “To be able to do this with a lovely child . . . whom you think of almost as your own, is truly an extraordinary experience. . . . I had the evidence that you could succeed.”

        The rest is history. VAMP has saved many lives.

        Freireich’s approach wouldn’t even be close to legal today. Not by a mile. It don’t know if he reached P < 0.05. (Probably not.) He evidently moved way too fast to keep the lawyers and bureaucrats at the FDA in the loop. Freireich didn't have any friends at the NIH, which hated the entire idea of combination chemotherapy.

        …But how important are those things, in light of the fact that he saved young lives that otherwise would have been lost to neglect?

        If the shackles on the pharmaceutical industry are loosened, we'll gain far more than we lose. Sure, there'll be malefactors — there'll be people like Shkreli — and we'll need to remain vigilant. That's just a fact of life. Besides, those malefactors are with us now, whereas the likes of George W. Merck appear to be bygone products of a lost Golden Age!

        Fundamentally, I think we need to remember that medicine isn't for p-values — medicine is for patients.

        And since others have repeatedly brought up ethics, let me ask you this: Is it more ethical to give a terminally ill man a placebo, and lie about it, than it is to make every effort to save his life — with heroic measures, experimental drugs, and polypharmacy, if needed?

        1. tangent says:

          Well that’s a lot of text (and haven’t I seen chunks of it before) to hang onto an “in the long run we’ll surely learn what works” handwave.

          Look at the ‘dietary supplements’ industry. Many of those drugs have been around a while. Do we have good knowledge of which ones actually work and which do not? More to the point, have we stopped spending money on the ones that don’t?

          I guess I should back up and ask, do you have any major concerns with the state of the U.S. ‘supplements’ industry, or is that all cool? Since it’s all voluntary actions taken by competent actors?

        2. tangent says:

          If you like economics, the issue is that efficacy data has big positive externalities — a lot more value to the world than market forces can channel to the generator of that data. Good proof of efficacy won’t sell much more of a drug than weak proof will.

          You argue that we can spend more on gold-plated proof than the gain over silver-plated is worth to us all. That’s true, we can, and “wah, I want the gold-plated” is not what I’m saying here. The problem is where the silver is worth buying, in terms of total social value, but too much of that value is externalized away from the patent-holder (or any other actor). So they don’t buy the silver.

          You’re arguing that the level of proof we’re buying now is way too high, that we’d do fine with non-RCT data. But whatever the optimal level, we’ll underproduce that. Unless you argue the benefits won’t be externalized much, within our market with our actual people making decisions.

          (You know, one of the best pragmatic arguments for a safety-only regime is one I haven’t seen you make here…)

          1. Erebus says:

            >”(and haven’t I seen chunks of it before)”

            If I need to repeat myself, it’s only because the same nonsensical counter-arguments keep popping up. “Thalidomide!” “Safety-only is not ethical!” Gimme a break.

            >”(You know, one of the best pragmatic arguments for a safety-only regime is one I haven’t seen you make here…)”

            Come on, don’t leave us hanging.

            As for the rest of your posts:

            The supplements industry is so far-removed from the pharmaceutical industry that no meaningful comparison can be made. All supplements are OTC, whereas the very vast majority of new drugs are only available via prescription. There are very many other differences; I wouldn’t want to burden you with another wall of text. Suffice it to say that the best comparison to the pharmaceutical industry of today is the pharmaceutical industry of 1962. Surely you don’t need me to remind you, yet again, that drug development was orders of magnitude faster, cheaper, and more effective back then.

            The above point is beyond dispute. Look again to Freireich. Not only would his actions be completely illegal today, but if you were to go back in time to 1962 and tell him that in 2017 — more than half a century later — VAMP would still be the standard of care for certain types of cancer, he’d think you were crazy. Things moved far more quickly back then. It would be possible for drug development to move quickly today, too, if that weren’t illegal.

            In any case, the “think of the supplements industry!” is an argument a la “Thalidomide!” It’s a non sequitur.

            Your second post is almost totally incomprehensible. I don’t know what you mean by “gold plated” vs. “silver plated.” I don’t know how you can assert that “whatever the optimal level, we’ll underproduce that.” Nowhere do you address a real argument. All you’re doing is pleading.

            If you’re suggesting that p-values are of “a lot more value to the world” than a growing drug industry, a more balanced and favorable regulatory environment, and allowing doctors to fight a war on cancer as though it’s a real war, with every tool at their disposal — including, when called for, combinations of new drugs that haven’t yet passed the FDA’s onerous efficacy review… then I just don’t know what to say. “Better 1000 die of disease than one of bad medicine,” eh?

  17. DanielT says:

    Erebus why being rather argumentative does raise an important point – has the 1962 efficacy requirement been an overall positive or negative? Is there any reliable data pointing one way or the other?

    1. tnr says:

      I think that would be very difficult to assess this because it’s difficult to tell how many more drugs would have been approved under a less stringent standard. Some of those would be efficacious but there would be some spectacular ‘bombs’ like the anti-arythmia drugs of the early 80’s.

      In my opinion, it doesn’t have to be the current system vs pre-1962. Some drugs should be required to be proven efficacious in randomized clinical trials while others could gain approval by well done observational studies. It should be based on a number of factors including seriousness of the condition, current trt options, size of patient population, and other factors.

  18. exGlaxoid says:

    I have to agree with Erebus on many points, I don’t agree with all, but I do think a compromise on trials would be to go back to safety and efficacy trials being on a reasonable number of people, such as 1000-5000, as was common in the 1970-1980 period, and stopping the idea of testing drugs in 10,000-50,000 people before approval. The statistical difference between 5000 and 50,000 people in finding safety issues is functionally nothing. And if a drug has efficacy in 5000 people, there is no need to go further to prove it. If it does not after 5000 people, then it likely won’t work in 50,000.

    If we could get the useful drugs to market for under $1 billion, that would cut the cost of medicines roughly in half, and if we could do trials on smaller numbers, the working drugs could be out to people faster than now also. We will never find a 100% safe drug, the idiots in the US now are demanding that, doing larger and longer trials will not improve the safety of drugs by any measurable amount, verses getting treatments to patients faster. At a minimum, the drugs for potentially live saving treatments should be made faster than the present. I do agree that drugs came to market faster and much cheaper in the 1960s, but much of that was through much faster in-vivo testing of research drugs, short development trials, and less bureaucracy.

    If people have not read about the discovery of bendryl and chlorotrimeton, those are excellent examples of that type of quick development, some were tested in inmates as volunteers, many on death row. I don’t really seem the clear difference in ethics of that verses paying stupid college students to test a new drugs. If someone about to be killed wants to help test new drugs, to help support their remaining family, I don’t see much difference from a college student doing the same for beer money.

    1. Hap says:

      Was there a difference in the target audience for medicines during the 1970s-1980s? If you are looking at a sick population, then a smaller test set makes sense, but if you’re dealing with large populations of potential patients (the push for blockbusters has generally meant aiming at larger target populations), then you will probably need a large population of test subjects to determine either safety or efficacy. In a large population receiving a drug for a long time, small frequencies of problems might not be seen unless you test in a lot of people. I guess the question would be whether you want the drug more quickly and are willing to take the chance of bad side effects in small groups of people or not, and that probably depends on the effectiveness of the drug.

    2. Michael Rogers says:

      The reason that some drugs require large cohorts for phase III has nothing to do with safety. It is a result of trying to treat populations with a small annual risk of a serious event (e.g. heart attack). To collect enough events in your control group, you need to treat large populations. If you had a therapy that dissolved all metastatic melanoma lesions quickly and completely (a “penicillin for melanoma,” if you will), you wouldn’t need a large cohort for phase III, because most late stage melanoma patients will die quickly. If the signal/noise is large enough, you don’t even need an RCS to get FDA approval. It’s just that the signal is rarely that large.

  19. Wallace Grommet says:

    Anyone want to develop some great neuralgia, congestion, and dyspepsia drugs? Prescription only? Spend a billion or two? Whether they work or not? Reminds me of Chairman Mao proclaiming,”let a thousand pharmaceuticals bloom!” I am only carpenter, but I can see the stupidity of Erebus’ position.

    1. Erebus says:

      >”Anyone want to develop some great neuralgia, congestion, and dyspepsia drugs? Prescription only? Spend a billion or two? Whether they work or not?”

      How does this make any sense? Surely you realize that very good treatments for congestion and dyspepsia already exist, and that if you intentionally set out to market a drug that doesn’t work, not only are you sure to become a laughing stock, you’ll also lose all of your money. Don’t forget that it’s not just the FDA which can evaluate drugs — insurance companies, physicians, epidemiologists, institutional investors, and even, in their way, patients, all regularly evaluate the efficacy of drugs.

      In fact, there’s more incentive to release ever more effective and safer drugs, as the market gets more crowded and competitive. I’d also add: More drugs — that is, disease treatments — is a better condition than fewer drugs. Certainly better for doctors and patients, if not for bureaucrats and regulators.

      I realize that “you are only carpenter,” and that your grasp of the pharmaceutical business may therefore understandably be quite poor, but I’d advise you to give these matters a bit more thought.

  20. Sulphonamide says:

    So if you wanted a system based primarily on post-market analysis, how many years/patients would it take before you “knew” that a drug worked? How many years/patients to “know” if Drug A was better than Drug B. I can see that these arguments might have some merit when talking about disorders such as DMD, but how would we have got to the current state of play with the treatment of hepatitis C, going from 40-50% “cure”, to 60-80% and finally to the >95% all within about 5 years. With post-market only, who would risk not taking a drug expected to be maybe 60-80% likely to cure them, compared the one for which there is only unsupported preclinical hyperbole (which proper clinical trials would have shown to be >95% cure rate – go back to hep C market analyses from only 6 years ago – NRDD 2011 – and what became sofosbuvir was not being talked about with any great excitement)?

    You can perhaps now argue that those for whom the earlier drugs did not work would have then moved onto the more recent ones and eventually we would see the light and would know that sofosbuvir was the best drug. However, how long would this have taken and how much would have been spent on less good drugs (combined with costly ribavirin and interferon presumably – as how would we know not to use them – and their side-effects) In the meantime, how many people would have continued to suffer from the disease and maybe progressed to later stage liver damage…never mind what all this trial and error would have cost. Can you really argue that much suffering, restricted access and extra cost has been incurred by clinical trials in this case, rather than in fact the complete opposite? OK, perhaps mention of hep C treatments is a red rag to a bull in view of their pricing controversies, but I just don’t believe that post-marketing can allow us to make the incremental improvements seen in many fields – e.g. which, in my opinion, slowly but surely will get us to a cure for all cancers. If we go for safety only, we may get lucky, but in all likelihood it will in fact cost us time and impede progress.

  21. Wallace Grommet says:

    Ok, I’m game, Erebus. Name ten drug candidates and their therapeutic goal you would begin producing without efficacy stage three trials.

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.