Skip to Content

When Small Trials Convince

This is a good piece by Bruce Booth in Forbes, and it points out something that’s changing in the biopharma landscape. Readers will have noticed over the years here the occasional eye-rolling at companies that run underpowered clinical trials and go to the FDA hoping for the best. That’s not a good place to save money, actually, and if it’s come to that then something is not going right. Statistics rule in drug approvals – well, unless you’re Sarepta – and you generally have to get some real N behind you to get solid, believable numbers.

But we’re used to what you have to do to get such numbers with small molecules, and small-molecule mechanisms. What about cell- and gene-based therapies (CGT)? I was just asked about these the other day, after speaking to a group about drug discovery, and I answered that I was very hopeful about what could be accomplished in this area (although I also mentioned that the general public has no idea of how complex it is, and how early things are). Live-cell and gene-delivery modes of action are (or can be) a different order of things entirely, and we may well have to adjust our settings.

As Booth mentions, it was (in retrospect) around 2009-2010 that the situation really began to look up in this field. In the late 1990s/early 2000s, patient deaths and general bad news had sent things into back-burner mode, and it took a while to move out of it. As positive data began to show up, though, it was interesting to see how few patients were in these trials – ten, maybe. Single digits. Maybe just one, the first time. A lot of drug mechanisms (and a lot of drugs) would have trouble convincing anyone at those levels.

But it comes down to effect size. If someone has Rare Genetic Disease X because Cell Population Y cannot produce Crucial Protein Z, the effect of replacing Y or Z can be enormous. We’re used to fighting biochemical mechanisms with all kinds of compensatory pathways webbed around them; there’s often only so much you can move the needle without it starting to move back. Or we’re fighting some awful shape-shifting pile of tumor cells, where wiping out 90% of them in one go only means that the patient lives another month or something. But when there’s an actual piece missing and that piece drops back in, or when you have some handle that lets you select every last cell you’re trying to target, that’s something else again. A small N can be enough to show you the way under such conditions:

One of the salient takeaways from the history of the CGT field is the catalytic impact of truly compelling early clinical data. An axiom in drug R&D is that “great drugs reveal themselves early” and this is never more true than in the CGT space. One doesn’t need to “torture the data” to make it yield answers. In many instances the answer is clear within a few patients. With such profound unmet needs in most of the grievous, debilitating conditions where these CGT approaches are appropriate, when CGTs work well, one can often see it quickly and with very small numbers (n’s) of patients. And that holds true even in later stages of development: take Luxturna’s Phase 3 study, which only has 21 patients; Strimvelis was approved with 18 patients.

The tricky part, as it is so often, is tox. The effect sizes there can be annoyingly small but show-stopping (ask Merck about that with Vioxx), and it can take a while for them to emerge. A ten-patient trial of a small molecule is unlikely to tell you much about efficacy, but it’s highly unlikely to tell you anything about toxicity, unless you’ve run into a grievous effect size indeed. What can mitigate this in the CGT field, though, is that you’re often replacing something that the body has been expecting to be there all along, not adding in a new drug substance with a less-worked-out profile in vivo. In that case, the big questions are whether your method of delivery is safe, and whether the abrupt replacement is handled well.

But in things like CAR-T therapy for cancer, the long-term effects are still an open question, because there aren’t any really long-term beneficiaries yet. It’ll be years before we know for sure. The mitigating factor there, of course, is that without the CAR-T treatment these people, every single one of them, would be long dead. Conventional oncology therapies have long made some pretty brutal tradeoffs of side effects and toxicity for much lower odds and payouts, so the CAR-T case is actually pretty easy to make.

As it stands, the sorts of conditions being treated by CGT all fall into this dramatic “death or glory” zone. No one’s going to go in with engineered T-cells or viral gene-delivery vectors to treat a (hypothetical!) condition that causes, say, shorter eyelashes. With the huge medical need on one side, and the dramatic effects of therapy on the other, the CGT field is going to be able go on for some time with smaller and shorter trials than just about any other part of the industry. If some new approach comes along and kills off all eight people in the trial, God forbid, that’ll be time to talk again. But so far, even though there have been deaths in some trials in this area, there hasn’t been a flat disaster of that kind, and let’s hope that there isn’t. Because the rewards of success in this area could be almost beyond measuring.

22 comments on “When Small Trials Convince”

  1. Barry says:

    we saw this last century with insulin. Sure, it’s toxic. But where you’re replacing a vital factor, the therapeutic effect isn’t hard to see. (Not quite as dramatic for vitamin C in scurvy, or vitamin D in rickets, where the drug substance couldn’t (initially) be given in pure form)

    1. Derek Lowe says:

      Insulin’s an excellent example – only in this case, it’s like suddenly having pancreatic beta-cells again. Mind you, people are working on that, too, although it’s a beast of a problem. . .

  2. Barry says:

    Insulin’s a beast because:
    1-it’s so toxic
    and
    2-production should be modulated with each meal

    More attractive to my mind are the clotting factors/haemophilias. Even supplying a percent or two of the normal concentration would be therapeutically useful, and they just have to be dumped into the plasma compartment

    1. Derek Lowe says:

      True. If you take twice as much ibuprofen as you planned to, you’re probably going to be fine. If you take twice as much insulin, you’re going to be on the floor. Very narrow window and very brittle response.

  3. Barry says:

    Whereas if I had 200%, or 50%, or 10% of the normal [Factor VIII], I’ll be fine.
    I note that there are fewer hemophilias than there are clotting factors. That suggests that knock-outs for some of the clotting factors are embryonic lethal, and we only see the disease where the absence is somewhat tolerable.

  4. Imaging guy says:

    Professor John Ioannidis did an empirical analysis of “very large treatment effect” (i.e. effect size) using Cochrane Database and published two papers (1,2). For the analysis a “very large treatment effect” is defined as clinical trials showing odds ratio of greater than or equal to 5 (or 0.2 depending on where you put the numerator and denominator) [statistically significant at 5% (P< .05)]. For example, if the median survival is 3 months in control arm and 15 months in treatment arm, a clinical trial will have an odds ratio of 15/3= 5 (or 3/15= 0.2) and would be counted as having “very large treatment effect”. What he found was 1) “very large treatment effects” are most likely to be found in small clinical trials and they are usually for non-fatal outcomes and 2) they are generally not reproducible in subsequent larger trials. So, you can’t use the argument that it would be unethical to do a larger study since you have already found a “very large treatment effect” in a small study.
    1) Empirical evaluation of very large treatment effects of medical interventions (PMID:23093165)
    2) Very large treatment effects in randomised trials as an empirical marker to indicate whether subsequent trials are necessary: meta-epidemiological assessment (PMID: 27789483)

    1. a. nonymaus says:

      I think this is an effect of Ioannidis setting too low a bar for “effect size”. When you’re talking about cure vs. death in an inborn error of metabolism, you can start talking about effect sizes of 10+ in terms of survival time.

  5. Mach4 says:

    Perfect example of small trials that worked is the recent success of Spinraza, the antisense drug that was discovered by Adrian Krainer at Cold Spring Harbor, acquired by Ionis and launched by Biogen. It corrects Spinal Muscular Atrophy (SMA) in neonates that otherwise would die young, and the videos of the children acting like normal kids after treatment are truly inspiring!

    The drug has a high cost, 750K for the first year, and 200K for yearly treatment, BUT- there is no other option, only death.

    Last week, at the Galien Prix Awards in NYC, essentially the Academy Awards for the Pharma Industry, the Best Biotech Product went to Spinraza, and all the executives went onstage to accept the awards.

    Except the inventor Adrian Krainer. Although in the audience he should of been there as well-instead the Execs acted like they invented it.

    I’d of preferred the Inventor speak over bloviating Execs anyday.

    1. ScientistSailor says:

      Heard a clinician say that only clinicians can invent *drugs*, researches only discover molecules…

  6. PorkPieHat says:

    Mach4, you beat me to it. How about gene silencing, which is the opposite of replenishing a missing vital factor. . .instead silencing a mutated gene which produces disease. I’d think the same benefits of clinical trial sample size should accrue, particularly the more severe the disease is, right? Huntington’s disease and the Huntingtin gene comes to mind. Or Primary Hyperoxaluria (PH) which arises from mutations in the enzyme alanine-glyoxylate aminotransferase.

  7. ex chemist says:

    Having move moved from medicinal chemistry to CGT. on top of very small trials, I have found another thing that is also quite counter intuitive with CGT: Very little preclinical data are required for a first in man phase I clinical trial. For CAR-T with the argument that it is a living autologous treatment, the traditional pre-clinical package are not required (seletivity, ADME, PK/PD, Cardiovascular, mutagenicity etc…). The main focus is really the CMC manufacturing of the CAR. My experience so far is very short timeline between hit finding and first in human, given that you have an efficient manufacturing team

    1. aairfccha says:

      What probably helps in that regard is that you are usually not introducing something new but replace something that should be there.

  8. milkshaken says:

    vey small trials can be used to demonstrate efficacy – but not safety. If you have the only available treatment for a debilitating genetic disorder or cancer, you can get away with all kinds of unpleasant side effects, risks and complications.

    Treating diabetes would be quite the counterexample, since there is a range of available therapies, many of which have decent safety profile. So you need to demonstrate that your patient (who can be elderly and have a range of additional complication – cardiovascular, renal, liver impairment, smoker with COPD ) – can remain safely on your diabetes drug for years and his/her conditions are controlled at least as well if not better than with the established treatments. You cannot demonstrate this safety profile within a small and short clinical trial

  9. Eric says:

    The ability to show convincing activity in early small trials has less to do with the modality and more to do with the disease being targeted. Cardiovascular disease, diabetes, osteoporosis, and arthritis will likely still need large trials even if newer technologies move into these therapeutic areas. That’s not to belittle the success of CGT therapies. Small molecules have had minimal success against these rare genetic diseases – so this is truly a huge step forward.

    1. plinytheotherelder says:

      The disease targeted makes all the difference. A clear example of this are antivenins.
      The current snake antivenin treatment (Crofab) had a grand total of 42 patients (cumulatively among all trials) prior to approval. But even the recently approved antivenin (Anavip, approved 2015) that will supplant Crofab only had 147 patients cumulatively prior to approval. Though a large portion of that is the relative rarity of snake bites in the US.

  10. sgcox says:

    N=1 is more than sufficient to convince if the effect is as huge as Fig.1 in this nature paper.
    https://www.nature.com/articles/nature24487

  11. steve says:

    The CAR-T story has generated excitement because it works in cases that have failed all conventional treatments and where there are strong data showing that those patients would die in 6 months. There aren’t too many cases like that. However, the downside that most don’t discuss is the relapse rate. Even CAR-T is not a cure (unless you define “cure” like Novartis does) and patients are beginning to relapse. All that is needed for recurrence is modulation of CD19 – mutation of the epitope or simple deletion of the antigen all together since it’s not needed for cell survival. What CAR-T and other immunotherapies are pointing the way to is the idea that the immune system can indeed provide dramatic improvements in even terminal cancer patients. Now the emphasis is on how best to apply this to solid tumors, how to reduce toxicities and how to sustain the response.

    1. Barry says:

      No one has yet–to my knowledge–shown that a CAR-T can target a tumor per se, rather than deleting a whole tissue. Most of what is distinctive about cancer cells is not displayed to immune surveillance on the cell surface. By not expressing MHC (or expressing broken MHC) the markers of disease are hidden from TCRs, or from CAR-Ts. And there aren’t a lot of tissues one could delete without grave repercussions. Gonads, I suppose. Pancreatic beta-cells.Thyroid…

      1. steve says:

        You’re not quite right as the whole point of CAR-T is to overcome the down-regulation of MHC by using a single-chain antibody chimeric receptor that avoids the need for MHC recognition. There are indeed a number of tumor-specific antigens in the works; probably the best is EGFRvIII for glioblastoma. However, your general point is taken as the immunotherapy approach (including checkpoint inhibitors) have an issue with regard to identifying true cancer-specific antigens. Tumors undergo a process called immunoediting in which cancer cells are specifically selected that can avoid immune surveillance. Still, there’s a lot of excitement about overcoming these defenses through inhibition of Tregs, MDSC, TGFbeta, etc. Lots of room to play for interested chemists!

        1. Barry says:

          Yes, the chimaeric receptor does away with the need for MHC recognition.But MHCs are not merely recognition sites for T-cells.Critically, they are how cytosolic proteins are displayed to immune surveillance. And most of what is distinctive about a cancer cell is cytosolic (or nuclear)
          Down-regulating Tregs or TGFbeta signalling, or blocking CTLA4, o PD1, or PD1L can disinhibit killer-T response.But one is then hoping that what the Killer-Ts kill is the cancer, and not some other self-tissue. If this is a “targeted” therapy, the targeting is not in our hands.

          1. steve says:

            We don’t want the targeting to be in our hands. The whole point of immunotherapy is that the immune system can recognize and kill tumors if it is disinhibited. T cells/MHC are not the only means; NK cells, innate immune cells and others can recognize and kill transformed cells. The advantage of the immune system is that it is not restricted to a single drug/single target mentality which is what has hampered cancer drug development. Nature is not unidirectional and while FDA may abhor pleiotrophic effects, they’re the only way to keep an evolving pathogen, be it bacterial, viral or malignant, at bay. The trick for immunotherapy will be finding the balance between removing the inhibition on tumor killing that cancers utilize while maintaining tolerance against host tissues.

  12. John says:

    I think one issue is that due to the heterogeneous population of the input, the output (modified cells) are also heterogeneous, which means (1) a large # of cells are required for therapy, >50M per dose, and (2) we could, potentially, be inserting unknown population of partially modified cells which could have an unknown response in the long-term. I don’t follow this literature and am curious to know whether these issues have been addressed.

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.