Skip to Content

What Passes For Positive Data

So can someone explain this one for me? Halozyme has a hyaluoronidase enzyme (PEGPH20) that they’re developing as a therapeutic agent. The idea is that it’s going to be an add-on with other cancer therapeutic combinations, and would increase the exposure of the tumor cells to the actual chemotherapies by breaking down hyaluronan (hyaluronic acid) around the tumor tissue and its associated blood vessels. Fine – this is not a ridiculous idea, and is worth testing.

Tested it they have, in a recent trial of pancreatic cancer with Abraxane (paclitaxel, or taxol) and gemcitabine. And the headlines say things like “Halozyme’s pancreatic cancer drug succeeds” and “Halozyme jumps on positive late-stage pancreatic cancer data”. And from that sort of coverage, you’d expect that the data were, y’know, positive. But this morning, the company released more information, and various biotech observers on Twitter began to roll their eyes.

Here, via AndyBiotech, is the survival curve for the Abraxane/gemcitabine group versus the group with PEGPH20 added. Right. The “positive data” are basically one point along in the trial where the treatment group looked better than the standard of care. But that disappears – check in a couple of months later, and everything looks identically bad. I realize that for pancreatic cancer the standard for “positive data” might be a bit low, since it’s very hard to get anything to work (you wonder what these curves would look like for untreated patients, to be honest). In that later stages of the trial, there are actually more people left alive in the standard-of-care group than there are in the experimental one. The numbers are so small that that’s probably meaningless, but then so is the earlier “effect” that the company is touting. As bio_closeau pointed out as well, this was an open-label trial, and has its own further problems because of that.

This is cherry-picking no matter how you look at it, and Adam Feuerstein was right to call it “shameful conduct” to press-release it the way the company did. But hey, the stock bounced (or it has so far this morning), and that’s what matters, right?

Update: as pointed out in the comments, the primary endpoint for this trial was progression-free survival, and I’m giving Halozyme grief here over their overall survival data instead. Fair enough – but it should be noted that they just barely made significance even for the primary endpoint. And if this drug (or any oncology drug) is going to progress, it’s going to need to show a benefit in overall survival, because (as has been the case for some years), in the end, if progression-free survival means “you die just as quickly, but with fewer tumors”, then odds are that no one cares. I still have a lot of trouble seeing this trial as good news, although I will freely admit that the stock market is disagreeing with me on that. For one of several counterarguments, see here.

39 comments on “What Passes For Positive Data”

  1. Fezziwig says:

    Halozyme received an accelerated Primary endpoint from the FDA- Progression Free Survival. HALO 202 was a halted trial with thromboembolic disease causing an interruption in Rx with pegpH20. In spite of this 4 month delay, HALO showed a significant improvement in PFS with addition of PegpH20 to Gem-Abrax. The 8.6 mo final PFS in this interrupted trial is actually the highest PFS ever reported in Metastatic Pancreatic Ductal Adenocarcinoma. The HR was 0.51. Margaret Tempero, MD Director of the Helen Diller Cancer Center, and Pancreatic Ca Center UCSF says 0.57 or less is “a dream HR” in this disease. FDA says PegpH20 P3 approvable based on PFS if OS “supportive” So I’m thrilled today as a HALO investor.

    1. Some idiot says:

      Please forgive me if I am being stupid (I am a process chemist, not med chem), but as far as I can see, these two curves are essentially identical. So if there is no difference between treatment on one side and standard of care on the other, then why should these results be seen as positive?

      Sorry if I am missing something obvious, but….

      (Disclaimer: I have nothing to do with this or any other company within cancer, as is probably also shown by my ignorance)

      1. Fezziwig says:

        I’m talking PFS and Derrick chose to focus on OS – please read my comment.

        1. Some idiot says:

          I read your comment. You stated that the FDAsays that it is approvable if the OS data is “supportive”. There is no difference between treatment and standard of care arms, so I have extreme difficulty seeing how this could be “supportive.” Hence my comment.

    2. PeptoidChemist says:

      What is HR?

      1. Anon says:

        Hazard Ratio (HR) a summary statistic in this case to tell us the chances of overall survival between the open label groups. Note the Confidence interval overlaps 1, meaning that it is well within the 95% CI that there was no difference, and due to the small nature of the study that low HR could well be due to chance. I’m with Luckless on the fact that it is a bit unfair to be reaming them for a exploratory endpoint.

    3. Anonymous Guy Who Can Understand Data says:

      I’d be trilled too; sell your stock now.

  2. Jamtomorrow2 says:

    Well we all knew that the data would be muddled. The Phase 1 & 2 trials with the halt and introduction of the Ventana diagnostic were bound to lead to muddled data. But I do think that HALO’s presentation of all the data was thorough and fair albeit confusing for those who can’t be bothered to understand all the implications of the trial problems. So now the big bad news data that AF and AB are focusing on is the overall OS data being worse than control, which HALO claims is skewed by one control patient living much longer than expected. I guess the claim about how one patient in a very small trial can skew the data of course could also work the other way round. I would therefore imagine that there is no chance of HALO submitting an NDA based on this data and therefore we are in for a long wait – hence the market disappoint.

  3. steve says:

    If I were running the trial I’d include some surrogate markers to give evidence that the stuff works as advertised. Are there increases in circulating hyaluronan fragments after enzyme treatment? If you alternate chemo treatment with and without PEGPH20 do you see greater increases in CA 19-9 (representing increased tumor cell death) when the enzyme is included in the treatment? If I were the company I’d certainly want to see those sorts of things to provide some confidence that I should go forward with a big trial.

  4. Anchor says:

    Derek: You must read..”You Wouldn’t Be Lying About That, Would You?” posted by Dr. Lowe, yesterday!

  5. Luckless Pedestrian says:

    I have no dog in this fight, but want to clarify a couple of things as I see them.

    The most important endpoint in oncology is Overall Survival (OS). But it can take a long time for studies to mature with an OS endpoint, so often trials (especially early trials) are designed with a different primary endpoint such as ORR (objective response rate) or PFS (progression-free survival), with the hope that these earlier endpoints will predict OS.

    As I understand it, study 202 had median PFS as its primary endpoint, and according to the press release the study met that endpoint (though just barely, with p = 0.05). The KM curve shown in the post is the OS result, which was not significant (upper bound of the Hazard Ratio 95% confidence interval was >1).

    With the caveat that the numbers are small, it looks to me that the study was positive for its primary endpoint, PFS, but the improvement in PFS did not translate to improved OS. This raises the question of whether the improved PFS, while statistically significant, is clinically meaningful. That’s the kind of question that FDA and other health authorities will wrestle with if results from pivotal studies look similar.

    1. Anonymous says:

      Yes I agree totally. This is the most insightful analysis I’ve read so far.

      In a nutshell their compound keeps disease away longer than control. But that didn’t translate to longer survival which is the gold standard. Is that bad? Well it’s not a slam dunk case for the drug. But, no survival improvement also isn’t necessarily a total loss for the drug.

  6. mallam says:

    This type of creative data analysis, presentation, advertising seems to happening more and more often. I find that such situations create negative impressions covering all of new drug R&D, picked up by the public and our politicians alike. Some of the problem is overly aggressive, hopeful, wishful people working in the companies. Some is due to bad training, or none at all with MBAs or MDs trying to be in charge of research organizations, and staff not having proper scientific training along with understanding of statistics, record maintenance, and scientific ethics. It all gives the entire industry a very large, visible black eye.

  7. me says:

    Improved PFS doesn’t always translate into improved OS, which has resulted in the long debate about whether to approve based on improved PFS. From a patient standpoint, improved quality of life, which might go with PFS, would be a benefit. Unfortunately they don’t seem to have asked that question. It is also not uncommon to see the same or even worse OS in combinations that give improved PFS, possibly because the side effects ultimately overwhelm any short term benefit. Companies need to present the entire data set, stop cherry picking, and let patients decide what their own priorities are.

    1. Anonymous says:

      Yep this is an insightful comment also. No difference in overall survival but at least the drug isn’t worse. Drug keeps disease away longer by a statistically significant amount. Although that didn’t translate into people living longer, it might have improved quality of life by improving disease symptoms. Also, if there are less side effects (I saw something about thromboembolism but didn’t r as carefully) that’s a definite plus.

      Of course none of this is reported so we won’t know until phase 3. But again, it’s not the end of the line for he drug and it’s debatable whether phase 3 is worth it. Not a sure fire ‘chuck it in the trash bin’ yet.

      1. RandomGuy says:

        I have heard from my connections in the biopharma world that unfortunately the FDA has not been accepting “quality of life improvement” as an endpoint for new drug approval without an improvement in OS vs. existing treatments. PFS is only considered approvable if there are no other treatments available – otherwise OS needs to be demonstrated as well.

        1. me says:

          It’s “non-inferiority.”

  8. anon says:

    Not much different than publishing papers with cherry picked data and yields.

  9. Captain of the Ship says:

    Mr. Lowe,

    You refer to of bio_closeau’s tweet that this was a poor survival result in an open-label trial, but in fact his *point* reflects a draconian ignorance of treatment for this deadly disease. Regardless of treatment arm, the doctors are trying to extend all patients lives. Thus, the open-label nature of the trial can have no bearing on Overall Survival.

    In addition, your support of Adam Feuerstein tweets that HALO’s “cherry-picking” of data was “shameful conduct” is absurd. In fact HALO has provided all data possible in what was a complex halted and resumed two-stage trial – it is in fact Feuerstein and you who are “cherry-picking” morsels of doubt in exploratory and/orsub-group data.

    The essential fact is that the Stage One PFS efficacy has been confirmed in Stage Two, effectively two independent outcomes. While often Adam Feuerstein is spot on in his commentary, in this case the “shameful conduct” is his journalism, referencing the shallow analysis on uninformed Twitter egoes.

    1. Hap says:

      Overall survival for a cancer drug is a pretty damn big morsel. No survival benefit isn’t going to get you a lot unless you can prove something else significant (probably cost of treatment – drug prevents other expensive secondary problems) to insurance companies. No overall survival and barely significant PFS = another place where your long-term stock portfolio goes to die.

  10. luysii says:

    A paper from Johns Hopkins cherry-picked their data exactly the same way when they reported on mortality vs. socioeconomic status on entry to JH Medical School — for details — https://luysii.wordpress.com/2009/10/05/low-socioeconomic-status-in-the-first-5-years-of-life-doubles-your-chance-of-coronary-artery-disease-at-50-even-if-you-became-a-doc-or-why-i-hated-reading-the-medical-literature-when-i-had-to/

  11. Jeff says:

    Ouch, Derek you and Adam are way off on this one (which is rather rare for you both). I’ve been doing pure cancer P2/3 trials for a bit now, and a trial of this sample size, with those event rates probably has like <30% power to detect a difference on OS. The fact they hit PFS in a study this small is a bit impressive, and a doubling of PFS and that low of an HR is a good sign. What you should concentrate on is their subgroup parsing, and the Stages those are a bit dicey.

    OS is so impossible in these trials for a ton of reasons I don't have time to lay out, but most notably patients go on to get other treatments after they progress. Several of the Avastin trial's are hit hard by this, as many/most of the patients on the control arm all start taking Avastin as soon as they progress, because the physicians were seeing it working during the trial. Dropouts/data censoring is another other big issue. They don't give full disposition information, so it is hard to tell how much that may impact OS.

    If you look into this trial specifically, 40% of the total patients discontinued active treatment due to a clinical hold that happened for safety reasons (Stage 1). Then they added a prophylactic treatment and restarted (Stage 2). So all those Stage 1 patients did not get full treatment with the drug, likely confounding the overall OS results which you are showing. In the Stage 2 group there is maybe a hint of an OS benefit, subgroup is so small you can't draw conclusions.

    As for improper scientific spin, they actually lay out a ton of data. My conclusion is that the trial indicates a likely active combination, but they direly need a large prospectively enriched for high-HA Phase 3 trial to prove it. Likely 150-200 patients.

    1. Anonymous says:

      Yeah you know what though. All the data are presented. The only thing they could be criticized for is not being more blatant with what it all means. Like the median overall survival is better with the drug, that’s no lie and the data is right on the slide. The lack of statistical significance is no lie, and the data is right on the slide with HR and CI. Could they be more clear? Yes, but they didn’t seem omit anything critical-they just left it there for you to figure out on your own.

      I dunno I think the message is just to wait for phase 3.

    2. Eric says:

      Insightful comment and it’s nice to see Derek acknowledge it above. It seems a little unfair to criticize Halozyme because the headlines cited above are so glowing. Halozyme didn’t write the Reuters headline after all. Also, I can’t fault Halozyme for being excited that the primary endpoint was statistically significant (even if it is just 0.05).

      This whole thing just strikes me as the media and the market watchers like Feuerstein being too reactive. It’s encouraging phase 2 data. It needs a bigger more robust phase 3 study to confirm it which is exactly what Dr Hingorani says in the Halozyme press release. If the overall survival doesn’t look better in phase 3 then it’s a bust. That’s science and usually the readers of this blog are well aware of it and a little more restrained.

    3. Hap says:

      What is the point of P2, though? If you can’t tell if your treatment changes PFS, let alone OS, then what are the results of the trial able to tell you?

      1. Eric says:

        The point of the phase 2 is to give the company a go/no-go decision on running phase 3. The study is too small to provide meaningful OS data, which is why PFS is used as a surrogate. Presumably the study was powered to measure PFS (they did find it to be significant although that doesn’t necessarily mean it was well powered). The assumption is that PFS is predictive of OS, which may or may not be the case. That’s where the sponsor has to make the decision on whether a phase 3 investment is worth the risk. The same approach utilizing a surrogate phase 2 endpoint is used in many indications with rare events such as osteoporosis or CVD. It’s just that the surrogates may be better or worse predictors in different therapeutic indications.

    4. anon says:

      No, it is not impressive to “hit” PFS in a small study. It is more likely that it is statistical drift of the two small groups.

  12. Anon says:

    Just waiting for the post-hoc analysis which shows that white men aged between 56 and 57, who happen to be called Cuthbert, show a pronounced survival benefit.

    Good grief, doesn’t anyone study basic stats at school anymore?

    1. Anonymous says:

      They all study it but when money is at stake, especially their own, then it Katy bar the door.

  13. FOTD says:

    While I have no doubt Halozyme will need ph3 results for any chance at registration, I also have no doubt the presented data is much more positive than what we knew about PAG in pancan last week. Last week we had stage 1 data from Higorani’s ASCO presentations. The data was weird and poor, especially with OS due, largely we presume, to 40% of the active cohort stopping PEG. There was also M&M associated with the increase in TEEs in the PAG arm. Those who follow this company were always ready to chuck the stage 1 data out the window. So, we waited for Stage2. Halozyme had to present the entire data set but AF and others perhaps don’t realize that only the new and cleaner data of stage 2 (on Lovenox) mattered much to shareholders.

    Unfortunately, the n is nowhere near large enough with stage 2 to make statist. But the numbers make sense in the context of historical data. Consider these generally accepted numbers: (RR/PFS/OS)

    G alone: 7-9.4% / 3.3-3.7 mos / 6.8 mos
    AG: 23% / 5.5 mos / 8.5 mos
    Mod FOLFIRINOX: 32% / 6.4 mos / 11.1 mos

    Now, know that these numbers are for all stage3/4 pancan and the third of patients who have high HA status consistently do less well than those with low HA.

    The stage 2 PAG numbers we saw yesterday look excellent in light of what we knew last week. Thus, the increase in SP. I anticipate now more rapid enrollment in the ph 3 trial and have more confidence going forward in its outcome, both PFS and OS.

  14. Michael McG says:

    Does the drug have to show a statistically significant difference in the pre-specified end-point in order for the company to proceed to Phase III trials?

    1. Fenichel says:

      The purpose of Phase 2 trials is to help the sponsor decide whether or not to keep spending money on the project. Phase 2 trials don’t have to have significant results; they don’t have to have worthwhile endpoints; they can do whatever the sponsor is willing to pay for. The only limits in advance are that they have to pass IRBs. FDA will be interested only if (a) the results are sufficiently bad to warrant imposing a clinical hold, or (b) the results seem to be sufficiently good that the sponsor wants to claim that they are sufficient grounds for approval or, at least, for a Breakthrough or Fast-Track designation.

  15. Paul Brookes says:

    So, going back to the basic biology, there are good reasons why this therapy appears to have failed. The naked mole rat is a long-lived rodent species (lives several decades), and this is in part do to an excessively high cancer resistance. My colleagues in Rochester have shown that this resistance is due to a very high molecular weight form of hyaluronan (http://www.ncbi.nlm.nih.gov/pubmed/23783513). As such, anything that decreases the MW of hyaluronan might be predicted to remove this cancer resistance pathway.

    1. Michael McG says:

      What about the research that says removing high-density HA increases tumor penetration of chemotherapeutic agents?

  16. Jamtomorrow2 says:

    I wouldn’t want to be working with you in Rochester. Your mole rat argument is laughable. These patients are dying of pancreatic cancer. What do you want to do – increase the HA levels? A bit late methinks.

  17. Mach says:

    Yeah, its great chopping up HA into little bits until those low MW HAs cause inflammation and tumor cell proliferation. While it may help it may also hurt.

    But with pancreatic cancer anything is worth a shot.

    1. Vader says:

      My thought is that injecting something that breaks down intercellular matrix is a pretty drastic measure. But, as you say, it’s pancreatic cancer, so desperation is understandable.

  18. E.T. says:

    Are the PFS survival curves available somewhere on the web?

  19. Scott says:

    “Good grief, doesn’t anyone study basic stats at school anymore?”
    No, most people don’t study ‘sadistics’, it’s not required in the most popular degree fields.

    Worse, it usually takes about 3 weeks into your second semester of stats before it suddenly makes sense.

    I default to quoting Mark Twain most of the time: “There are three kinds of lies: lies, damned lies, and statistics.” My father, a math PhD, calls statistics “creative lying with numbers.”

    And I almost always call out social-science statistics as violating the first rule of statistics: always compare like with like.

Leave a Reply

Your email address will not be published. Required fields are marked *

Solve the math problem. * Time limit is exhausted. Please reload CAPTCHA.