Skip to Content

A Clinical Trial Torpedoed By Fraud and Incompetence

Via @AndyBiotech on Twitter, here’s a story on some very troubling developments in offshore clinical trials. That Cardiobrief article is referring to this letter in NEJM, and the subject is the NIH’s trial of spironolactone in heart failure patients. The TOPCAT trial enrolled 3445 participants  in 6 countries (1151 in the US, 326 in Canada, 167 in Brazil, 123 in Argentina, 1066 in Russia, and 612 in the Republic of Georgia). These were supposed to be patients who had heart failure with preserved ejection fraction (HFpEF), a group that makes up about half the population that have congestive heart failure.

The problems with the trial were apparent rather quickly. There was a significant regional variation in its results, and attempts to iron those out by looking into differences in diagnosis and patient classification didn’t explain them. Spirononlactone showed statistically significant benefits (not huge, but real) in the North and South American patient cohort, but no difference at all in the Russia/Georgia group, and the event rate was much higher in the former as well.

This new letter advances the least welcome explanation for these discrepancies. The authors have gone back to the repository of blood samples taken from patients during the trial and analyzed them for canrenone, the main metabolite of spironolactone. It’s present in 97% of the blood samples from the patients in the Americas, and in a dose-responsive manner when compared to patient status. But in the Russia/Georgia samples, 30% of them show no metabolite at all, and there is no overall dose-response relationship. The authors conclude that “Our findings suggest that the trial results obtained in Russia do not reflect the true therapeutic response to spironolactone“, and that’s the sort of statement that’s made through clenched teeth.

These numbers back up an earlier article which drew what has to be the most likely conclusion: that a significant number of patients in Russia and Georgia never ever got the drug. The Cardiobrief piece goes further, airing out the speculation that the spironolactone was, in fact, sold off on the open market by trial personnel rather than actually used as intended. That’s the first thought that occurred to me, too – it had to go somewhere, right?

The whole debacle brings to mind a paper by John Ioannidis and co-authors a few years ago that looked at clinical studies in various countries. There were significant differences noted in the trials conducted in less-developed countries without a tradition of clinical research, and the paper suggested that these factors needed to be taken into account. As clinical trials become more globalized, it looks like these conclusions are more noteworthy than ever. It’s hard enough to get solid numbers out of human clinical data as it is. Trial design and patient selection are issues that any clinical team wrestles with extensively before the first patient is even dosed. But if all this time, effort, care, and money is then wasted by people who don’t even bother to give the drug to the study group, well. . .

So if you’re thinking of running a big clinical trial in Russia or perhaps one of the former Soviet republics, you might want to consider the example of TOPCAT. This warning goes for a number of other countries as well, but I suspect that the NIH investigators expected better when they chose Russia and Georgia for their own study. I have worked with a number of excellent Russian scientists during my career, and I’m very sad for their sake that their country’s reputation has ended up where it is. But if you don’t believe that this is what things have come to, go try to find some spironolactone metabolites.

34 comments on “A Clinical Trial Torpedoed By Fraud and Incompetence”

  1. Polynices says:

    It’s almost like doing stuff in lower cost countries is a false economy. Thanks for sharing the story.

  2. Eric says:

    Trust but verify.

    That’s why we take PK samples.

  3. oncodoc says:

    Fraud is a harsh word unless there was actual intentional deception. People are noncompliant with medication for lots of reasons. I remember studies by Louis Lasagna from forty years ago showing 30-40% of CHF patients without detectable digoxin metabolites when that was the standard treatment. The 97% rate in the Americas is exceptional, but I doubt that this is achievable in clinical practice. Ars longa; vita brevis est.

    1. tnr says:

      Yes, the reported compliance rate in Russia is much higher than in the US. It is possible that 30% are non-compliant in both countries but in the US, the majority of non-compliant subjects have no problem indicating they are non-compliant while in Russia, the majority say they are compliant.

    2. Kevin H says:

      The 97% rate in the Americas is exceptional, but I doubt that this is achievable in clinical practice.

      To be clear, the NEJM letter notes that they tested for metabolites only in the patients who self-reported compliance. In the U.S. and Canada, 76 patients out of 101 (75%) self-reported taking the drug as directed. The follow-up testing was only performed (or at least reported) for that group of 76 patients–not the entire cohort. Among those 76, 73 (97%) had detectable metabolites in their blood samples. In other words, in the U.S./Canada group, trial patients (or the trial staff collecting the responses…) were accurate in reporting compliance. Overall only about 73% of patients were taking the drug (not far from the Lasagna number you referred to), but those who stopped were adequately self-reported and identified.

      In contrast, 66 out of 70 patients (94%) in Russia were reported as compliant. When those 66 were tested, only 46 (70%) had detectable metabolites in their blood. Overall, that’s a compliance rate of about 66%, which isn’t substantially different from the North American numbers; by itself that’s not a problem at all.

      The problem arises because 30% of the Russian patients who were reported as taking the drug actually weren’t. When we don’t know who’s really taking a drug and who isn’t, we can’t determine if the drug has an effect. And we don’t know who was lying to whom here–did the patients discontinue the medication without telling the study staff, or did the study staff – for whatever reason – try to conceal the information from their collaborators? Quite strange.

      1. tangent says:

        > In contrast, 66 out of 70 patients (94%) in Russia were reported as compliant.

        Yeah, this is the red flag. I suspect we’d find that maybe the Russian patients were just assumed to be compliant unless they spoke up otherwise — or the staff inquired but applied some pressure to the patients or to the data afterwards.

        Was there a financial incentive to show higher compliance, such as perhaps payment per ‘compliant’ subject?

  4. PharmaHeretic says:

    It is interesting that you focus on poorly done clinical trials in countries other than USA.

    What are your opinions on clinical trials in USA where children as young as three were dosed with anti-psychotics to treat ADHD- by physicians with positions at teaching hospitals attached to Harvard? Or what about clinical trials for many SSRI-type anti-depressants where the test population (and data) were carefully manipulated to show efficacy when it was absent in the raw data? Or perhaps you might want to comment on how Merck used a variety of legal and extra-legal methods to suppress the disastrous incidence of side-effects with their COX-2 inhibitors?

    I could go on.. but you get the point. The days when people in rest of the world believed that people in USA, and rest of the west, were more honest are over- irrespective of what people like you believe.

    PS: I am not sure why doctors in Russia would sell Spironolactone to other patients in their practice since it is a pretty cheap drug, with many undesirable side effects- especially for men. Perhaps that is why the rate of compliance was so low..

    1. Undereducated_Observer says:

      He wasn’t disparaging non-US countries. The other participating countries referenced included not only the US, but also Canada, Brazil, and Argentina.

      “The Americas” refers to North, Central, and South America collectively, not just to “the United States of America”

    2. loupgarous says:

      “When you hear hoofbeats, think horses, not zebras.”

      I have to agree – not because Derek and the study organizers are ganging up on Eastern Europe and Russia, but because spironolactone isn’t a PDE5 inhibitor with a relatively low adverse effect profile and one splendid side effect, but a drug which makes you break out in a rash sometimes, puke sometimes, and not only doesn’t give you wood, but takes it away sometimes. So 30% patient noncompliance (especially among the men in the study) shouldn’t come as a huge shocker. The only thing that stuns me is that the compliance rate was as high as it was in Brazil and Argentina, given we’re talking about Aldactone.

    3. anon says:

      How long have you been following this blog? Derek never misses a chance to criticize Russia.

      1. loupgarous says:

        I’ve followed this blog for about eight years. I haven’t noticed Derek playing the dozens on anyone because of their nationality.

        In fact, I don’t recall that many articles about Russia at all here – critical, laudatory, or otherwise. Mostly, when he talks about folks from other countries, he’s been poking good-natured fun at Klapotke et al for trying to send postdocs and lab equipment into orbit, making azides that God clearly never intended to be made.

        1. Kommi says:

          Klapotke, the scariest chemist, in the wooorld. Really though, I love reading the madness that comes out of that lab.

      2. Derek Lowe says:

        Unfortunately for that argument, I’ve actually missed several chances, from what I can see.

    4. David Young says:

      Spironolactone is a great drug for acne. Maybe there is a market there among adolescents that allow the sale of Spironolactone at high prices to these kids

      1. loupgarous says:

        Unfortunately, spironolactone’s a great drug for acne because it competitively blocks the androgen receptors, causes hypogonadism in men and has some progesterone receptor activity in women. If it were just zeroing in on the acne disease process, that’d be great, but what it does is more broad – spironolactone affects a lot of things, some of which cause acne to be worse.

        Spironolactone’s also black-boxed here in the US for its tumorigenic potential (in animals) – FDA’s advice to prescribers is “Unnecessary use of this drug should be avoided”.

        1. anon says:

          For certain types of acne the anti-androgen mode of action is actually what makes it useful. I can’t exactly recall the discussion I had about it, but for people who have tried everything (and who aren’t in groups where the anti-androgen activity is a concern, particularly women who are pregnant or looking to get pregnant) for acne its pretty useful.

          The caveat: This is coming exclusively as a patient with a chemistry background. I had never been told by multiple MDs and derms that there was any concern other than that. To be fair though most of them keep telling me they are terrified of chemistry and maybe I should reconsider who I see and where.

  5. John Wayne says:

    A relative of mine used to work at nursing homes. If you knew how often nurses would hold back medications from residents and sell them on the side, you wouldn’t be impressed.

  6. Andy says:

    I’d be impressed if it was never.

  7. Thoryke says:

    Well, won’t _this_ make the pitch at the “Come run your clinical trials in Georgia” booth (say, at the upcoming DIA conference in Chicago) rather interesting?

  8. Abraxane 's real efficacy? says:

    “So if you’re thinking of running a big clinical trial in Russia or perhaps one of the former Soviet republics, you might want to consider the example of TOPCAT”

    Alternatively, remember the confusion when Patrick ShoShin’s pancreatic clinical trial was successful, mostly because of surprising positive results that came from Russia.

  9. Manonymous says:

    Regarding clinical trials run outside of the West, many remember the case of Medivation’s Dimebon. One of the worst has to be the example of Targacept’s TC-5214, which precipitated its slow and painful death, as well as caused major changes at AstraZeneca.

  10. loupgarous says:

    The CardioBrief article “Serious Questions Raised About Integrity Of International Trials” pretty much buried the real lede: the TOPCAT lead investigator Marc Pfeiffer and other prominent cardiologists are now advising those treating heart failure with preserved EF to consider prescribing spironolactone.

    While it’s usually considered dodgy to drop poorly-responding populations from a study post-hoc, in this case you have a really significant study compliance issue (either that, or thirty percent of the Russians and Georgian patients studied are non-metabolizers of spironolactone) for that population. The end result is the TOPCAT study results are now re-evaluated to reflect what happens in patients who take spironolactone and metabolize it, and it’s being recommended in heart failure with preserved EF.

    It’s bad news for the reputations for the study organizers, but if there was fraud, it probably didn’t involve the investigators from the Western Hemisphere.

    That’s a big “if”, by the way. I wouldn’t take spironolactone, there are other diuretics out there. And if 30% of the study cohort in Russia and Georgia either experienced the side effect that makes spironolactone a part of gender reassignment therapy or Googled the drug and decided “No, thanks!”, that’s not necessarily fraud. It could be a cultural variation in the TOPCAT study outcome.

  11. Anonymous says:

    I only read Derek’s summary and not the original articles. Derek wrote, ” The authors have gone back to the repository of blood samples taken from patients during the trial and analyzed them for canrenone, the main metabolite of spironolactone.” Present / not present in Americas / Euro samples.

    Does the paper say when the study was done (blood samples collected)?

    If they screwed up the original study, maybe they also screwed up some or all of the stored samples or had to come up with some samples in a hurry and drew some bloods from non-trial patients. Looking for other markers could help … and cost as much as running a new properly managed trial.

    If the hemispheric difference results are ever validated, it might mean the Euro group has suped up steroid metabolism taking down spironolactone and metabolites much faster than the Americas group.

    Spironolactone is also being studied for the treatment of Duchenne Muscular Dystrophy (not mentioned on the wikipedia page).

    1. loupgarous says:

      In the NEJM article describing the blood sample analysis, “Spironolactone Metabolites in TOPCAT — New Insights into Regional Variation”, blood samples taken from 366 patients (206 patients from the United States and Canada and 160 patients from Russia) at the 12-month study visit were examined for one active metabolite, canrenone, serum potassium and aldosterone levels were all examined. These patients had consented to the use of these samples in a “biorepository”, which seems to imply the main investigators had custody of them.

      Quoting the NEJM article: ” Regardless of the region, the participants with detectable canrenone concentrations had significant increases from baseline in potassium and aldosterone levels (a finding that is consistent with the expected effect of spironolactone), whereas the participants with undetectable canrenone concentrations did not ”

      So three markers, not just canrenone, were assayed for, and the relationship between them was consistent with what is seen in people who take spironolactone.

      There are several possible reasons why 30% of the Russian patient samples lacked detectable levels of canrenone at the 12-month study visit. The half-life of canrenone in the body is 16.5 hours, so that study non-compliance might have started not long after patients began having adverse events. If they were paid for their participation in the study, they might have shown up regularly to get their money regardless of whether they actually took the study medication. Or local investigator fraud may have occurred.

      Or there’s something odd about the metabolism of the 30% of the patients whose 12-month visit samples were clean of canrenone – though if either of the other two major active metabolites of spironolactone were present, you’d expect to see elevated potassium and/or aldosterone levels.

      Reading the NEJM letter, I don’t see evidence favoring either fraud by local investigators or by patients not taking the study drug early enough that canrenone couldn’t be detected 12 months into the study. Abnormal metabolism is possible, but it doesn’t look plausible from the abstract.

      I think that the talk about fraud’s premature – but the remarks about the wisdom of overseas clinical studies without at least a local representative of the group commissioning the study present to check for compliance are justified.

      I’ve worked on analyses of several multicenter, multinational clinical trials of new drugs or new indications for drugs, and I’ve seen inconsistent data have prompt visits to local investigator sites for examination of clinical research forms along with patient charts (the unfortunate expression “cleaning the data” was used more than once, which implies post-hoc manipulation of data I don’t think actually happened). And I’m talking about both company-run studies and CRO-run studies.

  12. ex medchem says:

    It reminds me of the Dimebon Trial for AD where the phase II, in Russia only, showed very positive results and the Phase III worldwide showed nothing.
    http://www.alzforum.org/therapeutics/dimebon

  13. steve says:

    Did they also assay for Smirnoff levels? The most likely explanation is that Smirnoff accelerates hepatic metabolism of canrenone. 😉

    1. loupgarous says:

      Steve: “The most likely explanation is that Smirnoff accelerates hepatic metabolism of canrenone.”

      I was once at the same table as a Russian cardiologist who ran a largish, prestigious clinic near Chelyabinsk and one of my bosses, a partner in one of the leading cardiology institutes (with many clinics, and an early participant in multi-center trials of intracoronary stents and new nuclear cardiology procedures) in the Gulf South.

      We were reducing the fluid level in a 750 ml bottle of vodka brought over by our guest, with me as translator. The topic turned to how alcohol consumption influenced late-life incidence of arteriosclerosis and coronary artery disease, and (however jokingly) there was some conjecture that early and regular drinking actually reduced cholesterol levels (as the liver converts carbohydrates to fats for storage in the human body).

      In fairness to all concerned, I don’t think anyone at the table would have advanced that hypothesis seriously, but Steve’s remark got me thinking – what if the thirty percent had impaired hepatic function and really weren’t metabolizing spironolactone to canrenone – or to anything else?

      Spironolactone’s a prodrug. Its action depends on uptake of its three major active metabolites. If those aren’t being made (for whatever reason) in 30% of patients, then you’d have treatment failure in 30% of the Russian cohort with nothing sinister happening. Liver enzymes, you’d think, would be as easy to assay for as canrenone. It’s something to investigate, and not just to either convict or exonerate the Russian site investigators of fraud, but to examine why spironolactone might not work for everyone.

  14. Anon_III says:

    In my mind one possible explanation is that the labelling was off somewhere, either in the drug supply or in the storage of the blood samples. It would be interesting to know if 30% of the blood samples taken from the placebo arm showed significant levels of Canrenone.

  15. Dan says:

    Academic trials like this one are very different to trials sponsored by pharma companies. Was this trial even monitored by the sponsor?

    1. loupgarous says:

      Good point. In my experience working with really serious CROs and drug companies, site visits to all centers in a multicenter phase III trial are done as a matter of course, You have to wonder, when the discrepancy in Eastern European centers in that trial came up, why there weren’t immediate site visits, with examination of patient charts for concomitant disease processes or other confounding factors.

      I’m very sure, had this been a new drug or an existing one being tested for a new indication by a top-line CRO or a major drug company, that site visits would have been scheduled the very week those results turned unaccountably negative for two national populations in the study cohort.

  16. Li Zhi says:

    It is a constant source of amazement to me that people appear to believe that honesty is the rule rather than the exception. Deceit is a competitive advantage, both from an evolutionary perspective as well as from a game theoretic perspective. Information is power. Who wouldn’t lie to protect their families or friends? And in an “us versus them” world, isn’t it the norm? An this doesn’t even include our ability and propensity to say one thing and mean another, eg. Climate Change is a fraud = I don’t want to pay more for energy. I don’t want to rock the economic boat. I don’t want to consume less.

  17. Cymantrene says:

    Up to my knowledge, since 2010 in order to register a medicin in Russia one should have a clinical trial in Russia.
    So pharmaceutical companies should have some experience on this topic. It would be interesting to compare it to the aboves.

  18. Paddy says:

    As someone pointed out there should have been some form of monitoring in place e.g. drug accountability logs . These should have been checked before people start talking about fraud and selling off medication in the local market

  19. Guppy says:

    Did they consider the possibility that the blood samples themselves were fraudulent? For instance, that the project’s phlebotomists didn’t bother to tap most of their patients, but just dispensed aliquots of blood from some other source?

    Or even that maybe a certain portion of the patients didn’t even exist (except on paper), and that the blood samples from these patients were sourced from some shady blood bank or such.

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.