Skip to main content

Academia (vs. Industry)

Wasted Money, Wasted Time?

Now, while we’ve been talking about how much basic research is done in industry, or how much clinical research gets done in academia, here’s something that might bear on the discussion. Too much of what looks like useful clinical research on the academic side is actually wasted effort. The New York Times has been running a series called “The Forty Year War”, looking at the history of the “War on Cancer”, and the latest installment is on clinical trials.
It’s been a problem for some time now that there aren’t enough patients to go around for many cancer trials. Breast cancer is an especially problematic area, last I heard. It’s high-profile, fairly high-incidence, and a lot of investigational anticancer agents are lined up to take a whack at it. So many, in fact, that there aren’t enough breast cancer patients available in the US, nowhere near, and the same situation obtains in a number of other areas.
Much of this problem comes from low recruitment rates. As the Times article makes clear, only three per cent of adult cancer patients are enrolled in any kind of trial at all. Many cancer patients want to stick with the best therapy that’s currently known, and don’t want to add any uncertainty to what they’re already dealing with. It’s hard to blame them, but that does make the state of the art advance more slowly.
Another factor that may come as a surprise is that many oncology practices find that they lose money by participating in trials. The reimbursement-to-paperwork ratio doesn’t always come out very well, especially for centers that don’t do a lot of clinical research and haven’t been able to streamline the process as much as possible. When they look at the number of patients that they can serve, given the time that’s taken up, the trials start to make less sense.
Finally, and this is the least excusable factor on the list, there are many trials that really shouldn’t be run at all. The Times does work in a line about how some studies by drug companies are just “designed to persuade doctors to use their drugs.” My take on that is that these studies usually are designed to do that by showing that their drug actually works better, which is not such a bad thing. But note this other problem:

There are more than 6,500 cancer clinical trials seeking adult patients, according to, a trials registry. But many will be abandoned along the way. More than one trial in five sponsored by the National Cancer Institute failed to enroll a single subject, and only half reached the minimum needed for a meaningful result, Dr. Ramsey and his colleague John Scoggins reported in a recent review in The Oncologist.
Even worse, many that do get under way are pretty much useless, even as they suck up the few patients willing to participate. These trials tend to be small ones, at single medical centers. They may be aimed at polishing a doctor’s résumé or making a center seem at the vanguard of cancer care. But they are designed only to be “exploratory,” meaning that there are too few patients to draw conclusions or that their design is less than rigorous.
“Unfortunately, many patients who are well intentioned are in trials that really don’t advance the field very much,” said Dr. Richard Schilsky, an oncologist at the University of Chicago and immediate past president of the American Society of Clinical Oncology.

I don’t want to dump a bucket of tar on all academic and publicly funded clinical research, because there’s a lot of good stuff that goes on as well. (And remember, the publicly basic research is very valuable indeed). But the next time someone tells you about the number of clinical trials run outside of the drug industry, you might want to keep those above figures in mind.
Not all trials are created equal, not by a long shot. But the ones that we run in industry, from what I can see, tend to have a better chance of relevance. That’s partly because we’re spending our own money on them, and with a goal of finding drugs that people will spend money on in turn. It focuses one’s efforts. It’s not like we never waste money in this business, but I’m very much willing to bet that we waste it less often than happens with public funds. Companies trying to get an agent through the clinic tend not to set up meaningless trials just to make everyone’s resume look better. That I can tell you.

24 comments on “Wasted Money, Wasted Time?”

  1. Don Monroe says:

    Don’t NIH-sponsored trials have to demonstrate adequate statistical power to be funded? Are this boutique trials funded independently?

  2. John Johnson says:

    Don – not small safety trials, typically run close to the beginning of a clinical development program and which make up a very large chunk of trials run. Think about it – not much information is known so showing adequate statistical power is like showing you’ve got a good chance to win at darts with the lights out.

  3. Tok says:

    Derek, could you please back up the second paragraph in your quoted passage? It sounds very much like the last paragraph you quoted in your “Where Drugs Come From, and How. Once More, With A Roll of the Eyes” post and has the same amount of evidence to support it. Did the author of the NYT article find NIH funded academics that admitted to wasting public funds to “pad their resume”? Sounds like this article may have committed the same sin as Domenech’s article. However, compare the response you gave that one with the response you gave the NYT article.

  4. Hap says:

    The only comfort in the low number of enrollees in clinical trials is that the hesitance of people to sign up for experimental therapies implies that they believe the ones currently in use to be pretty decent. People are likely to be fairly risk-averse about things they know little of (and when the stakes are high – life is what we know), and so the willingness to try something new implies that they think the therapies present aren’t good or have a really high cost. Thus, the low number of people in trials could be (at least in part) an implicit endorsement of the ability of drug companies and others to deliver effective cancer therapies.

  5. A testable implication of Hap’s suggestion is that enrollment should be easier to obtain the more hopeless the indication; does anyone know of a study of this?
    For something like adenocarcinoma of the pancreas, where median survival is well under one year & 5 year survival very near 0) would expect higher patient interest in experimental therapies versus a cancer with a high remission rate. But does this hold in practice?
    Another significant factor is the strong negative cultural bias against being in studies (“human guinea pig” is not a phrase most people seem to smile while saying), abetted by such horrors as the Tuskegee experiment.

  6. Karen says:

    One interesting note about cancer clinical trials. The majority of children with cancer are enrolled in clinical trials. And the cure rate for cancer in children is significantly higher than it is for adults. (Are the two things related? It seems likely.)
    But it’s more difficult for adult cancer patients. I was treated for cancer about a year ago, and when I was asked to be in a clinical trial, I accepted, even though it was comparing two known treatments. (As a chemist, I was familiar with clinical trials and I wanted to help.) But it wasn’t as easy as I’d thought. Money was a big issue. Most clinical trials will not pay for any of the tests or treatments, and most health insurance companies say they won’t pay for any “extra” bills related to the trial. So if you have to get a blood test just for data for the trial, your insurance might or might not pay for it. You might get stuck with the bill (and when it comes to scans, for example, that could be thousands of dollars.). In my case, my insurance covered it all, but that was one reason I hesitated before joining a trial.
    I also ran into the problem of having severe side effects. My doctor didn’t want to switch me to a different drug, because it would mean taking me off the trial. I felt like my care was being put second to the needs of the trial. (I wish I’d insisted that he switch me, but I wanted to be a “good patient”.) It’s hard to look at it objectively when you’re in the middle of treatment.
    Doctors are so hurried these days – my oncologist barely spent 5 minutes with me during my appointments – that I really didn’t get a chance to get information about the different treatments. The clinical trial stuff was something else my doctor had to fit into those 5 minutes before he rushed off to someone else. Being part of a clinical trial just added to the burden on me, without giving me any tangible benefit. No wonder patients are wary.
    (However, I was disappointed to read the comments from this article at the NY Times, and see how many of them said things like “pharmaceutical companies are covering up the cure to cancer so they can make money”. It’s sad to see how badly people view pharma and don’t recognize the contributions that have been made in terms of cancer drugs. I was frustrated by my experience with a clinical trial, but that has more to do with how they’re run.)

  7. Lucifer says:

    While I am no defender of academia, a lot of what you say about the role of academia in drug reserach would sound better if big pharma had not wasted the last 20 years on M&A, new paradigms, MBA bull and other assorted acts of management and legal insanity.
    Big centralized “scientifically managed” bureaucracies and fiefdoms do not encourage innovation.

  8. DylanE says:

    “Thus, the low number of people in trials could be (at least in part) an implicit endorsement of the ability of drug companies and others to deliver effective cancer therapies.”
    It matters more what the difference in expectations is between existing therapies and what a new therapy might bring, if that difference is perceived as negligible on the efficacy side, but you know there will be hassles involved with being in the trial (and oh yeah, the experimental drug just might make you sicker) I can definitely see people opting to stick with the devil they know.
    Still though, the numbers surprise me. I tend to think of America as a pretty optimistic country, especially where it comes to technology. You would think that Hollywood has trained enough people to expect miracle cures everyday that people would be jumping to be in trials though.

  9. Hap says:

    We like neat things, but we want someone else to pay for them, which makes it hard to appropriately allocate money and effort and time for things (because how much money is spent may not be an accurate indicator, nor is what we say we want). If this is true with money, which is replaceable, how much more likely is it with are lives, which are not?

  10. hibob says:

    @Hap: “the hesitance of people to sign up for experimental therapies implies that they believe the ones currently in use to be pretty decent”
    Well, the article gives as reasons for the lack of patients 1, money – putting patients in a clinical trial often costs both the patients and their oncologists money, (see Karen’s comment above); 2, time – patients complain about having to go farther away for more frequent treatments, oncologists lose more hours to paperwork; 3, fear of being put on a placebo, even though cancer trials are generally done with the standard treatment as the control arm of the study.

  11. Sili says:

    You’ll hate this, but my initial thought is that this stuff is too important to be left to individual egos.
    With a shortage of patients it would make sense to have a central centre evaluating the trials beforehand to make sure the useless resumefodder doesn’t get in there. I doubt it’s the case everywhere yet, but my impression is that these days in order to do experiments on animals it’s necessary to present ample evidence that 1) the test has not been done before (or reproduction is absolutely essential), 2) the animals won’t suffer needlessly and 3) the experiment can be expected to produce good results – that is, it’s conducted in a professional manner.
    Is it too much to ask the same be done for people?
    Secondly, by having all trials go through a clearinghouse, it should be possible to have a suitably randomised group ready beforehand. The patients can be signed up for a trial before assigning the drug in question (that will of course need tweaking in the case of semi-personalised medicine).

    My take on that is that these studies usually are designed to do that by showing that their drug actually works better, which is not such a bad thing.

    I should hope so in a matter so dire. I have to admit that even I, cynical that I am, was dismayed at reading about the tricks that can be done to make a drug look as though it’s performing better as described by Ben Goldacre.
    Yet another reason that I’m increasingly in favour of trials being ‘outsourced’ to someone less biased. Something like having the FDA do the trials rather than just have the results presented to them.

  12. Cellbio says:

    Isn’t one problem with enrollment rates purely logistics? CLinical trials are usually done at major medical centers with a research focus and experience in running trials, with enrollment criteria that aim to define a population with some boundaries. If you live nearby, or can make it to a major hospital, at the time enrollment is open, and meet the enrollment criteria, well, then you might make it on the trial.

  13. Sili says:

    (However, I was disappointed to read the comments from this article at the NY Times, and see how many of them said things like “pharmaceutical companies are covering up the cure to cancer so they can make money”. It’s sad to see how badly people view pharma and don’t recognize the contributions that have been made in terms of cancer drugs. I was frustrated by my experience with a clinical trial, but that has more to do with how they’re run.)

    I’m shocked to hear how poorly you were treated, Karen. I’m glad you made through.
    My own mother wasn’t treated too well (she wasn’t in a trial), either. I guess you can take that as an indictment against ‘socialised medicine’, but I’d have hoped that more care were taken of patients in trials. I’m a (failed) chemist, myself, and I don’t think I’d hesitate to offer to help out when I get cancer. (Yes, “when”, not “if”.)
    Anyway, the reason I quoted you was the shear illogic of that sort of comment. After all: Dead people don’t buy Viagra. What possible benefit can ‘Big Pharma’ have from people dying?

  14. CMCguy says:

    I feel a bit conflicted here as I think academic and NIH in general being overly focused on the late discovery/early development of drugs (which is where seems to be trending) would be a huge waste of resources. However such groups can play valuable roles in certain parts of the overall process for at two reasons:
    1) Because of unique or strong expertise that does exists in fundamental diseases and treatment knowledge ideally there should be significantly more direct collaborations with Industry to identify and work toward new medicines. Unfortunately companies and institutions both each tend to create barriers attempting to “protect” themselves that largely inhibits complete interactions and cooperation (my guess is Industry types could greatly aid the failure to recruit studies by better inclusion/exclusion criteria). At the same time there is also an existing attitude that anyone who works/receives funding from Industry in automatically “corrupt” that likewise making opportunities of working together more difficult.
    2) In terms of “too many trials” I would say as eluded by several above it is more an issue of “too few potential participants willing” which is a big problem in US. Often exploratory trials (at lease MD sponsored) involve approved drugs or at least compounds with demonstrated safety so as long as there is good legitimate basis/theory (assuming IRBs effective gatekeepers) it is a speedy way to “test in actual patients” which, although somewhat scary to consider, can be an effective means to new treatments. Although I have seen some cases that looked like MDs attempting to enhance their ego most of the time there is a deep concern for patients plus an idea they are convinced will help. The success rate is probably very poor yet since the whole process is scattered with pitfalls so any means to get to a useful drug should not be discouraged.

  15. Joe O says:

    Hi all,
    This is my first posting on this blog after discovering it a short time ago. I felt this was particularly relevant to my field of expertise, so I hope no one minds if I explain a bit about what I do.
    I recruit patients all-day long for almost every chronic indication. Including cancer and alzheimer’s and pediatrics and eldery…
    I work for a recruitment vendor, a company whose purpose is to fulfill trial enrollment needs, cut enrollment timeframes (or rescue time over-runs), and/or replace the need to add more sites to meet an enrollment target. I also perform retention work, but that’s for another blog post.
    I mention this specifically becuase my company’s foundation is built on metrics. We track everything. Every. Single. Thing. We use every large tactic to build study awareness, and we track every result from the cost of the tactic, how many people responded, how many of those respondents were local to a site and had the condition, how many pass a pre-screening and WHY did they fail out, etc. etc. all the way to how many randomize.
    This is unique for two reasons:
    1. I’ve worked with and bid against all of my competitors and no one else has the analytics and capabilities we do.
    2. As an external vendor who has run recruitment efforts for ~50 different companies and tracked all the results, I’m uniquely situated within the industry to perform wide-ranging analyses on what the actual problems are in fulfilling trial needs.
    To be clear, I’m not an investigator or site. They deal with certain aspects of recruitment that I don’t, and though I have some expertise in those areas I don’t want to represent them.
    Also, I represent the difficult studies. Sponsors don’t come to me for the easy ones….so, here’s my .02 on a few points raised above.
    I figured I’d give my opinion on a few of the comments:
    Hap said, “People are likely to be fairly risk-averse about things they know little of ”
    I totally agree that that is the conventional wisdom. My experiences lead otherwise. I use direct-mail alot, which allows me to send out millions of pieces of mail to people who have a condition (say, migraines). I track every piece of mail and confirm whether a person responds to it, so I know what percentage of the population is responding to a generic migraine study.
    For migraines, I’ll write a letter saying there is a local study opportunity, some barebone I/E criteria, and mention some barebones potential benefits (whatever they may be, based upon the study)
    On average, I’ll get about 1.8% response to that letter. Give or take.
    This is actually incredibly high…1 out of 50 people who receive a generic letter in the mail about a study opportunity want to know more about it and find out if their eligible.
    I’ve seen it be 2.5-4% as well for other studies (A stress urinary incontinence study I ran had a 2.7% response rate, probably due to people hating the social stigma of their condition).
    My point is: people are very interested in study opportunities, but they don’t have the information set in front of them.
    CMCguy said, “I would say as eluded by several above it is more an issue of “too few potential participants willing” which is a big problem in US. ”
    See the above….vast amounts of people are willing, if they have some basic trial information and a gateway to determine eligibility.
    Cellbio said, “Isn’t one problem with enrollment rates purely logistics?”
    Yes, distance and willingness to travel is a huge facet once someone has been pre-screened in some manner and is actually faced with the decision to come in.
    And it’s very dependant upon age population, reimbursement for travel, and the therapeutic area (people with seizure disorders and unusual cancers will travel hundreds of miles to be in the right study).
    The first hurdle there is the pre-screening. In one of my recruitment campaigns, I’ll take the individual through an IRB-approved pre-screening questionnaire designed to explore phone/on-line screenable criteria….On a good study, 3/4 respondents with the condition will phone screen fail. 1/4 will pass and be willing to travel to the site.
    On some difficult studies, that’ll drop to 1/10 or 1/20. Which means I need to find 20 people with the condition who are interested in a study opportunity for every one I send on to the site….let alone for every 1 that passes the actual screening and randomizes.
    Protocol design, protocol design, protocol design.
    Hope this helps and I’d love to see some thoughts or any questions.
    Love this blog.

  16. Hap says:

    It’s useful to know I’m dunningkruegering myself. The information is interesting though.

  17. milkshake says:

    A tidbit from my ex (who spent many years working in pediatric oncology): the significantly higher cancer remission rates in children – as opposed to adults – are related to the fact that children tend to develop a different spectrum of malignancies, with different etiology. Tumors and leukemias in children generally tend to respond to aggressive treatment much better than in adults, regardless whether it is the standard regimen or experimental protocol. Plus kids are incredibly resilient and recover faster than adults.
    Also, as a long time pharma chemist, I would be very reluctant to participate in clinical trials, and I would discourage other – unless a clear convincing argument is given why they should benefit from the participating in the trial. Very often the interests of a company and their clinicians are not quite the same as the interests of their trial subjects, and with so much money and career advancement on stake it is easy for the professionals running these trials to forget about patients actually being people in desperate need.
    Once think i worry about is the clinical trials patients being “blinded by science” – being oversold a hype how revolutionary the new tratment could be, without being given the caveats (majority of clinical candidates fail in the clinic, some arms of the study will have non-optimal dosage or the duration of the treatment might be too short to have a long-term benefit, unforeseen life-threatening complication are more likely etc).

  18. Joe O says:

    Milkshake – Are you familiar with the informed consent process (beyond the basic concept, I mean familiar with the full informed consent form)?
    It won’t talk about overall industry failure rates, but it’s very clear about known risks, efficacy, dosing, placebo, etc.
    It’s very detailed and written at a low-grade level. I actually try to develop tools for investigators to use to consistently administer the informed consent to each individual (tools to make sure they hit all the important points and don’t forget to mention something).
    On another note: I’ve personally recruited somewhere in the neighborhood of 3,000 subjects who wound up randomizing in a trial. Probably about 100,000 subjects who expressed interest in one of my studies.
    I do so by specifically mentioning the loosest of potential benefits:
    – altruistic participation
    – compensation for time and travel
    – medical monitoring of their condition
    – potential to receive an investigational drug (we generally avoid the term medication or treatment to avoid implying a benefit)
    Finally, anyone who tries to get someone to participate in a clinical trial based upon the revolutionary nature of medication is treading on unethical behavior.
    Just my .02

  19. milkshake says:

    yeah, these forms are drafted by lawyers to the mind-numbing detail to list any fucking conceivable complication, mostly to shield you from malpractice and product liability lawsuits. It gives the patient a wrong proportion, with extra dose of legalese. Its the same problem like with MSDS: you simply cannot have an objective, impartial informative document drafted by thevery parties running study that at the same time is designed to provides them with an aliby. (Not mentioning that when a physician tells the patient in re-assuring voice that this new treatment is really the best thing for the patient, the patient is unlikely to study the fine print too closely and just asks where to sign). Borderline unethical behavior in your field is very common because the incentives are there to reward it. every time I go to see a doctor I don’t really have much different impression than when bringing a car into a garage for repair.

  20. Still Scared of Dinosaurs says:

    You are WAY off base about informed consent forms. I can’t ever remember a lawyer having input to one and they are clearly not designed to shield liability because that wouldn’t even work. It’s the best attempt to describe the trial to a non-expert and that’s it.
    One interesting thing about factors that affect enrollment is who gets your disease. It’s easier to get patients onto trials when the disease is prevalent in the poor because they don’t have health insurance – at least for industry-sponsored trials. We know that some patients sign up in part because it means the opportunity to see a doctor they otherwise can’t afford. Wonder how the OBama effort would change that.
    I also wonder how much of the MD’s lose money on trial patients is restricted to non-profit trials. In every case I’ve worked on we’ve paid for everything we asked for (and a lot we didn’t) few questions asked.

  21. G1 says:

    I guess with all this discussion I need to sign myself up for the restless leg syndrome (RLS) study I keep on seeing on the T.

  22. Joe O says:

    Milkshake –
    I can’t believe you work in the industry and have that opinion of informed consent….i guess information is even more silo’d than I thought.
    Informed Consent Forms are drafted by people whose job it is to convert technical protocol script into a readable, and IIRC 5th-grade-level document. It talks about why the study is being conducted, what previous studies on the drug have yielded as far as side effects (doesn’t talk about benefits), how many total HUMANS it’s been tested on to date…
    It goes on to talk about the study requirements of both visits, phone calls, time out of the patient’s life.
    If anything, the informed consent is meant to DISSUADE people who aren’t serious about participating.
    It’s reviewed and modified by trial managers, study physicians, etc.
    It then goes to the Institutional Review Board (IRB) which is composed of people who are dedicated to ensuring that no part of the trial is suggestive of benefit or coercive in anyway. These are community leaders, religious figures, physicians, etc.
    And for the most part, these IRBs take their jobs ridiculously seriously and constantly seek to expand their dominion to make documents as innocuous as possible.
    I’m in shock, and slightly sickened, that a “long-time pharma chemist” is so repulsed about the very research you help build. And that you think one of the cornerstones of human research is drafted by lawyers to either bore someone into signing or entice them through false promises.
    Clinical Research has some unethical stuff happen in it, but it’s mainly people who are trying to make it work and keep it as pure as possible. Ironically, I saw more unethical stuff affecting patients at the premier academic research hospital I worked at, than with the pharma industry.

  23. Anonymous says:

    Still Scared of Dinosaurs –
    The main time when, in an industry-funded study, an investigative site loses money (and then quickly becomes disengaged and stops screening subjects) is when the sponsor improperly benchmarks their protocol.
    To give an example: sponsor’s, in my experience, rarely know how to properly establish an expected screen fail rate unless the study is a minor variation of something they’ve conducted a few times.
    So they come up with some very generic screen fail rate like 50%.
    And then they structure the site contract to pay the site for each subject who passes screening and goes onto randomize (or similar).
    Now when the protocol is actually a 75% screen fail rate, the site is now screening 4 subjects and getting full payment for the 1 that randomizes….when the contract was setup with the expectation that 2 subjects would screen for every one than would randomize.
    In other words: the sponsor established a site contract with an expectation of far less work than the site actually performs for them.
    Those are the cases where, if the sponsor isn’t pro-active in re-negotiating the contracts, the site’s become very dis-engaged and stop enrolling patients.
    In my opinion….engaged sites are one of the pillars holding up successful study enrollment. And nothing dis-engages a site faster than spending 40 hours and getting compensated for 20.

  24. retread says:

    As a neurologist in private practice ’72 – ’00, managing glioblastoma multiforme (treat is far too strong a word for what I was able to accomplish) I never recommended chemotherapy — unless the patients could be entered into some sort of study (when I would push for it). I didn’t have any evidence that chemo helped much, and it certainly made patients feel awful (unlike the corticosteroids I would give them — which for a time shrunk the edema around the tumor and made them feel a lot better).
    An intriguing paper looked at why the results of published studies of chemotherapy were so much better than actual experience. Even though the paper is now 20 years old, I think it’s still relevant to this discussion.
    [ Ann. Neurol. vol. 26 pp. 531 – 534 ’89 ] Anaplastic glioma patients don’t do well. However at a center in London, Ontario, they seemed to be doing much worse than the literature would imply. Since the center sees all anaplastic glioma patients from that area, they investigated why this was happening. Only the younger and healthier patients entered the study. Only 40% of those eligible for the study entered it, for various reasons (poor neurologic function , too old, chose not to). The point is that the median survival in the literature and study patients was 60 weeks, while for all patients the median survival was only 35 weeks! They saw 197 patients over a 5 year period.
    The authors speculate that the older, sicker patients never get to tertiary referral centers so the 60 week median survival holds up. However this center sees everyone in its geographic area in Canada.
    If we are ever to make progress in this horrible disease, it will have to come from controlled studies — this is exactly the way progress was made in childhood leukemias.
    Parenthetically, the husband of a musician friend is now 3 years out with the diagnosis, walking (with help) talking (slowly) and painting, not the man he was, but still alive. The tumor hasn’t grown in size in the past year. He’s doing so well that another doc and I wonder if the diagnosis was correct — but this was a university medical center. He’s had radiation. I’m not sure whether he had chemo, but I think he did.
    P.S. I hope to have my own blog up soon.

Comments are closed.