Skip to main content
Menu

Drug Development

The Sunk Cost Fallacy

Mentioning target validation yesterday led me to think about an even larger problem: the sunk cost fallacy. That’s a general human tendency, but (like several other human tendencies) it can lead to some wasted scientific effort. A “sunk cost”, in economic terms, is an unrecoverable one – it’s gone, it’s spent, and there’s no way to mitigate that. The classic sunk cost is time. If you spend a year working on a project whose target turns out not to be relevant, you’re not getting that year back. The moving finger writes, and having writ, moves on.
Of course, you’re not getting back the money that was spent, either, unless you can find some way to create value out of what was done during that year. And that’s the catch. Our human psychology is always ready to hold out the hope that something will occur to make it all worthwhile. Too many research projects (and investments, and relationships, and more besides) all slide down this same gentle slope into the pit. Back when I started in the industry, my workplace had a couple of programs that had been going on for years and years. No drugs had come out of them, not even clinical candidates. But they kept on going, because after all, we’ve come so far! Spent so much time! Surely there has to be something coming that will make good on all this effort?
Not in those cases, and not in a lot of others. An economist (or a behavioral psychologist) might tell you that the way to handle a sunk-cost situation is to ignore the amount of effort, time, and money that’s been spent. As painful as it might be, those aren’t really that relevant. Treat the project as if it’s just dropped out of the sky in its current form and landed on your desk. Look at the situation as it is right now – do you want to go on, or not?
For a drug discovery effort, that means that you might rate it a bit higher than usual, because this “sudden new project” has a lot of SAR and a big collection of compounds attached to it. Those can be good things. On the other hand, if so many compounds (and so much SAR) has already been explored without a good candidate turning up, what is there left to do? That’s also a notoriously dangerous question, because we medicinal chemists can always come up with more molecules to make and more things to try. But look these over – are they built on some sort of hypothesis put together from all the data, or are they just more rocks that haven’t been turned over yet? If this thoroughly-worked-on project still doesn’t have any clear direction to offer by this point, that’s a strike against it.
What you don’t want to do is factor in the time and money that have been spent, though. Those are sunk costs, and they have no bearing on what’s happening now. What matters is the potential that the project might have in the future, and the chances of realizing it. I freely admit that this is a somewhat inhuman way of looking at things, but that gets back to a point I made in yesterday’s post as well: not all human tendencies make for effective science. We have a lot of quirks in the way our brains work (confirmation bias, over-recognition of spurious patterns, faulty memory, loss aversion), and the importance that we attach to sunk costs is another one of those. A common response is “But you’re saying that all this time and work has just been wasted!”, to which the intelligent-alien reply could well be “Yep. And after looking all this over, the key thing is for us not to waste any more.”
What makes this especially hard is that (like many faults) there’s a virtue on the other side of it. Perseverance is its name, and without it, no drug would ever be discovered at all. Almost every research program in the history of pharmaceuticals has gone on for longer than people thought it would at first – sort of like a gigantic series of kitchen renovations – and that can’t be ignored when you’re thinking of killing one off. The only guidance is, I suppose, whether you seem to be getting anywhere, coming across any general trends or principles that give you some hope, or whether you’re just turning over more rocks and hoping that the next one has a gold coin under it.

30 comments on “The Sunk Cost Fallacy”

  1. The Iron Chemist says:

    From a demotivational poster (Despair.com):
    “Quitters never win, winners never quit, but those who never win AND never quit are idiots.”

  2. Wile E. Coyote, Genius says:

    This is exactly why the “quick kill” is so hard to implement. I’ve worked on projects where I advocated a quick kill, only to have it as a black mark against me, but then the project gets killed a couple years later when it became “obvious”.

  3. Anonymous says:

    This is what happens when your annual review is tied to the success of your project. You are better off to say “almost there” than to say “we need to kill this program”. The dead horses we ride still have short term value in that scenario.

  4. PorkPieHat says:

    Wars have been continued over sunk costs (“their sacrifice, however, was not in vain…”).
    My problem with this is that it fosters the “fail fast” mentality. And while “fail fast” works when you have a lot of optionality, it promotes a culture which focuses on failure and not on truly solving the difficult problems of drug discovery. This is why I left drug discovery in big pharma after almost a decade as a medicinal chemist there. Most biotechs dont have the optionality that big pharma do, so they cannot take a “fail fast” approach to absurdity the way pharma does, or they’d be as innovation starved as pharma.
    The survival of a given project is often corporate life or death to a startup or tiny pharma. While the sunk costs approach isnt what drives biotech, they certainly have to make better decisions on projects or die as a company (death is not really considered in big pharma – at least its not as much as an everyday reality to them).

  5. Anonymous BMS Researcher says:

    nor all thy Piety nor Wit
    Shall lure it back to cancel half a Line,
    Nor all thy Tears wash out a Word of it.

  6. Vader says:

    “But they kept on going, because after all, we’ve come so far! Spent so much time! Surely there has to be something coming that will make good on all this effort?”
    Put in cruder terms: With all that manure, there must be a pony in there somewhere.

  7. Cellbio says:

    Same for me PorkPieHat. Worked in a company that rewarded teams for never having chased away enough risk to move into the clinic. From every discipline there was doubt cast and a false ‘can do’ attitude to return to the process to improve some feature that was not known to be a game changer (potency, selectivity, physicochemical, etc.). The system favored the latest greatest target and celebrated the coronation of a new program, and importantly, elevated the status of the team ‘lead’ whose development skill was being the first to grab the on-line version of a journal and converting the figures to powerpoint. I grew tired of knowledge being translated into doubt, while ignorance was seen as opportunity.

  8. Hap says:

    Anything can be made dumb if implemented by people who don’t want to have to think about what they’re doing (even if that’s was they’re being paid to do) and would prefer others not to do so, either.
    “Fail fast” does seem to fit both the need for quick gratification and the risk-aversity people know very well, and it’s only the successes people have that pay for anything else. It just seems that pharma is spending a lot of money on things that probably can’t work, and at least some of the blame for that is scientific. Reasoned assessment of what you’ve done and what you’ve learned and what you can get from the work would seem to be reasonable. If reason is in short supply, then any method for evaluating work and deciding what to do will fail badly.
    If we have systematic errors in thought, they are likely present to avoid other errors, and so going hog-wild to avoid them without thinking is just going to unleash different costly errors.

  9. DCRogers says:

    “What you don’t want to do is factor in the time and money that have been spent, though.”
    Tell that to the management who authorized the spending of that time and money in the first place.
    I’d love to be the fly on the wall as they explained the sunk-cost fallacy to their horrified senior VP…

  10. Chrispy says:

    The most common organizational blunder I’ve seen in drug discovery is overextension. Zombie programs that should be killed, latest publication in Nature, whatever — pile it on. Before long you have an inability to really be effective at any one thing as you try to be effective at everything. And every program has a network of people who depend upon its survival for their own, so you have a set of biased internal champions. I have yet to see a high level manager, someone in a position to truly set priorities, actually do so. This is probably because they rose to their positions in large part by not making enemies within the company, a sure side effect of killing programs.

  11. JC says:

    True story. Compound looks great in October. Reviews in December yields promotions (mainly for the Biologists). Come February problems arise (DMPK, Tox, hERG et cetera) so the series gets an extension. More compounds made. Rinse, repeat. A few years of this and something makes it into Phase 1. Fails. Backup compounds go through the rounds for a few years. Program finally dropped after many years and flops but by then most people are Directors of some sort or VP’s.

  12. slcimmuno says:

    good read. guess Roche/Nutlins is a good case study of sunk costs… though the coin they were going after was perhaps the shiniest of all (drugging p53).

  13. Anonymous says:

    There is also the “lost opportunity” of working on Project A instead of the other Project B…so the “wasted” time/resource cost is doubled.
    However, Project B is probably just as likely to fail as A, which means that the virtual time lost on Project B is actually time you could have spent on Project C.
    However, Project C is…
    I’m going to lie down now…

  14. Anonymous says:

    There is also the “lost opportunity” of working on Project A instead of the other Project B…so the “wasted” time/resource cost is doubled.
    However, Project B is probably just as likely to fail as A, which means that the virtual time lost on Project B is actually time you could have spent on Project C.
    However, Project C is…
    I’m going to lie down now…

  15. Anonymous says:

    Plodding on in the hope you can get something out from all you have spent, is a bit like expecting the odds of getting heads in the next coin flip to be close to certain because all the preceding flips were tails. But alternatively, one could also expect tails to be almost certain. Thus rationality goes out the window.

  16. SomeGuy says:

    Our director likes to say “Invalidation is just as good as validation” – generally saves time and money in the long run.
    But he also pushes us to try and publish papers on invalidated targets, so at least we get a little something for the time we spent. Our group in the past year has gotten both a Cell and PNAS paper out of invalidation, which is pretty nice (and satisfying).

  17. Anonymous says:

    Value is created by answering questions correctly to get the truth, whether the answer is positive or negative. At least that way you can make the best decision, whether it is to stop or to continue.
    Conversely, value is destroyed by getting the wrong answer (with false positives or false negatives) or by asking the wrong questions in the first place…

  18. Anonymous says:

    The challenge is that our knowledge about how to successfully make drugs is so limited that we’re just not that good at evaluating projects. We believe we know what we’re doing (overconfidence bias), and we have really good reasons for the decisions we make (and, truly, they are good reasons), so we kill a project and move on to something else. Then we see a one-trick pony biotech company that has nothing else to move on to, so, out of necessity, they make the “bad” decision to stick with it, and they end up with a drug.
    So, I agree, don’t cling to sunk costs, don’t get too emotionally attached to your project, make the hard, science-driven decisions, but also at the same time keep going, be passionate, take ownership, fight and win at all costs. It’s a hard business….

  19. JC says:

    As long as you can flog that garbage to one of the bigs after some questionable Phase 2 data to eventually die after Phase 3 and all the people at the top make a few more million then who cares.

  20. Anonymous says:

    @18: “So, I agree, don’t cling to sunk costs, don’t get too emotionally attached to your project, make the hard, science-driven decisions, but also at the same time keep going, be passionate, take ownership, fight and win at all costs. It’s a hard business….”
    How to resolve the ultimate paradox and strike the right balance? How to know when to try again, try something different, or just stop trying?

  21. Anonymous says:

    @20: Define both success and failure before you start.

  22. Some idiot says:

    @21: Nailed it. And when (not if) something new and unexpected crops up during the project, treat possibilities that come out of it as _new_ projects with their own definitions of success and failure.
    Can be tough to do. And requires mental discipline (and understanding) in spades. All the way up the food chain…

  23. Anonymous says:

    @21: I think @17 has a better approach: focus on finding out the truth asap (whether there will be success or failure), without trying to force the outcome either way. That’s science.

  24. Some idiot says:

    @23: The approaches in @17 and @21 aren’t mutually exclusive… And in my book, acheiving an understanding of the the science behind it should be one of the goals.

  25. Anonymous says:

    Killing a project needn’t be a complete loss. It can be easier to hit the kill switch if you have a post-mortem process that captures whatever valuable data or lessons were encountered during the project. This assumes, however, that your organization can fathom failure. If people have careers invested in a project, it’s much harder to identify whether it has continuing value.

  26. Poison Ivy League says:

    People coming to grips with this phenomenon seems to be a recurring theme in the “I quit graduate school in chemistry” series on Chemjobber…

  27. Sympa says:

    Post-mortem is indeed important – just as in software. Sharing this (@16) is even greater – it helps science progress overall and not just withing the company.

  28. Anonymous says:

    @15. It’s different. Flipping coin is random, odds not changing.
    One would think, that your drug discovery efforts should be a bit better than purely random poking around. And information already found out in project should be used to guide further decisions.

Comments are closed.