Skip to main content

Animal Testing

Less Than Zero

When I wrote about lousy animal models of disease a few days ago, there was a general principle at the back of my mind. (There generally is – my wife, over the years, has become accustomed to the sudden dolly-back panorama shots that appear unannounced in my conversation). It was: that a bad model system is much, much worse than no model system at all.
I’ve been convinced of that for a long time. When you have no model for what you’re doing, you’re forced to realize that you have no clear idea of what’s going on. That’s uncomfortable, to be sure, but you at least realize the situation. But when you have a poor model, the temptation to believe in it, at least partially, is hard to resist. Even if it’s giving you the right answers at a rate worse than chance, you can still take (irrational) comfort in knowing that at least you’re not flying blind – even as you do worse than the people who are.
There are many reasons to hold on to an underperforming model. Sometimes pride is the problem. I’ve seen groups that stuck with assays just because they’d invented them, even though the method was slowly wasting everyone’s time. Never underestimate cluelessness, either. People will use worthless techniques for quite a while if they’re not in the habit of checking to see if they’re any good. But the biggest reason that useless procedures hang around, I’m convinced, is fear.
Fear, that is, of being left out in the middle of the field with no models, no insights, and no path forward at all. It’s a bad feeling, rather scary, and rather difficult to explain to upper management if you’re a project leader. Better, then, to hold on to the assays and models you have, to defend them even if you’re not sure you trust them. With any luck, the project will end (although probably not happily) before the facts have to be faced. As Belloc advised children in other situations: “Always keep ahold of Nurse / For fear of finding something worse.”

23 comments on “Less Than Zero”

  1. Harold says:

    I’ve been happily retired for almost a year now from the wonder drug factory, so I don’t remember specifics, but another reason to retain a worthless model of efficacy or of toxicity was that the FDA expects to see certain assays in applications and it is often easier to do the work than to make the argument that the work is less than worthless and it shouldn’t be required.

  2. milkshake says:

    The lack of curiosity (about whether the used model has any predictive power) is rather typical of a pseudoscience.
    I was making myself unpopular at the previous company – at the project meeting they were very seriously discussing how their candidate had to be above 30 min in human microsomes but their compound had only 16 min. They had these gorgeous Powerpoint slides summarizing (with bullets) that “we must improve the microsomal stability”. I asked them why wouldn’t they prefer to improve the rodent PK instead. They said the microsomes are like PK. I mentioned that I have seen plenty compounds with absolutely horrible PK but excelent microsomal stability, and vice versa. They said they had microsome assay set up and available every day, they were going to use it as a pre-screenbecause they cannot submit too many compounds for PK because there is a long waiting list.
    And so it goes.

  3. TX Raven says:

    Yet, there is the mother of all in vivo animal models (at least the CNS-ones), especially for those projects that don’t have one that works: ex vivo receptor occupancy!
    I have seen very well educated people, able to understand very complex scientific issues, come up with the most “sophisticated” reasons to perpetuate this type of ex vivo assays. In my med-chem biased view, these are just glorified in vitro binding assays, bearing very little relevance to the “real” in vivo receptor occupancy that drives behavior. Well, at least in theory 🙂
    I am talking about those studies where you dose the animal with your compound and you sac it, remove the brain, homogenize the region of interest and run a binding assay.
    Equating this in vitro determination with an in vivo receptor occupancy determination and using this numbers to draw conclusions is done all the time in a number of industrial research organizations.
    Wonder why our new drugs are taking so long to show up?

  4. NJBiologist says:

    Maybe it’s just me, but I’ve had a really tough time convincing some of my colleagues that there is a difference between a model’s resemblance to clinical disease and a model’s predictive capability.

  5. Kay says:

    Only the special few can broadcast doubt, especially when paychecks are involved. Perhaps this is one of the root causes for our current difficulties.

  6. LNT says:

    Milkshake raises a concern that I’ve had with my own “Wonder Drug Factory”. How useful is microsome stability data? Generally, we only carry compounds forward that have good microsome stability. Obviously, some compounds with excellent microsome stability have horrible PK due to solubility, permiability, excretion, non-microsomal metabolism, etc. However, is the reverse also true? How many people have encountered a compound with poor microsomal stability and genuinely good PK? Since we never carry unstable compounds forward, I really can’t answer that question based on experience at my company.

  7. MolecModeler says:

    Excellent point Derek. This is something I learned from my manager and through experiences: sometimes something is worse than nothing. Many times some higher up is invested in a certain project/assay and so it has to be used, this is Kay’s point in #5.
    To give you a comp chem example, this often happens when using homology models of family A GPCRs. They’re all based on bovine rhodopsin, no one really has any idea of their quality, and their predictive power is not statistically different from 0 imho. Yet they are used by many people who think they’re “real”.

  8. Hap says:

    “The greatest obstacle to discovery is not ignorance – it is the illusion of knowledge.” Daniel J. Boorstin
    In this case, it’s perhaps worse than an illusion of knowledge, because we know that some models are illusions, and at odds with reality, yet we keep them solely because the pretense that we actually know something useful comforts us. We would rather feel that we know something (even if at heart we know that we don’t really) than know that we do not. The knowledge that we could be wrong doesn’t help us if we ignore it.

  9. Cyan says:

    This is a point that goes far beyond just animal models. Will Rogers put it, “It isn’t what we don’t know that gives us trouble, it’s what we know that ain’t so. ”

  10. russ says:

    “The measure of a man’s greatness is how long his erroneous ideas retard progress in his field.” Frank Boardwell

  11. Clark Kent says:

    I have many examples on my current project of compounds which show low in vivo clearance (in multiple species) and high intrinsic clearance by S9 and microsomes. We were in the fortunate position of getting the in vivo data quicker than the in ivtro data, so we dont even send the compounds through the in vitro screen anymore because it was completely useless.

  12. Demosthenes By Day says:

    The problem of whether liver microsomes predict real PK is the same problem that crops up with the cell permeability assays. CaCO2 and PAMPA are both tragically wrong in a number of cases.I’ve been on two projects where the clinical candidate failed both permeability assays but in at least one case it sure seems to be hitting the target. But these assays are part of a company’s new automated high throughput way of anlyzing for pharmaceutical properties. What it really turns into is an almost random way of talking yourself out of making a compound because its quick not scientifically sound.

  13. SMIP says:

    Good post. I would recommend the book “The Black Swan” by Nassim Taleb. You will appreciate his writing.
    Good luck.

  14. Wavefunction says:

    However (depending on the situation) sometimes even a bad model is better than no model because it can teach you a lot (about what not to do)

  15. Hap says:

    I wouldn’t figure that a bad model is better than no model unless you know what its weaknesses are. With simple models such as the ideal gas law and Bohr’s model of the atom, you can use what you know and what doesn’t work to make estimates for the things you don’t know (like accounting for the volume of molecules and their attraction/repulsion), and even if you can’t do so, you can at least know that the limits of the model. I don’t know enough biology, but I figured that the biological models discussed aren’t well determined enough to know their limitations, and so you don’t really know anything other than what the model tells you, including what you actually want to know.

  16. srp says:

    My history of science professor in college described the phenomenon Derek describes in the field of anatomy. Apparently the Roman physician Galen’s writings on human anatomy, which was taken almost as sacred truth from the time of its rediscovery, was based on dissecting gibbons.
    People knew that, but they kept assuming that Galen was right until Vesalius and company came along and said “Wait a minute–that’s all based on gibbons. Maybe we should start dissecting human corpses.” Obviously, dissecting human corpses raised a number of thorny cultural and religious issues, but it did get around the bad model problem.

  17. Chrispy says:

    I hope all of you synthetic guys are familiar with the work of Alexander Shulgin. He tests all his stuff directly on humans — himself first.
    Phenethylamines I have Known and Loved (PIHKAL) is an absolute classic (haven’t read TiHKAL) for those in drug discovery.
    He is very careful and tries very small doses initially. But I don’t know anyone else doing drug discovery this way.
    I’m surprised (and glad) that it is legal!

  18. BCP says:

    Good call milkshake — I’ve also seen plenty of examples where either in vivo or in vitro PK over predicated the other. Bottom line, it’s very difficult to generalize – if you understand your SAR within a series there’s at least some hope.
    Nice Vonnegut reference by the way.

  19. DLIB says:

    Hi Derek,
    Off the subject, but I was just listening to the pfizer shareholder meeting and I think this blog gets a nod at about 9min 30 sec into it. Cancer patient.

  20. milkshake says:

    It’s like with the man who was looking for his keys under a lamp-post. When asked if he was sure that he had lost his keys right there he replied: “Not necessarily but at least I could see here”

  21. oldbio says:

    There is no single assay or model which tells you everything you need to know about a compound. Which in vivo model is the most important, the one that predicts some ADME properties or the one that proves the mechanism of drug action or the one that models a disease state? Judging and prioritizing compounds based on a large amount of data, some of which not meaningful or accurate, is where the art comes into the science. If drug discovery was solely a rational exercise, all our pipelines would be full.
    “To know what you do not know is the best. To pretend to know what you do not is a disease” – Lao Tzu

  22. Mark M says:

    That is how all respectable neuro-pharmacologists did their work back in the day.
    I am not sure how legal that is today; I think Sasha had some special “license” to do his studies.
    The point is that there is no substitute for whole animal work–problem is that is very expensive and not that amenable to high throughput.

  23. Psychedelic Biologist says:

    Shulgin does/did have some sort of licensure from the USA DEA for synthesis/possession. I’m not sure whether that entitled him to legally ingest likely-psychedelic analogs of schedule-one compounds like psilocybin or mescaline in the USA. The USA Analog Act (or something like that) is probably relevant. I haven’t read it.
    Personally I am very glad to see the new trials examining psychedelics such as MDMA for post-traumatic stress disorder. Many people may not know it, but prior to the prohibition of psychedelics, there was extensive experimentation by professional psychologists and psychiatrists interested in the medical use of psychedelics (not all of them necessarily as variably-flakey as Timothy Leary). Such research has been almost frozen until rather recently.
    (Regardless of medical utility or non-utility, the religious, as opposed to recreational, value of psychedelics is demonstrated by their religious use in numerous long-lived cultures worldwide.)

Comments are closed.