Very little time for blogging today (travel), but I wanted to pass on some words of wisdom from Kevan Shokat, from a recent Perspectives piece in Nature Reviews Cancer. Talking about chemical probes, and how to know if they’re valid enough to work with, he suggests that you need to see dose-response data (for one thing), and he’s much, much happier when there’s a crystal structure of the proposed probe with the target protein (hey, who isn’t?) But then there’s this point:
The third criteria is proof that the first drug can be modified and its biochemical and cellular activity improved in a manner consistent with the structural model. Note that I do not include a requirement that the molecule work in an animal model. In my opinion, too many first reports describe animal efficacy data, long before the first three criteria are established, leading to false-positive proof of target inhibition. Something to look for if the report was published more than a year ago, is whether a follow-up study has appeared showing an improved version of the molecule and further proof of target engagement. If nothing appears after several years in the peer-reviewed literature, bioRxiv, or published patent applications, you can bet the molecule was an artefact and the target remains undrugged.
I endorse both of those – the too-fast animal model problem and the lack of follow-up criterion. When the answer to “Whatever happened to. . .?” is “Nothing, apparently”, then it’s a bad sign. The only mitigating factor might be if the report was from a more obscure source or published in a more obscure journal. That gives you a possible out, in that other people may not have noticed it, but that could also be offset by the possibility that the group reporting it may not have had the resources or experience to do adequate characterization themselves.
This is not a mandate to ignore the literature wholesale. If you do that, you’ll end up stuck pretty quickly in this field. Richard Feynman was famous among colleagues, when he moved into a field of research, for deliberately not reading the literature and trying to work his way up from first principles. That, though (as one of those colleagues remarked in James Gleick’s biography) only worked if you were as smart as Feynman. And at any rate, it doesn’t work at all in biology, where there are no first principles, at least by the standards of physics. No, we’re stuck with the literature, and we have to keep up with it, but we also have to remember that a reasonable percentage of it is wrong, and be prepared to ignore parts of it as needed, and with cause. If you try the opposite, and decide that every report you read is completely correct, you will end up stuck as firmly in the mire as if you don’t read the literature at all. It ain’t easy.