Skip to main content

Academia (vs. Industry)

More Odd Compounds

When you look into the literature on small-molecule agents for really tricky targets, something stands out to medicinal chemists immediately: the structures start to get strange. Examples of this sort of thing are beyond counting, but this recent paper will serve as well as any. It’s from a large multicenter academic team, and proposes several compounds as ligands for Bax, a protein of the Bcl family that’s involved in apoptosis and is a potential target for a range of lung cancers. There’s a particular serine residue whose phosphorylation has been shown to alter Bax function significantly, and the present paper notes that there may well be a small-molecule-sized pocket nearby.
The work starts off by computationally docking a large collection of molecules from the NCI database to a model of this pocket. 36 out of 300,000 were found to score well, so the team exposed Bax-expressing cells to all 36 of them and looked for effects. Three of the compounds showed apoptotic effects, which were less marked in lung cancer cell lines that expressed less Bax. The compounds were able to compete with a fluorescent protein that also binds to Bax, and this assay gave affinities of around 50 nanomolar. Update: see the comments. These compounds give weirdly similar data.
These results sound promising. But let’s look at the structures of the compounds. SMBA1 is not very attractive, for sure. You’d want to make sure that any fluorescence assay involving it doesn’t suffer from some sort of interference, because it sure looks like a compound that would make its presence known in the UV/Vis spectrum. I have no experience with these sorts of fluorenylidene phenols, but some stability and reactivity checks would not be out of place.
Bax compounds
The problem is, SMBA2 and SMBA3 make that first compound look like ibuprofen. I know that the NCI compound collection has a lot of wooly stuff in it, but come on. These are formaldehyde/amine condensation products. In aqueous solution, they exist in various equilibria depending on pH, and such structures are known to be reactive reagents in organic chemistry. (See the Delépine reaction, among others). SMBA3, in fact, is an intermediate structure from that very reaction, as if someone were trying to synthesis allylamine. I would be very worried about exposing such compounds to cells – there are no guarantees that they’re going to remain the way that they’re drawn, and the species that form can be quite reactive. Figuring out what’s really going on with them would be quite a job.
I can see no indication from the paper that any of this bothered anyone. The only characterization that seems to have been done with these compounds was some DLS, dynamic light scattering, to look for aggregation. Nothing wrong with checking that, but there’s no other chemical characterization or purity assessment of the compounds at all, or at least none that I can see. I mean, they’re probably what they say on the label, but who can tell? And even if they’re pure, how do they behave under the assay conditions?
The problems don’t end there. SMBA2 is said to have 57 nM affinity to Bax. But that’s an extraordinary value for a compound with a molecular weight of 168 – that kind of ligand efficiency should make a person suspect covalency. Covalent compounds are not necessarily bad, but you do want to examine them closely to see if they’re working on the target you think they’re hitting. I don’t think the paper has any tests for that, such as checking for time-dependent inhibition or looking at the isolated Bax protein from the assay by mass spec. Meanwhile, SMBA3 has 54 nM affinity, and is shown as having a molecular weight of 308. But that includes the iodide counterion, and that doesn’t count: this thing is not floating around in cells, nor binding to its putative target, with its iodide partner. Its real molecular weight is about 181, and that also represents a wildly high ligand efficiency. (There’s also the question of how a charged quaternary compound like this gets into the cells so easily, but that’s another issue).
The docked structures of these three compounds with Bax are shown in the paper. The two formaldehyde/amine compounds are shown interacting with two Asp residues, and there had certainly better be some strong interactions if you’re going to pull out potencies like these. But the positively charged nitrogen in SMBA3 would surely seek out a negatively charged Asp if this were the case, a classic salt bridge, but that’s not what the docking shows. Meanwhile, SMBA1, the fluorenyl compound, is shown just sort of floating there in space, not doing anything with the Asp residues, and not showing any strong interactions that I can see from the figure at all. Even the phenol doesn’t seem to be doing anything, unless that’s a pi-stack with Phe176. These dockings structures do not, unfortunately, inspire confidence.
So in light of all these objections and complications, let me get back to the point from the first paragraph of this post. When you get weird-looking structures out of a screen for a difficult target, you can explain them two ways. One possibility is that such targets, and such binding sites, are not evolutionarily optimized to bind any particular small molecules, and that regular “drug-like” chemical matter should not then be expected to hit them any more than anything else. Odd sites, in this view, will generate odd molecules. The other possibility, though, is that these things are false positives. The ways in which compounds can fool you are legion – in this case, I’ve mentioned some of the possibilities already, but there are more where those came from.
One thing that’s for sure about targets like this one is that their intrinsic screening hit rates are very low. So that means, necessarily, that if the false positive rate is at any realistic level, then the bulk of what you get out of a screen will be just that: false positives. The challenge in screening these things is to dig through the garbage heap in search of the few hits that might be real. It’s not a lot of fun, in some cases, because the list can be long and playing endless whack-a-mole with the compounds on it can be wearying. But you have to give all your hits the same tough love, or you run a significant risk of wasting your time.
So yes, I think that the odds are good that the compounds reported in this paper are false positives. They clearly seem to have cellular effects, and these may well be mediated by the Bax pathway. But it’s a big leap, at least for me, to believe that everything lines up the way it’s supposed to here. Odds are that anything reported for this Bax binding site is a false positive, and (although I hate to say it) this paper hasn’t convinced me that its authors have given this problem sufficient attention.

28 comments on “More Odd Compounds”

  1. MedChemNewbie says:

    Thanks for highlighting this paper Derek. I enjoy your insight and writing. Based on reading your post, I expected the authors (and reviewers) to have avoided any of the expected experiments to show Bax specificity with MEFs derived from WT, Bax-/-, Bak-/-, and Bax/Bak-/- MEFS. However, these experiments were done…. as were cyt c release experiments with isolated mitochondria from these cells. I am new to drug discovery, so I would be curious to hear how else you would want them to test off-target effects inside cells. I guess these compounds could hit some up-regulated factor that compensates for the null, but this seems unlikely. Do you or others know of such examples? Thanks again for your wit!

  2. David Borhani says:

    Derek, is your point that the review process failed?
    It is remarkable that such chemically distinct compounds as SMBA1 vs. SMBA2 and SMBA3 have such nearly identical IC50 values for displacement of the fluorescent peptide (Figure 1b) from Bax.
    An overlay of the middle (SMBA2) or right (SMBA3) panels onto the left panel (SMBA1) indicates, at the resolution of Figure 1b (blue square symbol size, and error bars) that the actual data for the three compounds are essentially identical. That seems a bit odd, don’t you think?
    I’ve posted the Photoshop overlay of Figure 1b (original PSD, plus each overlay [PNG] in turn) here:

  3. Anonymous says:

    Ah, more academia=evil nonsense.

  4. Hap says:

    No, it’s academic drug discovery = crap. Which this paper does not exactly refute.

  5. Derek Lowe says:

    #3 Anon –
    Not at all. This isn’t evil. It just should be better than it seems to me to be. Anyone who follows up on this, be they in academia or industry, will have to address these issues.

  6. SpaceDocker says:

    I haven’t done a huge amount docking and modeling, but I have done enough to know this: docking is pretty frequently BS. There’s a whole lot of things that need to fall into place before it’s even remotely useful. (eg, I think you’ve posted on xray structures taken as fact, when they are still very much model representations, Derek.)
    Otherwise, you can do a LOT of hand-waving and structure-massaging to dock molecules into the most atrocious structures in the PBD. But a predicted binding pose doesnt necessarily make it so.
    These guys started from 1F16 – a protein elucidated using solution-phase NMR spectroscopy. I plugged that structure into MolProbity just to see whats up, and the geometry and steric clash scores are garbage. GFL.

  7. Wavefunction says:

    The problem here is not with docking or HTS or any specific techniques: it’s with taking the results at face value and not having the knowledge of chemistry and drug discovery to flag these molecules and all their potential problems. This paper demonstrates why every member of a drug discovery team needs, at the end of the day, a healthy dose of chemical intuition and experience.

  8. Barry says:

    none of these compounds are lead-like, much less drug-like. But the assumption that a quaternary amine must pair with an Asp or Glu side-chain to make a productive contact hasn’t been borne out in the lab. X-ray diffraction studies of acetylcholine bound to its target show that the quaternary amine binds to an uncharged box formed by aromatic side-chains. The binding is not nanomolar

  9. mousmous says:

    As an experience medicinal chemist desperately seeking employment, the fact that so many talented academic biologists can get together and produce collaborative work which is so fundementaly flawed, but could have been resolved if a med chemist had been consulted or involved, makes me pretty mad. I’d have happily done it for free.

  10. Anonymous says:

    #3 and #4 – Please do not paint every academician that does drug discovery with the same brush. Ignorant nonsense. It is as insane as using scandals like the Vioxx scandal as evidence that every big pharma scientist is corrupt. Also as ridiculous as saying that pharma scientist should do basic science because that is not their specialty.
    No doubt the pressure to publish leads people to do bad things, but having spent years in pharma before going back to academia, I can tell you that this is not that different than what people do when “bonus time” rolls around in companies. All of the sudden, people start having experiments with greater impact. Hmm. The difference in most cases is that this never gets out because it does not get peer reviewed.

  11. Anonymous says:

    #3 and #4 – Please do not paint every academician that does drug discovery with the same brush. Ignorant nonsense. It is as insane as using scandals like the Vioxx scandal as evidence that every big pharma scientist is corrupt. Also as ridiculous as saying that pharma scientists shouldn’t never do basic science because they only know drug discovery. Give your head a shake!
    No doubt the pressure to publish leads people to do bad things, but having spent years in pharma before going back to academia, I can tell you that this is not that different than what people do when “bonus time” rolls around in companies. All of the sudden, people start having experiments with greater impact. Hmm. The difference in most cases is that this never gets out because it does not get peer reviewed.

  12. milkshake says:

    these hits are eye-brow raising, and if they are real rather than artifacts (confirmation of which has not been done) I would hate to take them as leads – but they are not totally implausible. The first hit compount: a fluorene-derived stilbene looks a little bit like Sulindac NSAID, and making these analog compounds require just one step – why not. The third hit compound is a quaternized urotropine – these compounds are actually quite stable, to hydrolyze them one really has to cook them under harsh conditions. Urotropine itself is a really useful drug for treating bladder infection – it is orally available and makes its way unchanged all the way to urine, where it behaves as a bactericide.

  13. dr z says:

    adding to the comments of David Borhani:
    Yes, it is remarkable that the Ki’s are so similar for these (in fact identical within error for 2 and 3).
    Perhaps a silly mistake, but in Figure 1b the FP curves that should correspond to SMBA2 and 3 are labeled SMBA2 and 2, although they do not appear to be the same graph as the other curves are different.

  14. Hap says:

    I don’t think that academic drug research is crap, but it does seem that there is a lot of it published that seems unaware of its limitations. That should be a problem in any field.

  15. Danon says:

    @2: Similar to Borhani, here is all 3 overlayed:
    Perhaps a careless mistake? They definitely appear to either be identical or an artifact of the assay.
    Rest of the paper I am still reading, but found it a bit surprising that in the DLS experiment the fluorenylidene didn’t aggregate until 500uM! It doesn’t look that soluble.

  16. anon the II says:

    I wonder what happens to the authors of such papers when Derek sends them down the “Walk of Shame”. Do their colleagues come in and give them a hard time? Are they aware how crappy this “science” really is? Maybe they know and just want to get it out there and move on.
    Is this blog big enough to have an impact and slow the tide of fluff science?
    I hope so.

  17. Anonymous says:

    #15 Danon
    Please send it to RetractionWatch and PubPeer !
    It should be retracted faster than STAPs.
    Bax/Bcl is a hot field and many good teams have assays in place. Should be easy to check this crap.

  18. Project Osprey says:

    The third compound (SMBA3) looks very similar to Quaternium-15, which is used as a a surfactant and preservative in a variety of settings, including some cosmetic applications. It’s a known allergen but for some strange reason Americans appear to be more allergic to it than Europeans.

  19. A chemist says:

    Derek’s point seems to be lost to some commenters. The subject of the paper is the agonists (see title), i.e. the chemicals. Yet the chemicals are totally uncharacterized. This is a systemic problem in this kind of literature (see any issue of Chemistry & Biology for examples), and is a contributing factor as to why so much of the oncology literature is irreproducible.

  20. A chemist again says:

    Amusingly, the “article metrics” not only links to page views and news mentions, but also blog mentions, so Derek’s piece is directly linked from the paper!
    One step closer to direct feedback.

  21. sucks when you do good work says:

    and Derek totally ignores it, but the utter garbage in your field (like this, which is 95-99% of it) gets press.

  22. Pat says:

    If they developed some SAR around these compounds I’d be more convinced.

  23. daen says:

    The first thing I thought when I saw SMBA1 was “What’s that FMOC-like protecting group doing there? And what’s it protecting?”
    Too much time spent around peptide chemists …

  24. Nick K says:

    #12 Milkshake: I thought urotropin worked by hydrolysing and generating a low concentration of bacteriocidal formaldehyde in the bladder. None of these three compounds is remotely lead-like.

  25. Anonymous says:

    I think you have just immediatly looked that these compounds and decided to confirm your own biases here. A bias that you give private companies a much easier time with. But they have much different priorities, and they are not held to peer-review or outsiders scrutinising their work
    Yes, your critisisms are sound but I think in this case this paper has enough different assays that it should still have been published, for someone else to at least work on. I think this one doesnt’t deserve the ‘walk of shame’.
    It seems like all your critisism comes down to is that the authors should have had a bit more humility with their claims, but does that not apply to all science now?

  26. A Nonchemist says:

    #25, When private company employees publish, they are held up to the same scrutiny as anyone else. Anyway, I think we get more peer review in industry than in most of academia, I’d say, having seen plenty of both sides.
    Unfortunately this is just one example of many such papers, not the worst, where observations on cells at high concentration are attributed to a molecular target based on flimsy compound data. I agree that this does contain things that could have been published, but to me this should’ve never get into a Nature journal the way it is without major revision. It’s literature pollution, in my opinion, and it says a lot about the state of peer review for high-profile journals.

  27. nerd v geek says:

    As a chemist who works in a biology focused research institute, this is the type of thing I see too often: people with little chemical knowledge trying to do drug research without involving someone with even basic chemical knowledge. It would be miraculous if any of these compounds worked as described, despite the limited data presented. And that this type of crap gets into Nature Comms is outrageous. The sooner editors insist that chemists are involved in the review process of papers where new chemical leads are discovered, the better. Unfortunately it won’t be long before a reagent supplier starts selling these compounds as Bax inhibitors and a lot of labs will end up chasing shadows.

  28. Jose says:

    Seriously if you don’t know enough not to include the counterion in the binding (especially when it’s a freaking I-), or can’t look at those structures and know things are deeply askew, you have no business trying to publish (pseudo) medchem. Sorry, that’s brutal, but you’re only embarrassing yourself.
    “Really, the skiing is Maine is just as good as anything out West.” which means, ahem, you’ve never skied out West.

Comments are closed.