Skip to Content

Drug Assays

PAINs Filters In the Real World

Here’s a look (open access) at Eli Lilly’s screening collection in terms of PAINS filters, and there are things for everyone to argue about in it. The entire concept of these filters has been occasion for argument, of course. Allow me to caricature some of the opinions that you hear in these: at one extreme, it would go something like “Damn right. The screening collections have compounds in them that never should have been put in there in the first place and just waste everyone’s time. Filtering those out at the start is a public service.” The other end of the spectrum would be “Barbaric. We’re supposed to be scientists and make our judgements based on evidence. Crossing off compounds just because you don’t like their structures (or even worse, just because somebody else didn’t) will reduce your chances of ever finding chemical matter to work with. Get your noses out of the air”.

There’s something in both of these exaggerations. My own opinion is that it’s a sliding scale. As you progress along it, the chances that a given compound (or whole structural class) is going to waste your time increases. If you want to work on them past a certain point (a point of your own choosing, mind you) then you should do so in full knowledge that there are people who would not, and know why not. I’m sliding over a number of important details, of course – how that scale is constructed, who says what compounds are on it, the issues with them varying according to the assay, their intended use, etc. But in the abstract, that’s my own take.

The Lilly group took a look at their compounds via the original PAINS filters, and evaluated things based on their behavior in six assay types, including the AlphaScreen format that started the whole thing), compound stability, cytotoxicity, and Hill slope. A high value in that last one generally indicates promiscuous/multivalent binding as opposed to a 1:1 complex – details here and here. This represents around 14 million data points (often dose-responsed) out of over 3,000 assays, so it’s a pretty good-sized selection. Compounds that had been flagged as impure were excluded (but see below). Calculating the hit rate of the PAINs-flagged compounds versus a random set showed that AlphaScreen, FRET, and fluorescence polarization assays (in that order) showed the most enrichment in promiscuous compounds (recall that the original PAINs paper came out of an AlphaScreen campaign). Looking at the specific structural alerts, it also appeared that these are more AlphaScreen-focused. The only two that really seem to cause pan-assay trouble are 1,4-diamino aryls, with one nitrogen substituted by dialkyl, and our good old friends the rhodanines (with the exo alkene bond).

There’s another effect at work with the structural filters, though. This paper suggests that some of them may not be flagging promiscuity across assays as much as they are chemical instability, with the parent structures falling apart to the (reactive) species that are really causing trouble. That’s a worthwhile distinction, especially since the amounts of these bad actors don’t need to be very large to blow an assay. Many readers here will have had experiences with these things, and a couple of my own are here. You cannot assume that you are always screening a pure set of compounds, because there will always have been some interval since these things were last checked (and that’s assuming everything was clean to start with, which is a mighty generous assumption).

Once you move on to cell assays, you have cytotoxicity and other such off-target garbage to worry about. There are far, far too many papers in the literature that report Compound X as hitting in a primary assay of some sort, then show that Compound X is active in some sort of cell assay, and conclude (or let the reader conclude) that these two things are necessarily connected. When it ain’t necessarily so. That is especially true for things like growth assay in tumor cell lines and the like. Just assuming that such activity is due to your primary assay readout is. . .ambitious. The Lilly group found that several PAINs filters seem to be associated with general cytotoxic effects, with the anilines mentioned above and some quinone structures as particular offenders.

As for the Hill slope data mentioned above, a number of PAINs substructures seems to be associated with high values, which is generally not a good sign. The FP and FRET assays seem to be especially sensitive to this sort of thing, but overall, I’d say that this is the category that the filters deliver pretty good value for the effort. To use that sliding scale view that I was talking about before, this would mean that if you find an interesting compound in one of these classes, an early action item should be to check the Hill slope that you got in the assay and keep an eye on it in follow-up assays.

And that leads in to the overall lesson, too: (1) there are active compounds from assays whose structures lower their chances of success. (2) Some of these are true hits that will nonetheless be difficult to progress, and others are flat-out false positives. But (3) these bad-actor structural classes can all be context-dependent, and some of them are intrinsically more worrisome than others. So (4) use the filters and literature reports on them as guidelines for how to deal with them. You don’t kill a compound in the absence of data, but the filter tells you what data you may need to pay attention to immediately. (5) Check the purity again, on both the solid and DMSO solution samples (and do a quick silica gel plug or HPLC for good measure). Check the Hill slope. Check for aggregation in your assay buffer. Check for redox cycling. Check for activity in other assays, if you have such reference data. You actually should be doing these things for all your hits of interest, but setting off one of the various PAINs alerts is a reminder that you ignore them at your peril.

17 comments on “PAINs Filters In the Real World”

  1. Peter Kenny says:

    Nobody who has worked with HTS output would deny the problems caused by badly behaved compounds. In general, I would be wary of any compound whose chemical structure suggested that it would be strongly colored, reactive or have accessible redox chemistry. Some of the compounds identified by PAINS filters (e.g. quinones) would have been regarded as potentially nasty long before the PAINS filters were published. In general, I would treat a PAINS match as soft information to be used in conjunction with other information. One concern with J Med Chem editorial policy is that the lack of a PAINS match appears to absolve authors from doing confirmatory work (e.g. purity checks) that they should be doing anyway. In general, I would want to use SPR to examine all starting points for hit-to-lead work since this technique can provide useful information regardless of whether the problem is interference with readout out an undesirable mechanism of action. I have linked my comments on the ACS assay interference editorial as the URL for this article.

  2. G says:

    I remember watching a case-study talk using a HTS hit. From ‘big pharma’. It was the only hit they had. To be sure, they carried out some serious beutification of the molecule. The result was they got a drug against a very hard target with the only hit from the HTS.

    The molecule was very big. Looked like a dimer byproduct with two nitro groups and some other stuff like in dantrolene

  3. SP123 says:

    I think Hill slope is only a major concern in biochemical assays- after all, it was derived as a measure of binding cooperativity. In cell based assays, especially cancer tox assays, it’s common to have a steep slope, which doesn’t necessarily mean there’s something wrong with the mechanism. In a cellular environment over the course of 24-72+ hrs it’s not surprising that an on-target cell active compound would cause a response in a pathway that would tip suddenly to cell death.

    1. MrRogers says:

      For cell-based assays, demonstration of lack of activity in control cells (you do have such cells in your pipeline, right) will get you much of the same information. Specifically, it will tell you whether you’ve just rediscovered cyanide (or a similar general toxin) or a specific inhibitor of your molecule/pathway/function.

    2. skrinah says:

      Conversely, don’t assume that a compound must be okay just because it has a Hill slope of 1. We routinely counterscreen our hits with biophysics assays and regularly find compounds that give reasonable biochemical curves that bind but don’t saturate by SPR.

      Also, worth saying that most non-saturating compounds act in the 1-10 micromolar affinity range, which is exactly the same range as most new screening hits. Unless your compound extremely soluble it’s quite unlikely that the binding curve has been very well fitted at the top end, so the slope is probably not well determined.

    3. skrinah says:

      Conversely, don’t assume that a compound must be okay just because it has a Hill slope of 1. We routinely counterscreen our hits with biophysics assays and regularly find compounds that give reasonable biochemical curves that bind but don’t saturate by SPR.

      Also, worth saying that most non-saturating compounds act in the 1-10 micromolar affinity range, which is exactly the same range as most new screening hits. Unless your compound extremely soluble it’s quite unlikely that the binding curve has been very well fitted at the top end, so the slope is probably not well determined.

  4. MrXYZ says:

    A more general question from someone who is not a medicinal chemist (I’m in the biologics world). At what point in the process do you start re-synthesizing a compound so you can properly confirm that it truly is what you think it is?

    1. AR says:

      Immediately

    2. b says:

      Before you ever tell your boss you found a hit.

  5. med chemist says:

    MrXYZ, The very first thing we do, after dose-response assays suggest a compound is a validated hit, is to obtain an independent sample, preferably by internal synthesis, with full characterization. We call it “round zero of SAR studies”

    1. MrXYZ says:

      Thanks for the answers. Similar to antibody discovery where you get hits from your screening method of choice (phage, hybridoma, B cell cloning, NGS) and then immediately(!) sequence and recombinantly express to make sure the hit is really a hit. In general, antibody binding assays are a bit more reliable than cell functional assays although we have our share of nasty surprises.

  6. another med chemist says:

    As a med chemist who was harangued by a receptor pharmacologist for three years on a project early in my career this has stuck.
    We dose animals but not cells, cell membranes etc so it is concentration response NOT dose response.

    1. SP123 says:

      I’ve heard that complaint but it’s really unnecessarily pedantic. Everyone uses that terminology anyway, and it’s not like there’s potential confusion with the pharmacologist’s meaning of dose- “Oh, you said DRC not CRC, so were you crushing different size pills onto your cell culture?”

      1. Says who? says:

        Words matter.
        Like using “molecule” as a synonym of “compound”.
        Eating garbage is not ok because a million flies do it.

  7. Adam Shapiro says:

    At the very end of the Hill slope paper by Prinz linked above, he asks “Had there ever been a successful drug developed from a compound which had shown a Hill coefficient different from 1?” I suspect that the answer is “Yes, lots of them.” The concept of Hill slope applies only to equilibrium binding, but many drugs aren’t in equilibrium with their targets (on the assay time scale) because they form covalent bonds (mechanism-based inhibitors like aspirin and penicillin) or are slow, tight binders. Moreover, if the IC50 is near the target concentration (tight binding), the Hill slope will be elevated.

    In my experience of HTS hit evaluation from enzyme assays, we used Hill slope >2 as an early criterion for identifying non-specific inhibition by weak inhibitors (IC50>1 µM) because we assumed that weak HTS actives either were in rapid equilibrium with the target or were non-specific inhibitors. (More potent hits were not subjected to this filter, just in case they were slow, tight binders.) The likelihood seemed very low of finding a mechanism-based inhibitor or slow, tight binder in an HTS screen of a generic library against the novel targets we were interested in. You might think about it differently if you are working on a member of a well-known target class, like protein kinases, because the library probably contains lots of potent inhibitors. So you have to take a nuanced view when it comes to Hill slopes of IC50 curves in enzyme or receptor binding assays.

    You also have to take into account interference by the compound with the detection method, and compound insolubility.

  8. pottered says:

    Derek makes a good point when he says “There are far, far too many papers in the literature that report Compound X as hitting in a primary assay of some sort, then show that Compound X is active in some sort of cell assay, and conclude (or let the reader conclude) that these two things are necessarily connected.”

    I’d include in this the kinds of papers where someone runs a virtual screen, tests 5 hits in a biochemical assay, finds 2 of them active (IC50 150 uM, naturally) and writes it up. If these people had ever bothered to question this, they could have run 100 random compounds through their assay and probably would’ve found 40 hits. It’s junk science, and the lower end of medchem journals are full of it.

    1. Peter Kenny says:

      If journals want to address problems like these, they need to set criteria for validation of ‘activity’ that must be satisfied for all reported ‘actives’ (as opposed those that match PAINS substuctures). If journals use predictive models to make editorial decisions to enforce standards then these predictive models also need to be subject to standards. If journals use the term PAINS in author guidelines then they need to say exactly what term means. For example, do compounds actually have to exhibit pan-assay interference in order to be described as PAINS?

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.