Skip to main content

Drug Assays

Has AI Discovered a Drug Now? Guess.

Here is an interesting paper in Nature Biotechnology on computational drug design, and if you read it without reading any of the accompanying articles about it, you will have a perfectly good time. There are things that you will be impressed by, and there are things that you will argue with, but that’s how most papers go, right? But if you read the articles about the paper, your experience will be very different indeed. So let’s do the authors (a team led out of Insilico Medicine) the favor of first talking about what they actually wrote, then we’ll get to what a lot of people seem to believe.

This is work on “generative” drug design, which as the name implies, is trying to generate new structures rather than evaluate existing ones. The idea (which does not apply only to chemical structures) is that you train a computational system on all sorts of existing knowledge and data, and it then comes up with new stuff that seems as if it would fit the existing framework. Now, this is what human medicinal chemists do all the time – look over a bunch of compounds that are ligands for a certain protein and say “Hmm, I think we should try a seven-membered ring here to change that dihedral angle” or “What happens if we make that aryl more electron-poor?” and the like. So generative methods are one of the big things that people have had in mind over the years when they think about what they’d really like computational methods to do for them: “Find me some ligands” And some major subsets of that request are first “Find me some more ligands, because we need some patentable chemical matter”, then second (harder) “Find me some more ligands, because the existing ones all have problems with selectivity/metabolism/stability”, then third (even harder) “Find me some ligands, because no one has ever screened against this protein and we’d like to save time and just go right to active compounds” and fourth (hardest) “Find me some ligands, because people have screened actually this before and always come up empty”.

You can also consider these “virtual screening” efforts by the sorts of chemical matter they’re going to be searching through to come up with those ligands. You could start with a list of actual compounds that you have more or less on hand, such as a list of known drugs, a chemical supplier’s catalog(s), or your own screening collection. Or you could extend into not-made-yet territory, first in relatively easy ways (here’s another hundred amines that could be used to make the amide at this position), and moving on to harder ones that don’t resemble existing chemical matter so much. That’s the “generative” part.

By those classifications, the current paper fits into the “Find me some patentable chemical ligands, which means that you’re going to have to come up with new scaffolds” class, because it’s targeted at discoidin domain receptor 1 (DDR1), a tyrosine kinase target that’s been the subject of quite a bit of drug discovery work already. As the authors note, at least eight different chemotypes have been reported for it in the last few years, so it’s safe to say that finding DDR1 chemical matter is not in itself an outstanding problem in drug discovery. But it’s a good proving ground for a technique like this, which is really on the outer fringes of what’s currently possible and thus needs all the help it can get. The authors describe a software approach acronymed GENTRL (generative tensorial reinforcement learning), which involved training the system up on all the existing DDR1 literature, the larger set of kinase inhibitors in general, databases of medicinally active structures, and an even larger set (17,000) of compound structures that have been specifically claimed in all sorts of med-chem patents. This allows the software to do a multilayer optimization to propose structures that are (1) more likely to hit DDR1 itself, (2) more likely not to hit other kinases, and (3) more likely to not resemble structures that are already patented. And those are just the sorts of starting points that you’d be trying to find.

How did it do? The initial output was 30,000 structures, so the problem then became how to narrow this down. It’s worth noting that a high-throughput screen that produced 30,000 hits would be considered to have failed, but that’s because in that case you’re screening against a large assortment of chemotypes, the great majority of which are surely not going to be real hits – in this case, the idea is that the software is supposed to be zooming in on those real hits to start with. Still, 30,000 starting points is perhaps not all that much of a zoom. They cleared out structures based on molecular weight, number of polar groups, etc., and that knocked it down to about 12,000 compounds. These cutoffs were far more generous than “Rule of 5” cutoffs (cLogP values from -2 to 7 were deemed OK, for example), so the original set must have had some pretty shaggy structures in it. The next round cleared out structures that seemed unstable or reactive, which took it down to about 7900 compounds. They then applied clustering and chemical diversity sorting to the remaining structures, and cleared out the too-similar ones to leave about 5500 molecules, and these were reduced to 4600 by scoring for too-close similarity to commercially available compounds. This set was then scored again through the program’s kinase-evaluating filters and also fit to pharmacophore models derived from known DDR1 ligand-bound X-ray structures. 2570 compounds were called as potential kinase inhibitors by the model, and 1951 of these were specifically called as likely DDR1 compounds. The pharmacophore-based screening took these down to 848 candidates, and the group then picked 40 structures that scattered across the chemical space.

These were almost entirely outside of currently patented chemical matter at this point, and the team chose six of them for experimental validation. I tried running the resulting chemotypes through Reaxys, and it’s true, you don’t find much, as advertised. Of the six compounds, two of them hit DDR1 at around 10 and 20 nM, two others were up in the hundreds of nanomolar range, and two were completely inactive. What I haven’t been able to find is how these six compounds scored in the evaluations above: it would be very interesting to know if the two complete misses had any distinguishing signs versus the two very solid hits, or if they were all in the same basket before testing. My guess is the latter. Overall, if I think about the (many!) kinase inhibitors I’ve seen and about DDR1 inhibitors in specific, and you show me these compounds and ask if they look like they could be added to the list of known ligands for that target, I’d say “Sure, why not?” They look perfectly reasonable (not otherwordly in any way), but of course there are heaps of other structures you could say that about, too – which is why you do screening, of course.

Update: see this blog post by Pat Walters on the similarity of the reported compounds to ones already known in the literature.

The results of a kinase screening panel are provided for the most potent compound, and it’s pretty good against 44 other kinases. But none of the compounds have data against any more general screens, which would be interesting to see. It would also be worth knowing how some of the reported DDR1 chemical matter looked in that same kinase screen, for that matter. The paper goes on to do some microsomal stability assays, mechanistic cell assays, and even doses the lead compound in mice, but to tell the truth, I’m actually less interested in those parts. Metabolic stability wasn’t one of the things that the virtual screening selected for, so it’s the same crap shoot as with any other set of fresh compounds, and if a 10 nM kinase inhibitor has enough stability to be dosed and doesn’t look obviously crazy (and these compounds don’t) I assume that it will show some effects in a cell assay and indeed, even in a rodent. As this one does.

So my evaluation is that this is one of the most interesting virtual screening papers I’ve read. People have been working on virtual screening for a long time now (decades) and it’s been slowly improving the whole time. I take this paper as another step down that long road, and I’m glad that the field is moving forward. The authors do slip some of what I would call headline-bait into the last paragraph, though, saying “In this work, we designed, synthesized, and experimentally validated molecules targeting DDR1 kinase in less than 2 months and for a fraction of the cost associated with a traditional drug discovery approach“. Ah, but how long would it take you to find DDR1 chemical matter by traditional means, if you weren’t doing anything else? Not a heck of a lot longer than that, honestly, and the costs at this point (by any technology) are but tiny little roundoff errors in the total cost of a real drug development project. I like the statement above the last paragraph a lot more: “Despite reasonable microsomal stability and pharmacokinetic properties, the compounds that have been identified here may require further optimization in terms of selectivity, specificity, and other medicinal chemistry properties“. Yes indeed, and that is where you will start to begin to commence to spend the vast piles of money that you will eventually go through in trying to get a drug to market.

Let’s also remember that DDR1 is a very well-trampled area for small-molecule drug discovery. Where would you be, for example, if you didn’t have a big ol’ list of good-quality X-ray crystal structures in order to build yourself a pharmacophore model? If you didn’t have a set of hundreds of known kinase inhibitors to help train your software on (not forgetting either that kinase inhibition is by now one of the most well-worked-out areas at a binding-model level in all of medicinal chemistry). I don’t fault the authors one bit for using this as a proving ground; that’s what you have to do with new technology. But neither should anyone overlook the fact that this example was grown in a very well-maintained greenhouse, not in a clearing hacked out of the jungle.

And that brings us to the press coverage. Oh, dear. The coverage from people who know what they’re talking about is here and here, and I strongly recommend those pieces by Ash Jogalekar and Andreas Bender respectively. I think that they’re roughly where I am on this one: very interesting paper, with some real strengths and some real limitations, but geez, the headlines. The worst I’ve seen so far is this hyperventilating article at LinkedIn, which informs us that “This is Pharma’s AlphaGo moment when the potential for AI to radically transform the operating procedures and business models of the entire industry becomes obvious to the public” and takes off eventually into statements like “By using AI in drug development, it’s possible to accurately predict which drugs will be safe and effective for specific patient subgroups“. Why yes, this was written by someone who helped fund InSilico, why do you ask?

Let’s get this straight: this paper did not discover a drug. It discovered what might be a drug candidate, after a lot more work is done on it. (But no one’s going to do that work, because at the moment the world does not need another drug candidate for DDR1). It did this on a very well-worked-out drug target in an extremely well-studied target class, and generalizing these techniques enough to take them into new drug discovery territory is going to take a lot of time, money, and effort. To get personal, I myself am working on the sort of target that makes anybody’s virtual screening technology choke, turn purple, and fall over. We have plenty of those. So all these folks going on about huge transformative revolutions and all the rest of it should go take a cold shower or something.

The good news, though, is that there is no reason that virtual screening can’t do great things, eventually. We just have to get a lot better at it than we are now, and that’s as true as it was when I first heard about it in the mid-1980s. The academic and industrial groups that have been working on it over the decades have advanced the field a great deal, but there’s plenty more advancement needed. I liked this paper because it shows that very advancement in action, not because it heralds the end of the process. That end, folks, is not yet at hand.

34 comments on “Has AI Discovered a Drug Now? Guess.”

  1. Xavier says:

    Isn’t the 10nm compound they report quite similar to this compound from chembl with known DDR1 activity ?
    (Similarity: 74.597,

    Maybe a 2D molecular similarity against known inhibitors would have sufficed for the priorization step ?

  2. Bioisosteric replacement says:

    The linked post by Andreas Bender shows that their best compound is just two atoms different from CHEMBL1172720, a known DDR1 inhibitor. Would medicinal chemists really treat this as a novel molecule, and therefore a real discovery, or is it an obvious modification? Any patent lawyers in the house?

    1. a says:

      which in turn is weak analog of ponatinib

    2. Anonymous says:

      To crudely summarize a half century of case law, nearly nothing is obvious. Likely to be patentable.

  3. Wavefunction says:

    “The good news, though, is that there is no reason that virtual screening can’t do great things, eventually. We just have to get a lot better at it than we are now, and that’s as true as it was when I first heard about it in the mid-1980s.”

    And that’s the crux of the matter. There’s been a number of advances in virtual screening over the last twenty years; induced fit docking, polarizable docking (whatever happened to that one), inclusion of water thermodynamics, ensemble MD, FEP, QM/MM, covalent docking…the list goes on. Some of these have been less than useless, some of them have dubious utility and the rest are incrementally useful and have their own domain applicability. Those last ones have taken their place in the armamentarium of other tools that drug designers use, but absolutely none of them have transformed drug discovery by themselves. Given this history, is there a good reason these new ML tools will be different, and do we know which one of the three categories listed above they will fall into?

  4. A Nonny Mouse says:

    I did some synthetic work in the dying days of an in silico company after they had blown though almost £40m. Even the pre-screened molecules that I was given to look at (about 200) were mostly a joke in terms of synthesis or stability.

    We eventually made 8 from different classes, but not of them showed a trace of activity against the intended target!

  5. Michael J Olson says:

    Where is your article on Compugen? Of course they are developing against “AI” discovered targets, but do have two now in trials, one bought by Bayer and the other in collaboration with Bristol. What is your perspective on their approach and “success”.

    1. Uncle R says:

      Assume referring to COM701 (and COM 902) as per Compugen website?

      Status: Phase 1 study
      COM701 is a first-in-class therapeutic antibody targeting PVRIG, a novel immune checkpoint target candidate discovered computationally by Compugen.

      Looks like chemical structures undisclosed (patents anyone?).

      If so, I’d be inclined to reserve judgement pending definitive evidence of novelty (i.e. not more examples of minor iterations from what’s known already, cf comments above from Xavier and Bioisosteric).

      One other query comes to mind (which only Compugen can answer) – are COM701 and COM902 “compounds bank” designation numbers, or “development candidate” code numbers?

      If the former (and compound bank numbers are linear), then an old hack once of Big Pharma pretty seriously compellingly overarchingly impressed to see not one but two clinical candidates in the first 902 compounds banked away…

  6. Jakob says:

    I know next to nothing about drug development, but the AI hype is getting a little boring – especially when you are old (and grumpy) enough to have heard it all before. It is the same story as in AI for discovery of biomarkers for personalized medicine etc. Lots and lots of candidate biomarker patterns are being published for virtually any kind of disease. Now all we need is for “someone” to develop a robust and high throughput method for measuring this pattern so that we can move into actual method validation in order to find out if this new technology is good enough for clinical measurement and, if it is, then to advance the new candidate molecular pattern into clinical validation to find out if it is at all useful when integrated into the clinical context. How great it would be if AI could find a way to reduce the work-effort and costs associated with these challenges also… – and not only for discovery of candidates.

    1. Mostapha Benhenda says:

      Yes, AI (or a very very simple version of it) is also useful for clinical biomarkers, read my piece:

  7. loupgarous says:

    The problem here is journalists do journalism (selling ad space around clickbait, usually), scientists do science. These domains are skewed non-congruent planes in most cases. Good science writers don’t generally work for Big Media outfits.

    1. Jim Hartley says:

      “Skewed, non-congruent planes”, I like it!

    2. matt says:

      I think this is unfair. There are, in fact, some pretty good science writers writing for big enough media organizations. This blog has mentioned, in fact, how the quality level of American science reporting is light-years above the level found in British media.

      Instead, this is attention bias. If you look for ignorance and bias, you will certainly find it, and no shortage. Somebody writing for LinkedIn–not really even in the media business, is it? or have I missed some transformation?–should hardly be used to tarnish journalism. I see similarly biased articles on the breathless-science-is-now websites, because like you say they aren’t doing journalism they are doing page impressions and ad sales. And because hype is what they do, the singularity is near, thanks to silicon valley finally revealing how everything (including our fecal material) should be growing at an exponential pace.

    3. Scott says:

      I’m pretty sure you could extend that to “non-intersecting planes” and still be 100% accurate.

      1. loupgarous says:

        Parallel planes can be nonintersecting but only apart by an offset distance in one axis. Apply the offset to one plane, they’re congruent. If I’d chosen “nonintersecting planes” as my metaphor, one could argue (and other posters here have) that the scientist and the journalist do basically the same thing at different levels. I don’t think that’s always true.

        Writers like Gina Kolata and Richard Preston are careful to be methodical in their work and convey accurate impressions. But they’re out-shouted by other journalists and editors after splashy headlines and who need to sell eyetracks to advertisers in journalism that may intersect science on a aingle line, but fails to be recognizably congruent with science. That’s what the Washington Post did in an article accusing Pfizer of concealing data that its drug Enbrel (etanercept) prevented Alzheimer’s disease, under the headline “Pfizer had clues its blockbuster drug could prevent Alzheimer’s – Why didn’t it tell the world?”.. That article was based on internal Pfizer documents referring to a study in which fewer people who took Enbrel were diagnosed with Alzheimer’s disease than those who took Enbrel and were diagnosed with Alzheimer’s, and a corporate decision not to perform a study to confirm or deny what might have been a causal link between taking Enbrel and not getting Alzheimer’s disease.

        Most people reading this blog know correlation doesn’t imply causation. The post hoc, ergo propler hoc and cum hoc, ergo propter hoc fallacies have led people astray for centuries. A large newspaper like the Washington Post could have interviewed researchers within easy driving distance (say, the National Institutes of Health at Bethesda, MD) and been told that the study merely suggested a correlation – not a cause. Instead, they falsely accused a major drug company of concealing a possible preventative for Alzheimer’s disease in a lurid headline. The Washington Post did to Pfizer what they’ve done to too many others – tried them in the court of public opinion with misleading arguments.

        1. loupgarous says:

          Oops. Second paragraph ought to have ended

          “That article was based on internal Pfizer documents referring to a study in which more people who took Enbrel were not diagnosed with Alzheimer’s disease than those who took Enbrel and were diagnosed with Alzheimer’s, and a corporate decision not to perform a study to confirm or deny what might have been a causal link between taking Enbrel and not getting Alzheimer’s disease.”

    4. Isidore says:

      Scientists though are complicit to the exaggerated claims of the PR people, whether ignorant journalists or university publicity hacks. I recall a conversation with an acquaintance at a major university whose team’s discovery had been hyped to ridiculous proportions by their institution’s publicity office. I pointed out the exaggerated claims and asked him if he or anyone else on the team (he was a senior member but not the “big name”) had seen the press release in advance or at least had ben consulted by the PR people. He was rather sheepish about it and as it turned out they had, in fact, seen the text ahead of time and if there had been any objections (he was rather evasive about it) they certainly had not been expressed strongly enough. I don’t suppose industry scientists are any less willing to go along with the hype.

      1. Ian Malone says:

        It’s a complex process behind this. If you talk to our university press office they’ll tell you they are interested in getting the science right to maintain their credibility. They also keep in touch with serious science journalists, because you can’t control what people write and inevitably there will be those who sensationalise, but at least you can try to get the accurate version out first.

        It’ll vary from place to place of course, and there is something of a chinese whispers aspect, where academics are encouraged to produce impact statements for their work. These, quite reasonably, have a ‘big picture’ component; understanding how protein X works will improve our understanding of disease Y and may ultimately lead to a cure. Nobody approving the grant or working in the lab is going to think getting a better understanding of the protein’s binding site (or in my line of work, improving the quality of the measure on a PET scan) will directly cure a disease, it’s a small part of a much bigger scaffolding (the same applies for cell and animal work). But as things travel up the chain and get simplified all the details and caveats gradually fall off and you’re left with X could lead to a cure for Y.

  8. Jay says:

    Just saw in FierceBiotech today that Astra Zeneca did a deal with Schrodinger. On the deal sisde there is also Atomwise and Exscientia. Some of those employees read this blog and have responded, though I never found their comments to be helpful, and one comment from Exscientia was a smug one word reply. The only people making any money out of these are the employees while the gig lasts.

    1. Passerby says:

      One of the big problems of our times is that deal-making and VC funding are considered proportional to the value of the science or technology. Keep in mind that VCs are like gamblers or horse bookies; they have no problem throwing $50 million at a company if there’s a 0.0001% chance of it working, so their investment says very little about how good the technology actually is. Good VCs can actually assess the real science and statistics, not just the buzzword and the hype (Theranos, anyone?).

      1. Earl Boebert says:

        Passerby says: “Keep in mind that VCs are like gamblers or horse bookies…”

        Precisely. In the early 1990s I had to do the VC circuit to keep my team together. Neither the happiest nor proudest moments of my career. It became very clear that the question in their mind was whether our enterprise was worth a bet. And the bet was not on whether our technology would work or be worthwhile; it was on whether we would catch the eye of a potential buyer. I bailed shortly after that.

  9. loupgarous says:

    I’ve quoted John D. Clark’s book Ignition! on this issue before, but that was three years ago:

    “Just as Wharton was starting his IBA work, there occurred one of the weirdest episodes in the history of rocket chemistry A. W. Hawkins and R. W. Summers of Du Pont had an idea. This was to get a computer, and to feed into it all known bond energies, as well as a program for calculating specific impulse. The machine would then juggle structural formulae until it had come up with the structure of a monopropellant with a specific impulse of well over 300 seconds.

    It would then print this out and sit back, with its hands folded over its console, to await a Nobel prize. The Air Force has always had more money than sales resistance, and they bought a one-year program (probably for something in the order of a hundred or a hundred and fifty thousand dollars) and in June of 1961 Hawkins and Summers punched the “start” button and the machine started to shuffle IBM cards. And to print out structures that looked like road maps of a disaster area, since if the compounds depicted could even have been synthesized, they would have, infallibly, detonated instantly and violently. The machine’s prize contribution to the cause of science was the structure,

    H—C= C—N N—H
    O O
    F F

    to which it confidently attributed a specific impulse of 363.7 seconds, precisely to the tenth of a second, yet. The Air Force, appalled, cut the program off after a year, belatedly realizing that they could have got the same structure from any experienced propellant man (me, for instance) during half an hour’s conversation, and at a total cost of five dollars or so. (For drinks. I would have been afraid even to draw the structure without at least five Martinis under my belt.) ”

    Since 2016, the last time I quoted this passage, progress has been made. The compound can be made, it’s stable, even druggable to the extent it shares structure with existing DDR1 kinase inhibitors, and shows the same sort of activity in cell assays and mice.

    It’s not a patented compound, but other posters here wonder aloud if it’s patentable when it differs from an existing drug by two atoms. Still… the field of AI drug design’s come a significant way in three years. Compare and contrast with a 2016 “In The Pipeline” article.

    1. NJBiologist says:

      If I was biology lead for a program, and one of the chemists came to me and said that the compound that they came up with–the one that passed the in vitro screens, was active in cellular assays and showed concordant activity in an in vivo assay–turned out to have a lethal FTO issue, I’d probably say something to commiserate and then buy them a beer in recognition of what they actually had delivered.

    2. DTX says:

      loupgarous – thanks for that fascinating piece of history. Knowing this has been going on since at least 1961 gives much context.

      A friend of mine is a science writer, previously for both for the journal Science and several newspapers (and she’s a responsible reporter). She pointed out that the newspaper editor generally creates an article’s title, not the writer. Titles often hype things much (as noted above). In addition, often it’s the scientists themselves that hype their work/their findings. It can be challenging for a writer to find a scientist with an alternative view who will publicly dismiss the hype of another scientist.

  10. Dominic Ryan says:

    Because I’ve been on the AI/ML/QSAR/CADD/MedChem ride for decades I’ll take the chance of stating the obvious.

    Success at marketing does not equal success at science.
    Nor is the opposite true, and in either direction!

    To me this is the sad part of the story because it is a local manifestation of a bigger problem. Society, perhaps rightly in many cases, has put pharma in the doghouse. This is just the most recent manifestation of science getting dumped on.

    Aah, it is the journalists’ fault!

    No, I don’t think so, at least not mostly so.

    The problem is that science as a whole is too eager to brag, too eager to want that Gold Medal, too eager to proclaim that ‘we are not like the last batch of blowhards’. Perhaps much is well meaning but we are doing this to ourselves. Who is the audience for the NBT paper? Journalists? Not primarily I hope. We need to come up with a better way of filtering or tempering grandiosity -no easy task and I don’t have the answer.

    Tolerating this is only hurting ourselves. The paper in question may have in fact added something useful buried in the ML details. But once word gets out that the ‘new drug candidate’ is a compound extremely similar to the data feed stock and that the problem had a uniquely, perhaps embarrassingly rich training data set, the community is likely to look at it differently, perhaps dismissively. That’s a shame. I’d love to see a ‘sensitivity’ analysis for this kind of work: at what point do the results change in a meaningful way as you loose data quantity or quality? If your method is much better than another it should do that better. Lets not even get into data curation, feature generation, selection and filtering. Perhaps a metric could be robustness until some cliff when randomness is introduced? The topic has been beaten on by many for a very long time.

    Stepping back, Derek points out how ludicrous it is to call this screening hit a candidate and with no data! This merely adds fuel to the pharma pyre. I really do get, and support, that a startup needs to demonstrate progress. Perhaps this ticks that box, but please don’t sell it for what it is not.

  11. EB says:

    Regarding DDR1 as a target, in Davis et al., 2011, Comprehensive analysis of kinase inhibitor selectivity. Nat. Biotech. 29:1046-51 (, over a third of the reference compounds tested had a Kd of 300 nM or less on DDR1.

  12. dipthroat says:

    call me old fashion, but when I read about this in the news, my first thought was:” here we are again with some BS claims about AI achievement”. I am glad, you are confirming my initial feeling. And, the compound apparently hasn’t even be facing the most important test, that is, toxicity in humans.
    No question, that one day in the future, all biology and chemistry will be done in silico. However, 99.99999999% of today’s AI wonders, and startups, are just poppycock.
    It also makes me giggle, thinking I clashed with a company, during the hiring process, which claims of using AI to generate new protein therapeutics from molecular libraries. I guess, nobody told them, and their clueless investors (including pharma companies), that in order to train their system, even if it was a good one, it would need a number of suboptimal binders, far larger than what is possible to provide experimentally.

    1. Frank says:

      Exactly! There are not enough positive and negative controls to train the “smart” AI yet. There is no obligation for effective drugs to follow a set of parameters.
      One of the dumbest AI buzz I have heard was ML/AI based-discovery of “novel” ADC drugs– a drug class with only five positive controls as June 2019, and none of them target same receptor!. So the learning sample size was magic….

  13. Anon says:

    I had two initial responses:

    1) At least it’s not p38.
    2) Funny how they don’t flag that, at one point, they were trying to train GENTRL by going around to Big Pharma, hooking up some skullcap thing with electrodes to the heads of medicinal chemists, and showing them structures of molecules while asking them to think “is this a drug?”.

    (No, I’m not kidding.)

  14. BioDude says:

    Through a combination of smart math and substantial improvements in computational resources (memory, GPU, CPU), Neural Nets/Machine Learning/AI is now a fairly mature platform. The key component is whether you have the right kind of input data and the right kind of output read to create a prediction engine of real value.

    The article being discussed took what might be the lowest hanging fruit of all–a system quite well understood, in a class quite well understood, and with very large characterized data sets–and created some “hey, that’s pretty cool” predictions by applying AI. But the entirely of the exercise was to identify additional leads, judging primarily by the criteria of binding constant and stability of the molecule. As has been noted, this is frequently/usually the easiest part of the drug discovery process for a well defined target. The harder benchmarks with respect to ADMET, promiscuity, etc. haven’t been addressed. Nor does this method speak to hard targets with substantial flexibility and few/no known binders, “undruggable” molecules and the like. It is down those alleys where the real pots of gold lie if an AI machine can suitably trained.

    And therein lies the real issue: When it comes to those high value predictions, there’s a paucity of training data. And and few things are more worthless than an insufficiently/improperly trained ML model.

    I like this paper. I definitely think it’s cool. I wish I were an author. But I agree the claims surrounding it, that it demonstrates that AI WILL lead the way, are still very premature.

  15. Old says:

    I won’t belive any of this AI magic until they show the selection of a novel compound that binds selectiviely and has favorable drug-like properties consistent with investing in initiating a lead optimization program against a unprecedented target with no previous chemical equity and minimal structural information, you know real targets for drug discovery. Nature Biotechnology should be ashamed for publishing this work on another kinase

  16. Pat Fan says:

    Pat Walters has a great blogpost debunking these results. What were the reviewers thinking?!

  17. VC says:

    Insilico is known to hype highly dubious data. They seem to be a marketing shop with some “AI” on the side.

  18. loupgarous says:

    The other posters have cited a great list of ways for this AI approach to go wrong. At this point, the sheer lack of positive and negative training examples seems to be the main problem here.

    But, as some of us spoke of in this blog three years ago while commenting on the Theranos debacle, the more general issue of pseudovalidation of a process by computer looms large. Computer programming processes – “object-oriented code” as a common example – can be complex and poorly-documented, so that the programmers and users of some software have difficulty tracing program execution through complex software architectures to validate program results.

    I saw that happen while validating a minor program for generating tables of the top ten adverse events observed during a clinical study. The team’s well -used-for-years reporting macro it called was producing tables of AEs which seemed to be in error.
    Fortunately, the macro and the tabulation program which called it were simple enough I was able to manually discover the issue by using a subset of the data and walking through the program logic with pencil, paper and a hand-held calculator. Thank God it wasn’t AI in that case, just arithmetic.

    I shudder to think how many statistical analysis functions we developed in-house and which were validated by pairs of programmers (sometimes, four of us working in stages) just didn’t work the way we thought they did. We’d have had to walk back many reports of safety and efficacy if our faith in the programming processes we were using seriously wavered.

    I hope others here who are much better-informed than me in what the AI is intended to do are right, and that it stays a tool to identify lead compounds. If we lean on it to assess safety and efficacy, we could see the same sort of dangerous errors that the users of highly automated beam radiotherapy devices saw roughly ten years or so ago, and even earlier than that during the Therac-25 incidents. Safety and efficacy are best looked at through simple lenses.

Comments are closed.