Skip to main content
Menu

Drug Development

Arguing on AI Drug Discovery

Here’s a letter from Pat Walters and Mark Murcko of Relay Therapeutics on the September report from Insilico Medicine (blogged here) of a drug discovered by AI, specifically generative methods. Here’s their working definition of what that means, which I think most folks in the field can agree with:

. . .In this technique, a deep learning model is trained based on a corpus of existing molecules. The model typically ‘encodes’ a higher-dimensional representation, such as a SMILES (simplified molecular-input line-entry system)10, into a lower-dimensional representation, often referred to as a latent space. This latent space can then be ‘decoded’ back to the higher-dimensional representation to create new molecules. The exploration of this latent space can be coupled with a predictive model with the aim of discovering novel, active molecules. In a sense, generative models can be seen as a variation on the de novo design11 programs that were in vogue during the 1990s and early 2000s. As with de novo design, evaluating the significance of the output of these models is not straightforward. Although two groups have made initial efforts at developing methods for benchmarking generative models12,13, evaluating the novelty, and ultimately the significance, of the molecules generated by these methods remains an open question. . .

As Walters pointed out at the time, the best structure from Insilico for their target DDR1 was actually rather similar to the marketed kinase inhibitor Iclusig (ponatinib). It is (as shown at right) an inhibitor of DDR1, but of plenty more kinases as well, which is why it ended up with a black-box warning for toxicity. Now, the Insilico compound was reported with 44 kinase activities, Walters and Murcko point out that none of these overlap with the reported activities of ponatinib, which leaves the question of selectivity unresolved. A pointed (but important) question is brought up about the criteria for evaluating such work: “One has to ask whether a paper reporting work in which a team of chemists substituted an isoxazole for an amide carbonyl to generate a compound that is roughly equipotent with published compounds would be reviewed, let alone published” And that’s true, but one counterargument is of course that this was software doing it, not humans – but the counterargument to that is that we don’t really need fancy software to tell us to try such an analog, and that maybe we should get more excited about generative models when they suggest something to us that’s a little less obvious.

A larger question is about training sets for such generative models. The Insilico team provided references for the sources of the data used for the model, but did not give a detailed breakdown of just how these structures were used. As generative drug analoging grows in importance, it’s going to be crucial for people to make the entire training set available in detail when such work is published. The letter suggests a set of rules for future publications in this area, which seem reasonable and worth following up on.

Two of the key authors on the paper (Alex Zhavoronkov and Alán Aspuru-Guzik) respond in a back-to-back letter, which is good to see. Some of their points:

The critique of Murcko and Walters, and many similar online commentaries, fails to recognize that, as we state in our paper, our goal was to provide the first demonstration of the effectiveness of a novel generative approach; as such, in-depth validation of the molecules produced was not the main goal of our paper. We readily acknowledge that the compounds require further optimization. . .Regarding the statement that “compound 1 is selective,” it should be emphasized that selectivity versus DDR2, as well as against the small panel of kinases provided by Eurofins, is exactly what was claimed in our paper. There were many structures generated by GENTRL that were substantially different and likely to be more selective, but these were more difficult to synthesize in the short self-imposed ‘race’ mode of our original work.

Perhaps the “race” mode is part of the problem here. One hopes that as such methods improve and become less newsworthy that they can be used less in what might be termed “make a big splash” mode, which is what we’ve been seeing a lot of. Once we’re past demonstrating first uses, we can get down to business.

To that point, readers will recall that I expressed skepticism that the startup EQRx would be able to generate new follow-on drugs at the ferocious pace claimed in their publicity (a pace that is said to be enabled by advances in computational drug discovery). I wanted to mention that my prediction on this is now posted at Longbets.com, ready for all comers to challenge. The next step is for anyone who wants to take the other side of that wager to make themselves known on the Longbets site, at which point we will negotiate terms to have an officially posted wager, winnings to go to charities of our choice. Anyone want to take me up on it? I’m doing this because I have an allergy to the hype I detect in such cases, but on the other hand, if EQRx can prove me wrong, I will not be upset by that, either, because it will mean that we really have been making breakthroughs in how quickly drugs can be developed. But I don’t think that’s the case – not enough to enable ten new ones in ten years. Step right up!

21 comments on “Arguing on AI Drug Discovery”

  1. Jack says:

    In Derek’s entry from July 17, 2018, “The Case for Virge Genomics” I pointed out in the comments section that the future of drug discovery was already here in Compugen, which will soon have three in silico discoveries in the clinic, notably COM701.

    Derek, in that thread you mentioned that you would follow the company’s development…have you done so?…if so, I’d love to hear your viewpoint.

    Thank again for a fabulous blog!

    1. Druid says:

      COM701 is an antibody, like all Compugen’s late stage projects. Compugen says it “discovers novel drug targets through a unique, predictive, computational process …”. That is great, but how is it related to computers designing NCE drugs? And your previous publicity-seeking was 18 months ago – not so fast, eh?

      1. Chrispy says:

        Looks like Compugen jumped on the hype bandwagon.

        If they were designing antibodies in silico, I’d be impressed!

        1. Jack says:

          They have.

  2. tlp says:

    I guess, equivalent in face recognition would be a model trained on pictures of US presidents spitting out photo of Donald Trump with mustache. It is either super exciting (if Donald Trump was excluded from the training set) or completely trivial (if he wasn’t).

    1. Natalie says:

      Not really. Trump with mustache with Trump in the training set was MIT Tech Review’s innovation of the year in 2016. And here is the fun part:
      https://www.technologyreview.com/s/611726/the-defense-department-has-produced-the-first-tools-for-catching-deepfakes/

      The #1 most popular paper in 2019 was published by another Russian guy who used Mona Lisa (they love Trump):
      https://www.altmetric.com/details/60733304

  3. anon3 says:

    Good to know that AI is way behind DEL when it comes to discovering novel chemical matter:
    https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6343110/

    1. Dr. Liu says:

      DEL is expensive. GENTRL is free if you modify.

  4. anon says:

    I’d take that bet but 1:1 odds aren’t going to cut it. I’d rather buy TSLA at 1k.

  5. Wavefunction says:

    I am skeptical of the line in the response saying “Similarity is in the eye of the beholder”. That can often be true in case of molecular similarity, but not in this case in which the compounds are similar by both chemical similarity as well as PK/PD similarity. I think most competent medicinal chemists would reach a consensus that these compounds are similar.

  6. Hungry Chemist says:

    The element of generation of the structure is only part of the problem – as is rightly pointed out, many medicinal chemists would propose the same change. The question is also whether these algorithms are capable of prioritising ideas with higher accuracy than a medicinal chemist. Would every medicinal chemist pick out this change as an early one to make? Would they follow it up straight away even if they did? Did the algorithm pick this up ahead of other bioisosteric replacements or was it one of a set?

  7. Michael J Olson says:

    Jack
    I’m not sure you actually understand what Compugen does. They identify novel “targets” insilico and contract the development of corresponding antibodies (drugs) to others. Then do the validation and clinical work (in collaboration with Bayer, Bristol, etc.). What they have accomplished is however quite remarkable.
    The early data on their anti PVRIG (DNAM axis) drug in KRAS CRC is particularly noteworthy.

  8. MedChemist says:

    AI shows promises indeed, but it is all over the place and starts to be boring. Can we talk drug discovery again and use AI the same way we use TLC, SciFinder, spotfire and common sense?

  9. Ozymandius says:

    If a straightforward logic tree (algorithm) can design new chemical entities starting from some generated model or lead compound, then what is the novelty in the design. Specifically, what are the patent implications if the design is a straightforward (and thus predictable) outcome from known starting points? Where’s the art? If multiple companies use the same algorithm, will they naturally be guided to the same designs? I can see a future where computational design will be challenged in the courts based on the obviousness of the invention. Just more risk to add to an already risky venture of drug discovery.

    1. Dropin says:

      AI didn’t just design one compound. It could in theory suggest a vast array of possible analogues to make of every compound in its training set. As you say, that isn’t patentable until someone decides which ones to make, makes them, tests them, and finds that they work.

      I don’t see anything in this paper to suggest that AI decided that this compound was the one to make out of all the other possible analogs it could have suggested. Until the day that happens, which won’t be in my lifetime, I think there’ll still be employment for patent lawyers.

  10. michal says:

    “There were many structures generated by GENTRL that were substantially different and likely to be more selective, but these were more difficult to synthesize in the short self-imposed ‘race’ mode of our original work.”

    Does this translate as “we wanted to publish ASAP so we did chose the simplest/easiest molecule we could make” ?

  11. Bioduuude says:

    Is AI as applied to drug discovery still at the first peak in the Hype Cycle? Absolutely. Was the Zhavoronkov paper a bit wobbly with respect to broader significance? Sure.

    The ultimate issue with AI as applied to drug discovery is that we don’t have big data for training, and it’s unlikely we ever will. Certainly not on the scale of the data used to train AI models for language processing and image scanning–the poster children for asserting that AI has arrived. (It has arrived, but not for every application).

    The suggestions in the Walters/Murcko letter are fine: More transparency with regard to data, broader comparisons to existing matter. All fine. Just the latest in a long line of such proposals as applied to computational chemistry. Most of which get nodded heads and then get ignored. But these are always good points to make. I have been a party to dozens of such roundtable discussions over the years at conferences. Everyone in the room agrees and nothing substantive ever happens.

    Digging a bit deeper into the Zhavoronkov paper: I have had many discussions with scientists in the field of drug discovery since it was published. And I’d say that the issues many have with it are a combination of frustration from the hype that surrounded the paper when it was published, and if we’re being honest, a bit of jealousy that they were first out of the gate. As I’ve said about the paper: It’s not nearly as amazing as you might imagine from the hype surrounding it (for the reasons that Walters has described), but I think it’s a nice proof of concept piece and if I’m being honest I wish I were on the author list. It’s an interesting paper and I would assert it’s an important paper because it DOES effectively prove the concept. If AI for drug discovery ever amounts to anything real, that paper is justly going to get cited in most papers and every review article in the field for decades.

    1. Passerby says:

      It proves that you can use a complicated method to generate a result that can be arrived at using many vastly simpler methods. That is not a “proof of concept” for me. A proof of concept for a purportedly novel method is to generate a minimal version of the *novel* result that the method is supposed to provide. This is validation at best.

      1. KG says:

        I guess it depends on how you classify novel.

        I mean, adding a single pendant group to an established platform can totally change PK, and can be patentable.

        Just saying “this method produced a molecule that looks ‘like’ something that is known doesn’t mean the method hasn’t identified novel and valuable matter.

        My reading of the original paper and of the Walters/Murcko letter is that they did identify patentable matter,. The letter questions whether it would hold up in terms of specificity, and I agree that’s a question not satisfactorily addressed in the original paper.

        But I don’t think the bar for proof of concept is that you ID a wholly novel scaffold.

    2. The biggest joke is that this is not even by far the first paper that does prospective “AI” driven design. Gisbert Schneider has a bucketload of prospective (Angewandte) papers out where they generated nanomolar compounds. Under the lead of Daniel Merk they did this even with Neural Generative Models. But for some reason the community seems to ignorant of this.

  12. Dr. Liu says:

    We modified GENTRL code and use it a lot. Very appreciate the paper.

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.

This site uses Akismet to reduce spam. Learn how your comment data is processed.