Skip to Content

An IBM-Watson Collaboration Goes Under

I’ve written several times about IBM’s Watson machine learning system and its potential applications to health care. To be honest, many of these applications sound unlikely, at least at present, and that skepticism doesn’t apply only to IBM by any means. Now word comes that a collaboration between IBM and the M. D. Anderson people (announced in 2013) has fallen apart.

But to be fair to IBM (and to Watson!) a similar arrangement from around the same time with Sloan-Kettering seems to have been more successful. Along with Quest Diagostics, they seem to be in the process of launching an actual product. How good it is, and how successful it will be, there is no telling yet, but that’s the end result of a development program, and it’s a goal that the M. D. Anderson work seems to have missed quite thoroughly.

That failure seems to be partly (or perhaps more than partly) on the Anderson side of the collaboration. Matthew Herper quotes reporting from the Cancer Letter and the Houston Chronicle that makes things sound quite odd:

In a strange twist, MD Anderson would pay for the whole thing, eventually giving $39.2 million to IBM and $21.2 million to PricewaterhouseCoopers, which was hired to create a business plan around the product. According to the Washington Post, at least $50 million of the money came from Low Taek Jho, a flamboyant Malaysian financier whose business dealings are reportedly now under investigation by the U.S. Department of Justice.

Usually, companies pay research centers to do research on their products; in this case, MD Anderson paid for the privilege, although it would have apparently also owned the product. This was a “very unusual business arrangement,” says Vinay Prasad, an oncologist at Oregon Health & Science University.

According to the audit report, Chin went around normal procedures to pay for the expensive undertaking, even making sure individual payments to IBM were below a threshold that would have required her to get approval from MD Anderson’s board. She also didn’t get approval from the information technology department.

To round it off, it appears that the donations from Jho may not have even been received yet, which really does make a person wonder what’s going on. But it’s worth noting that there was good press about this effort along the way, with stories about how well the software was doing at diagnosis, etc. Something to consider when you come across stories written in the same vein, for sure. Wait until the project delivers before making it down as a success. . .

20 comments on “An IBM-Watson Collaboration Goes Under”

  1. Jeff says:

    Maybe Watson doesn’t have enough computational horsepower to both cure cancer and figure out tax returns for H&R Block.

  2. anon the II says:

    Seems that I remember from a few years ago, the new head of MD Anderson spouting off about how they were going to find cures for cancer using computational methods. I was disappointed but was forced to write him off as a moron. I didn’t pay much attention to it all after that except to feel sad any time I heard someone mention MD Anderson, knowing it was being run by an idiot. I wonder if this was what he was talking about.

  3. Peter Kenny says:

    When an organization spends large amounts of money on acquiring capability it is usually in the interests of both customer and vendor that the purchase is seen in the most positive light. With computation in drug discovery there tends to be a fair amount of “X worked here so it will disrupt/transform/revolutionize drug discovery”. The Open Source folk talk about a Linux for drugs and others tout mobile apps, AI, gaming and VR. Don’t get me wrong, I’m not saying that these are not useful technologies. However, we don’t have experts that can simply gaze a molecular structure and tell you whether or not it will cure cancer in humans (we have a lot of ‘experts’ who would challenge this statement). We can set up in silico games but how relevant is in silico victory to curing cancer? I have linked my contribution to the JCAMD 25th anniversary issue as the URL for this comment

  4. Anon says:

    “How good it is, and how successful it will be, there is no telling yet”.

    Let me just say, Watson couldn’t even tell me who won the US Presidential election 3 days after the results were pasted all over the news. Instead it gave me all sorts of stats on Twitter activity on the subject, but to answer ma actual question of who actually won, it took one second via Google. Watson is crap, it can only tell you what you first specifically feed it, like some retarded parrot. Garbage in, garbage out, and then some.

    But don’t take my word for it, you can actually sign up for a free trial run. So go on, give it a go, and report back here on the kind of crap it churns out. I bet it won’t even come close to answering even your most basic questions.

    1. Anon says:

      P.S. If I ever interviewed a human candidate that gave the kind of dumb and irrelevant answers to my questions that Watson gave, I would throw them out the room, and I’ve never done that with ANY human candidate (and believe me, there have been some pretty thick ones over the years). So the chances that Watson or any other “AI” programs will take over any human jobs are between miniscule and zero. Unless you’re thick enough to think otherwise.

      1. Pennpenn says:

        To be honest all you’ve really shown is that you’re willing to blow up and dismiss AI concepts just because they don’t immediately act and respond like a human, which is kinda childish. We’ve been working on this, what, a few decades (at the most generous) as compared to millions of years of development (technically undirected, but still development) that went into the human brain. So yeah it’s not going to act like a human. And as if you wouldn’t build an AI to purpose if you were going to use one. And as if Watson wasn’t a work in progress.

        Sheesh, we’d not have gotten anywhere if we had your attitude of shouting down stuff that’s weak because it’s still being developed.

        1. Anon says:

          No, I dismiss it because Watson couldn’t even answer the most simple questions. Now there may well be a future in AI, but IBM should not be hyping and marketing Watson as such an advanced super genius solution, because it really is crap. Apart from playing a couple of games with predetermined rules. And MD Andersen seem to agree.

          Have you actually tried it like we have, or are you just defending it as a theoretical concept?

          1. David says:

            Didn’t it beat humans on Jeapody? Granted, not exactly useful, nor incredibly simple as far as questions go, but still “can’t answer simple questions” seems to not always have been the case?

          2. Anon says:

            True, but then Google could perform just as well. And it’s free.

  5. hn says:

    Real story here is Lynda Chin, who also happens to be wife of MDACC president, Ronald DePinho.

    1. MrRogers says:

      That’s exactly right (although DePinho is also to blame). First an $18 million no-review grant, then a $2 million office that coincided with layoffs due to financial pressure, and now this. You’ve got to wonder how long the board will continue to have patience.

  6. loupgarous says:

    What’s with MD Anderson, anyway? Copyrighting “moonshot”… now jumping into bed with Malaysian money men to to make computational snake oil with Watson? Makes me glad that when I went to Houston to have my cancer treated, it was with people at St. Luke’s.

  7. Anon22 says:

    There is a lot of controversy at the physician/MD heavy M. D. Anderson. Essentially every physician is a millionaire through the high salaries, bonuses, and consulting. One guy by the name of Laurence Cooper took technology from University of Minnesota, slaved his grad students and postdocs with typical MD “I can’t do it myself, but you better make it work or else” approach and he is now reaping the rewards. It was licensed to Intrexon/Ziopharm and then they made Cooper the CEO…no one knows how that works…he also maintains treating/patient privileges at M. D. Anderson…and they are doing trials at M. D. Anderson. Any who.
    Chin is the wife of the Depinho. The greater context is that the institution recently lost over $100 million and had to lay people off to make up that difference.
    http://www.chron.com/news/houston-texas/article/MD-Anderson-set-to-announce-layoffs-today-10837056.php
    Not sure how being conned by business consulting firms is tolerated, but there they are.

    1. Cato says:

      Thanks for mentioning that–didn’t realize about that huge conflict of interest at MD Anderson on CAR-T trial

      Here’s a link from Houston chronicle

      Every time I hear about MD Anderson in the news recently, it seems to always be about an ethical violation…

  8. Glen says:

    The primary use of EMR systems is to provide documentation so that the hospital gets paid. They can also streamline physician orders and improve access to information from other other providers.

    Financial reasons forced MD Anderson to shift to a popular EMR to improve billing efficiency and collections. It is unfortunate, but not surprising, that Epic will not work with Watson. While there are a relatively small number of major EMR providers, the systems are structurally different, and don’t work well together.

  9. Dominic Ryan says:

    Artificial Intelligence is not magic but we want it to be, at least that is what popular culture seems to make of it.

    AI is essentially an inference engine. It makes connections that go a couple of steps beyond an index lookup by picking the most probable edges connecting nodes of data. The probability attached to those edges comes from an analysis of distributions of information and the recognition of outliers as significant.

    Frankly I am not at all surprised that Watson didn’t give you the election answer. Google is built to score web pages based on a massive database of people making connections. That’s not AI, that’s I.

    I rather doubt that Watson has attempted to replicate Google’s mining of web pages.

    I think Watson’s key is in fact demonstrated in Jeopardy. It has a strong ability to parse language (well perhaps mostly English?). That is no mean feat. NLP, or Natural Language Processing is a difficult but evolving computer science area. Doing it well means that you can aim it at complex text and hope to extract connections that would otherwise take a person reading it to accomplish. So, why not just read it? Well, let’s say you want to read all the literature on P53 and that includes published sequences, gene and protein names, clinical trial results etc. You and a small army could do it or, perhaps, you could give it a try with Watson?

    In my past employer we were looking at that type of question. I found it worth investigating based in part on data from another Watson project but I think the jury is out on just how much signal it needs above the noise of irreproducible results to provide actionable information. I think it will be like most computationally-assisted problems. If you just push a button it is quite likely to fail. If you get closely involved and guide at every step it might open doors for you that you didn’t know were there.

    Anon’s comments above reflect a misunderstanding of what to expect from AI

  10. Argon says:

    “[…] $21.2 million to PricewaterhouseCoopers, which was hired to create a business plan around the product.”

    Most (all?) corporate consultants and hired business planners are always happy to take as much money as you’re willing to provide. Doubly so for products that don’t exist.

    I wonder if IBM applied waterfall or agile development strategies to biological research.

  11. Li Zhi says:

    What is the simplest question? My vote:”Are you there?” or “Hello?”

  12. fajensen says:

    Not surprising they went tits-up: If they *really* need to pay the likes of PwC almost *half their budget* to do the business thinking, then they either haven’t got much of a clue at all to begin with.

    … or there was actually a scam going down and they needed a “bullet-catcher”, the top-tier accounting bureaus will price this eventuality into a contract too, since nobody ever goes to jail over financial fraud, the whole thing is just another risk on the risk-register for them.

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.