Skip to Content

Theranos And Its Blood Test Machine

Theranos is a very high-profile startup doing blood tests. You’ve probably come across them (or their founder, Elizabeth Holmes), promoting “X number of tests from a single drop of blood” and so on. The company has been making a big splash since at least 2013, when this article in the Wall Street Journal came out (I keep thinking that I’ve written about them here, but I can’t find a post, so apparently not). In that article, things are working fine:

Theranos’s technology eliminates multiple lab trips because it can “run any combination of tests, including sets of follow-on tests,” at once, very quickly, all from a single microsample. Ms. Holmes estimates that patients and doctors will receive readouts in “as little as two hours” and can even do so before an office visit based on their physician’s recommendation for better, or at least less ad hoc, consultations. . .

. . .Another Theranos advance is its testing’s accuracy. Ms. Holmes believes the chain of conventional laboratory custody introduces too many opportunities for error, “which is basically wherever humans are involved.” . . .Theranos’s technology is automated, standardized, and attempts to subtract human error from the process. It can thus achieve much lower variance ranges for a given test. Ms. Holmes says its tests have margins of “allowable error” targets less than 10%.

How they’re getting this to work has always been a bit of a mystery, though. And a much different article in this morning’s WSJ by John Carreyrou suggests that they may not be getting to work very much at all. (If that link doesn’t work for you, try going through Google News, which often does the one-time-access trick). It’s a grenade, all right:

But Theranos has struggled behind the scenes to turn the excitement over its technology into reality. At the end of 2014, the lab instrument developed as the linchpin of its strategy handled just a small fraction of the tests then sold to consumers, according to four former employees.

One former senior employee says Theranos was routinely using the device, named Edison after the prolific inventor, for only 15 tests in December 2014. Some employees were leery about the machine’s accuracy, according to the former employees and emails reviewed by The Wall Street Journal.

In a complaint to regulators, one Theranos employee accused the company of failing to report test results that raised questions about the precision of the Edison system. Such a failure could be a violation of federal rules for laboratories, the former employee said.

It goes on, in great details, with many insights from former employees and regulatory filings, and the picture does not look very good at all. Particularly concerning are the proficiency tests, required by Medicare and Medicaid for testing services to show that they’re producing data that are close to their peers. There are detailed allegations that Theranos tried these with both their own system and with current commercial testing systems, with widely varying results, and that the company tried to gloss over these discrepancies. And there are other allegations that the company, since it collects such a small amount of blood, has to dilute these samples to what could be problematic levels to run the other standard blood assays on them, and these could (according to some in the article) be as much as 90% of all the tests in their portfolio.

This has been a key part of the Silicon-Vally-Meets-Medicine story, and a lot of people are going to be very upset if it falls apart. Theranos, or at least their press coverage, has been as much a cult of personality, though, as a biomedical or analytical chemistry story. But analytical chemistry – which is what all this comes down to – is not something that responds very well to well-crafted TED talks, high-profile magazine pieces, and photo shoots. The company has reached the point where they need to demonstrate, in great detail and at excruciating length, that their technology works the way that they say it does.

Update: Matthew Herper has more, including some accusations by Theranos about the story, and some rather defensive conspiracy-mongering by Holmes herself.

Update 2: The WSJ is now reporting that Theranos, at the request of the FDA, has stopped using its nano-scale testing in all but one of its offered assays. This took place fairly recently, after agency inspections, but the company has never disclosed any of the details (or that any such changes took place). Their web site was reworded sometime during the last two months, however. . .

50 comments on “Theranos And Its Blood Test Machine”

  1. Aaron says:

    FYI, Theranos has released an statement claiming vague factual errors, but seems to miss that the actual problem is nobody can independently verify their claims.

  2. Martin says:

    Let’s see how long it will take to have similar news about NantKwest..

  3. Manoj Jadhav says:

    These days scientist/people want everything fast, on finger tips, cost effective, reliable, reproducible, and reachable and so forth and so on. ..
    The technology is being used “in excess” to solve all the problems of mankind. And it is wisely said ” anything excess creates problems”. This is what happening everywhere… e.g. too much cleanliness… symbol of being more civilized…. on the contrary other segment is studying the benefits of the microbiota on health and several diseases…

  4. TimmyG says:

    Nothing about this company makes sense. $9B valuation but extremely slow commercial roll-out? A 2-log advance in analytical chemistry capabilities but NO validation in peer-reviewed journals? A CSO who commits suicide, in spite of “revolutionary” opportunity to improve human health? BoD with Henry Kissinger, Sam Nunn & George Schultz? (Great if you’re looking to sign an arms treaty, bad if you’re testing for glucose levels.)

    I wish I could short this.

  5. blogceutico says:

    Dear Derek,

    thank you for this sentence:

    ” But analytical chemistry – which is what all this comes down to – is not something that responds very well to well-crafted TED talks, high-profile magazine pieces, and photo shoots. ”

    This just made my day!

  6. johnnyboy says:

    In total agreement, the Theranos story seemed fishy from the get-go, and another case of Silicon Valley overhype crashing against the wall of reality. Unlike for Tech, in the medical field, having a young, PR-savvy and photogenic CEO will only get you so far before you’re found out.

    1. renata says:

      Photogenic?! Since when trouts with deep male voices are “photogenic?

  7. Nobody says:

    I interviewed with Theranos a couple years ago for an organic chemist position. They were looking for someone to help synthesize reagents for their assays, but nothing more specific was told to me. As TimmyG noted, the BOD isn’t your conventional group of MDs and PhDs, but rather military men. When I asked about this people either seemed to have no idea what I was talking about or brushed it off quickly.

    What Theranos has going for it is that it’s a very feel-good story. A female who drops out of a top university to found a company that has potential to revolutionize medical diagnostics? Buzzfeed and Upworthy couldn’t have dreamed up a better headline. Sounds like it was a lot of flash and no substance though.

  8. SomeGuy says:

    I’ve long suspected this was a load of hooey. Company always seemed to have a Madoff-ish feel to it to me.

  9. NJBiologist says:

    “Ms. Holmes says its tests have margins of “allowable error” targets less than 10%.”

    Talk about solving the wrong problem. If they’re profiling liver transaminases, the variation due to diet and exercise will be several fold. I think that’s the biggest non-pathological variation regularly encountered, but maybe a third of their analytes will vary a lot more than 10% with things attributable to the patient (did they actually fast, for how long, did they get drunk last night) or person collecting the sample (hemolysis is a major-league headache on most of these measurements).

    So they’re attempting to build an ultraprecise tool to measure ultra-variable biological parameters.

  10. someguy says:

    I have yet to have anyone demonstrate to my satisfaction that you can actually build a small-volume blood test device for rare analytes that has anything close to a respectable ROC. Particularly, as in this case, if you just flat out refuse to disclose what your secret sauce is.

    Apparently most people involved in this field have never heard the words “Nyquist rate” or “Bayes Theorem”.

  11. Erebus says:

    That Theranos statement is a case-study in how not to reply to accusations from the press.

    -It asserted that the WSJ article was “factually and scientifically erroneous” — yet did not offer any facts or scientific evidence to disprove the WSJ’s claims. Only jargon . (Why not make portions of that 1000-page document available to the public?)

    -It engages in a sort of ad hominem argument where it asserts that “disgruntled ex-employees” are spreading “false allegations” — and that said ex-employees were “never in a position to understand Theranos’ technology and know nothing about the processes currently employed by the company.” Sure thing. If you believe that, I’ve got some oceanfront property in Nebraska I’d like to sell you.

    -It alleges a conspiracy, repeatedly blaming “industry incumbents” for trying to damage Theranos’s business. The closing sentence is a particularly good example of this: “Stories like this come along when you threaten to change things, seeded by entrenched interests that will do anything to prevent change, but in the end nothing will deter us from making our tests the best and of the highest integrity for the people we serve, and continuing to fight for transformative change in health care.” (Italics added.)

    I was willing to give Theranos the benefit of the doubt… until I saw that disgraceful rebuttal.

  12. Mark Thorson says:

    Ruth Drown developed a machine that could do all sorts of analysis from a drop of blood many years ago. 🙂

    http://www.chirobase.org/12Hx/drown.html

    1. Gene says:

      Wow! Wall O’ Text.

  13. pete says:

    Not sure if this comes into play in this saga but I know there’s been concern about using small volume blood samples from “fingerstick” due to claimed potential for altered blood values. That is, it’s been argued by some in the clinical testing community (admittedly, they have a biased POV) when you go the conventional route of taking a relatively large volume sample blood by i.v. needle, any “tissue plug” resulting from inserting a needle through skin and into vein will have only a trivial effect on blood chemistry. But when you only take a small volume of blood via fingerstick, the claim is that damaged finger tissue can have a comparatively large effect on blood samples and, for certain analytes, be a particular problem.

  14. Hap says:

    Boy, it’s always a sign of legitimacy when your critics are “disgruntled”, and a good credit to the hiring process that you hire so many people that supposedly suck (because they didn’t know anything despite working for you). Perhaps they could get together with Moderna?

    I guess it would be unfair to say that “data talks and (something else) walks”?

  15. Hap says:

    @9: Why is the hypervariability of the biological data relevant? I assume that 10% would be the desired maximum difference between measurements from small blood samples using their machine and measurements from larger samples using standard methods. If the measurements are taken at the same time, the variability of the underlying quality with time, etc., shouldn’t matter. Am I missing something?

    I assume you could compare measurements using small blood volumes and big ones to see if the small samples give significantly larger variance. If you have a problem with skin or something else messing up assays, shouldn’t you see it overall in the data (well, when it exists)?

  16. DZ says:

    Abaxis has been doing blood drop testing successfully for well over a decade. Most of their business is veterinary, but they also can do people. A public company market cap $1 billion with $50 million in sales.

  17. Andy II says:

    Is it only me that this story would remind that infamous VW story?

    CEO said that in spite of no publication FDA had validated the effectiveness of their test.
    “The Food and Drug Administration has just posted a detailed explanation of its decision to clear Theranos’ one approved test, and, to my eyes, it does give some validation of their technology.” (http://www.forbes.com/sites/matthewherper/2015/07/15/giving-theranos-a-few-tests/)

    1. Norbert Schtumpf says:

      The single Theranos test that has been cleared is for Herpes Simplex 1 and gives a yes/no answer, not a variable quantity.

      1. renata says:

        Not to mention, could the FDA be trusted as a 100% incorruptible?

  18. Hap says:

    If you’ve got real data (or at least the FDA thinks so), then why not point to that, rather than talking about disgruntled employees and competitors? Theranos ought to have decent PR, and that seems like a dumb move if it has something.

    FDA approvals are a much better measure of legitimacy than publications when you have them, but you have to have them first before you start running smack. If you don’t, then people have nothing to judge your word by. People are allergic to hype here, with good reason.

  19. johnnyboy says:

    @Pete: it is entirely justified to have doubts about the validity of blood results obtained from finger pricks. The blood you get from a prick will be a mixture of blood from small venules, small arterioles, capillaries and lymph vessels, mixed with whatever loose tissue and cells have been cut by the prick and carried over into the blood. No way you could ever get a reliable CBC from such a sample. Since the kind of general blood tests that this company is targeting are usually CBC + chem panel, even if you could get your chem results from the prick, you still need a normal blood draw for the CBC, so the competitive advantage that they are trying to push (quick sampling by patient at pharmacy, without need for phlebotomist) is lost. On top of that, the variability inherent in chem results from prick blood will be significant for several analytes, like CK and AST (present in muscle so could be raised from prick injury), glucose and lipids (different levels in venous or arterial blood or lymph), etc… They are focusing on their fancy technology without much consideration of the underlying biology, typical error of engineering types.

  20. bad wolf says:

    I remember that New Yorker article but thanks for the refresher. Just enjoyed one gosh-wow paragraph where (white) girl barges into Stanford, takes Mandarin classes while still in high school, everyone is amazed.

    Funny how no one blinks an eye at boy/girl in Shanghai learns English while still in high school. Must be a genius!

  21. Pierre Menard says:

    But she started it when she was a SOPHOMORE at STANFORD. How could it not be gold, baby?

  22. Eric says:

    A $9 billion dollar valuation? I always wonder how they arrive at that number for a privately held company.

    I’m sure it’s a much smaller number today 🙂

  23. Chrispy says:

    This seems to be the new way to get rich in biotech: go public with a slide deck, no real science necessary. It seems that the venture capitalists seem to know so little about the science that they bet on people and hot ideas. Why will these companies not just all become like Cell Therapeutics, with wealthy founders trying to preserve capital and a dwindling number of workers in a pathetic scramble to make something work before the lights get shut off?

  24. SP says:

    I for one am shocked, shocked that a company with Henry Kissinger on the board might be less than forthcoming with facts.

  25. Sam P says:

    Eric: In principle, that’s actually pretty simple. You use the most recent investment. Say 10% of the stock was exchanged for $1 million => valuation of $10 million. Of course, if the last round was a long time ago, the valuation might be very inaccurate. Having complicated stock structure (common, founders, preferred, convertibles, multiple series, etc) also makes valuation more difficult.

  26. Seb says:

    I always had mixed feelings about Theranos and Ms Holmes. Contrary to the typical Silicon valley “let’s write an app and cure cancer” banter I thought (and somehow still think) she knows what she’s talking about, and she had a professor in chem eng with her from the beginning, which I think is a good sign.

    Now, is it over-hyped? Yes. Is what they claim actually possible? Yes as well. I’m convinced that with very smart engineering and microfluidics kind of stuff you can do wonders (cf David Walt at Tufts for example), but then, we need data! Peer-reviewed studies, FDA accreditations, you name it, but we need more than the responses seen so far which are on the school level of “it’s not me it’s you”, or the alternative (and convenient) excuse that trade secrets require hiding everything!

    For now I am willing to wait before making a final opinion. But I want to believe Elizabeth Holmes has a good goal in mind (in very strong contrast to a couple people that have been mentioned on this blog recently) and still has a chance to achieve it.

  27. Soon-to-be-former BMS person MOLS says:

    One obvious possibility would be for somebody to send BLIND proficiency test samples to Theranos, handled in such a manner that nobody at Theranos could know they weren’t ordinary patient samples.

    In fact, as the recent news about VW demonstrates, it might be a good thing generally for regulators to do more secret testing.

    Ironically enough, some product defects in other fields have been detected by people who were not even trying to detect them, only doing research when they hit something odd.

    The people who caught VW cheating were just evaluating their portable testing machine; they bought several diesel cars in the US for their first tests because they expected cars sold here to be the cleanest in the world. Purely by chance, the first three cars they tested were two VW diesels and one from BMW. The BMW’s emissions were in compliance most of the time, except that when going up steep hills (which was one of the conditions when they had suspected cars in the real world might do worse than in the lab). When their two VWs emitted way more NOX under a wide range of driving conditions, at first they thought there must be something wrong with their device.

    Similarly, the mathematician who discovered first-generation Pentium chips did long division incorrectly was not trying to debug Intel chips. He was doing calculations involving prime numbers for his research, and when he added a new computer with a Pentium chip to his compute farm its results were inconsistent with the output from older computers. At first he thought there must be a bug in his own code, or the compiler, or the OS. Only after eliminating all other possibilities did he call Intel and say “I think I’ve found a bug.”

    Incidentally, to this very day every time Linux starts up it checks whether the floating point unit does long division correctly, which it started doing in 1994. No Pentium with a clock faster than about 150 MHZ has failed that test.

  28. OrganicChemist says:

    There was a Hawaii Five-O episode (the ORIGINAL series, with Jack Lord) in Season 1 called “Once Upon A Time” with Joanne Linville as
    a quack-doctor that tested ONE drop of blood in a mammoth “computer” to diagnose any disease…sounds a lot like this.

    Joanne Linville was also the love-interest Romulan Commander that Spock developed feelings for in “The Enterprise Incident” – one of my
    favorite Star Trek (TOS) episodes….

  29. Canman says:

    I know you have a “things I won’t work with” section… Sounds like you need to start a section called “companies I won’t work for (with)”.

  30. assaydeveloper says:

    If you look at the fundamentally different kinds of analytes that Theranos is offering, it is kind of obvious that not everything will be happening in one machine:
    1. Small molecules
    2. Elecrolytes, urea and co
    3. Relative and absolute cell counts
    4. Proteins and peptides (collectively spanning 7 orders of magnitude in conc.)
    5. Active enzymes
    6. DNA and RNA
    7. Steroid hormones
    8. Lipids
    9. Immunglobulins
    Taking just the different types of immunoassays, it is still quite the challenge to squeeze them into one platform. Even if you start with well characterized and validated reagents and don’t do any homebrewing.
    I don’t think that Theranos and their founder have been around for long enough to manage all that.
    OTOH: 15 assays, maybe 30 assays, probably of similar types, maybe some of them still slightly wonky; That feels about right.

  31. Mark Thorson says:

    No, they’re talking about doing it using lab-on-a-chip technology. There’s different ways of doing this — one way is to break up the sample into tiny droplets and move them around electrostatically. You could have several different reactions and many reagents, all on the same chip using very small volume.

  32. dbcooper says:

    George Church’s comment:

    http://www.washingtonpost.com/news/wonkblog/wp/2015/10/15/the-wildly-hyped-9-billion-blood-test-company-that-no-one-really-understands/

    George Church, a Harvard Medical School scientist who has invented many new genome technologies and been involved in the inception and leadership of dozens of companies noted that the board was an unusual cast of characters, without much real medical expertise.

    “Usually such deficits in the board of directors would be offset by an equally stellar scientific advisory board and/or medical advisory board,” Church said in an e-mail. “I don’t see either for Theranos.”

  33. Renee says:

    Glassdoor.com has a plethora of reviews from Theranos employees, most posted in the last year. They fall into two categories – 5 star and 1 star, very little in between. The 5 star reviews read like blather from recruiting brochures and company Linkedin sites; the 1 stars are somewhat disturbing to read, and some state that the company tells employees to post fake reviews. Very interesting reading.

    One of the more disturbing tidbits, from a lab associate in Arizona: “They have no knowledge of infection control.”

  34. Mark Thorson says:

    Note how many “reviews” on the first two pages were posted on April 1 or April 2 of this year. Some bad news must have come out around then.

  35. David Cockburn says:

    I can’t see how ‘micro-drop technology’ is such a big deal except in the case pediatrics. I certainly don’t like needles being stuck into me but once it is in I don’t really care if they take 1 tube or 5 tubes of blood.

    1. dbcooper says:

      Yeah, or “Nanotainers” vs. existing small tubes. e.g. BD’s “Microtainers”.

  36. skeptique says:

    many questionable technology and therapeutic platforms out there, so no surprise that some will falter because they’re not based on well-founded science.

    @dbcooper sadly, a few (such a Pronutria which claims to be developing therapies using combinations of amino acids/peptides!) include good scientists such as George Church on their advisory boards, which in itself allows them to persist and be funded longer than they should.

    1. dbcooper says:

      Yeah, that’s a bit sad. He also has some involvement with the rather dodgy sounding BioViva:

      http://www.technologyreview.com/news/542371/a-tale-of-do-it-yourself-gene-therapy/

  37. Renee says:

    @Mark Thorsen,
    I also noticed that most of the 5 star reviews were posted in April and May of this year. Perhaps that’s when management at Theranos became aware of the plethora of one star reviews?

    On a related note – the NY Times link above leads to another link, that shows Elizabeth Holmes will be featured in the Oct. 25th issue of the NY Times Style Magazine.

    Oddly, I’d never heard of her until the article in the New Yorker magazine came out about a month ago.

  38. bellwether says:

    Here’s a very interesting reported comparison of Theranos test results:
    http://www.mondaynote.com/2015/10/18/theranos-trouble-a-first-person-account/

  39. Oliver H says:

    According to their website, they offer five part WBC differential counts. I’d have serious statistical concerns if they really do that from a microdrop…

  40. Nickels says:

    Elizabeth Holmes is not the company. It’s become all the rage in silicon valley for VC to have a ‘young genius’ front your product because it makes for a good story.

    It’s marketable. It might work a bit for software apps, but biochemistry?.

    Anyone who’s tried to start a business or is engaged in research knows that youth and inexperience are not the way to do it. Again, it makes for a good story (young super-genius saves the world), But how many 19 year olds do you run to with difficult problems?

    Due to zero % interest rates there’s trillions of dollars out there looking to create ‘value’. Some of us are well connected enough to get a piece of it.

  41. Anonymous Researcher snaw says:

    Forbes asks “Could Theranos Go From Unicorn To Unicorpse?”

    http://www.forbes.com/sites/chrismyers/2016/01/28/could-theranos-go-from-unicorn-to-unicorpse/

  42. Greedy Narcissist Child says:

    Yeah right Elizabeth your a genious at 19. We all love and adorne you. Please get off your greedy little self absorb narcissistic high horse and go through finish unversity and earn a PhD. Learn something. Your not the saviour of the Universe. Makes me sick that money is pissed away on such filfthy greedy narcissistic meglomaniacs….

Comments are closed.