Skip to Content

Expertology

I’ve been thinking about “right to try” laws, the ones that are trying to open up access to not-yet-approved drugs for patients who wish to take their chances with them. There are a lot of practical considerations that bear on this idea (differences with existing “compassionate use” programs, effect on clinical trial enrollment, the ability of companies of various sizes to satisfy demand, etc.) But recent events, and the fact that this is an election year, have me thinking about a larger issue that this one is just a part of. Bear with me – it’s a holiday weekend coming up, so this will be a long post. The issues I’m discussing have been thought about since the ancient Greeks at the very least, but I’m not up for a formal survey of epistemology and I’ll bet you aren’t either. So here goes.

The Appeal To Experts

The big underlying question here is “Who gets to make the decisions?” Right-to-try advocates would say that the patients themselves should be making those calls (presumably with some advice from their physicians). Others would say no, we need an FDA, some sort of regulatory authority to make sure that the therapies people are choosing from are actually worth choosing. Another argument is that the underlying medical and clinical issues are complex enough to make some sort of expert review worthwhile, and that we can’t necessarily expect “informed consent” to always be informed enough under some of the more wide-open proposals. Choosing among unapproved drug therapies, in this view, may not be within the range that we can expect people to operate in usefully.

And that takes us to the general phenomenon of distrust of experts, which has cropped up many times in recent years. Inevitably, a big triple slug of politics gets poured into this particular blender, because people on any side of a given issue are generally happy to adduce the support of experts (and to question the credentials of the experts brought in by the other camps). We’ve seen an awful lot of people using scientific evidence in particular (or the other side’s lack of understanding of same) as a club to beat over the heads of their political or policy opponents. It happens in both directions: there are people on the right who think that all climate-change data is a devious hoax, and there are people on the left who think that all evidence in favor of GMO crops is an evil Monsanto conspiracy.

Calling on expert evidence is a tricky business, because even if everyone involved is still appealing to expertise, not all experts are created equal. Many of the “experts” you can bring in are not above trading on their affiliations or degrees, even when those don’t have very much to do with the issue under discussion (“. . .in much the same way that the ocean is not above the clouds”, as Douglas Adams put it). The proliferation of junk pay-to-publish journals means that (to the untrained eye) both sides of a medical issue, even one that’s ridiculously lopsided, can appear to cite perfectly scholarly papers in support of their position. One side might be real journals and real research and the other side might be J. Steaming Manure and Wire Transfer Lett., but if you don’t know anything much about the subject, you can’t tell. It looks just as convincing as the real stuff to you.

The Limits of Expertise

Past that, though, you run into the folks who aren’t even bothering to appeal to experts at all, because they simply don’t trust them much and don’t believe what they have to say. Here, too, we have gradations. If we’re talking about the Poincaré Conjecture, to pick an issue on one end of the scale, the only people that will be of much use while discussing the details of Grigori Perelman’s proof of it will be those who have devoted serious time to the study of topology. No one who is not comfortable dropping the phrase “Riemannian manifold” into their conversational flow can really have a seat at that table. Even world-renowned prize-winning scientists from many other fields are not going to able to pull up a chair. Now, since many of these people do have a fair amount under the hood, if they wanted to take a year’s sabbatical and devote themselves full-time to advanced mathematics, some of them could indeed come back and at least contribute intelligent remarks, but very, very few (if any) could do it cold.

For a topic like that one, or any similarly advanced topic in any field, there is really no substitute for expertise. The origins of Basque declensions? Pulse sequences for four-dimensional NMR experiments? Anomalous element distributions in globular cluster star populations? You’d better know what you’re talking about, and it’s going to take some time to get to the point where you do. No one’s born knowing this stuff. We’re not going to take a poll and get the answer; the wisdom of crowds will be of no help at all unless it’s a pretty damned weird crowd. This category I will call, naturally enough, “Matter for Experts”.

Now let’s go to the other end of the scale. There are actually several different other ends; this is not a one-dimensional problem that fits on one line. One class of nonexpert evidence is in matters of pure fact, about which there can be little argument. Expertise is not a big issue when we’re discussing if the sun is shining or not. Many people will try to drag other arguments into this realm, at least rhetorically, but “What is to be done about climate change?” or “What is the best policy regarding the Syrian War?” are not the same sorts of questions as “What day of the week is it”, no matter what you think about those topics or how much you might like for your preferred answers to be in this category, which I’ll refer to as “Just Plain Facts”.

Then you have situations where there are no facts to be established – that is, when something is unknowable. “When exactly will this carbon-14 atom I have here decay?” is an example of that sort of question, although you need a bit of expertise to understand why. But there are plenty of examples, both complex and trivial. Taking a poll of what’s in this here sealed box here will probably not be of much use, because I could have all sorts of junk in there, and I’m not aware of any random-sealed-box expertise to be had. But you have to make sure of the unknowability: at one point, everyone would have told you that the question “What is the Nth digit of pi?” (without just calculating out that far) was a perfect example in this same category, but in 1995, to everyone’s surprise, that changed. I’m going to call this category “Flat-Out Unknowable”.

A third “no-fact” place where expertise starts to break down is in matters of pure opinion. “Who is the greatest baseball starting pitcher of all time?” is an example of this sort of topic, partly because it immediately devolves to an argument about the meaning of the word “greatest”. But expertise is not totally valueless here – you can adjust your respect for the answers that might come out. Experts are people who presumably know a lot of facts about a subject, but there’s such a thing as expert opinion, too. Someone who proposes “Ethelred the Unready” or “Sofia Vergara” to answer the above question is not worth taking as seriously compared the people who immediately start arguing about Nolan Ryan versus Grover Cleveland Alexander versus Sandy Koufax versus Cy Young versus Bob Gibson versus Warren Spahn and so on. These folks are never going to come to an agreement, because the question is intrinsically unsettleable, but they clearly at least have some idea of what they’re talking about. A vast poll for a question like this might be of interest, assuming that the people who are motivated to vote in it are baseball fans and not a bunch of internet zanies who have been waiting to write in good ol’ Ethelred, Sofia, or Pitchy McPitchface. I’ll call this category “Matters of Opinion”.

Update: as a reader pointed out, a good example of this category is English usage. In fact, the way that people speak and write any language is a sort of continuous public opinion poll, and there are, in the end, very few “facts” to be established. A good article on this is David Foster Wallace’s “Authority and American Usage”, which appears in his collection Consider the Lobster and can also be found here, and in its original magazine version here.

The Battle of the Categories

These categories, then, matter a great deal when it comes to expert versus lay opinion. This means that many people will like to think that their particular topic of interest is surely not in the “Matters of Opinion” or “Flat-Out Unknowable” areas. They might talk as if it’s in the “Just Plain Facts” one, although most things aren’t, but it’s almost for certain that they’ll also act as if it’s in the “Matter for Experts” one. I would like to propose (this is not an original suggestion) that a lot of time, effort, and oxygen is actually spent arguing about these categories and their boundaries rather than arguing the questions per se. That includes trying to move a question from one category to another, or denying that a particular category has any real validity. For example, a religious fundamentalist (pick your religion here) will tend to put anything that has any connection to any part of their preferred sacred text in the “Just Plain Facts” category, like those bumper stickers I used to see once in a while (“God said it, I believe it, and that settles it”), while nonbelievers will not have any of this at all.

How Much Is An Expert Worth?

It’s the “Matter for Experts” category that’s getting a real workout these days. Even if people take it as a valid category and are willing to fight within its borders, we then get mighty disputes about the experts themselves. The anti-vaccination crowd and the glyphosate-is-poisoning-everyone crowd are often both examples of this. The counterarguments are often “We have our own experts, and yours are wrong and stupid and corrupt!” There’s another common counterargument, though, that denies expertise from the start, as in “Your ‘medical experts’ are worthless. I don’t care what all these doctors have to say, I’m a mother and I know that this shot gave my child autism”. To do a full swan dive into politics, Donald Trump’s entire campaign partakes deeply of the whole “down with the so-called experts” view. (Someone really needs to rush out a new campaign edition of Ortega y Gasset’s Revolt of the Masses, but I digress).

You don’t have to set the value of expertise to zero – you can argue about its value versus other considerations as well. The arguments for “right to try” laws are at least partly based on the premise that expertise, in the form of clinicians and the FDA, is not worth as much as we’ve been saying it is, and that a patient’s right to be able to use experimental drugs is actually of more immediate import. Similarly, the recent vote by the UK to leave the European Union partly hinged on whether a given voter believed the economic experts, who largely warned that this would be a costly mistake, and/or whether they believed that other concerns (which the experts had, in this view, failed to consider or value properly) still outweighed those in the end. Not everyone was devoting this much thought to the question (on either side of the vote), but many did.

So how much are experts really worth? At this point, I should note that there are, in fact, experts who aren’t so sure of the answer to that general question. I would take opinions like those seriously, given that the default setting, when an expert is asked such a question, is a reflexive attempt to show that, gosh, they’re actually worth even more than you think and should be valued accordingly. Expertise, for all its value, has its downside, and as that link illustrates, a big downside can the inhibition of new ideas in a given field of study. But if it’s going to be worth something, it’s going to be worth the most in just the sorts of highly technical areas that I used as examples earlier in the “Matter for Experts” category. This will be seen, perhaps, as special pleading, but I tend to think that choices of medical therapies fall into this area. Not everyone agrees, or agrees on the barriers to entry.

Run Along Now

But you have to be careful when starting this category fight, because you risk getting the opposite result from what you intended. In the same way that a religious fundamentalist is not going to persuade people that his stuff is all “Just Plain Facts”, you can have a hard time making the case that your stuff is in the “Matter for Experts” pile. Persuading people that you’re an expert, or that there are actually experts, or that their opinions might be worthwhile, is a delicate business. (In fact, as mentioned here, it’s been found that trying to push straight ahead with these claims often hardens the opposition to them). That, I think, was a big factor in the recent EU vote. I don’t often find myself agreeing with Matt Taibbi, who’s well to the left of me politically, but I think he’s right on target in this article:

Were I British, I’d probably have voted to Remain. But it’s not hard to understand being pissed off at being subject to unaccountable bureaucrats in Brussels. Nor is it hard to imagine the post-Brexit backlash confirming every suspicion you might have about the people who run the EU.

Imagine having pundits and professors suggest you should have your voting rights curtailed because you voted Leave. Now imagine these same people are calling voters like you “children,” and castigating you for being insufficiently appreciative of, say, the joys of submitting to a European Supreme Court that claims primacy over the Magna Carta and the Bill of Rights.

The overall message in every case is the same: Let us handle things.

We have plenty of that sort of thing here, as well, from both ends of the political spectrum. Run along, little people, your betters have work to do. This tension is built into any democracy – even a republic, where the idea is that the people vote for representatives to work on these decisions on their behalf. Such a buildup, I and many others think, is one of the big reasons that we have Donald Trump’s candidacy at all.

But let’s talk about that. As much as I would like to think, as much as it would make things simpler for me, that people who are voting for Trump are just morons doing moron things off in moron-land, that’s not going to cut it. I do think that they’re disproportionately low-information voters, and I certainly do think that voting for Trump is a serious mistake, but there are also plenty of reasons to be pissed off at some of the things that the opinions of the great and the good (of both major parties) have wrought. His voters see these reasons as perfectly legitimate ones, and in a few cases, I actually agree with them, although God knows I don’t think that Trump is the answer. It’s also true that many of these reasons are, in my view, dumb ones, and some of them are based on completely wrong information, but not every single one of them is either of those, and I can’t just make myself feel better by assuming that they all are.

I’m always struggling between my libertarian impulses to let people make their own decisions and my feelings when those decisions include things like voting for Trump, but I can’t pretend that I can have one without having the other. That doesn’t mean I’m voting for the guy (not that I can stand Hillary very far, either, damn it all), but if I want to keep anyone from voting that way, I’ll have to think of a way to do that without coming across like I’m lecturing to the Great Unwashed about what they’d be doing if only they were smarter. Because that, like the EU vote, is one of the big factors that got us here. It’s the same impulse that makes a person just go off on a rant about people who buy electro-clustered super water, or use magnets to deactivate the Monsanto gluten genes in their food (I made that up, but it’s probably out there), or think that vaccines are a poisonous UN plot or something. Believe me, I feel like ranting about these folks, too, but it doesn’t really do much good. The choir’s been preached to many times already.

Facts and Politics

Political issues are especially prone to the category fights I’ve mentioned. Some of these really are “Matter for Experts”, but not all of them are, and if you try to pretend that all of politics is that way, you’re making the same mistake as people (such as the religious types mentioned) who try to drag more stuff into the “Just Plain Facts” category. I would, as mentioned, also like to think that medical questions are far more “Matter for Experts” driven than most political questions, but I have to take into account that I myself would be seen by most people as one of those experts, so my point of view has a real chance of being skewed. (“Skewed? Knowing something about a subject makes your opinion about it skewed?”)

In the end, then, I think that there are several stages in dealing with such problems. First off, you have to make sure that the issue under discussion is in a category where expertise is valuable (or available at all). Then you have to make sure that the experts you’ve found really are such, and not charlatans or people who are hoping to charge you a consulting fee, and you have to see if there are equally plausible people who would disagree with them. Then you have to weigh those expert opinions versus all the other factors you can think of, deciding what each is worth and how much it should be part of the final decision. This is not a brief process, nor is it a lot of fun. Just arguing with people or making fun of your opponents, on the other hand, is fun (at least for many people when they’re in the mood for it). So it’s no surprise that things tend to slide off in that direction a lot of the time.

What to do, then, about “Matter for Experts” decisions where the people affected by them aren’t experts themselves? That’s where the “right to try” laws fit, I believe, and many other issues as well in a large, developed country like ours. The only recommendations I have are to (1) hold the line on it really being an area that expertise is valuable in, but (2) try to lower the barriers as much as possible to letting people get enough knowledge to at least see the outlines of the problem, while (3) not making them feel as if they’re being railroaded or mocked. This is work. To use my earlier example, not everything is as hard as the Poincaré Conjecture (although math has a lot harder stuff than that waiting for you), but I think even in that case that a hand-waving-level explanation can be gotten across to an intelligent person inside of a half hour (although not of Perelman’s proof, I would have to think). The reaction of said listener is fairly likely to be “Who cares?”, but they’ll at least have some idea of the question.

There are whole books written to lead nonexperts through things like Gödel’s Proof and the Riemann Hypothesis, and political and medical issues are not, in the end, as bad as those. Not everyone is going to want to hear about the details, but if a good-faith effort is made to supply them (and they’re then still rejected), that’s a point in favor of one side of the issue already. Perhaps the deal to be made is “I’ll listen to how you feel if you’ll listen to how I think”. I realize, though, that good faith seems to be in short supply this year. . .

81 comments on “Expertology”

  1. Anon says:

    Without the time to read beyond the first paragraph, my response to this is that patients should make the ultimate decision based on the risks, which must reflect the real uncertainty. Thus, the role of the FDA should be to ensure that the risks are clearly and accurately spelled out given what we know and don’t know, and they could even give their own independent recommendation based on this info, BUT they have no right to make the ultimate decision for the patient. Ultimately, it’s the patient that must live (or die) with the consequences of this decision, and not the folks in tge FDA, no matter how great their expertise and intent to do good for patients. To me it’s really that simple.

    1. Phil says:

      The FDA’s job (and therefore pharma’s job) is to develop a treatment that is sufficiently safe and efficacious across the patient population. However, the individual patient is not concerned with the effects across a population, she is concerned with the effect on herself. Risk tolerance is something that is highly varied from individual to individual, and on top of that, we are talking about life and death as each side of the bet in some of these cases. If on one side the prognosis is “you have 6-12 months before your tumor kills you” and on the other you have “taking this experimental drug has a 10% chance of stopping the tumor and giving you another 10 years of life, but it will probably make your life hell for the next 6-12 months, and 90% chance that you die anyway.”

      To Anon 1’s point, if the patient is informed as accurately as possible about the possible outcomes and the relative probabilities, I don’t see why we can’t let each one make his own decision.

      1. Phil says:

        I left a hanging “if, then” statement. Oops. My point is the value the patient places on her own life vs comfort can determine which option is preferable, and if that doesn’t fall under the Matters of Opinion, I don’t know what does.

        1. Anon says:

          Fully agree, it’s a personal choice, albeit a fully informed one.

    2. SirWired says:

      The motive behind the idea that you cannot sell a drug until you’ve proven it works and is safe is because if you permit the sale of treatments before that process is complete, you’ve removed the incentive to ever actually complete the approval process.

      Let’s say you are a drug company and your initial results suggest your treatment doesn’t work. If I want to recovery at least SOME of my cost, I can simply let my clinical trials drag on, while I sell my treatment outright to desperate patients.

      Or, even worse, lets say I’m a complete charlatan selling Snake Oil. I can start “clinical trials” for treatments that I know will never work, and in the meantime raise cash from vulnerable populations, and drag things out for years.

      I’m not sure how to let patients get a chance to use useful drugs that simply haven’t finished the process yet, while barring those that would prey on the innocent.

    3. Jim says:

      The major concern I have with the “right to try” approach is that you’re going to end up with a lot of case studies if your lucky and a bunch of anecdotes if you’re not. These will end up being incredibly powerful in the general population and could lead to some decisions made regarding safety and/or efficacy that could be quite ill-informed.

      1. tangent says:

        Yes. We’ll end up with a lot of low-grade evidence, a shortage of high-grade evidence, and patients will make their decision based on the evidence that’s to be had.

        I don’t think it’s condescending to say that *none* of us is good at correctly weighting uncontrolled anecdotal evidence. Anecdotally, we often overweight it.

        If “right to try” tends to leave drugs longer without a final high-grade judgment, this is the tradeoff: X number of patients have the option to take the drug using inherently flawed data instead of not at all, but Y number of patients must make medical decisions on the flawed data instead of having Phase III results back earlier. For a lot of X and Y values that’s a bad tradeoff.

        If we could set up “right to try” so the company truly accrues no benefit from it (even in visibility? I don’t know how), so it creates no incentive to linger along the approval, I could deal with that. In the way of “well, it’s not as bad as selling homeopathic placebos.”

  2. Anon says:

    I feel like a related (and important) point is the selective appreciation/call on experts. People arguing against the FDA/experts being able to control access to unapproved drugs are still relying on similarly trained experts who have developed the supposed drug in the first place. Politically, the same people voting against the experts/establishment are simultaneously complaining to those very people when the (economy, climate, immigration, etc.) situation isn’t to their liking. You can’t have your cake and eat it too-if you want someone (in this case, the experts) to help fix your problem, you have to give them the power to do so.

    1. Anon says:

      Fine to give experts the influence, but NOT the ultimate decision!

  3. Argon says:

    The wonders of Google University.
    See also ‘Biohacking’.

  4. former GSK says:

    Derek,

    As a preliminary matter, what does the phrase “low-information voter[]” mean to you? I think this term is often used to describe (allegedly) “stupid” out-groups. Thus, people on the traditional right will use it to describe certain minorities, and people on the left will use it to describe those on the right who live in southern and/or rural parts of the country.

    Is the implication of this term that if these groups were presented the “right” information, and they used “reason” to process this information, then they would inevitably reach the same conclusion? Or, are they inherently incapable of understanding the modern world and thereby are not really qualified to vote in matters of national significance, as many who were pro-Remain quite plainly argued? There may be other alternatives, but I think this is what people mean.

    Contrary to what you may believe, there are plenty of highly educated people, even those with PhD’s in organic chemistry, who support Trump (I am one of them). I think many people who oppose Trump have a certain perception of him (and things in general) that causes them, rightly or wrongly, to react with revulsion. Others, such as myself, do not have such a strong reaction. However, it is generally difficult to have calm conversations about this issue without it quickly devolving into ad hominem attacks. I think this blog may be one of the few places where such a discussion is possible.

    I don’t try convince people of anything, and I respect those who have a different opinion, by which I mean I am not going to denigrate them for supporting someone else (my significant other if a Clinton supporter, for example, and we have no issues whatsoever). This begs the question of whether people *should* be trying to convince others of anything? It’s analogous to the iPhone/Android wars on blogs, where people have intensely passionate and personal “discussions” regarding why the other side are idiots for buying an iPhone or an Android phone. I don’t view it as my job to convince anyone of anything. That is Trump’s job and Clinton’s job (assuming both end up being the candidates). I believe if you take such an approach, you will be happier.

    A lot of this has to do with perception of reality. You can perceive one issue a certain way, and I will have a completely different view. But that does not mean your view is “wrong” or mine is “right.” For example, as you are aware, much of peer reviewed literature is rife with lack of reproducible or outright fraud. Accordingly, I find the appeals to “studies” and purported claims of statistical significance with great skepticism (by either side; confirmation bias is a deadly trap). I’m in the Nassim Taleb camp with many things, although I recognize he too is not the Delphi Oracle.

    1. Derek Lowe says:

      Good questions. “Low-information”, in the Trump case, would be people, and I’ve run into them, who would freely admit that they don’t know much about economics, don’t know much about foreign policy, etc., but just like Trump. Some of them (and I’ve run into these, too) believe that Trump himself is a wonder-working businessman who can make a success of the US government just like he’s made a success of (basically) everything he’s ever touched. All these propositions are in trouble with the evidence, from what I can see. There are also Trump voters who seem to believe that Mexican immigration is a swelling tide that’s getting worse and worse, when it in fact peaked years ago. (You see this sort of illusion at work on the other end of the political spectrum when you look at the statistics on gun violence and gun ownership in general, both of which have been going down for some time, which I’d bet that a lot of people with more left-wing views don’t realize, either). Trump himself worries me in part because he doesn’t always seem to know that he doesn’t know something, and can act like what he doesn’t know isn’t worth knowing, anyway (he’s had a number of statements to this effect).

      1. Phil says:

        “Low-information” shouldn’t imply stupid. In most cases, people could find and understand the information if they wanted to, they just don’t want to.

        Given that you’re an informed Trump voter, could you lay out your reasoning for thinking he would be a good president? I’m earnestly interested.

        1. Anon says:

          Because you’re more likely to get a change from the status quo, and the status quo is shit? That’s not to say Trump will definitely make things better (they could also get worse), only that there is *some* chance they won’t carry on being shit.

          1. Phil says:

            So you’d rather have something different than the status quo, even if it were worse than the status quo? You’re for 10 feet of shit over 6? This is not a well-reasoned opinion.

        2. Gene says:

          One problem is the status quo is not an option. I don’t believe either candidate is up to the quality expected of an American president. However, I have seen poor performance from Clinton, and I think while Trump is a really crappy choice, I’ll be voting for him as a roll of the dice. Maybe it works out, maybe it doesn’t. There is no good choice.

          As my friends say, “we’ve got a choice between Stalin and Hitler here”

          1. Phil says:

            He’s a crappy choice, but you’re rolling the dice on him anyway? Please lay out a scenario where Trump being president is good for this country, and explain why you think its likely.

            The status quo is shit because of the irresponsible decisions made by Wall Street with other people’s money and the fanatical worship of “shareholder value” above all else in Corporate America (at the expense of consumers and employees). How does Trump help the situation? The guy’s a con artist and part of the problem.

          2. Bagger Vance says:

            You don’t sound super open minded, Phil, but fwiw i’m with former GSK here too.

            I wasn’t paying much attention to the early campaign (expecting the legacy vs legacy candidates to win) and so i didn’t expect much from Trump either. He wouldn’t have been my first choice but the voters in the early states had plenty of opportunity to listen to him and form their opinions. First off, he’s completely fearless, and doesn’t let opponents beat him with labels or force him to play on their terms.

            While i’d be happier if he gave a speech about NIH/NSF funding Tbh i’m not that focused on finer points of policy proposals because those have never amounted to anything. The bigger picture this time seems to be more about representation in a culture war and identity politics. Societal trust in institutions is at an all-time low, and given endless wars, endless economic doldrums, being priced out of housing and experts that more and more seem to be outright lying, a breath of fresh air is urgently needed. Trump is personally and symbolically a rejection of PC culture and a pushback against trans-national elites that have no loyalty to America, who seem to be undercutting it more than they’re improving it. People have noticed that international trade is not always working out in their interests, and would prefer someone with that sort of knowledge and a clear loyalty to them to level the playing field (or at least slow the acceleration).

            The American Presidency is constitutionally limited. It is smoke-and-mirrors, a lot of persuasion and social influence. Trump is apparently very very good at that. Trump is a businessman, and while that includes a level of salesmanship it is not necessarily a “con”, and is to my mind more honest than turning government experience into $$$ then returning to elected politics. When people complain about Wall Street, or war-minded neocons, or simply having their jobs sent overseas, over and over again you hear rejection of out-of-touch elites. That in this election the Democrat is the pro-war big money candidate is amazing.

          3. Phil says:

            BV: I do understand and share the frustration, but I find Trump to be the epitome of an out-of-touch elite. And on top of that, his blatant racism and baseless braggadocio make him physically repellent to me.

            But, like most of the commenters on this blog, I am a scientist. And I am willing to set aside my emotional reaction to Trump if there is a case to be made for him. “He’s not Hillary” is not a satisfying argument to me. I’m no Hillary fanboy, either, but she has a track record of reasonable policy (and no, I don’t agree with everything she’s done from a policy standpoint). Trump’s “policy” is largely nonsense (though you say this doesn’t matter, even though it’s the only indication of what a candidate would actually DO).

            So again, please lay out the case FOR Trump. I know you think Hillary would be bad. But why would Trump be good? I am listening (despite not “seeming” so, apparently).

          4. Gump on Trump says:

            “Mama says stupid is as stupid does.”

            For a position like the presidency, a strong moral compass is more important than expert knowledge – e.g. Forrest Gump. If Trump demonstrated a strong disposition to be a moral and good-hearted leader, regardless of his expertise, he would be worthy of consideration. But he’s a selfish, bigoted narcissist – so was Hitler.

          5. Bagger Vance says:

            Phil, you are probably asking the wrong guy but maybe i can make some observations. While I have not thoroughly researched positions and policies I am at least satisfied that Trump angers all the right people–both the regressive left but also the ‘establishment’ GOP (including the pundits) that have run it into the ground as well. I generally think of Trump’s policies as center-right a.k.a middle-of-the-road, not socially extreme, with some real populist appeal to things that people have been asking for for years. He definitely seems less theoretical/ideological and more practical, which sounds good. I think you might try watching one of his recent policy speeches (you can find them on youtube) for the serious side, unfiltered by the sound-bite media. I have also been noticing a lot of articles that criticize him while accepting the problems that he’s brought up. But as i say, leadership isn’t just getting your homework in on time, it’s a connection to an audience. Trump apparently just wings the majority of his speeches, which of course makes them sound less polished. But people really seem to like it.

            I try to not focus on bashing the opponent either but vide supra nowadays everyone who doesn’t agree with the left is Hitler. Literally! I am also pleased to see some real excitement and youthful energy on Trump’s side, which is a real change from the past.

          6. Phil says:

            BV: Thanks for your view. Some of the problems Trump brings up are relevant, others not so much. But, identifying the problem and solving the problem are two very different things. Here is the best I can come up with regarding his positions:

            https://www.politiplatform.com/trump

            Legalize pot and maybe other drugs too. This may be his most reasonable position.

            Take better care of veterans. No shit, who’s against that (other than the legislators who weasel their way out of funding it every. god. damn. time.)

            Make China play fair. Good luck with that one.

            Close the parts of the internet where the terrorists are. lol.

            Defund Planned Parenthood. Good for the red meat Republican voters, bad for their pregnant teen daughters who could’ve gotten condoms or an education about where babies come from since it’s not coming from their parents. It’s also not a “middle-right” position. I get opposition to abortion, but how much evidence do you need to believe that >95% of what PP does PREVENTS abortions from being needed in the first place.

            Build a wall! Deport all illegals! Do I have to explain to an educated person why these are not realistic solutions?

            15% flat tax on businesses, except the ones who outsource jobs because they’ll pay an extra 15%. Or something like that. And rich people won’t be able to dodge their taxes (but their putative tax rate will go down). In principle, I’m for simplifying the tax code and closing loopholes, we’ve just heard this so many times, why do you think Donald Trump would actually do it? Bernie sure. The Donald? I don’t buy it.

            Finally, the Left does not have a monopoly on Hitler comparisons:
            http://theweek.com/articles/568774/why-republicans-are-obsessed-comparing-obama-hitler

    2. another former GSK says:

      No wonder GSK R&D is such a hot mess. Imagine the Donald in charge of R&D.

      Project Leaders: “Emperor, we seem to have hit a wall. The MOA of our project appears to be invalidated and the PFI of our compounds is trending too high?”

      the Donald: “WTF is wrong with you people? You’re fired! I don’t give a &^%$ about the PFI or whatever! Fix the damn biology and get me a drug pronto!”

      err wait… maybe the Donald is actually overly qualified

      1. former GSK says:

        Are you implying that because I support Trump now that when I was at GSK people like me led to the downfall of it’s R&D (because I am stupid or something like that?). Irrespective of your answer, it doesn’t really matter to me. I ain’t mad. 🙂

        I actually voted for Obama in 2012. I also despised GWB when he was president. Does that make things clearer or better or worse? GSK’s problems have been well documented by Derek, and I don’t think they really have anything to do with the discussion here.

      2. Peter Kenny says:

        I see The Donald as more of an LELP than PFI sort of guy since that the Voodoo Thermodynamics of the flagship metric from the Hogwarts School of Thermodynamics matches the Voodoo Economics of his rhetoric. Nevertheless, I have used a blog post on PFI as the URL for this comment and I should warn you that using this seminal contribution from the Stevenage School of Metricology may cause you to lose your Barings in Singaore.

      3. Corey Lewandowski, Ph.D. says:

        “Fix the damn biology and get me a drug pronto!”

        That’s not realistic Donald Trump dialog! Here:

        Fix the damn biology and the fire damn idiots who came up with it. They’re awful, terrible people. I want a great drug, a drug that is incredible, one that is very, very good! The people who take this drug will be great people, just the absolute best, the same people who bought Trump steaks, drank Trump vodka and wine. Really great people, I love these people, they are the best!

        1. another former GSK says:

          Dr. Lewandowski – I stand corrected. Thanks for making this blog great again 🙂

    3. Chemluvr says:

      You lost me at NNT. Anyone who’s primary arguing point is “you’re an expert in field X, therefore you are biased and cannot debate legitimately” doesn’t really engender much respect. Doesn’t help that he’s completely insufferable on top of that.

    4. Dr. Manhattan says:

      ” I think many people who oppose Trump have a certain perception of him (and things in general) that causes them, rightly or wrongly, to react with revulsion.”

      I think the certain perceptions are based on evidence such as cited in these articles:
      http://www.nj.com/opinion/index.ssf/2016/07/trumps_ac_windfall_left_others_broke_editorial.html
      and: http://www.philly.com/philly/business/20160628_ap_10bbe40a86774bac9ad1fbd3a936c808.html

    5. also former GSK says:

      A couple of comments. “Low-information” is just that – low information, which doesn’t imply stupidy, just lack of information. You can believe anything you want, but if you can’t back it up with a reasonably cogent argument based on something resembling a fact, it’s just an opinion. Most of the arguments put forth by “low-information” folks consist of “this is what I think, because it’s what I think, and who believes all this so-called experts anyway?” Climate change for example – I’ve read lots of comments by people who truly believe there is a conspiracy by thousands of climate scientists to make up the data. This is not a argument.
      On the argument that so much science is irreproducible, so why believe any of it? Having waded in the waters of difficult biological data for a long time, I would say that the details are often difficult to reproduce, but the gist of it is generally reproducible. Chemotherapy works in many cancers much of the time. Does it work exactly the same way in every study in every population – no, and there are pretty good reasons. If you do the same experiment in mice five times, will you always get the same result? No, and anyone who has worked with mice can tell you that in their own hands they might not get the same result five times running. What is often flawed is our use of statistics in evaluating the merits of data, and the standards of review, not the underlying principles.
      So to address the point of the article – right to use “unproven” drugs. Often the drugs that patients want to use are in a generally relevant bucket (as studied by the experts), not just something pulled out the sky. For example, a cancer drug may have been studied in a population subset, but not your subset; or a related cancer, but not yours, etc. There should be a mechanism that allows patients to try, because the reasons for not having the data are often commercial ones, not scientific ones. The tricky part is to decide who pays for this, and my suggestion would be that insurance companies set up a fund that pays for it. Pharma are not the main bad guy here. Sometimes though there is really a novel mechanism that has never been proven in anything – and this is where I would make some distinctions about the right to try. There should be an expert panel that offers them an opinion based on the science, in lay language that lays out not just the risks but the reasons, and also some possible options that might be better. A lot of cancer patients have spent a lot of time studying their disease, and while they may not be scientific experts, they are often well-informed – it is a matter of life and death. Every single community oncologist cannot be up on all the latest ideas unfortunately, and access to scientific experts would be of value to patients.
      As to why people are so skeptical of experts, well, our brains are not wired for careful logical thought. Even with a PhD and having practiced the art of thinking about complex ideas for many years, it is hard not to fall into the gut responses that we are wired for – lots of experiments that demonstrate this. Experts don’t appeal to the instinctive thinking part of any of us, and even less to people who have not spent a lot of time being forced to make logical arguments about their ideas and beliefs. To return to politics, whatever you think you believe, if you sit down and really challenge yourself to support your beliefs with facts, you might find that the situation is muddier than you would like.

  5. SirWired says:

    I think the biggest drawback of “Right to Try” laws is that it removes the incentive for those possessing unpromising treatments to ever complete the NDA process. I mean, if you can have a bunch of patients pay out of pocket to buy something while you pretend to continue to run clinical trials on it, why ever finish the NDA process if you think it’s going to fail? How does one ward off the quacks pushing stuff they KNOW is going to fail NDA, if they ever had any intention of getting it that far?

    1. anon says:

      most “right to try” laws state that the drug must have completed an FDA safety trial, so I guess the regulatory burdens to even getting that far might push a lot of the quacks out of the business.

    2. anon99 says:

      most “right to try” laws state that the drug has to have passed an FDA safety trial (phase 1). The regulatory burdens to get something this far might ward off all but the most sophisticated quacks.

  6. Sok Puppette says:

    Erm.

    I tend to believe that most decisions about what drugs are worth taking are either Matters For Experts or at least Matters of Opinion with relatively high barriers to entry.

    But whether that means people should be told what they should take is still a Matter Of Opinion.

    It is a fact that many people will make dumb decisions in this area. It is likely that very permissive “right to try” would cause net negative health outcomes, taken over all people, because of the people who do make dumb (or even just wrong-but-not-really-dumb) decisions.

    It does NOT follow that you should be telling people what they can and can’t do. There are tons of possible objections to that step. Assuming it is cheating.

    1. Phil says:

      Please define “dumb decision” and “wrong-but-not-really-dumb decision.”

    2. Derek Lowe says:

      Yeah, that’s a big problem. It may well be that there’s a situation where people are likely not going to make the “best” choices, but you still shouldn’t be making for them. This comes up with the topic of (say) personal investments for retirement as well. That’s where the libertarian and technocrat sides of a guy’s personality can really start yelling at each other.

      1. Phil says:

        Well, with personal investments, at least there is an objective best outcome (maximize the money in your retirement fund when you plan to retire). Choosing between living and dying, comfortably or uncomfortably and rolling the dice on any of the permutable outcomes – there’s not going to be agreement on which is the better probability-weighted outcome because there is not a uniform measure of value.

  7. Squealer says:

    wrt the attack on experts and expertise, I think that the Dunning-Kruger Effect plays a major role. It is much easier to rally behind the Manichaean worldview preached by the instigators of populist movements than to struggle through the nuance that experts recognize in complex issues. Four legs good. Two legs bad.

    1. cthulhu says:

      But when the so-called experts (who may actually be experts) start in with “all animals are equal but some are more equal than others”, my gorge becomes buoyant and I tend to stop listening to them.

      1. Philosopher King says:

        Perhaps it is time for a Noble Lie. I say that we resurrect the myth of the metals, where Jenny McCarthy and The Donald will be firmly placed in “Bronze”.

  8. steve says:

    Let’s say a person exercises their “right to try”, and there is a serious side effect. Was it caused by the medicine? Does their insurance company have to pay for these new medical costs? Not that I care too much about insurance companies, but how deep is the rabbit hole?

  9. Alan Crowe says:

    I think we can identify a single piece on knowledge that qualifies a patient to exercise a “right to try”. It is whether the patient can grasp that entering a stage III trial is an altruistic act.

    Some patients seem to think that getting into a stage III trial is to their personal advantage as they get the latest and best drug first. Not really. Most drugs fail in stage III, turning out to be useless or worse. Joining a trial is a matter of gambling with the odds against you.

    After the results from the trials are in, we have new knowledge and can recompute the odds. For the drugs that fail the odds go from unfavorable to very unfavorable. For the drugs that pass the odds go from unfavorable to favorable. Taking a drug that passed goes from being an altruistic act, taking an unfavorable gamble in the hope of advancing medical research, and becomes a gamble with a positive expected value, even on a narrowly personal basis.

    A patient who understands the above (perhaps the test is: can they explain it, not have they listened to it) is expert enough to understand why their own doctor is mildly discouraging and, in my opinion, is entitled to say “Fuck it! Time for a reckless gamble.”.

  10. Christophe Verlinde says:

    Anon-1 advocates “Thus, the role of the FDA should be to ensure that the risks are clearly and accurately spelled out given what we know and don’t know, and they could even give their own independent recommendation based on this info, BUT they have no right to make the ultimate decision for the patient.”
    That approach can only work IF the FDA has an infinite amount of resources and time to study potential cures and spell out the knowns and unknowns, WHICH IT DOES NOT. Access to expertise costs money and many Americans are loathe to pay more taxes to enable an ideal FDA.

  11. mb says:

    a personal anecdote for what is worth, as the FDA already has a right to try if you are close enough to death.

    My cousin recently died from liver cancer most likely linked to Hep C infection. Back when the first Hep C drugs were hitting the clinical trials, he tried to get in to one, but had no luck. Some were eventually approved, but it was too late for him, he already had cancer. Eventually, he exhausted all possible cancer therapies and given the right to try anything anyone would give him. That was two weeks before his death. A pretty meaningless choice at that point, nothing to lose, he would have taken anything.

    Back when he actually had a choice and could weigh the risks, he was not given the choice.

  12. Peter S. Shenkin says:

    Perhaps something would be gained by separating permission to sell a substance from approval of the substance as a drug. Presumably only the latter would imply consideration for insurance coverage.

  13. Peter Kenny says:

    Worth remembering that there is no shortage of ‘experts’ in drugt discovery. I have linked a blog post on ‘expertitis’ as the URL for this comment.

  14. New kid on the block says:

    It’s Sofia VERGARA btw

    1. Another Nitpicker says:

      It’s also Grigori Perelman.

      1. Derek Lowe says:

        Fixed, thanks. My non-mathematician credentials remain in force, I guess.

  15. Sic Freat Crustulum says:

    Excellent article. It’s refreshing to read a longread that not only addresses some of the problems facing scientists, but also the difficulties in communicating those problems.

  16. Curious Wavefunction says:

    Another way to think about voters is not in terms of “low” and “high” information but through the lens of psychologist Jonathan Haidt’s moral foundations theory which I have found to be an excellent tool to dissect political differences. In Haidt’s view, conservatives and liberals are usually divided on the basis of what human traits and moral qualities they value: for instance liberals might value the moral foundations of care and harm more while conservatives might value authority and loyalty more.

    In this world neither side is necessarily higher or lower in information (although specific individuals certainly can be so) but each side values certain aspects of human nature and relationships more, and this can even color the way they approach and process information (this, the information component might be true but it could have more fundamental roots). Interestingly in his research, Haidt found that conservatives value a plurality of moral qualities while liberals place a lot of emphasis on a select few. Haidt’s book “The Righteous Mind” as really worth reading, especially in today’s political climate.

  17. Ryan says:

    I am fine with right to try and letting patients try medicines they want to. I am not fine with anyone charging them to try medicines without FDA approval of safety and efficacy. If companies can charge patients to “try” unproven medications, the conflicts of interest will lead to a disincentives to prove anything about medicines and the selling of a lot of snake oil. So let patients try a medicine if and only if a company is willing to give them the medicine for free.

    1. tnr says:

      I believe this is consistent with the current FDA Compassionate Use program. Patients can petition FDA for compassionate use and if the company agrees to participate, then the company provides the drug for free. However, many companies (especially small biotechs) do not want to give their drug away for free. It is also a cumbersome, time consuming process as it currently stands. The ‘right to try’ folks want to bypass the FDA and make it mandatory for a company to participate.

  18. Peter says:

    You say Goedel’s results or the Riemann Hypothesis are harder to understand than political or medical issues. I would, very strongly, disagree.

    Goedel’s (famous) results are really not very hard things to understand: just that most people never tried to understand. What it means for say philosophy or physics is a bit harder to understand, and usually badly misunderstood by the people who talk about them; that’s what most of the book is about. There is a bit of a barrier to getting into the subject, which is that unless you work with mathematics you are not used to kicking around abstract concepts (whereas everyone argues politics in the pub). But one can easily give, in plain English and in less than ten pages, a fairly accurate description of what the result is and why it is true (in other words, more or less give the proof). And the issue is settled – in some sense, after those ten pages there is not much more to say.

    The Riemann Hypothesis is harder to talk about; partly because it assumes (much) more background in mathematics even to state usefully, partly because we don’t know if it’s true and therefore have to discuss a bunch of ways we tried to prove it that didn’t work (or at least we couldn’t get to work), and partly because it has a lot more connections to things we are interested in. Briefly, Goedel’s results are an end goal interesting on their own; they don’t so much lead anywhere. The Riemann Hypothesis isn’t so interesting on its own, rather it’s a tool whose (positive) answer would allow us to do a whole lot of things in number theory we care about.

    Now compare this to trying to seriously discuss a political issue. In ten pages you’ve not even begun to cover the different positions and the basic facts, let alone tried to assess which parts of the different viewpoints are more or less reasonable. Probably you will have to offload the background history and the economics and law understanding (which your readers have to know to really understand what’s going on) to other people’s books: to do just the current politics properly you need to write a long book, and even then someone will come along with points you didn’t cover.

  19. former GSK says:

    I think the term “expert” has become very diluted. I think real experts are those that a) have a lot of experience in their field, b) have some stake in their expertise, for example, NBA basketball players, surgeons, etc, where a negative outcome has appreciable negative consequences to their livelihood, c) have a verifiable track record of success that is far beyond average, and optionally, d) have made predictions in the past that have proven correct.

    Lebron James is an expert basketball player. David Boise is an expert attorney.

    When it comes to economics (the dismal science), I very much doubt most people’s expertise. I would certainly give more credence to someone who has actual business experience than an academic because I value actual doing more than theoretically doing.

    1. Alan Crowe says:

      I like the jargon from education that distinguishes between norm-referencing and criterion-referencing. see perhaps http://www.edpsycinteractive.org/topics/measeval/crnmref.html.

      Your suggestion a)experience is norm-referenced: what is a lot of experience? We answer by comparing one expert to another, who has more experience?

      Your suggestion d) is criterion-referenced. If our criterion for expertise in economics is making money by shorting the US housing bubble, then we have very few experts, mostly mavericks.

      Criteria-referencing can be quite narrow. Mark Carney did a good job in Canada so he was a criteria-referenced pick for governor of the Bank of England. I don’t think that validates his opinion on broader political issues.

      I don’t think the problem is dilution in the sense of having low standards for who is an expert and hence too many, poor quality experts. I think the problem is expecting too much from experts selected by norm-referencing. They may be the best humans on a subject, but if the subject is too hard for humans, nobody really knows and one shouldn’t go all in on trusting them.

      I’d like to add a fifth point to your list e)Do they ask the right questions?

      For example, what will become of labor market regulation in the EU? Will it go Dutch/German or Spanish/French? My guess is that the EU is above the size for coherent policy making, so that it ends up with some kind of political average that nobody thinks is a good idea and which works badly and which the citizens of the EU are basically stuck with, because the decision makers they elect are trapped within the system and cannot over-ride the dynamics implicit in the system.

      Whoops! I need to try to stick to Derek’s problem: How to choose experts in general. It is natural for an expert to ask the kind of question that their expertise allows them to answer. So maybe they ask the wrong question, answer it correctly, because they are genuine experts, and then we correctly ignore them because the correct answer to the wrong question doesn’t actually help 🙂

  20. Grookology says:

    “Experts”

    Experts have
    their expert fun
    ex cathedra
    telling one
    just how nothing
    can be done.

    -Piet Hein

  21. DJK says:

    Cool article, Derek. Really interesting.

    Brexit and the American presidential election are both incredibly complex issues and made more complicated for the voter by misinformation and scaremongering, all of which is put to the population in binary terms. Of course both end up as an emotional vote – the number of variables make an informed opinion all but impossible, even if you simply wanted to decide based solely on your own interests. But that’s not to say these are issues to be left to the experts. Perhaps the Brexit fallout and the Trump ascendancy illustrate the divisions in our society and the situation that we find ourselves is closer to the “religious fundamentalist” examples Derek mentions; the distrust of experts he refers to has expanded to tribalism and distrust between sectors of society.

    I think right to try laws can be distinguished from the above because these rules are the recognition of the emotion we attach to our health, and that we can be easily taken advantage of without regulatory protections. Everyone, even experts, faced personally with a terminal disease, can be driven to Vitamin C infusions and colour therapy, knowing the infinitesimal chance of success.

    This issue comes down to balancing the freedom of choice against the protection of the vulnerable, which is not a matter for experts but rather a matter of opinion (informed by experts).

  22. Insilicoconsulting says:

    For me the quality of experts, their exposure to the entirety of the subject, the agenda and intrinsic beliefs of both the experts, their personality and risk averseness and those commissioning/questioning them and the inherrent known-knowns—–>unknowns-unknowns (degree of information available to make an informed decision), all play a role. The last factor is often the most important even if the expert is unbiased.

    If the right to try compounds are considered to be safe there’s every reason to give patients the right to decide. The data can be collected and better policies can be evolved based on this real world evidence very quickly.

  23. Erebus says:

    Philosopher and futurist Robin Hanson just posted an interesting opinion piece, which is something of a counter-argument. http://www.overcomingbias.com/2016/06/against-prestige.html

    In any case, Derek, your long post is something of a non-sequitur. On the whole, it has very little to do with right to try, which is a downright simple issue: If the drug is proven reasonably safe, to prevent access on account of “efficacy” is reprehensible. This is because the FDA is a very poor judge of efficacy, which employs measures as mercurial and arbitrary as they are costly, lengthy, and inefficient. Many sick people don’t have a decade to spare, and thus simply cannot wait for the FDA’s experts. (Who are an excellent example of the Iron Law of Bureaucracy. And, contra Robin Hanson, do not appear to be preoccupied with improving outcomes or optimizing the approval process they oversee.)

  24. kjk says:

    Of all STEM fields, medicine is one of the least experts-should-have-the-final-word.

    It mixes the puritan impulse with extreme risk. The Impulse has worked for air travel: it’s 99.999997% safe without excessive costs. But the treatment for xyz disease is coin flips with the grim reaper. Drugs wait years to be >99% sure before FDA approval, but the terminally ill only have months. It may be DALY-average better (or better according to their own risk-preferences) to try a phase I drug moonshot.

    Secondly, people are experts about the quirks of their own body and symptoms. Telling their doctor all of this during a 20 minute appointment is like trying to write down all the English words we know.

    Thirdly, medicine is mostly memorization of easy, nearly independent facts. A patient can become knowledgable about i.e. ILC breast cancer in much less time than an oncologist can learn about a hundred different cancer types.

    Fourthly, there is a fair amount of division among experts. Even well-respected journal paper articles are often wrong or in contradiction. And experts make mistakes. One place where the studies DON’T contradict is that there are lots of lives lost due to medical errors:
    https://hub.jhu.edu/2016/05/03/medical-errors-third-leading-cause-of-death

    There should be a right to try. We can buy Miracle Water (now with 50% more holiness) but can’t use a drug-candidate that actually underwent some theoretical work and experimental testing. Yes, patients should be sternly told it probably will not work. Yes, there should be some limit on when a drug is “tryable” for a given disease. Telling patients it’s hopeless rather than giving them a drug that has a 5% chance of working is a good way to boost sales of Miracle Water.

  25. David says:

    First of all I would like to state that there is something horribly wrong with the system that gives us these choices. Why do no supremely qualified people want to run for president?
    That said, I have no idea who to vote for.
    I understand why people want to vote for Trump. They see a system where cops are stealing money from citizens through civil forfeiture. They see a system where the government believes it has the right to spy on its citizens. They see a system where policemen are killing civilians and getting away with it. They believe that one indisputable function of government is controlling the borders, and don’t see it being done. They see H1B visas being abused, with former employees having to train their replacements. They just want a change. Hillary is saying things will stay the same. Trump is saying things will change.
    Is that a convincing argument? Not to me. I don’t know who I will vote for. I really detest EVERYONE running.

  26. Mark S. says:

    IMO the nub of the problem is that Derek’s perceptive category “Matters for Experts” really should be labeled “Matters for (Intellectually) Honest Experts”.

    Gresham’s Law applied to information leads directly to https://en.wikipedia.org/wiki/On_Bullshit . Once we have large segments of the populace, both the general populace and the expert populace, who are not morally & intellectually invested in honesty as a first principle, then every statement or act by every actor becomes legitimately untrustworthy until proven otherwise.

    Just as no then-Soviet or now Russian citizen trusts their police to be law-abiding, honest, and fair, very few citizens of the West now trust their elites to be anything other than self-serving.

    IMO it is impossible to run a complex society, including as a tiny subset a complex drug discovery and safety regimen, without honesty as a core value / virtue. There will still be plenty of room for disagreement amongst the honest folks as Derek skillfully outlines. But that legitimate signal will be utterly overwhelmed by dishonest noise in any environment where dishonesty plays more than a trivial role.

    How we might improve this state of affairs is mostly a mystery to me. Clearly we need the incentive structure to be very different than it is. One hopes we won’t need to upgrade H. sapiens from P. troglodytes v2 to v3 before it’s doable.

    On the specifics of “right to try”, IMO the only way to get to a net-good result is to ensure that any “right to try” drugs are provided free of any payment to the drug supplier, no matter how convoluted the business arrangements. That is the only way to keep the honest/dishonest ratio even remotely under control.

    Distilled water will pass any FDA safety screen with truly flying colors; it beats aspirin by a mile. Anyone proposing a profitable variant of “right to try” for any/everything passing a safety screen is sowing the seeds of a health (and health industry) disaster.

  27. MCS says:

    Until the completion of phase III, no treatment is more than a plausible conjecture. This is when people, other than those invested in it’s success, have actual numbers to compare to the alternatives. And the present evidence is the most of these conjectures are wrong.

    When considering informed consent, think about the last time you handed a few coins to a clerk after he entered the whole dollar tender in the terminal and the painful pause. Or talk to me about the general population’s grasp of the finer points of statistics and probability while standing under a lottery billboard.

    In many cases, it will come down to desperate people spending whatever money they can get either on the plausible conjectures of drug developers or on people in conveniently lax jurisdictions selling ground peach pits or frog secretions.

    I, unfortunately, don’t have even a plausible conjecture of a solution. The main advantage of the present “system” is that it allows someone with a possible cure to look a desperate person in the eye and say, “its not my decision”.

    1. Erebus says:

      Phase III is the problem. (Even when shameful debacles like the flibanserin approval are not taken into consideration at all.)

      “Right to try” arose due to the failures of the FDA’s approval process — particularly, indeed solely, where their flagrant inability to measure drug efficacy is concerned. They can’t do it in a timely manner, as drug approval timelines continually creep upwards and now stand at an average of 12 years. They can’t do it in a cost-effective manner, either, as it now costs an average of approximately $2B to get a drug approved, and that cost, which is still rising an order of magnitude or two faster than the rate of inflation, is almost entirely due to the onerous demands of efficacy testing.

      …That this is extremely bad for patients and bad for innovation in the pharmaceutical industry should go without saying. If patients are inclined to roll the dice themselves, can you blame them? The FDA is, very clearly, not doing a good job on their behalf.

      If one grants that the FDA’s efficacy testing paradigm is very deeply flawed — as I believe one must — one must also grant that safety testing is the only reasonable barrier to “right to try.” If the FDA has demonstrated anything, it is that it is incapable of judging drug efficacy in a reasonable manner. (In their defense, it’s a very tough job.) Vastly expanded right to try, combined with intensive postmarketing surveillance, is the better option. It’s more humane and compassionate, and it’s ultimately better for the drug development industry.
      The other, still better, option would be to roll-back the FDA’s mandate and re-establish it as an agency which safeguards the safety — and only the safety — of America’s drug and food supply.

      1. Fenichel says:

        I don’t understand the notion that safety is easier to establish than efficacy. Most of medical practice was fairly safe for thousands of years, and most of the nostrums of modern charlatans are probably pretty safe, too. Efficacy was, and is, the rub.
        Some efficacy effects are obvious (for example, those of general anesthetics), and so are some safety effects (for example, renal toxicity of amphotericin), but these are the exceptions. Still, most efficacy is substantial: An analgesic that beat placebo in 1% of patients wouldn’t get very far into development. When efficacy is required to be demonstrated, it’s usually possible. If it’s not possible to demonstrate efficacy, one must wonder why some people believe that it’s present.
        We care about some safety effects that are much, much smaller than that, so they are harder to demonstrate. We use isoniazid with care, because it we think its hepatotoxicity (around 0.1%) is important. The efficacy of chloramphenicol could be strongly suggested in vitro, and rigorously demonstrated in a few dozen patients, but we use it with care because of its hematological toxicity (less than 0.01%).
        This is not to say that isoniazid and chloramphenicol shouldn’t be used. Either one’s safety (or efficacy) can be interpreted only in the light of its efficacy (or safety).

        1. Erebus says:

          An argument from first principles? Although you have a point, we must look at the way things actually work: Safety testing — Phase I — is both rather quick and rather cheap. Many small firms finance it themselves. If the FDA were so inclined, it is even amenable to standardization, for e.g. on a points-based system. Efficacy testing — Phases II and III — is astoundingly expensive, beyond the capabilities of most smaller pharmaceutical firms to finance, is extremely slow, is utterly impossible to standardize, and can be difficult to fairly interpret. Efficacy testing can keep drugs mired in the regulatory system for a decade or more — and these are drugs that have already been proven reasonably safe, and which may be able to save lives or improve quality of life in patients.

          …Even if we weren’t discussing matters of life and death, would it be worth the trouble? Low-hanging fruit notwithstanding, the zenith of the Golden Age of the pharmaceutical industry was the period from 1950 to 1960 — an era where the FDA did not police drug efficacy.

          There are other considerations as well. Among them, the ultra-long regulatory approval process is increasingly at-odds with the 20-year patent lifespan. Another thing worth mentioning is that broad and fast right-to-try coupled with stringent postmarketing analysis could provide better efficacy data than the FDA’s current paradigm.

          Of course, unsafe drugs will slip through the cracks, the best efforts of all experts and all self-serving bureaucrats notwithstanding. It’s inevitable. This is tinkering with biology, after all.

          All that aside, when you say “We care about some safety effects […] We use isoniazid with care […]” and so forth — who is this “we”? Physicians? Speaking for myself, I very strongly believe that physicians should be granted more leeway in how they prescribe drugs. (Including experimental drugs.)

  28. First, this is great. Thanks, Derek and all the commentators for their thoughtful and CIVIL discourse on some of the more emotional and difficult problems of today.

    Another analogy for this whole question of right to try and when that’s appropriate might be found in baseball. There’s been some great research identifying stabilization rates for various batter skills. Stabilization refers to the point at which a given statistical metric is “stable” in the sense that if you split a player’s discrete actions into two pools at random the two pools would correlate at a level of 0.7 (good explainer here: http://www.fangraphs.com/blogs/when-samples-become-reliable/).

    In other words, when can we start “trusting” things like batting average, swing percentage, etc as being indicative of true talent level? Interestingly different metrics stabilize with different numbers of plate appearances. Within 50 plate appearances we’ll have a pretty good idea of how often a player tends to swing at pitches. Slugging percentage, on the other hand, can take over 500 plate appearances before we have a good idea of a player’s actual power.

    The takeaway for right to try: Phase trials can be thought of as a drug making its way through more plate appearances, and as it goes forward we become more confident about safety and efficacy. We might say that safety has a faster stabilization rate than efficacy. At what point in Phase trials does efficacy have some confidence around it? As MCS points out, it’s often not until Phase III trials that we start to have relatively strong confidence in a drug’s efficacy. My point here is that we know so little about efficacy before the approval process has concluded, that right to try is enough of a shot in the dark that it shouldn’t be done. If Robinson Cano goes 0 for 10 to start the season, you don’t drop him from the Seattle Mariners. Conversely, if an Independent League batter hits two home runs in his first game of the season, you don’t immediately sign him to a $10M contract. If we want to make data driven decisions, that right to try doesn’t work because there isn’t enough data.

    I’ve also enjoyed the discussion about experts. For those interested in a real-life science story about expertise, about a month or so ago Neuron published a paper claiming an MS risk variant. Commentators in the genomics realm such as @dgmacarthur (Daniel Macarthur at Mass General) pointed out that this study was flawed from a statistical genetics standpoint and within a month a preprint had been submitted to the preprint biorxiv (http://biorxiv.org/content/early/2016/07/01/061366) by Chris Cotsapas and others (@ccots) raising a counterargument to the Neuron paper. What I found interesting after the fact was a conversation among some in the genomics field saying that the authors really should have learned more statistical genetics and genomics before getting involved in this study. One comment I read was “…basic stats and sound hum gen study design should not require an expert but be the norm,” to which I had a visceral reaction of, “well yeah, but you could say that about a lot of different topics in biomedical research.”

    Clearly the authors of the neuron paper could have used more expert help, but there are two ways that could go. One could say that everyone should become expert (which I’ll define operationally in this case as someone who knows enough about a given field to be able to perform scientifically sound experiments) about everything that’s an important topic these days. Or one could say we should all recognize the limits of our expertise and make sure to find the right people at the right time. I fall in the latter camp. Would love to see our curricula emphasize a broad enough knowledge base to allow scientists to have a better chance of identifying their gaps in knowledge, and also the tools to figure out who to contact.

    Circling back to the question of skepticism of experts, I think this is another example of why this trend is so alarming. It suggests not just that a lot of people don’t have great confidence in institutions that previously had strong support and trust, it also suggests people may believe expertise is more easily gained than is actually the case. While skepticism is a useful trait, when it’s used to provide a blanket dismissal of things that are close to “Matters of Fact,” because they do not agree with one’s personal opinions, or when it leads one to conclude that being an expert isn’t that hard, that’s a path to bad science and bad policies. The question, the really hard question, as Mark S. says, is what’s to be done?

  29. Sean says:

    How COULD these people balance the risk. If you don’t work in drug discovery, how do you know? Look at the rodent studies. This has ended badly at least twice in stage 1, crippling 11 and killing one.
    First into man should be done the old fashioned way – the designer is subject 1. See Bentley et al who, nearly wiped out the whole team by allowing them to make tea in the lab. It was stirred with a rod previously used for etorphine. Then they all went and tried bupreorphine and pushed it as opiate replacement therapy – only took 30 years to happen.
    In the UK, stage 2 can go to stage 3 for lifesaving drugs… don’t know about US. This site is in the US, so I guess that everyone has the right to comment

  30. DoctorOcto says:

    If an expert is foremost in their field, then is anyone else really fit to judge the level of their expertise?

    1. Hap says:

      Doesn’t that depend on how their expertise is defined? If everyone has a common set of ideas but no validation with experiment, then being the best in your field doesn’t (shouldn’t) make you immune to data. It might also depend on how much better the expert is than others in the field (a lot or a little), whether he’s of narrow or broad superiority, and, if narrow, how many other people are competent in his subfield.

    1. Derek Lowe says:

      I was reading that last night, and found it truly irritating. If I can put the reasons why into words, I’ll blog on it.

      1. RM says:

        (Replying here to possibly help Derek crystallize his thoughts.)

        My biggest issue with the article is that it takes a rather patronizing tone. “Oh, what have you come up with, my dear scientists? A “scientific method”? Good job! We’ll put that one up on the fridge, now, won’t we?” That’s not helped by the (intentionally) inflammatory headline.

        The other issue – the main one where it falls down logically – is the casual dismissal of empiricism. That’s a key difference which separates the scientific method from other investigations. Yes, you can talk about what “justice” means by consolidating various opinions and finding a consistent intersection, but the orbit of Mars is the orbit of Mars, and it doesn’t matter how many learned scholars you get to agree: if their agreement differs from experiment, it’s wrong. It’s that objective experimentalism which makes “the results of science [more] reliable than what is provided by these other forms”.

        Other than that, I don’t have many complaints. It’s basically what Chad Orzel has been saying for a while now (his take on the article): the scientific method isn’t some rarefied concept – it can be applied to ordinary life.

      2. Xplo says:

        The first problem I see with the article is that it fails to correctly observe that “scientific method” is a term of art, not colloquial language:

        “Maybe the method on which science relies exists wherever we find systematic investigation. In saying there is no scientific method, what I mean, more precisely, is that there is no distinctly scientific method.”

        On the basis of this equivocation fallacy, the author goes on to demonstrate how definitions are hard by making a great show of being unable to define courage… in much the same way that an infomercial sells a product that solves a problem that nobody actually has by portraying an ordinary, simple household chore as an insufferably difficult one.

        The final problem, as RM points out, is that the author concludes his argument about how “scientific method” can’t be adequately defined by dismissing empiricism altogether.

        The article is a fine example of the sort of bullshit described in “On Bullshit”, referenced elsewhere in these comments: there’s nothing particularly true in it, but it’s impressive enough to confuse the ignorant into supposing there is.

  31. b says:

    Two points I struggle with regarding Right to Try:

    A) If a company is trying to develop a legitimate drug, what is their incentive to provide said drug in this case? Presumably, if somebody gets to the point that they need to bypass standard of care, their outcome is much less likely to be a positive one to begin with. Is the data generated to be relied upon? This could end up killing a drug in development that legitimately had a chance of helping others in the future.

    B) Why do patients have a divine right to a treatment in development? Patients today have access to many more treatments than existed 50 years ago, and the same can be said 50 years from now. We’re all unlucky bastards at some point along the time continuum. This ties into arguments for the patent system/generics. “Sorry you can’t get it today, but it’ll be really cheap for others in the future.” It’s tough to figure out where the line is.

  32. Blissex says:

    I am sad to read these kind of arguments, because of the common convention to avoid pointing out some important topics, which have however been addressed by some commenters:

    * We should never forget that “official” chemical medicine did not have *any* efficacious compounds (that is, it was 100% quackery, whether dishonestly or merely deludedly, for many thousands of year) before penicillin, yet “experts” like doctors and pharmacists usually became quite rich. As someone above hinted by writing «Matters for (Intellectually) Honest Experts».

    * The “right to try” is primarily a topic of interest not to patients, but to pharma executives, who dream of being able to market and sell as “medicine” placebos proven safe by the FDA: huge repeating revenues and immunity from lawsuits! As implied by the comment «Distilled water will pass any FDA safety screen with truly flying colors».

    * It is necessarily fantastically hard to find single compounds that are both generally safe and efficacious, because body chemistry is both a highly complex system and there is a lot of variation in the details of body chemistry, and most low hanging fruit has already been plucked. As implied in the comments saying «its hepatotoxicity (around 0.1%) is important [ … ] its hematological toxicity (less than 0.01%).»

    * The reason why efficaciousness matters is that even after safety trials nearly all medicines in essence are or can be also “poisons” and have some bad side effects for someone, and therefore the potential risk of taking them cannot be evaluated without also considering the potential gain from efficaciousness. That I think comes under the “first cause no [pointless] harm” principle. As the comment above saying
    «safety (or efficacy) can be interpreted only in the light of its efficacy (or safety)».

    Thus the banner of the chemical medicine should be “precautional principle”, rather than “do not restrict the sales opportunities of placebo marketers”.

Leave a Reply

Your email address will not be published. Required fields are marked *

Solve the math problem. * Time limit is exhausted. Please reload CAPTCHA.