This one will be good for a wry smile, a roll of the eyes, or perhaps a knowing shiver. The British Medical Journal has published a “Key opinion leaders’ guide to spinning a disappointing clinical trial result”, and many are the times that such a handbook is needed, unfortunately:
When key opinion leaders are asked to comment on disappointing trial results in news reports or at conferences, we have observed that they seem curiously unable to recognise that the treatment doesn’t work. They prefer to argue that the trial design was wrong, drawing from a set of stereotyped criticisms. Using cardiology as an example, we have systematically analysed the excuses they provide to compose the Panellists’ Playbook, an anthropological classification that will be useful not only for readers but for key opinion leaders in need of inspiration (or backbone). . .
. . .We found comments on 321 trials from the 15 international scientific congresses [in cardiology] held during 2013 to 2017. Of these trials, 127 (40%) had negative results and received a total of 438 remarks from key opinion leaders. Excuses were provided for 108 (85%), with a mean of 2.5 published excuses for each trial. We defined an excuse as any explanation given for a trial’s result other than the treatment not working. . .
The most common excuse was that the sample size was too small. Interestingly, the authors found only one instance where anyone suggested at the time what the sample size should have actually been. “Follow up too short” is up there, too, but apparently with no estimates of what an appropriate length would be. Other high-scoring excuses are that the trial was too inclusive, the comparator therapy was too good, or that the wrong doses were used. “More studies are needed” is also popular, as anyone who’s followed the medical literature for more than twenty minutes can well believe, but the authors put that one in the “vacuous” category, suggesting that “the key opinion leader simply does not like the result and wants another throw of the dice“. This seems hard to refute.
The authors suggest that wild-type Key Opinion Leaders have been selected for over time for their ability to mobilize these explanations under journalistic time pressure. But this handbook should make anyone capable of such performance. Indeed, they say, with its help “no intervention is too ineffective for an excuse“. I fear that they are correct!