Skip to Content

The Dark Side

Systematic Fraud

Here’s a personal experience with fraudulent scientific literature, as reported in Nature:

In 2015, I discovered several papers had been written about a gene that I and my colleagues first reported in 1998. All were by different authors based in China, but contained shared and strange irregularities. They also used highly similar language and figures. I think the papers came from third parties working for profit, fueled by the pressure on authors to meet unrealistic publication expectations. . .

The author, Jennifer Byrne at Sydney, and a colleague from France, Cyril Labbé, went on to develop software tools to spot this sort of thing more easily. But as this latest article shows, finding this kind of work is one thing; getting authors and journals to do anything about it another. They wrote to 48 authors after doing their first run through the literature, asking them to explain problematic results, and not a single one responded. So far, there have been 17 retractions, but not one of these has led to any sort of investigation into the research practices involved – the authors have relied on “Whoops, our mistake” as the explanation, apparently.

The problem is that working in this area can get you a reputation. People are uncomfortable talking about research fraud, for one thing, and the fact that many of the papers committing it are scattered through obscure low-impact journals makes it easier to dismiss as something unworthy of attention. And as Byrne says, “It is even more uncomfortable to think about organized fraud that is so frequently associated with one country.”

Indeed. It will be very interesting to see if the recent efforts by the Chinese government to crack down on shifty research and publication strategies will have an effect. The latest announcements on the system include public naming of those found guilty of misconduct and punishment that not only includes research funding and promotions, but can extend to things like ability to get personal loans or jobs outside of academia. It’s not like there isn’t fraud coming from other countries – there sure is – but the sheer volume of it from Chinese sources is a clear problem, and one that the authorities have apparently recognized. There’s a possibility that this (as with many another political initiative in countries around the world) could turn out to be window dressing, a way to look make it look like something is being done. But the Chinese government has – to put it lightly – demonstrated in the past that it has the willingness to follow up with severe measures if it really perceives a threat.

The fundamental problems, though, are the incentives that exist for starting and running journals that look the other way on this stuff, and for researchers to devote time, money, and effort to filling up the literature with swill. As long as there are rewards that can be obtained by fraud, people will commit fraud. I’d rather not have a corpus of scientific literature that has to be worked on just to clean the lies out of it, but that’s what we have now, and it’ll get worse.

30 comments on “Systematic Fraud”

  1. loupgarous says:

    At this point, I remembered your previous blog “Nationalist Medicine? What Crap is This?”:

    “Not only are these preparations getting easy regulatory treatment – the government is making it harder and harder for anyone to express any contrary opinions about them:

    With strong government support for the alternative medicines industry, Chinese censors have been quick to remove posts from the Internet that question its efficacy. On 23 October, an article on a medical news site that called for closer attention to the risks of aristolochic acid was removed from social media site WeChat. The story had been viewed more than 700,000 times in three days.

    Debate over TCMs has been silenced before in China. Last year, a Beijing think tank — the Development Research Center of the State Council — proposed banning the practice of extracting Asiatic black bear bile, another common ingredient in TCMs. The think tank’s report questioned the remedy’s efficacy and suggested using synthetic alternatives. It was removed from the think tank’s website after the Chinese Association of Traditional Chinese Medicine, which supports the development of TCM, called it biased and demanded an apology.

    You can go to jail for criticizing this stuff. Now, I think the way that the US regulates (or doesn’t regulate) “dietary supplements” is a disgrace, but this is insanely worse.”

    At this point the question “Does the Chinese government see systematic fraud to gain priority over discoveries in scientific fraud as a threat, or an opportunity” comes up.

    The sheer number of prominent Chinese citizens – including the head of Interpol – who’ve “disappeared” without what the rest of the world recognizes as due process or any other sort of protection of their human rights argues that whether systematic fraud is punished by the Chinese government, or protected by it is a very open question.

    1. MCS says:

      About the fifth time today I’ve said to myself: “sounds like China to me.”

    2. Paul says:

      There are a lot of things where “just wait long enough and these things will self correct” when talking about rapidly growing economies (China being the current one) is both an accurate observation and yet at the same time woefully inadequate in the consequences of humans broadly and individuals specifically, not to mention that established habits are hard to shift once they’ve been ingrained…

      “just because we’ve always done it that way doesn’t make it a good idea” and “just because it’s a really bad idea doesn’t man they won’t do it”

  2. loupgarous says:

    Wups. The second-to-last sentence ought to have been:
    “At this point the question “Does the Chinese government see systematic fraud to gain priority over discoveries in science as a threat, or an opportunity?” comes up.”

    1. Patrick says:

      Yeah, but your original phrasing makes a nice point as well, even if unintentionally.

  3. luysii says:

    A friend and college classmate, an emeritus prof of chemical engineering, referees a lot of papers. He estimates that 80% of the papers in his field, quantum chemistry, coming from China are absolute trash. According to him China gives bonuses to people getting published in high impact journals. What he finds particularly appalling is that he writes up a detailed list of corrections and improvements for the paper, and then finds it published totally unchanged in another journal.

    1. John Wayne says:

      People behave in a way consistent with how you reward them. I suggest that the mistake here is rewarding people for the impact factor of their publications. This causes three problems, (1) impact factors are not a great substitute for looking at a research group as a whole, (2) it indirectly encourages gaming the system (via fraud or other methods), and (3) it consolidates power into the hands of editors who get to put a strong hand on who the winners and losers are. This almost invariably leads to academic nepotism.

      “What he finds particularly appalling is that he writes up a detailed list of corrections and improvements for the paper, and then finds it published totally unchanged in another journal.”

      This happens to me all the time, except that it gets printed in the journal it was sent to. I think I annoy editors by giving them a bunch of work do to; I don’t get asked to review nearly as much as I used to. It is a shame that people don’t look at useful comments made by reviewers as an asset. Several reviewers have greatly improved the quality and clarity of several of my publications, and I appreciate the time they took to do so.

    2. fajensen says:

      There could be a mercantilist strategy behind this. Every garbage paper ties up western expert resources on the review and subsequent refutal. Whenever a garbage paper slips through and is cited, western experts will be scratching their hair for a while trying to find out why they can’t replicate the results.

      After a while of this, the real experts can’t be bothered any more since they have paid work to do, dogs to walk, family meals to attend, beer to brew, and so on, and “the filter” breaks down and everything is unchallenged no matter how scummy it is.

      The Chinese spam-paper-factories has the information advantage of knowing what is the garbage and what is good.

      Of course it is more likely “just” the normal competition-game local-utility maximisation, but, in the way that is worse because when the collaboration-game becomes a competition-game everything becomes chaos.

  4. Christopher Wood says:

    Sounds like one more argument for aggressive funding of replication studies. I have found in my field (biofuels / biochemicals) that inability to replicate other peoples work is a real problem. It is definitely worth putting some more funding and effort into replication, and maybe finding a way to get that kind of replication work into journals (maybe a big open access journal of nothing but replications?).

    1. JK says:

      I like this idea but it may be more complicated in reality.

      In order to build a critical mass of change, for every group caught with irreproducible data, there would have to be one or more who start verifying their own. That means consequences – losing respect from one’s peers is probably not enough, and there are some legitimate defenses, such as, “my grad student falsified data without my knowledge” or “the supplier of X must have changed their formulation/synthesis which changed the results of the study.” But the truth of these may not be verifiable.

      At least, one might say the PI is responsible for his grad students data which would undercut the validity of excuse #1. But then what? PIs begin a new standard practice where one researcher in a lab would be a check on another. But this would be inefficient and would change long-standing expectations for productivity.

      And if we’re being fair, there should also be some kind of standard for determining irreproducibility – the work would need to be done by someone trustworthy (who’s to say who that is?) or by 2-3 separate labs.

      Making academia even less efficient in the short run is a hard sell, even if it makes things more efficient in the long run (which wouldn’t clearly be the case but it would certainly remove confusion caused by false reports.)

  5. Uncle Al says:

    If plagiarism were mandatory, then the most agile not the most intelligent would be rewarded. The second, deserving rat would get the cheese. Further the daily struggle! Restore historically factitious pharma.

    The Green Deal is living off your body fat, ending farm emissions.

  6. JB says:

    Fraudulent work coming out of China? What about work from China done technically correct, but performed unethically? There are many issues with respect to the state of Chinese research besides just those surrounding data manipulation. There is also a big clash of ethics between what westerners deem acceptable vs. what those in China think.

    http://www.sciencemag.org/news/2017/02/study-retraction-reignites-concern-over-china-s-possible-use-prisoner-organs

    which has now been updated

    https://www.theguardian.com/science/2019/feb/06/call-for-retraction-of-400-scientific-papers-amid-fears-organs-came-from-chinese-prisoners

    1. a. nonymaus says:

      This is, assuming that the results are as stated and relevant to non-stolen organs, not a case for retraction. It is a case for the same authorial damnatio memoriae used in citing data obtained from the abuse of concentration camp inmates.

  7. “Shifty research.” I appreciate your polite misspelling.

  8. MoMo says:

    So what level of fraudulent research is Pharma experiencing?

    Lets hear some stories.

    1. David in Kent says:

      Years ago, I was involved with a study in California, let by a famous doctor. The CRA got suspicious because the routine chemistry was too perfect. It turned out that most of the ‘patients’ had been invented. Called into question a lot of drug submissions.

  9. Anonymous says:

    John Wayne — I refereed for a journal that insisted on sending me crap which my detailed reviews conveyed. One of the papers (out of India) was a repeat of some older chemistry (out of the UK, 1960ish, as I recall). My report was politely critical of their poor lit skills and failure to cite prior art. I was actually thinking of accusing them of plagiarism but I held back. After rejecting many mss, the journal no longer needed my services. 🙂

    Chris Wood — Organic Syntheses is a reliable compendium of checked (tested and verified) methods. Is there anything else like it in other fields? (There are some Mol Bio methods collections that I have seen that appear to be at least annotated or updated with user comments.) Now, if Crook B confirms the method M of Crook A and method M turns out to be inexplicably irreproducible, then both Crook A and Crook B should pay a penalty.

    JK — There should be penalties to be borne by the institution for fraudulent or irreproducible results from their staff. Not just payback of the original grant, but additional $ penalties or temporary loss of grants to the entire department that did not oversee their members; or beyond the department if applicable. Some administrators do not know how to evaluate scientific research so they do apply pressure to boost publications or other simplified metrics.

    PIs should absolutely bear responsibility for work of their grad studs. They take all the credit, don’t they? When all you give is praise and rewards for good outcomes and punishment for failing to get the “right” answer (product, yield, effect, …), many students get the message: “Tell the PI the wrong thing and you are finished around here. There is no penalty for lying. There is a reward for telling the best lies.”) If punishment has to be enforced on the Prof by outside agencies, the message to the students should be, “Lie to me and I am toast … and so are you” (not “LIe to me but make it a good one!”)

    Kent Budge — I noticed the same spelling error on the first pass!

    Off topic: Google Doodle celebrates “Friedlieb Ferdinand Runge: The Man Who Gave Us Caffeine.” This article (linked in my handle) notes, “Despite his [many] contributions [to chemistry], many of his accomplishments went unrecognized and were often attributed to other scientists. He eventually had a falling-out with his employer and lost his job. Runge died in poverty on March 25, 1867, at the age of 73.”

  10. It take too says:

    Quoting Anonymous: “There is no penalty for lying. There is a reward for telling the best lies.”

    So true. Couple thoughts on incentives.

    I’ve always seen the model of ‘no paper(s); no phd.’ Hypothetically, if after years of work, a student has piles of high-quality data wherein everyone can confidently agree there are no positive (read: publishable) results then the question becomes: ‘Do we send the student packing sans phd, or ask them to take a few more years rolling the dice???’ Wouldn’t it be better to put the manuscript in an open access journal and reward the student with a degree for having reliably documented things that don’t work? This could be a great model for biology where industry frequently takes good hypotheses (drug targets) from the literature only to move them to the column of ‘things that don’t work’. Does this happen sometimes and I’ve just missed it?

    Grad students are less incentivized to do good science than they are to graduate in as little time as possible in order to move on to something that actually pays. What I have seen (certainly not always) are students running a large number of poorly-controlled, underpowered studies then hacking the p-values into publishable shape until they have enough papers for a phd. I’ve also seen NIH funding and timelines that are utterly incompatible with running well-controlled, properly powered studies. Thus, literature littered with drug targets just waiting for pharma to jump on them and find out — they’re duds.

    I’m not a PI, but it seems like the incentives can range from ‘Publish copious amounts of anything so you can get tenure and keep the grants coming in’ to the slightly more useful ‘Do quality work so that your start-up actually has an outside chance of making you rich.’ Harkening back to the AOC thread, maybe allowing PIs to ‘pick the pocket of the NIH’ to set a foundation for their private ventures at least encourages those funds to be spent on good science rather than the irreproducible byproducts of phd-factories.

    To be fair, there are PIs who seem to want to do high-quality research simply because they make a living and they like it. I’ve met many such folks from the grey-haired demographic, but perhaps that’s simply an artifact of tenure (a grey-hair-inducing process indeed!).

    1. anonymous says:

      Grad students are less incentivized to do good science than they are to graduate in as little time as possible in order to move on to something that actually pays. What I have seen (certainly not always) are students running a large number of poorly-controlled, underpowered studies then hacking the p-values into publishable shape until they have enough papers for a phd.

      Worth also considering the strategy of get multiple MSc students or research assistants, have them run different experiments. The ones that work are the good students and you encourage them to stay for a PhD…

  11. It take too says:

    Subtangent: It seems easy enough for folks to start up predatory publishing groups, so why is it so hard to get someone to invest in the Journal of Confirmed Negative Results (JCNR)??? Just think of the publishing fees you could collect….

    1. the benefit of the doubt says:

      You’d be surprised by how many people can’t run routine experiments properly at various levels. Technicians right out of undergrads, graduate students, and Ph.D. chemists from top 10 programs, etc. In science, negative results are far too common and I’m not ready to write off published results because someone ran an experiment twice and didn’t get the expected results.

      1. Some idiot says:

        Yep… I still almost die laughing when I see yield with decimal places on a reaction that has been run once… I know, I know, they are reporting what they got, but how on earth can they justify decimal places after only one experiment???

        1. fajensen says:

          My very “own” TBTF-project insists on having measurements stated in ‘mm’ on all construction drawings … for buildings! Since nobody in charge sees why this is a problem, I am updating my LinkedIn page instead!!

  12. Niftyniall says:

    Sadly these issues are rampant across the sciences. Retraction Watch has many examples: https://retractionwatch.com/

  13. dennis chute says:

    For more than 27 years I ran my own bio-tech company (just retired). The company is 100% owned by myself and my partners. We have provided all the equity the company has ever had.

    We make money if our ideas, processes and products work.

    Now what you need to know is that freed from publish or perish and the tyranny of investors our failure rate is stunningly high. I mean they all seemed like good ideas at the time.

    For example, I have been trying for years to get iron loving bacteria to help us mine oolitic iron. The literature would imply this has a) been done by Mother Nature b) been done on a bench top and possibly c) been done on a pilot basis. It may not scale but the theory seems sound. I have sort of been assuming I just have bad technique or a poor understanding of critical conditions that Mother Nature can achieve and I am not.

    If I could publish negative results I would now have literally hundreds of published papers. I can’t of course publish at all. Too good a way to lose control of the IP.

    On the other hand if I can’t trust the literature that lead me here (much of which was done by Chinese scientists) I have wasted years of my life wandering around collecting bacteria, culturing them out, and then testing them as candidates. Not to mention lost economic opportunities. It makes me curious to know if it is me or them.

    1. Anonymous says:

      There are Magnetotactic Bacteria that sequester iron from their environment and make magnetite and greigite in their magnetosomes. You can harvest the bacteria and their natural multicellular aggregates (MMPs) with a magnet! I’m still searching for engineered bacteria that can sequester gold. Any investors? If it helps to reassure you, I’ll promise to use Artificial Intelligence and Machine Learning in the research program.

      (I am assuming that the poster dennis chute knows a lot about magnetotactic bacteria. I was posting a bit of info for other readers In The Pipeline.)

  14. eugene says:

    If you’ve been recently following Leonid Schneider’s blog ‘For Better Science’, then you will not be surprised by China due to the amount of systematic fraud in Spain and France in biochemistry. Schneider probably single-handedly brought down the CNRS president. In those two countries the government also seems to protect the perpetrators and that stakes to being successful in fraud are higher, since you get much better rewards.

    I was really surprised how people who get EMBO grants built their whole scientific careers on fraud, and when found out, the whole Spanish scientific community gathers to protect them. It’s very depressing and made me wonder if there is big spillover into Spanish chemistry in my field, but from the papers that I read, it seems they are trustworthy and good science…

  15. Bryan says:

    it’s obvious to me that if you come up with genuine interesting results that help medical science, then you will be credited appropriately. It’s only when you get into the politics/fights/he-said-she-said that fraud/awards/papers become a factor…..by that time, you already lost.

  16. Paul Brookes says:

    Nature, of all people, should not be taken seriously on any effort to rid the literature of fraud. Take for example this paper (https://pubpeer.com/publications/D569C47E7BE09AD9D238BA526E06CA) which accumulated a multitude of very serious allegations of image tomfoolery almost immediately after publication.

    8 weeks later Nature published a “cautionary comment” (which doesn’t show up on PubMed). No comment was added to the “puff-piece” they wrote about the paper.

    5 months later the paper is still out there for download, and has already been cited 5 times, including a glowing editorial in the New England Journal (https://www.nejm.org/doi/full/10.1056/NEJMcibr1811991)

    One might conclude the “investigation” underway at the journal is not about how exactly the fraud was committed, but rather “how long should we wait ’til this blows over before issuing a correction and quietly sweeping it under the rug?”

    1. old biologist says:

      One of the best arguments I’ve seen for publishing the names of the reviewers with the paper. Anyone who was responsible for letting something like this into the literature should be seriously shamed.

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.