Skip to Content

Run It Past the Chemists

I missed this paper when it came out back in October: “Reactome Array: Forging a Link Between Metabolome and Genome“. I’d like to imagine that it was the ome-heavy title itself that drove me away, but I have to admit that I would have looked it over had I noticed it.
And I probably should have, because the paper has been under steady fire since it came out. It describes a method to metabolically profile a variety of cells though the use of a novel nanoparticle assay. The authors claim to have immobilized 1675 different biomolecules (representing common metabolites and intermediates) in such a way that enzymes recognizing any of them will set off a fluorescent dye signal. It’s an ingenious and tricky method – in fact, so tricky that doubts set in quickly about the feasibility of doing it on 1675 widely varying molecular species.
Reactome slide
And the chemistry shown in the paper’s main scheme looks wonky, too, which is what I wish I’d noticed. Take a look – does it make sense to describe a positively charged nitrogen as a “weakly amine region”, whatever that is? Have you ever seen a quaternary aminal quite like that one before? Does that cleavage look as if it would work? What happens to the indane component, anyway? Says the Science magazine blog:

In private chats and online postings, chemists began expressing skepticism about the reactome array as soon as the article describing it was published, noting several significant errors in the initial figure depicting its creation. Some also questioned how a relatively unknown group could have synthesized so many complex compounds. The dismay grew when supplementary online material providing further information on the synthesized compounds wasn’t available as soon as promised. “We failed to put it in on time. The data is quite voluminous,” says co-corresponding author Peter Golyshin of Bangor University in Wales, a microbiologist whose team provided bacterial samples analyzed by Ferrer’s lab.
Science is also coming under fire. “It was stunning no reviewer caught [the errors],” says Kiessling. Ferrer says the paper’s peer reviewers did not raise major questions about the chemical synthesis methods described; the journal’s executive editor, Monica Bradford, acknowledged that none of the paper’s primary reviewers was a synthetic organic chemist. “We do not have evidence of fraud or fabrication. We do have concerns about the inconsistencies and have asked the authors’ institutions to try to sort all of this out by examining the original data and lab notes,” she says.

The magazine published an “expression of concern” before the Christmas break, saying that in response to questions the authors had provided synthetic details that “differ substantially” from the ones in the original manuscript. An investigation is underway, and I’ll be very interested to see what comes of it.

47 comments on “Run It Past the Chemists”

  1. FME says:

    To me that cleavage looks, how shall I put it… unlikely, to say the least

  2. Carmen says:

    Here’s a link to those updated synthetic details (apologies if this posts twice)
    http://tr.im/JuXI

  3. anchor says:

    Derek : Thanks for bringing this paper to our attention. As you have pointed out there are lot of errors in this paper. As some one working in the area cyanine dyes, I also see that the the dye structure is also missing the “double bonds”. We can be assured that this paper did not go through the chemist !

  4. Dennis says:

    Hmmm, I bet you could get that cleavage to go by adding silver oxide in paraffin wax.

  5. Vader says:

    Hmm. So you all are saying that that cleavage is about as authentic as Sheyla Hershey’s?

  6. KinaseNerd says:

    I could not access the synthetic procedures above. However, I am not as sceptical about the fragmentation reaction as everybody else around here: Essentially, we are looking at a retro-Michael addition with an amine nucleophil. It becomes more obvious if one draws the Hydroxyketone as the endiol tautomer, then you can easily “push the electrons” to get rid of the ammonium leaving group. What keeps me wondering is that you would then require a subsequent 4-electron-reduction of the alpha-Hydroxy-Michael system. As the resulting bicyclic system would be very electron rich, the required reduction may be feasible for many enzymes, but not all the way down to the hydroxy indane. I do not see how this reductive requirements would fit to the claimed biological applications (I have to admit that I have no time to read the whole paper …). More likely, the structure of the indane “catalysis product” is wrong (also missing some double bonds).

  7. Northern_Chemist says:

    Page 29 (of 553!) of the supporting data has the synthetic scheme. It’s complete nonsense.

  8. Hap says:

    But getting to the aminal seems sort of problematic (it doesn’t seem as if it would require a protein to make it go away, and making it selectively doesn’t seem easy) – maybe you could use a pendant ammoniumalkyl group on the aminal instead of using an aminal of an ammonium group, but that isn’t what they show.
    I don’t know if the research is bogus, but the figure should have raised eyebrows.

  9. RB Woodweird says:

    As someone who makes cyanine dyes all the time, I just facepalmed. anchor is totally on the ball. The cyanine 3 dye structure is obviously flawed (lacks the unsaturation in the carbon chain linking the two indole halves to provide the conjugation and thus the (duh) color).
    Doesn’t fill me with confidence in the rest of their structures. And if you can’t get the structures right, then how could you get the chemistry right?

  10. road says:

    everyone keeps focusing on Figure 1 — totally overlooking the fact that the syntheses of all 1600 of their metabolite conjugates are patently absurd. even if Figure 1 made perfect sense the rest of it would be total nonsense.

  11. Rhenium says:

    Comment #7 is correct, the scheme within the supplementary material is baffling…

  12. Hap says:

    I think, based on the figure (apparently there are 672 pages of SI now), that the ammonium-functionalized aminal isn’t an aminal but a dimethylammoniumethoxy group. I’m not a big fan of the diiodoindanone, though.

  13. NorthwestT says:

    Science doesn’t seem to really get chemistry. All they had to do was run it by a couple of reviewers that do get it.

  14. Sili says:

    Is this where we shout “hoax!” and insist that all of Pharma hands over all their data, and that no prescriptions be filled until we have all the numbers? Also it must be true – it was peer reviewed!
    But yeah, embarrassing. I guess they could defend calling the tertiary amine a weak base as a leaving group, but even that makes little sense.
    Did Krishna Murthy ‘do’ their crystallography?

  15. milkshake says:

    The funny thing is: even if the biologists habitually get their double bonds wrong they supposedly had someone to synthesize all this stuff for them – I would expect them to involve that person in reading the manuscript before submission.
    This reminds me of the NMR work was done by Dr. Voss of Berlin, where the spectra signals have no satelites and the chloroform solvent signal was added by hand at 7.5 ppm…

  16. J-bone says:

    This only adds fuel to my fire that peer review is fundamentally flawed. Didn’t journal editors learn their lesson about supporting info with LaClair? This is plain stupid. On the other hand, it gives me a great idea about what to do with all that bunk chemistry I thought was unpublishable.

  17. MTK says:

    I’m not as pessimistic/indignant as most of you over this. In many ways, I think this is a good example of the scientific process working.
    The peer review system isn’t perfect. It’s pretty good and it’s better than no review system, but unless we all want to be formally reviewing papers all the time, things are going to slip through.
    Upon publication the paper can then be informally reviewed by the scientific community as a whole. Questions were then raised, additional information requested and scrutinized, and eventually someone will have to repeat the work. If it’s unrepeatable or unusable, the work is essentially deemed worthless.
    So yeah, it may have wasted your time in reading it and I feel bad for anyone who has started a line of research based on this paper, but the reality is the flaws have been pointed out in a matter of months, so little real harm has been done here.
    Certainly much less than in other cases.

  18. Moody Blue says:

    Well I must say this is not a case of fraud or fabrication, but some serious errors in chemical structure drawings – still present even in the updated SI info.
    1) As pointed before the amine linker is an ethanolamine type and not an aminal.
    2) Histidine is attached to the indanone via its -NH2 and NOT via its -CO2H as shown in the MAIN article.
    3) The error(s) in the SI: histidine N is linked to 3-position of the indanone (step 8 as they call it; p17). It should be N attached to C-2 (ie alpha to the ketone). As a consequence of this mistake, they made another one – attached the NH2 of the ethanolamine to C-2 (step 9; p18).
    4) However, they have it all correct in fig S1 A (p 46) in my reckoning.
    Also, enzymatic reduction of indenones to indanols (presumably via indanones) is known. see for eg: doi:10.1016/S0957-4166(98)00128-1
    Therefore the “catalysis product” as drawn would seem correct.

  19. S Silverstein says:

    Perhaps a chemistry/biology analog of this tool was used to write the paper …
    Seriously, though – there was an editorial in today’s Wall Street Journal pondering if we’ve reached the point of societal incompetence (“Our Incompetent Civilization”). Perhaps we’re getting there. I, a nonchemist, would have at least ensured the paper was reviewed by a chemist with the appropriate background. Stating the obvious, it’s scientific common sense.

  20. KinaseNerd says:

    OK, with Moody Blue’s help and my initial comment (#6) we have now figured out the cleavage step. With the correction to the overall structure (ethanolamine linker & corrected amide bond & corrected dye) this makes a lot more sense. Having now also reviewed the updated SI, I think that the synthetic procedures are feasible. It looks pretty much like the work of a biochemistry group being dumped into synthetic chemistry. All those mM-stuff in the descriptions are pretty much how biochemist like to write down what they do. In addition, I think that a lot of the materials (aka metabolites) described are actually commercial from various sources so that I would raise doubts that all this stuff was really synthesized. This is however not really essential for this paper.
    In conclusion, it is astonishing (to say the least) that this manuscript made its way past the reviewers (and editors). But I do not see major hints for what I would call fraud. It’s just an example for editors and reviewers who did not do there job and should therefor in future spent there time with doing something different.

  21. J-bone says:

    HA HA, SciGen! I haven’t seen that since my 2nd year of grad school, I forgot about it! I might just try to put together a chemistry paper and see if I can slip it by somebody.

  22. ian says:

    As Bond said in Tomorrow Never Dies, ‘They’ll print anything these days’. Perhaps a more Juvenal movie-related paraphrase might be ‘Who reviews the reviewers’?

  23. Ale Gore says:

    And everyone agrees that all those Global Warming…er..I mean Climate Change manuscripts were rigorously reviewed?

  24. cientificorojo says:

    Errors in the figures are small potatoes. In the SI, the authors pretend to have regioselectively attached their fluorophores in all of their metabolites. It’s absurd, really.

  25. Anonymous says:

    The chemistry in this paper is nuts. The original Figure S1 has at least 5 or 6 impossible reactions or implausible structures, many of which have been corrected (my favorite: the divalent hydrogen that appears in one compound for no apparent reason; this makes it into the new version of the supp. info as well).
    But the bigger problem is that the concept of the reactome array **makes no sense**. The authors assume a bunch of things about enzymes that are totally crazy, such as (1) most enzymes catalyze the hydrolysis of their substrate, (2) most enzymes will still catalyze that hydrolysis when one half of the substrate becomes Cy3, and, most bizarre, (3) cobalt is some sort of univeral affinity ligand for enzymes, such that a cobalt bead will capture any enzyme from a lysate.
    Do these assumptions seem reasonable to anyone? They seem to me like the kind of mistakes that flunk a graduate student in his candidacy exam. It is interesting that the chemical errors in this paper have been noted by so many, but these larger conceptual problems are completely ignored. Am I missing something? Maybe we don’t require that an idea make any sense, just that the structures are right?

  26. cookingwithsolvents says:

    i stopped flipping through the SI when I hit the structure on pg 27 that has parts with the atom labels *upside down*
    I don’t know enough about the biology to comment on the concepts. The figures certainly are sloppy/wrong/weird enough, though.

  27. Hmmm says:

    It’s the everyday achievements that I admire: 1,3-dichloropentane: pentane + chlorine. Just like we are taught in week one of organic chemistry as being impossible to control.
    1679 compounds were purchased or synthesized and all were specifically attached by complex reactions and characterized. There are 18 authors and if 16 participated in the chemistry that requires that on average each one made and characterized 100 conjugates and whatever precursors were necessary. All this while someone was having to express all those proteins with His tags that love cobalt. And someone else is detecting fluorescence at the array where the item that is released is what is giving the signal. No entropy problems in that lab.

  28. Arjun says:

    Whether or not there is fraud involved, this a major embarrassment for Science. In fact, I think things would look better for them if it turns out that there was actual fraud, since at least then somebody was trying to fool them.
    If, instead, it turns out that they simply screwed up and let obviously shoddy work slip through, it’s a major blow to their position in the scientific publishing pecking order.

  29. Jason says:

    Is El Naschie one of the authors?

  30. Hmmm says:

    Arjun thinks that it would be better for Science if there is fraud involved. Fraudsters don’t provide glaring evidence against themselves. The only deception is Science’s appearance that it indulges in appropriately supervised peer review. When confronted with criticism, the editor states that the paper wasn’t reviewed by chemists (who then?). This demonstrates that Science needs an editorial overhaul. The reactome array has served the purpose of demonstrating the need for some scrutiny of the process of publication at Science, just as the reduction of sodium hydride has done for JACS.

  31. Javaslinger says:

    I recently defended my ORP in front of my committee and while being grilled on enzyme mechanisms, one of my committee members remarked that biochemists are really just very, very good organic chemists…. This evoked a chuckle from the strict organic chemists in my committee….

  32. J-bone says:

    “one of my committee members remarked that biochemists are really just very, very good organic chemists”
    If “very, very good” means not knowing why you’re using certain reagents (other than “that’s what the protocol says”) and assuming that you’re doing a good job because an enzyme produced 99% pure product for you, then yes, he’s right.
    Of course, his whole statement could be corrected by replacing “biochemists” with “enzymes”

  33. triisynthon says:

    At first I thought this paper was an outright fraud. After the revised SI I’m still not sure. What it really all hinges on is their magical “iodinase” enzyme, which will install 2 labile iodine atoms onto ANY substrate. The enzyme “will be reported later.” At least there is a hint of logic to using a high-energy reagent to render any molecule reactive (the 10-methyl group on 2 bromodecane!!??). But someone has to ask the question, where did those 1600 MS values come from? That’s got to be fabricated, or at least they blew up the baseline and found rubble with the calculated mass.
    This is what happens when biologists play with chemists toys. I also think it’s a product of too-much cross disciplinary work among too many group. Everyone thought the other guy knew what he or she was doing. But no one did…

  34. Hap says:

    I think a lot of their functionality is already there (they use a 10-hydroxydecanoic acid, for example – I think the actual sources of the functionalized substrates are listed). I don’t know about some of their selectivities, let alone the bio, but I don’t think the functionalization is pulled from one of their orifices.

  35. trisynthon says:

    You guys can debate the feasibility of the one cleavage reaction they show, but remember they are invoking 1600 similar cleavage reactions on 1600 structurally distinct substrates, using presumably different enzymes and mechanisms. Just take the 2nd one, caprolactone. Asides from how they got the iodine there in the first place, what enzymatic reaction will cleave that bond? Where’s that reasonable arrow push?
    More to the point, where is the proof-of-concept experiment? Where did they take a model metabolite, subject it to their method, and show cleavage and fluorescence with a known enzyme?
    I don’t doubt they actually did what they say. They probably did expose 1600 compounds to this iodinase enzyme and a dye and histidine. They probably did put that witches brew onto slides and the slides probably did look cool after cell lysates were added. It’s fun and cool, but it’s not science.

  36. road says:

    @trisynthon
    you nailed it. perfectly stated. thank you.

  37. trisynthon says:

    Road-
    Just following up on your initial insight, my friend.

  38. Oy Vey says:

    I agree with commenter #25. Even if we disregard the absurdity of their synthetic schemes (and these routes are truly absurd), the whole concept is fundamentally flawed. In addition to the improbability of enzymes catalyzing the reactions shown, their assay cannot work. Supposedly, the fluorescence of the dye-containing substrate is quenched by being bound to a cobalt containing surface (bound to a glass slide or nanoparticle) and an enzymatic reaction releases this quenching by disrupting this interaction. However, now the dye will diffuse into solution. So how do the spots light up? A spatially addressed micro-array is pretty useless if the signal diffuses away. This is very embarrassing for Science. A fundamental failure of their review process.

  39. Mtweedle says:

    Capturing an enzyme would indeed occur if during the enzymatic process, the Co(II) became Co(III) due to inner sphere electron transfer through a bond between the enzyme and the Co(II). The difference in ligand exchange rates (labililty) of these two ions is on the order of 10E12, with Co(II) highly labile and Co(III) highly inert.

  40. petros says:

    and another story about a questionable (non-chemical) paper in Science
    http://www.guardian.co.uk/lifeandstyle/2010/jan/06/chronic-fatigue-syndrome-xmrv-virus

  41. Richard says:

    The additional material at “http://www.bangor.ac.uk/biology/staffdetails1.php?ldapfilter=uniqueidentifier=025123&ldapou=people&personid=025123” is strange. File “Additional Material (Beloqui et al., Science 326, 252-257 (2009)).pdf” describes details about the synthesis and purification of the coupled substrates. It has 330 times a piece of text that includes the phrase “The folates were then eluted from the column using isocratic program of 88% A and 12% B in 30 min at a flow rate of 3 ul/min. Mobile phase A consisted …”. This text and ist surroundings are identical to a piece of text in a paper by Chang and Gage published in Nature and Science (2003) 1:32-36, also available as a .doc document via the web. The Chang and Gage paper is about tetrahydrofolate metabolites in Arabidopsis. “Elution of folates ..” does not make sense in additional material for the reactome array paper since there are no folates in these syntheses. The authors seem to have used the CTRL(V) keys too often.

  42. trisynthon says:

    Nice catch Richard! The copied text is actually appropriate the first time they use it (isolation of 10-formyltetrahydrofolate), but then it’s used for every different kind of molecule. Maybe they really did use that method for the folate prep. But for the remaining preps, one must ask did they record the procedure wrong, or is it all just made up?
    Another interesting bit from the revised SI. They got some flack for coupling an amine and a sulfonic acid without activiation. So in the revised SI they cite two methods for this conversion. The first is from the journal Pigments & Dyes, and it is a plausible method of forming a sulfonyl chloride in situ by treatment with POCl3. So far so good. The second method is from the Journal of Nuclear Medicine, in which dye localization to lymph fluid is studied. The authors, presumably non-chemists, observe a correlation between the protein-binding propensity of dyes and the number of sulfonate groups they contain. And so they invoke sulfonamide formation with lysines as a possible explanation, with no data, no controls, no reagent or mechanism, just a speculation. The fact that someone would cite this paper as a method of synthesis is in itself disturbing. But even more incredible is that they actually found this paper through a literature search, read it, and realized it supported their shaky assertions. Truly a little knowledge is a dangerous thing.
    Which brings a final thought. Journal quality often correlates with the reliability of methods therein. So for those young organic chemists out there, if the only precedence for a desired transformation is the Journal of Nuclear Medicine, you may want to rethink your strategy.

  43. trisynthon says:

    Nice catch Richard! The copied text is actually appropriate the first time they use it (isolation of 10-formyltetrahydrofolate), but then it’s used for every different kind of molecule. Maybe they really did use that method for the folate prep. But for the remaining preps, one must ask did they record the procedure wrong, or is it all just made up?
    Another interesting bit from the revised SI. They got some flack for coupling an amine and a sulfonic acid without activiation. So in the revised SI they cite two methods for this conversion. The first is from the journal Pigments & Dyes, and it is a plausible method of forming a sulfonyl chloride in situ by treatment with POCl3. So far so good. The second method is from the Journal of Nuclear Medicine, in which dye localization to lymph fluid is studied. The authors, presumably non-chemists, observe a correlation between the protein-binding propensity of dyes and the number of sulfonate groups they contain. And so they invoke sulfonamide formation with lysines as a possible explanation, with no data, no controls, no reagent or mechanism, just a speculation. The fact that someone would cite this paper as a method of synthesis is in itself disturbing. But even more incredible is that they actually found this paper through a literature search, read it, and realized it supported their shaky assertions. Truly a little knowledge is a dangerous thing.
    Which brings a final thought. Journal quality often correlates with the reliability of methods therein. So for those young organic chemists out there, if the only precedence for a desired transformation is the Journal of Nuclear Medicine, you may want to rethink your strategy.

  44. TFox says:

    Just an update on this paper: the institutions’ review committee is recommending a retraction. http://www.nature.com/news/2010/100728/full/466540b.html

  45. null says:

    Good post. I study something more challenging on completely different blogs everyday. It can all the time be stimulating to learn content material from different writers and follow a little something from their store. I’d want to make use of some with the content on my blog whether or not you don’t mind. Natually I’ll provide you with a link on your internet blog. Thanks for sharing.

Comments are closed.