Skip to main content
Menu

The Scientific Literature

Dealing With the Literature

I’ve requests from time to time to share some tips about dealing with the scientific literature. Which is indeed a problem, on several levels, and has been for a long time now. So here’s what I have to offer, and I hope it helps.

A large-scale problem is what journals to even look at. I use an RSS reader to try to keep up, although I will freely admit that I have not been as diligent with the general literature during some of this pandemic period. But I’ve found that RSS is congenial to the way I scan things (it may not work the same for everyone). There are a lot of sites that will handle feeds for you – I use one called NewsBlur, so you might start there if you’re looking to get going. What I’m trying to do with RSS is to replicate the older experience of flipping through the paper journals as they came in, because I found that I came across interesting things that way that I might not have seen otherwise. A similar experience that I’ve never found a way to recapitulate in the digital world was the way that you’d come across other papers while looking up other references in the bound journals, but that one may be gone forever.

I would recommend having some of the big, general ones in there, such as JACS and Angewandte Chemie and Nature Chemistry for chemistry. It’s true that many of the papers will be on topics that you know little about (or might care for even less), but you’ll see some things you wouldn’t otherwise see, while also getting quality work that’s relevant to your own interests. Science and Nature work at an even higher level, of course, but I have a general scientific curiosity, so they work well for me. I have gone back and forth on PNAS over the years. There are some really interesting papers in there, but it’s a firehose, and the signal/noise for me is not as good. Past the higher-level journals, you’ll naturally want to pick out the best ones in your own specialty, be that medicinal chemistry, organic synthesis, chemical biology or what have you.

My own RSS feed (for now) is ACS Chemical Biology, ACS Med. Chem. Letters, Bioorganic Med. Chem. Letters, Cell, Cell Chemical Biology, Current Opinion in Chemical Biology, J. Med. Chem., JACS, Nature, Nature Biotechnology, Nature Chemical Biology, Nature Chemistry, Nature Reviews Drug Discovery, Organic Letters, Science, Science Translational Medicine, and Angewandte Chemie. Other journals go in on a trial basis; I see how the feed is working out and make a call after a while. Are there interesting papers? Do I have to scroll past too much other stuff? Are there just too many papers to deal with? Right now, BMCL is on the bubble, I have to say. I like the short reviews that they publish, but a lot of the regular papers are of reference interest only, if that. At any rate, I just click on a title and scroll backwards through the feed, looking at titles and graphical abstracts. I miss things. Everyone misses things; don’t beat yourself up about it. But if I see something interesting, I call up the paper itself.

Now, that’s for the general scientific literature. For specific topics, I tend to set up Pubmed alerts, although the rise of preprint servers is shifting the timeliness of those around a bit. The NIH has noticed this as well, and has started adding preprint-server versions of work funded by the NIH into the results. But if you’re in a fast-moving field, you’re going to have to do regular searches in BiorXiv and/or MedrXiv as well to make sure you haven’t missed something that’s potentially big. The patent literature is another matter, of course. Many libraries or companies have a mechanism set up to do regular patent searches and alerts, and you’ll definitely want to take advantage of that. But you can always go hit the WIPO and USPTO sites yourself and search for applications on a regular basis, and I’ve definitely been on projects where that’s been worthwhile.

Here’s how I deal with any individual paper, then. The first thing is to keep in mind what it is you’re trying to get out of it. Is this a topic you’re new to and looking to get your bearings? Or is it something that you’re reading to get a specific practical suggestion about something you already know fairly well – how did these folks run that assay, how did they get that catalytic reaction to work? Are you perhaps just adding more knowledge to some field you’ve been reading about for a while, as in “You know, I don’t actually know so much about the cellular turnover of this protein target – I should have a look”. You’ll approach these in different ways – if it’s a new field for you, you’ll want to read the author’s intro and scene-setting, and maybe go check out some of their review references. If it’s a field you know well, you can scan past that stuff as boilerplate and go straight to the conclusions, to just tease out the one thing that’s different or interesting about this work that you might want to know. But one thing that I only do rarely is actually read the whole paper from start to finish. Ars longa, vita brevis, and life is indeed too short to plow through every paper every time. It can’t be done. So in general, what I tend to do is read the abstract while having a glance the graphical abstract, if there is one – unfortunately, too many of the latter are of poor quality and don’t explain much. Then I tend to jump to the conclusions of the paper, flipping to the end to see what the authors, at least, think that they’ve been able to show, and whether it is indeed of any interest to me. I’ll work backwards up through the paper to see how specific interesting claims were established, if need be, but one thing that I almost never, ever do is read a whole Methods/Experimental section.

Then you have to decide if you’re going to take their word for it or not. Most of the time, I do, as most of us do most of the time in science, but if it’s a particularly unusual result (or one that’s going to be directly relevant to my own work) I will dig in and see if I think the paper has really demonstrated its conclusions. And here’s where you hit the bimodal distribution that’s common to the scientific literature. There’s a lot of crap in the low end of the scientific publishing world, that’s for sure – but there’s also quite a bit of it in the very high-end journals, too. The problem is, that latter crap isn’t always as apparent as it is in the lousy journals. The high-end crap consists, all too often, of experiments that look exciting but can’t actually be reproduced, or of conclusions that could have been invalidated by some other experiments that just didn’t get run. You don’t get as much of that in the respectable-but-mid-tier journals like (say) the Journal of Organic Chemistry. Pretty much all of that is exactly what it says it is; it’s just that you may or may not be interested to start with. You don’t run into as many reproducibility problems with a JOC-type journal for the same reason that you don’t run into many counterfeit $10 bills. So it’s the high-end papers that I tend to subject to more scrutiny, because they tend to make more unusual and significant claims.

Now, once I’ve looked over a paper, what do I do about it? I use a freeware literature manager called Zotero, and I’ve been using it for ten to fifteen years now. I can recommend it; it does everything I want a program like that to do, and it has saved me a lot of time more times than I can count. That said, there are other similar programs, and I think it’s more important that you use one of them than it is which exact one you use. If you’re logged in through some sort of institutional account, you can automatically grab the PDF of the paper you’re reading and have it stored for later use. I have a whole list of folders for topics of interest, and when I come across a paper that I know I may need to refer to in the future, I chuck it in there without a second thought. Every so often I go in and prune things, getting rid of topics that are no longer so relevant, or clipping off the older papers in a folder that might have been largely superseded. But I still have a lot of stuff in there, and having it organized by topic, in a way that I can search every aspect of the text and bibliographic information, is crucial. Use a literature manager program. Get used to using it, keep adding to it, and it will get more valuable to you every month. In Zotero I sometimes write note documents to my (future) self about why a particular paper looked important, or conclusions that hit me while I was reading it, and those have saved my rear on several occasions as well. It’s important, I find, to write these things down when they occur to you and while they’re still in some organized context in your thoughts. You cannot keep all of this organized in your head without such help, not over a period of years. No one can.

If I’m getting into a completely new area and have to learn it from the ground up, I will sometimes sit down and sort of write a blog post to myself about it. That’s to try to build up a coherent picture of what’s going on, with literature citations, and occasionally my own drawings get added. To be sure, I can count the number of times I’ve done this over the last thirty years on the fingers of my two hands, but if you’re diving into a big field for the first time and you know that you’re going to be spending a lot of time in it (or if you know that it’ll do you good to have it more settled in your mind in general, because of all the other topics it touches), then it can be worthwhile. I realize, though, that writing things down like this is probably my own response because writing and composition come relatively quickly to me, and that others may prefer a different approach. You might screen-grab some particularly good illustrations of key points from other papers, for example – whatever works!

Overall, I would say that you have to come to terms with the reality of the literature. You’re not going to be able to read everything you (ideally) should. You’re not going to be able to catch everything of interest as it appears. Neither are you necessarily going to understand everything you read the first time you go through it – you may need background that you don’t have yet, or for some reason the paper is going into your brain from the wrong angle that day. I’ve had that happen, where a repeat visit makes things clear that didn’t click the first time I tried it. I think it’s important to realize these things so you don’t get demoralized (or outright paralyzed) by feelings of inadequacy when trying to deal with the ceaseless torrent of scientific knowledge. No one is truly up to the challenge if you measure yourself against the unattainable ideal of seeing it all, reading it all, and understanding it all. But we can all do a better job – more comprehensive, more efficient with our time – than we do.

43 comments on “Dealing With the Literature”

  1. Ian Malone says:

    Another long-time Zotero user! It seems people are gradually discovering it.

    “but one thing that I almost never, ever do is read a whole Methods/Experimental section”

    Maybe this is a difference in fields, for me results are usually where the interest lies and I may need to glance at methods to interpret them. In my area discussion sometimes seems to be an opportunity to hang a lot of opinions on weak data.

    1. Mister B. says:

      I second this. As a bench chemist, I rush to the SI to assess its quality.
      I can read tables of compounds (methyl, ethyl, futile… mostly useless btw) but I do want clear data in the SI.

    2. Nat says:

      As a methods developer, this is often the only section that I read closely!

      1. Ian Malone says:

        I suppose it does depend what you’re reading for. To be across swathes of literature the way Derek is you do probably have to focus on discussions, if it’s for techniques and findings directly relevant to what you’re doing you’ll want to look a little more closely.

  2. Erwin says:

    I also miss the experience of finding something interesting as I looked for a specific article in a bound volumes. What I don’t miss is standing at a copy machine with a stack of bound volumes, each with a bookmark, feeding nickels into the machine as I copied one page at a time, pressing hard down on the spine to be sure the text close to the inner margins was legible. If you fumbled at all, the people standing in line behind you would yell at you for wasting their time.

    One of the great things about keeping your references in an electronic database like Zotero is that you can assign each one multiple tags, making it much easier to find them in the future. A paper copy could be in only one manila folder (or stack of papers on your desk) at a time, and where you put it depended on what you were thinking about last time you handled it.

  3. luysii says:

    That’s an amazingly wide net Derek. The great thing about the high tier journals (Nature, Science) is that you have a bunch of high level people reading a vast amount of stuff from all fields, and publishing only what they think is new, significant and, more importantly, credible. They’re doing a lot of your work for you.

    Agree about PNAS. In particular they’ve recently allowed papers to double from 5 to 10 pages, leading to all sorts of bloviation so you need to plow through it to find the basic points of the paper. I subscribe to keep the memory of my classmate and friend Nick Cozzarelli alive. https://en.wikipedia.org/wiki/Nicholas_R._Cozzarelli. He edited the journal for 10 years and really improved things.

    A lot of the discussion in this blog is about neurologic disease, and I’d add Neuron to your list for some hard core neuroscience (although Nature, Cell, PNAS and Science all cover it).

    The medical literature was awful when I had to read it, with results cherrypicked and conclusions which didn’t follow from the data. Hopefully it’s better now. Here are two examples — cherrypicking and missing the forest for the trees

    https://luysii.wordpress.com/2009/10/05/low-socioeconomic-status-in-the-first-5-years-of-life-doubles-your-chance-of-coronary-artery-disease-at-50-even-if-you-became-a-doc-or-why-i-hated-reading-the-medical-literature-when-i-had-to/

    1. Husky Voice says:

      “High tier” journals definitely publish “new and significant” findings, but they are most definitely NOT always credible. A lot of this is the result of the incentives in these “journals”. Just look at the Benveniste saga

      https://en.wikipedia.org/wiki/Benveniste_affair

      Click bait has been around in different forms for a long time.

  4. Bannem says:

    Not something I’ve ever had to do professionally, but a fascinating insight in how to deal with the rising tide of information that we all have to deal with these days . . .
    Oh, and kudos for using the correct word ‘specialty’, instead of the horrible neologism ‘specialism’.

  5. Tocrat says:

    Thanks Derek, a very useful insight, I wish I’d had about 30 years ago during my own graduate study days when it seemed that everybody else was across everything and I drowned trying to keep up. Is there here a specific reason that JOC itself is not on your RSS feeds list?

  6. Jose Blo says:

    It’s interesting that you (in the small molecule field) mention reading the patent literature. I also tend to read some relevant patents, but I’ve been at companies (usually the larger ones) that specifically discourage reading any patent that isn’t held by that company itself (or perhaps your own issued patents) for legal reasons. I assume that it has mostly to do with 3x damages for knowing infringement, even though I doubt my ideas of infringement would match a court’s decision more than half the time.

    I like your description of reading papers, but perhaps just because it largely matches my own (in a completely different field).

  7. Ross says:

    @Derek, @Erwin:
    > I also miss the experience of finding something interesting as I looked for a specific article in a bound volumes.

    It seems to me that this could be, sort-of, reproduced with some automation, like maybe a plugin for Zotero? My concept:

    1. User files away a PDF using Zotero.
    2. Plugin extracts the original journal issue that published that PDF.
    3. Plugin retrieves the original table of contents for that journal issue.
    4. Items from the TOC are turned into RSS items for your feed reader.

    If such a plugin appeared tomorrow, do you think it’d be useful, or just nostalgic?

    Disclaimer: I’m a software guy who loves this blog, not a research scientist of any kind

  8. Doug H MD says:

    one problem: i have read almost no solid analysis of the major published and prepublished trials, short of some letters to the editor and a few bloggers who are not well followed.

  9. Jonathan B says:

    You prompted a bit of nostalgia there. Now I am retired, I get my scientific “fix” reading other people’s recommendations, such as from you, but I have vivid memories of making sure each week as a graduate student I put some time into checking the contents list of the latest issues in the library. And then a stage of technology beyond, getting my turn with the department copy of “Current Contents” before passing it on to the next person in the list (for those in the electronic age, this was a publication consisting of facsimiles of all the contents pages of journals in a field, but of course you had to write yourself notes of which to follow up and if interesting photocopy).

    As you say, while that was a chore it did have the benefit of serendipitously discovered papers adjacent to your immediate interest – which quite often widened one’s perspective in a positive way.

  10. Phthor Quiddity says:

    Every project in my research lab originated with serendipitous finds in bound journal volumes. Stumbling across facts that lead to ideas with traction always struck me as a stochastic, irreproducible process, and I’m not sure modern “library work” affords the same entry points. That may just be superstitious.

    Podcasts can help maintain a breadth of awareness even while multitasking: Nature has a good feed, and Shortwave from NPR is lighter fare but does biology particularly well (plus one great episode on helium).

    If your interests bend that way, the American Society for Microbiology and Vincent Racaniello produce an absolute firehose of content, in a handful of “This Week in X” podcasts, typically focused on discussing recent papers or interviewing experts. If you like geeking out over fresh, cool results, preferably with a beer, this may be for you. The TWIV (Virology) podcast has traced out this coronavirus year in granular detail, and they’ve left a clear record of where the conventional wisdom among virologists has been on- or off-target about Covid-19.

    Not sure if there are chemistry equivalents, especially without the visuals that make blogs like Baran’s Open Flask so good.

  11. Jordan says:

    Some great recommendations here.

    I’ve tried Zotero in the past and found it a bit cumbersome — it seems to only want to “play nice” when you are starting a library from scratch. If you already have PDFs neatly organized into folders somewhere, you have to add them all in manually. Or am I missing something?

    1. Mister B. says:

      I did import my own library into Zotero in 2017 and it worked like a charm.
      Selected the whole package, import, done.
      Some were lost in the process, a few duplicate were found but honestly, import 3k+ papers was super easy.

  12. Not-an-epidemiologist says:

    I’m surprised nobody’s mentioned Google Scholar yet. Once you’ve been in research for a while, it’s great for automatically suggesting new recommendations based on your previous publications. (At least it is as of a fairly recent update, but it’s now working very well.) If you don’t have research career of your own (or if you are also interested in areas tangential to your own work), you can also get it to send you the recommendations determined from other researchers’ profiles. (There was a wonderful but little-known and now dead service called PubChase that used to do this very well — in my view, Google Scholar has finally achieved recommendation parity with it. And that’s a big compliment.)

    Google Scholar is great for literature searches also, as its “page rank” algorithm for papers appears to be strongly citation-weighted (it lists citation numbers on the search page — an incredibly useful feature). It’s easy to date-limit these searches to — say — the last ten years. Plus it includes preprints. I never use pubmed for literature searches any more.

    Nature and Science both have an excellent “News and Views” section that I used to read all the time, back when tea rooms had physical journals lying around. I keep meaning to digitally subscribe just for those sections — they contain short, well-written, accessible summaries of recent, broad-interest, ground-breaking work (and not just from their own journal stables), and I miss reading them.

    Also, re. the papers themselves: “Then you have to decide if you’re going to take their word for it or not. Most of the time, I do, as most of us do most of the time in science …” — everyone is different on this, but I personally find that if I do not begin reading a paper from the outset with a highly sceptical attitude, I do not take the details of the research in. By questioning everything, by the time I’ve finished I’ll either fully understand the reasoning or will have found the flaws (or both). It becomes a bit of a game — can I tear this paper down to size? — which helps bypass the deadly tedium of scientific writing.

    1. also not an epidemiologist says:

      Yes — Google Scholar is the best way to keep up to date in particular with your own field of research. It also gives you related work that may be relevant but you otherwise wouldn’t have read. Love it!

    2. E.T. says:

      I find following key Authors in the field very useful.
      Another useful feature of Scholar is finding open versions of articles that are behind paywalls.

  13. Not a scientist says:

    Molecular biology experts dont exist, or naturally COVID19 would not be a thing. Cars that cant be fixed dont exist, because mechanics exist.

    1. Anonymous says:

      Car’s that can’t be fixed are everywhere. Every seen the aftermath of a head-on collision?

  14. Druid says:

    In the ’70s when I was in entomology, the group pass around a set of printed record cards from UKCIS, an early computer search for keywords in the literature. My favourite results were for “mam-moths” and I also learned a lot about vanadium in “fly ash”.

  15. Some idiot says:

    FWIW, as a process chemist, OPR&D is a must…! 🙂

    1. Mister B. says:

      FWIW, as a non-process chemist, OPR&D is a must…! 🙂

      1. Derek Lowe says:

        Agreed, it is a very good journal for organic synthesis info. . .

  16. Project Osprey says:

    For general technical reading I would suggest ACS editors choice

    https://pubs.acs.org/editorschoice/

    Articles are open-access, diverse and notionally chosen by an experienced someone who thinks them interesting. A new paper is added each day, so you’re drip-fed content rather than hosed-down with it.

    A recent example would be “Review on Ammonia as a Potential Fuel: From Synthesis to Economics” which I’m sure would support a passable news article somewhere

  17. anon says:

    Things are much easier in fields like theoretical physics and (at least some subfields) of mathematics, where everything is posted on arxiv often months before their publication in journals. I haven’t read a new article from a journal in ages.

  18. In Vivo Veritas says:

    OK, here’s the place to ask: How do y’all pronounce “BiorXiv”? I’ve literally seen hallway arguments over this. Can we get a consensus ruling here?

    1. Mel says:

      Bio Archive. How else do folks say it?

      1. In Vivo Veritas says:

        “Bio-Rik-Siv”, “Bio-Ex-Ive”. I’ve even heard the more obvious arXiv referred to as “Arks-Iv”.

        1. Marko says:

          BIORelaXatIVe : a preprint server with very loose bowels.

  19. lfert says:

    I do miss feet up on my desk, leaning back in my chair paging through printed journals. I would always keep a look out for an interesting synthesis scheme. Just browsing through the table of contents, you miss so much. What I don’t miss is the copy machine!

  20. Mel says:

    I’m curious whether readers of this column are finding value in semantic search engines, e.g., Embase. I get enthused when I hear the capabilities described, but end up reverting to my usual approaches – like those described here – as the semantic engines never turn up more than I’m finding anyway.

  21. DH says:

    Thanks, Derek, for the Zotero recommendation. I just downloaded it and indexed a bunch of PDFs on my local disk. Zotero supports fulltext search, but I cannot figure out how to get it to open the document with the search text highlighted so I don’t have to open the PDF myself and do a separate search within it. Does anyone here know whether this is possible in Zotero, either natively or via a plugin?

  22. MTK says:

    I use Google Scholar Alerts for specific topics, although it doesn’t cover pre-prints, but every few days sends an email with articles based on whatever search terms you choose.

    As a reference manager I use Mendeley which like Zotero is also free. It’s quite intuitive. The only real downside is that the Word plug in is/was awful. They may have fixed it since the last time I used it which was years ago.

  23. Daniel says:

    Can you please provide the actual rss feed links so we can easily subscribe? Thanks!

  24. Red Fiona says:

    The pubcrawler web search is an absolute life-saver if you’re a biologist and need to keep up to date with what’s new on PubMed. (www.pubcrawler.gen.tcd.ie if anyone interested)

  25. Jackie says:

    I wish the QAnon types and anti-vaxxers would at least TRY to filter out low-end crap

  26. TPO says:

    Like you, RSS is central to my literature awareness methods. You didn’t mention how you process PubMed alerts, but I also do this through RSS, and I have hundreds of searches converted to feeds, some on authors, some on keywords, even a few on journal titles if I can’t manage to find the RSS feed for a particular journal. A few of these need pruning because of high volume / low S/N, but the “Mark All As Read” shortcut is always there (I use Feedly with the Reeder app on Apple devices).

    I’ve tried many reference managers over the years, and the most important thing for me is efficiency of capture, going from an RSS item to capturing it in the reference manager. I’ve now settled on ReadCube Papers, and the browser plugin for Chrome/Edge/Firefox makes all the difference in the world. One keyboard shortcut on an RSS item (literally the letter b) opens the article page in Chrome, and one click on the “Add to Library” button that is generated by the Readcube plug-in and the reference is captured. There’s no copy/paste or fiddling around with menus that I had to do when using EndNote.

    There’s optional work left to do in Readcube, including reference tagging or filing away in folders. I use tags for topics and folders for projects (one for each manuscript or grant).

    Readcube also makes it easy in most cases to capture papers referenced in a manuscript. If it doesn’t automatically create links to a reference, it’s easy to select the text of the reference and use a contextual menu command to send it to a Google search, and from there it’s almost always a single click to an article with its own “Add to Library” button.

  27. pfizer joe says:

    who owns the pdf you download? you or the company you work for?

  28. Jim Hartley says:

    I switched into cancer biology 8 years ago and evolved a method that works for me. I have dozens of alerts set up, through PubMed and bioRxiv. I download pdfs and take notes from them, mostly copy and paste, into a text document, highlighting phrases and noting important concepts. I find papers I’ve read by searching the text document, and I attach identifiers (either PubMed IDs or random character strings) from the text document to each pdf file. I started in Word but it became painfully slow as my document got large, so I switched to Pages. The document is now >800K words. Whenever I see something new and intriguing I search my big document to see what I have read about it previously. Big help.

  29. YohananW says:

    medical search links – a retired librarian’s old 2¢

    * The NIH entrez databases
    https://www.ncbi.nlm.nih.gov/search/

    * MyNCBI storage of queries, collections alerts… https://www.ncbi.nlm.nih.gov/account/

    * MeSH Database, NLM’s keywords search PubMed with the Medical Subject Headings and subheadings https://www.ncbi.nlm.nih.gov/mesh/

    RTM – Read The Manual = help support pages. e.g. https://scholar.google.com/intl/en/scholar/help.html

  30. Chemist says:

    For all of you who travel a lot and Play with their phone while doing so, i would recommend the „researcher“ app.
    You can choose the journals you want to follow and set up some filters with keywords and author names for a more refined search. You can also link your OrcID and your Mendeley Account. With this, you can flag interesting Papers which are directly transfered to your library

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.

This site uses Akismet to reduce spam. Learn how your comment data is processed.