I’ve requests from time to time to share some tips about dealing with the scientific literature. Which is indeed a problem, on several levels, and has been for a long time now. So here’s what I have to offer, and I hope it helps.
A large-scale problem is what journals to even look at. I use an RSS reader to try to keep up, although I will freely admit that I have not been as diligent with the general literature during some of this pandemic period. But I’ve found that RSS is congenial to the way I scan things (it may not work the same for everyone). There are a lot of sites that will handle feeds for you – I use one called NewsBlur, so you might start there if you’re looking to get going. What I’m trying to do with RSS is to replicate the older experience of flipping through the paper journals as they came in, because I found that I came across interesting things that way that I might not have seen otherwise. A similar experience that I’ve never found a way to recapitulate in the digital world was the way that you’d come across other papers while looking up other references in the bound journals, but that one may be gone forever.
I would recommend having some of the big, general ones in there, such as JACS and Angewandte Chemie and Nature Chemistry for chemistry. It’s true that many of the papers will be on topics that you know little about (or might care for even less), but you’ll see some things you wouldn’t otherwise see, while also getting quality work that’s relevant to your own interests. Science and Nature work at an even higher level, of course, but I have a general scientific curiosity, so they work well for me. I have gone back and forth on PNAS over the years. There are some really interesting papers in there, but it’s a firehose, and the signal/noise for me is not as good. Past the higher-level journals, you’ll naturally want to pick out the best ones in your own specialty, be that medicinal chemistry, organic synthesis, chemical biology or what have you.
My own RSS feed (for now) is ACS Chemical Biology, ACS Med. Chem. Letters, Bioorganic Med. Chem. Letters, Cell, Cell Chemical Biology, Current Opinion in Chemical Biology, J. Med. Chem., JACS, Nature, Nature Biotechnology, Nature Chemical Biology, Nature Chemistry, Nature Reviews Drug Discovery, Organic Letters, Science, Science Translational Medicine, and Angewandte Chemie. Other journals go in on a trial basis; I see how the feed is working out and make a call after a while. Are there interesting papers? Do I have to scroll past too much other stuff? Are there just too many papers to deal with? Right now, BMCL is on the bubble, I have to say. I like the short reviews that they publish, but a lot of the regular papers are of reference interest only, if that. At any rate, I just click on a title and scroll backwards through the feed, looking at titles and graphical abstracts. I miss things. Everyone misses things; don’t beat yourself up about it. But if I see something interesting, I call up the paper itself.
Now, that’s for the general scientific literature. For specific topics, I tend to set up Pubmed alerts, although the rise of preprint servers is shifting the timeliness of those around a bit. The NIH has noticed this as well, and has started adding preprint-server versions of work funded by the NIH into the results. But if you’re in a fast-moving field, you’re going to have to do regular searches in BiorXiv and/or MedrXiv as well to make sure you haven’t missed something that’s potentially big. The patent literature is another matter, of course. Many libraries or companies have a mechanism set up to do regular patent searches and alerts, and you’ll definitely want to take advantage of that. But you can always go hit the WIPO and USPTO sites yourself and search for applications on a regular basis, and I’ve definitely been on projects where that’s been worthwhile.
Here’s how I deal with any individual paper, then. The first thing is to keep in mind what it is you’re trying to get out of it. Is this a topic you’re new to and looking to get your bearings? Or is it something that you’re reading to get a specific practical suggestion about something you already know fairly well – how did these folks run that assay, how did they get that catalytic reaction to work? Are you perhaps just adding more knowledge to some field you’ve been reading about for a while, as in “You know, I don’t actually know so much about the cellular turnover of this protein target – I should have a look”. You’ll approach these in different ways – if it’s a new field for you, you’ll want to read the author’s intro and scene-setting, and maybe go check out some of their review references. If it’s a field you know well, you can scan past that stuff as boilerplate and go straight to the conclusions, to just tease out the one thing that’s different or interesting about this work that you might want to know. But one thing that I only do rarely is actually read the whole paper from start to finish. Ars longa, vita brevis, and life is indeed too short to plow through every paper every time. It can’t be done. So in general, what I tend to do is read the abstract while having a glance the graphical abstract, if there is one – unfortunately, too many of the latter are of poor quality and don’t explain much. Then I tend to jump to the conclusions of the paper, flipping to the end to see what the authors, at least, think that they’ve been able to show, and whether it is indeed of any interest to me. I’ll work backwards up through the paper to see how specific interesting claims were established, if need be, but one thing that I almost never, ever do is read a whole Methods/Experimental section.
Then you have to decide if you’re going to take their word for it or not. Most of the time, I do, as most of us do most of the time in science, but if it’s a particularly unusual result (or one that’s going to be directly relevant to my own work) I will dig in and see if I think the paper has really demonstrated its conclusions. And here’s where you hit the bimodal distribution that’s common to the scientific literature. There’s a lot of crap in the low end of the scientific publishing world, that’s for sure – but there’s also quite a bit of it in the very high-end journals, too. The problem is, that latter crap isn’t always as apparent as it is in the lousy journals. The high-end crap consists, all too often, of experiments that look exciting but can’t actually be reproduced, or of conclusions that could have been invalidated by some other experiments that just didn’t get run. You don’t get as much of that in the respectable-but-mid-tier journals like (say) the Journal of Organic Chemistry. Pretty much all of that is exactly what it says it is; it’s just that you may or may not be interested to start with. You don’t run into as many reproducibility problems with a JOC-type journal for the same reason that you don’t run into many counterfeit $10 bills. So it’s the high-end papers that I tend to subject to more scrutiny, because they tend to make more unusual and significant claims.
Now, once I’ve looked over a paper, what do I do about it? I use a freeware literature manager called Zotero, and I’ve been using it for ten to fifteen years now. I can recommend it; it does everything I want a program like that to do, and it has saved me a lot of time more times than I can count. That said, there are other similar programs, and I think it’s more important that you use one of them than it is which exact one you use. If you’re logged in through some sort of institutional account, you can automatically grab the PDF of the paper you’re reading and have it stored for later use. I have a whole list of folders for topics of interest, and when I come across a paper that I know I may need to refer to in the future, I chuck it in there without a second thought. Every so often I go in and prune things, getting rid of topics that are no longer so relevant, or clipping off the older papers in a folder that might have been largely superseded. But I still have a lot of stuff in there, and having it organized by topic, in a way that I can search every aspect of the text and bibliographic information, is crucial. Use a literature manager program. Get used to using it, keep adding to it, and it will get more valuable to you every month. In Zotero I sometimes write note documents to my (future) self about why a particular paper looked important, or conclusions that hit me while I was reading it, and those have saved my rear on several occasions as well. It’s important, I find, to write these things down when they occur to you and while they’re still in some organized context in your thoughts. You cannot keep all of this organized in your head without such help, not over a period of years. No one can.
If I’m getting into a completely new area and have to learn it from the ground up, I will sometimes sit down and sort of write a blog post to myself about it. That’s to try to build up a coherent picture of what’s going on, with literature citations, and occasionally my own drawings get added. To be sure, I can count the number of times I’ve done this over the last thirty years on the fingers of my two hands, but if you’re diving into a big field for the first time and you know that you’re going to be spending a lot of time in it (or if you know that it’ll do you good to have it more settled in your mind in general, because of all the other topics it touches), then it can be worthwhile. I realize, though, that writing things down like this is probably my own response because writing and composition come relatively quickly to me, and that others may prefer a different approach. You might screen-grab some particularly good illustrations of key points from other papers, for example – whatever works!
Overall, I would say that you have to come to terms with the reality of the literature. You’re not going to be able to read everything you (ideally) should. You’re not going to be able to catch everything of interest as it appears. Neither are you necessarily going to understand everything you read the first time you go through it – you may need background that you don’t have yet, or for some reason the paper is going into your brain from the wrong angle that day. I’ve had that happen, where a repeat visit makes things clear that didn’t click the first time I tried it. I think it’s important to realize these things so you don’t get demoralized (or outright paralyzed) by feelings of inadequacy when trying to deal with the ceaseless torrent of scientific knowledge. No one is truly up to the challenge if you measure yourself against the unattainable ideal of seeing it all, reading it all, and understanding it all. But we can all do a better job – more comprehensive, more efficient with our time – than we do.