Skip to main content
Menu

The Scientific Literature

PNAS: Read It, or Not?

You know, every time I point out a paper from PNAS, there are always a few comments to the effect of “Why do you bother reading that garbage heap, anyway?” Since I keep citing papers from the journal, it’s obvious that I disagree, but I suppose I should take a minute to explain why.
The reason people are down on PNAS is the way that members of the National Academy can, if they choose, sort of jam things into the journal through a side entrance. Here are all the details. The unusual thing about the journal is the existence of “Track I”. Basically, a member of the NAS can publish up to four of their own papers per year. Each of these have to be submitted with the comments of two qualified referees, but the author gets to pick them. So a reasonable member should be able to get any sort of interesting or at least non-insane paper in there, by judicious choice of colleagues for review. Members can also pass along up to two papers a year by others in their field, with a similar review process (Track III). Some NAS members take full advantage of these privileges, and some hardly ever do, even (so I’m told, in some cases) for their own papers.
It’s a lot less rigorous than the open (Track II) submissions, that’s for sure. For those, you’re supposed to name three editorial board members, three NAS members, and five external referees, and the editorial board can still do whatever it wants with your paper or with the lists you’ve sent. (To be sure, they can also reject those direct-submission papers from members, although no figures are available on how often that happens). Two thirds of the Track II submissions are rejected before being sent out for review at all.
But hold on: according to the journal, 80% of the submissions are via Track II, but those make up only 40% of the published contents. Doing the math, that means that the most of Track I and Track III submissions have to get in. Assume 80 Track II manuscripts and 20 of the others. Rejecting two-thirds of the first group will give you about 27 papers to send out for review. If you’ve accepted all 20 of the others, that means that about half of those 27 will have to get canned during the later review process, to make that 40/60 proportion come out right. So the overall acceptance rate for open submissions has to be, at most, 16%.
But if you ditch some of the 20 member-track papers, you have to come down even harder on the open submissions, of course. If you only (only!) take 75% of the member submissions, that gives you 15 manuscripts. Now you have to reject not half of the open submission papers that made the first cut, but 63% of them, to knock it down to ten published Track II submissions. So with an acceptance rate of 75% for member submissions, it has to be about 12% for everyone else. And so on.
So much for the numbers – it’s clear that NAS members must put a lot of things of their own (or from their friends) into PNAS. The real question is: what does this do to the quality of the journal? As far as I can see, it’s still a very interesting read, and definitely cannot be safely ignored. And the publication routes are out on the table: if you want to keep score and adjust your perceptions accordingly, the Track I papers are identified as “Contributed by” the member, and the Track IIIs are “Communicated by”. I think, myself, that the advantage of letting members publish unusual or possibly controversial work outweighs the temptation to fill the journal with junk.

24 comments on “PNAS: Read It, or Not?”

  1. Hap says:

    I would figure that if enough bad papers get through, PNAS’s review policies would be modified to change that. If no one reads the journal or cares about it, the NAS is weakened a little, and one would think they would try to avoid that.
    Knowing which papers are which is also helpful – supposedly stringent review policies don’t help if they can be circumvented or ignored (ACIEE?). If the type of review a paper receives can be readily assessed by readers, they can also easily decide whether the paper is worth reading or following. The lack of ability to know the subsequent reproducibility of results (unless the paper is retracted or corrected) affects all journals, and makes it hard to determine if Type I or III review yields less reproducible papers than Type II review, or the review processes of other journals.

  2. Bob Hawkins says:

    You have to figure that the “Proceedings of” a given body will reflect the prejudices of that body and its members. Otherwise it would be just another journal.
    As long as the reader can identify which track a paper traveled, fine. Let a hundred flowers bloom.

  3. qetzal says:

    You can tell which papers are which. Track I papers say “Contributed by …” directly below the author affiliations on the first page. Track III papers say “Communicated by …” instead. Track II papers say “This article is a PNAS Direct Submission” in the footnotes.
    Of course, you have to have the full paper to know that. I don’t think you can tell just by looking at PubMed or Medline.
    I think PNAS is a worthwhile journal. Even if NAS members can get things published with much less rigorous review, they are still NAS members. In theory, they have a track record of top quality research in their field.
    Besides, the normal peer review process doesn’t always do such a stellar job screening out the crap.

  4. Alan Lehman says:

    There ought to be a fast way to get hard numbers regarding the Track I, II and III submissions. Sadly my first attempt to do it seems to have produced ambiguous results. Using the online PNAS search engine, I limited my search to 2007 and then searched for the phrase
    For Track I, I searched for “Contributed by” which got 811 hits.
    “This article is a PNAS Direct Submission” got 1941 hits and
    for Track III, I searched for “Communicated by” which got 746 hits for a total of 3498.
    I am in no way implying that these numbers are valid especially since searching for “www.pnas.org” (which ought to be in every publication) only got 2596 hits and by my back of the hand calculations, PNAS publishes 85 papers a week or about 4500 papers per year. I am certain, however, that better minds than my own could come up with a more authoritative set of search strings and get the actual numbers.

  5. Chrispy says:

    PNAS is one of my very favorites, just precisely because there is some crazy stuff which gets in there. Anyone see the article on cattle and deer aligning themselves with magnetic north in the last issue? I’m not saying that it is wrong, mind you, but it IS pretty crazy. And I certainly agree with qetzal that the normal review process doesn’t do such a great job of screening out the junk. Working in early research at a large company has required frequent duplication of published results, and all I can say is just because you see it in a top flight journal: ’tain’t necessarily so!

  6. Jose says:

    Along the same lines, an Early View:
    The Effectiveness of the Peer Review Process: Inter-Referee Agreement and Predictive Validity of Manuscript Refereeing at Angewandte Chemie
    10.1002/anie.200800513

  7. German says:

    Hi Derek, thank you for the blog – I am working on making a submission to PNAS, and this was very useful. However, I think you may have switched the numbers of the tracks: Track I is “Communicated by” and Track III is “Contributed by”, not the other way around.

  8. Still Scared of Dinosaurs says:

    Now that the intelligent posters have chimed in I feel less guilty about pointing out my complaint with the journal – PNAS has to be a leading candidate for worst journal initials ever.
    Yes, it’s pathetically sophomoric, but every PhD (almost) was once a sophomore.

  9. chris says:

    re: TI/III rejections. I have only ever heard of one case of a “Contributed by” article being rejected from the journal, which caused a bit of chitter on the grapevine. I was highly amused.
    Not that I hobnob with NAS members often enough to be an authority on rates.

  10. But also keep in mind that all the research in PNAS is freely available after 6 months. And some of course is also available immediately. I wish more journals would take this approach to open access.

  11. chris says:

    Passed Over By Nature and Science (PNAS) (And this is exactly what happened in my last PNAS paper….).

  12. Anonymous BMS Researcher says:

    PLOS ONE

  13. Jordan says:

    We used to say it stands for “Papers Not Accepted in Science“. Similar to #11. There’s some good work in there, though.

  14. Morten says:

    Love the fact that their articles are free after 6 months… hate that you don’t have to deposit data from xray structures. Considering the kind of fraud and failure going on recently with regard to crystallography papers, a policy of not having to release structure factors is just wrong.

  15. Marc D says:

    Interesting thread… If this is of any use, using impact factor data you can see that PNAS published 3306 articles in 2006, 3200 in 2005, and 3084 in 2004. I don’t have data for 2007, but 3200-3400 should be a good estimate.
    Wasting a bit more time and looking at citation reports for 2006, as of today the top paper was cited 301 times, #2 was cited 139 times, and the next 10 all about 100-120 times. The paper ranked 330 (or ranked at 10 percentile) was cited 34 times. 419 papers were cited twice or less. The latter may be used to calculate a novel (?) junk index …
    Wasting even more time, there are quite a few authors who published 4-8 papers in PNAS in 2006 and I rapidly checked their citations and a bunch were never cited (not even a self citation…duh).
    I published a paper in PNAS in 2003 and this was one of the best experience ever. I had numerous journalists call me (including one from Science, which had turned down the paper!) and I will try to publish there again.

  16. Chris Neale says:

    If it’s a “proceedings” then let them send it out only to their members as a “newsletter”, which is exactly what it is. While PNAS is at it, why not tell everybody about how to enlarge their **nis.
    Chris.

  17. Xuebing Wu says:

    one way to assess the quality of track i/iii papers is to check their citations (such as ISI/google scholar) and compare with those of track ii. has anyone done this before? I’m really interested in it.

  18. Xuebing Wu says:

    oh, I find such a paper, here is the link http://biology.plosjournals.org/archive/1545-7885/4/5/pdf/10.1371_journal.pbio.0040157-L.pdf
    They show that track ii paper indeed are much more likely to get more citations, especially after a relative long time. Interesting.

  19. kevin says:

    i published in PNAS. it was a “communicated by”. but i have to say the reviews were quite rigorous. and yes, my paper was reviewed and rejected by nature, science, and nat med but i find that the reviewers from PNAS were very good and improved my paper..

  20. Brian says:

    I think lots of people who criticize track1 and track3 in PNAS editorial process are usually those who have never had a chance to publish their paper in a multidisciplinary journal. Meanwhile, it is hard fact that (1) PNAS is a multidisciplinary journal; (2) multidisciplinary journal has a higher impact factor than speciality journal and greater prestige; (3) the majority of the journals is “speciality journal”.

  21. Giovanni says:

    Considering that PNAS is publishing manuscript in mathematics geology and Physics (which are less cited in average than biomedicine) an impact factor of about 10 tell us a lot about the average interest raised by PNAS manuscript. Obviously direct submission is a very tough track with low acceptance rate. It is true that for a NAS member life is much easier on this journal but it very difficult to become a NAS member! I do not think that papers communicated by NAS member are overall of lower quality of lower quality than direct sumbmission. It will be interesting to compare citations of the two group of manuscript to test if this is the case

  22. Igor says:

    PNAS eliminated Communicated submissions in July 2010. I find the following quiet interesting: “Academy members continue to make the final decision on all PNAS papers, unlike the process in place at such journals as Nature, Cell, and Science, where final editorial decisions are often made by staff rather than practicing researchers”. http://www.pnas.org/content/106/37/15518.full

  23. Ross Boucek says:

    Hi all, great information! I have learned alot!
    Myself and others have just submitted a paper to PNAS for the first time. The paper we feel is best fit for this journal. It has been with the editors for 15 days now. Can anyone who has experience publishing with PNAS provide insight to how long papers usually sit with editors?
    PNAS says 18 days on average papers are with editors, but I have read rejections without review are usually sent within a week.
    Needless to say we are biting our nails waiting to hear from them.
    Best!

  24. Ross Boucek says:

    Hi all, great information! I have learned alot!
    Myself and others have just submitted a paper to PNAS for the first time. The paper we feel is best fit for this journal. It has been with the editors for 15 days now. Can anyone who has experience publishing with PNAS provide insight to how long papers usually sit with editors?
    PNAS says 18 days on average papers are with editors, but I have read rejections without review are usually sent within a week.
    Needless to say we are biting our nails waiting to hear from them.
    Best!

Comments are closed.