Since I don’t have to write NSF grants, I haven’t had to wrestle with “Criterion 2”. But ask anyone in academic science about it. The first criterion is intellectual merit, as it darn well should be. Here’s the NSF’s own description (in full):
How important is the proposed activity to advancing knowledge and understanding within its own field or across different fields? How well qualified is the proposer (individual or team) to conduct the project? (If appropriate, the reviewer will comment on the quality of prior work.) To what extent does the proposed activity suggest and explore creative, original, or potentially transformative concepts? How well conceived and organized is the proposed activity? Is there sufficient access to resources?
But the second criterion, while initially worthy-sounding, invites trouble. It’s “What are the broader impacts of the proposed activity?” Here’s more description:
How well does the activity advance discovery and understanding while promoting teaching, training, and learning? How well does the proposed activity broaden the participation of underrepresented groups (e.g., gender, ethnicity, disability, geographic, etc.)? To what extent will it enhance the infrastructure for research and education, such as facilities, instrumentation, networks, and partnerships? Will the results be disseminated broadly to enhance scientific and technological understanding? What may be the benefits of the proposed activity to society?
To me, that puts the most important question last, and even that one can be hard to answer. As for the rest, this would seem to be an open invitation to insert all sorts of nice-sounding boilerplate, or to just start making things up. The NSF itself seems to have realized this, and has been working on a revised version of this language, but here’s a column from Dan Sarewitz that says that “Criterion 2.1” isn’t a bit better than the old one:
At the heart of the new approach is “a broad set of important national goals”. Some address education, training and diversity; others highlight institutional factors (“partnerships between academia and industry”); yet others focus on the particular goals of “economic competitiveness” and “national security”. The new Criterion 2 would require that all proposals provide “a compelling description of how the project or the [principal investigator] will advance” one or more of the goals.
The nine goals seem at best arbitrary, and at worst an exercise in political triangulation. . .Yet, more troubling than the goals themselves is the problem of democratic legitimacy. In applying Criterion 2, peer-review panels will often need to choose between projects of equal intellectual merit that serve different national goals. Who gave such panels the authority to decide, for example, whether a claim to advance participation of minorities is more or less important than one to advance national security?
. . .Motivating researchers to reflect on their role in society and their claim to public support is a worthy goal. But to do so in the brutal competition for grant money will yield not serious analysis, but hype, cynicism and hypocrisy.
One of the comments to that article points out that this isn’t the NSF’s fault, in a way, because this exact language was mandated by Congress. And so it is – take a look at Section 526 of Title V of H.R. 5116, the “America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science Reauthorization Act of 2010”. There’s all the same language. Not only that, but the Act directs the NSF to assign people and funds to evaluating how well all these “Broader Impact” measurements are going. The director, within six months, is supposed to have implemented a policy that:
. . .requires principal investigators applying for Foundation research grants to provide evidence of institutional support for the portion of the investigator’s proposal designed to satisfy the Broader Impacts Review Criterion, including evidence of relevant training, programs, and other institutional resources available to the investigator from either their home institution or organization or another institution or organization with relevant expertise.
So, in case you’ve lost track, the NSF is supposed to train people to implement a policy that requires grant applicants to show that their institutions are training people to implement a policy that requires grant applicants to show evidence that their work involves training people to implement a policy. I think I’ve got that right. A greater invitation to bullshit I cannot picture.