There's been some discussion on various blogs about the "OMG! NSF is Completely Broken and the World is Going to End!" forum headed up by the disgruntled "Aureliano Buendia." See Prof-like Substance and DrugMonkey for some interesting comments.
Something mentioned in that forum and in a comment at DrugMonkey's piqued my interest. QoQ over at DM's asserted that the NSF is indeed broken and noted in support:
First, the identity of the reviewers is not public and changes from submission to submission -- so you can't target a grant.
You can't target a grant.* I'm not entirely sure what QoQ means by this, but I suspect they want to write their proposals for specific reviewers on the panel. Perhaps so they can try to "butter up" the reviewers by citing their work favorably and often, or to avoid having to write a proposal in more general language that reviewers not experts in the sub-sub-field can understand. Or maybe even both. Or neither.
It doesn't matter really, because targeting reviewers on the panel is WRONG, WRONG, WRONG!!!!!!!
Why?
'Cos there ain't no guarantee that the reviewers you are targeting will get your proposal.
In fact the odds are not in the least bit favorable. Review panels at the NSF (and study sections at the NIH) cannot have experts from every sub-sub-field on them. Unless you want either panels with many dozens of members, or many, many more panels than currently exist (and there are already a lot). So the odds are there may be one reviewer at most who is an expert in your particular sub-sub-field (and that person is likely a competitor...).
Targeting panel members would be a particularly stupid thing to attempt (if you could) at the NSF where you could have anywhere from three to ten people reviewing your proposal. Usually three or so on the panel, and the rest as outside, "mail in" reviewers. Even if there is an expert on the panel you could target, one good review isn't even close to being enough to land funding. And let's be realistic - a panel member might be somewhat flattered by some "buttering up" in a proposal, but they're generally smart enough to recognize it for what it is.
Write your proposals for people only somewhat conversant with your corner of the field (and cite all the relevant literature, and none of the irrelevant). If you can't do that, chances are, you won't stand a chance of being funded.
* Actually, the NSF does let you do a form of targeting that a proposer would be foolish not to take advantage of. When you submit your proposal you are given the chance to suggest reviewers. In my experience NSF PO's do actually use some of these suggestions as outside reviewers. Obviously these need to be reasonable suggestions...
10 comments:
In NIH land, it is absolutely essential to "target" your proposal to the study section that is going to review it. This does not mean targeting it to the presumed expertise of particular individual members of the study section, but it *does* mean writing your application to appeal to the scientific emphases of the panel.
For example, if your application is about some ion channel mechanisms of cardiac arrhythmia, and it is assigned to the Cardiac Arrhythmia study section, then you would be wasting valuable page space justifying the importance of ion channels to cardiac arrhythma. Or, if your application is very basic science oriented, and it is going to be reviewed by a study section that contains a lot of clinical scientists, then you better do a really good job of justifying the potential clinical relevance of your proposed studies.
I don't know if this is what the NSF-sucks disgruntlenut is talking about, but people who know what they are doing take this approach to "targeting".
(And BTW, if you request a specific reviewer to review your NIH application, you actually bar that person from reviewing your grant by creating an apparent conflict of interest.)
I was going to comment, but CPP pretty much said it all, at least for NIH study sections. An NSF panel is pretty much a crapshoot. As an applicant, since you don't know the composition/expertise in the panel at all, you pretty much have to cover all bases. That takes away valuable space for that critical figure that you might want to put in or some such.
Funny, I'm on the other side of the Atlantic and on every grant writing workshop I've attended (both for national sources and EU funding), the first thing they always tell you is: do not write for specialists. Write for reviewers in your general area, because you never know who you're going to get. Sounds like good advice to me (and it kind of seemed to have worked for my funded grants).
Anyway, for all I know (and see CPP above), things might be different where you are, so not sure if this is of much use.
CPP,
Agreed - it's essential you target a study section at NIH. It's harder at NSF - they don't allow cover letters - and it's the PO's who decide which panel your proposal goes to. But I don't think that's what QoQ was referring to - he/she specifically noted that panelist names are not published. The NSF review panels themselves are pretty set, it's the panelists that change. Also, the NSF oversees a very broad range of sciences, so the panels are less focused than NIH study sections, making it more important to pitch your proposal at a somewhat more general level.
NSF specifically asks for reviewer sections. There's a form on Fastlane devoted to just that. You can also name reviewers you don't want, but in conversations with PO's I gather that's not necessarily a good thing to do.
Anon.@12:45:
An NSF panel is pretty much a crapshoot. As an applicant, since you don't know the composition/expertise in the panel at all, you pretty much have to cover all bases. That takes away valuable space for that critical figure that you might want to put in or some such.
It's a different style of proposal writing. One could argue the merits or pitfalls of the two approaches ad infinitum. In the end you have to write your proposal to fit the agency you're applying to.
And if the figure etc. is absolutely critical, you find a way to get it in.
Also, the NSF oversees a very broad range of sciences, so the panels are less focused than NIH study sections, making it more important to pitch your proposal at a somewhat more general level.
Not all NIH study sections are narrow. Some are quite broad. I recently had a grant reviewed by an NIH panel that did not have a single person on it whom I would consider an expert in any of the overlapping subfields that encompass the subject matter of the grant. I knew this ahead of time, and wrote the grant accordingly. (And it received quite a good score.)
I actually like targeting study sections like that, because they are not going to get caught up in subfield-specific minutiae. Less charitably, one could say that it is easier to pull the wool over their eyes.
I actually like targeting study sections like that, because they are not going to get caught up in subfield-specific minutiae. Less charitably, one could say that it is easier to pull the wool over their eyes.
This is spot on. I lean toward the former, more charitable interpretation myself. Navel inspecting minutia flogging on the part of reviewers is one of the biggest flaws of NIH review. Why do you think the recent change in review has tried to de-emphasize the evisceration of "Approach"?
(oh and don't take your toys and leave the disgruntleforum, odyssey, there's a rich vein of DAOTI to be mined...)
CPP and DM:
That kind of study section targeting sounds eminently sensible. As I noted in a comment above, that simply isn't an option at the NSF. As far as focused vs. broad study sections go, I admit my experience with the NIH is far more limited than with the NSF.
DM:
They're my toys! MINE!
I haven't completely abandoned the disgruntlement forum - I've been keeping an eye on the posts there. The very, very few posts. Given all the publicity that site has gotten (including on TheScientist.com), surprisingly few people are bothering to have their say.
I am reluctant to post over there. As you noted on your blog, Dr. "Buendia" has made it clear he won't tolerate smug NSF-funded investigators like me.
Another point here--different reviewers will necessarily have different biases. If you know the biases of your reviewers, you can take them into account and not risk going unfunded because you've unknowingly hit someone's pet peeve (said peeve, of course, may or may not have any scientific merit).
Whether this is a good argument for knowing who reviewers are or not is not clear to me; but certainly there are reasons to want to target a proposal to specific reviewers that don't involve "buttering them up". Any communication can be made more effective through knowledge of to whom you are communicating.
Patrick,
Certainly knowing who will be reviewing your proposal could help in terms of communicating. But the point is, even if you know who is on the review panel (or study section), there is absolutely no guarantee that the people you target will be reviewing your proposal. By targeting specific reviewers you run the very significant risk of pissing off the reviewers you actually get.
Post a Comment