A while ago, I had a conversation with a colleague that really bugged me.
This scientist submitted a biology proposal to NSF this January, which wasn’t invited for a full submission. I understand that the bulk of preproposals aren’t invited, but the rationale in this situation was messed up.
The reviews were clear that the research agenda and approach was excellent, and that the PIs were academically prepared to conduct the research. My colleague was told that the thing that tanked the preproposal was the lack of preliminary data. The panel thought that there were inadequate resources or facilities that enabled the PIs to get the project done, and that the junior faculty PIs were not prepared to run the project. Keep in mind that this was a project that was proposed to take place with senior collaborators at large research organizations.
I thought the purpose of preproposal was for the PI to argue for the importance of the question and the validity of the approach. Those conditions were met, quite well, according to the reviews.
If you’re at a small institution without a huge research infrastructure, did you know that you need to dedicate verbiage into your preproposal to defend the infrastructure at your institution? I didn’t. My colleagues got dinged on this at the preproposal stage.
Did you know that if you proposed an ambitious but eminently do-able project, that you needed to explain your qualifications in detail at the preproposal stage? I would have thought that that both of those issues would be something to deal with a full proposal rather than a preproposal.
To be clear: If the PI was a tenured professor employed by a large research institution, with access to more preliminary data, then it sounds like this proposal would have been given an invitation to submit a full proposal. In my view, this is an unfair bias against the PI, who wasn’t even given a full chance to propose a research project on account of institution size and seniority.
I was really mad when I heard about this – and I would have taken a few days to call down, before yelling at my program director. Am I off, or are these good reasons to triage a preproposal? They keep saying it’s about the concepts and experimental approaches (and broader impacts) at this stage, but it sounds like it’s just a regular NSF review.
I didn’t submit in the last round, but I am preparing a submission for the next round. I realize there are all kinds of great advice from Prof-Like Substance on how preproposals were handled in deed during this last round; I’m just wondering how much verbiage out of five pages you have to spend on things that aren’t the concepts, experiments and broader effects.
This does not surprise me. People are going to review as they have always reviewed. They are trying to shorten the stack, and are looking for any reason to chuck out pre-proposals.
My sense is that success with pre-proposals (and proposals) is based upon how excited people are about the proposal. Flaws can be found with any proposal; often the reviewers are stretching to find a rationale for why it just wasn’t exciting enough. hey can’t say, I loved the xxx proposal and only yyy proposals could make the list, so while this was fantastic, it just didn’t quite make the cut.
I worry that the uproar over the pre-proposal process obscures the fundamental problem– funding rates are so low that high quality proposals don’t get funded; the review process is necessarily arbitrary because so little money is available for a large amount of high quality proposals.
Perhaps it would be more honest to use a random number generator?
“Perhaps it would be more honest to use a random number generator?”
I’ve suggested that a small number of grants be awarded by a lottery system. They do it for green cards. More here: http://neurodojo.blogspot.com/2008/01/rethinking-granting-part-2.html
Sounds about right to me, too. Though they can claim that a preproposal is being screened for X but not Y, really it’s just a miniproposal that’s being evaluated in every way. And I don’t know if there is even the possibility of an RUI impact statement at that stage. There’s more justification to be done for RUI proposals, in my view, and the skeptical reader of preproposal isn’t keen on that stuff within just five pages. (Neither am I.)
I served on a Preproposal panel for this latest round, and I can say I didn’t hear that rationale (“Great idea etc., but no preliminary data”) expressed against a full proposal invititation.