I started this morning with tremendous news: a student of mine, who left my lab for a PhD program last year, let me know that his NSF Graduate Research Fellowship was funded!
I had two other former students who put in applications. I downloaded the big list from NSF, and — alas — they did not have the same fortune. So, I was 33% happy.
I sorted the list to see if any other CSU Dominguez Hills students got an award. (NSF lists the baccalaureate institution of the awardees.) I emailed relevant campus people to share the new that our campus had a great year – two CSUDH students had success!
At this point, I have this list of all 2,000 GRFP award offers in an excel file. It’s hard to not fuss around a bit with these data.
As the largest state university network, I wondered how California State University students compared with other universities. Among all of the CSUs, there were 37 awards. Then I scrolled through my sorted list, and this was my reaction, which I shared on twitter:
https://twitter.com/hormiga/status/583123206887755776
Undergrads from all Cal State campuses had a combined total of 37 awards. Harvard undergrads had 37 awards.
According to Wikipedia, the undergraduate enrollment of the CSU campuses is 392,951. Whereas the undergraduate enrollment at Harvard is 6,700.
Considering that I spend so much of my professional life preparing my undergraduates for career in science, this makes me realize that I’m pushing up against a goddamn unmoveable object. The tremendous success I had this year — with one student getting a GRFP! — would simply be the notch on the belt of PIs who are working at institutions with so much more support at every level.
For new students at my university, what this number tells them is that the deck is stacked against them. This is, of course, obvious. These numbers don’t come as any surprise to me. The lack of surprise doesn’t mean that it should be accompanied by a lack of outrage.
I am sure that the NSF panels that decided the awardees did their job quite well, and I have no doubt that everybody who got an award deserves it. Getting one of these things is not easy, surely!
In the middle of my twitter angst, a colleague reminded me of this Joe Biden quote, that I shared with him while the faculty of his university were on strike:
https://twitter.com/hormiga/status/573983878118948864
Which leads me to the obvious thought:
https://twitter.com/hormiga/status/583123668638638080
While deserving people got graduate fellowships this year, many deserving people did not get fellowships.
Moreover, many deserving people didn’t apply for fellowships. Many deserving people didn’t even know that NSF has a graduate fellowship program. Many deserving people don’t have access to well-supported faculty members that can provide individualized research experiences.
If we want to diversify STEM, we’ve got to cultivate a broader foundation. If we want to diversity STEM, we can’t expect that the next generation of graduate students will come from the same elite undergraduate institutions.
Yes, I wish this was a joke, but it’s not.
NSF isn’t going to fix this problem by saying, “We just funded the best applicants and we can’t help it if most of them came from elite institutions!” I don’t know if that’s what they’re saying, but I can’t imagine how else they could rationalize that Harvard undergrads get the number of GRFPs as all of the California State University undergrads.
Graduate committees can’t just say “We would take prepared applicants if they existed!” They need to reach out to institutions like mine and build substantial bridges to genuinely recruit applicants — and build in support for the labs that are generating these applicants.
Undergraduate admissions committees can’t say “We’d have good students if only we could find them!” Yale clearly got caught in a rare moment of honesty with this magazine cover.
The United States has a grievous problem with diversity in STEM and we have a great need to broaden access to higher education to women and men from underrepresented groups. This movement cannot limit itself to recruiting the small proportion of ethnic minority students who attend elite institutions, but must truly broaden access to those who come from disadvantaged backgrounds.
Disadvantaged students come from disadvantaged universities.
NSF Graduate Fellowships are definitely not a part of the solution. If the Graduate Research Fellowships aren’t a genuine part of the solution, than where the heck can we expect a solution to come from within the NSF?
Not that long after I started my job at CSU Dominguez Hills, a good friend of mine came to visit campus and give a talk. I was griping about a series of challenges I consistently face, like riding a bike into a very strong wind. He was telling me how he was thrilled for the potential in front of me. I remember how he said it: I had the chance to literally remake the [white] face of ecology. Every student that I send on to graduate school would have a measurable effect. If I wanted to make change, then, he argued, then this is the perfect place for it. And I’m a guy who can make that happen.
I think he’s right. As several years have passed, I draw on this conversation for inspiration. I really need that inspiration for moments like these, when I realize how hard I have to pedal into the wind, when students at more privileged institutions have the wind at their backs. If we are going to make science equitable, then it must come from institutions like mine. If opportunity continues to overpass us, then the injustice persists.
Sometimes, I really feel like I want to stop pedaling. I have that option, but my students don’t.
Do you know how many students applied for the GRFP from Harvard versus how many applied from CSU? Obviously, the denominator matters here.. Not that I’m arguing your point (it’s accurate), but what you really want to compare is the rate at which students from these 2 institutions get fellowships.
It’s worth mentioning that NSF used to have a separate Minority Graduate Research fellowship Program – that’s the program though which I got mine – that was eliminated following a lawsuit alleging it was discriminatory to have a seperate program for URMs. From a 2011 NAS Report: “Despite these challenges and the continuing dialogue about the effectiveness of race-neutral policies, no one denies the fact that we had the most rapid growth of minorities in STEM fields during this period.” (http://www.ncbi.nlm.nih.gov/books/NBK83366/).
To this day I still make sure I refer to it as my “Minority GRF” and I can’t overstate how important it was atthe time that I got it – not just because of the support, but because at the time I felt that there was a real effort to diversity graduate student cohorts and faculty. Today I sense the feeling of urgency to do so is gone, especially at the faculty levels.
Great comments on this post already. Agree that the number of minority students / students from non-prestige institutions that apply needs to increase, and good mechanisms need to be identified to increase that number.
Re: the Harvard / Ivy League point. Even if the reviewers of GRFP applications think they are immune to its effects, there is certainly a prestige hierarchy that influences who receives these awards. I’ve seen it in action on grad admissions committees at my last job. Degrees from certain institutions are highly recognizable, brand-name luxury goods that signal prestige and quality to reviewers. That effect was recently quantified for tenure-track hiring in certain fields:
http://advances.sciencemag.org/content/1/1/e1400005
I wonder if GRFP reviewers should be blind to the institution from/to which an applicant is associated.. Not 100% fix, but maybe better. Still, the effects of nearly unlimited resources will be apparent especially in 1st and 2nd year grad student applications, but that is a harder problem to fix.
Take an application from an elite institution and one from CSU. Remove all identifying info. Elite-app is likely to be better written. The applicant likely entered college with better writing skills, worked with a prof who has time to do research and write grants, interacted with intellectually elite peers and grad students and postdocs who challenged her, received extensive critique from those people, attended fast-paced classes that assumed all-elite students, etc. It amazes me anyone can overcome this disadvantage. The best solution I’ve observed is for disadvantaged students to “catch up” by doing a masters at an R2 school. Then they are on a better footing entering a PhD program – better writing and research skills. I’m not saying that’s fair, but it’s true. My husband has had multiple students in this position. Of course, you can’t get a GRF with a masters.
It is a real shame that the minority GRF was eliminated. If NSF wanted to, they could give “special consideration” to GRF applications from disadvantaged students in a way similar to how NI/ESI applications are evaluated for NIH grants.
Let’s insert some data. NSF has a pretty detailed study of GRFP from last year (comparing four cohorts from mid-90s to 2011).
http://www.nsf.gov/ehr/Pubs/GRFP_Final_Eval_Report_2014.pdf
Section 6 is the most relevant to the current debate: it compares fellows to a national sample of Ph.D. completers. You can definitely see the “heritability” of academic privilege, since parents of fellows are more likely to have advanced degrees and less likely to have bachelor and less than the overall Ph.D. completing population (which itself is highly skewed in that regard). The only data about UG institution attended is whether fellows ever have gone to Community college or not, and fellows are slightly less likely to have, but that could be in part due to a difference in wording of the question (GRFP survey more stringent about what counts as having attended CC). NSF must have much more detailed data about educational backgrounds. Hope they will analyze/release it.
I think Emilio makes an excellent point about the importance of a unified, strong effort towards diversity. I received my GRF in the days after the minority grant program existed and I am grateful for the opportunities that support provided. However, as I have gone on in graduate school, there have been several situations revolving around race and gender that have presented themselves. It is very difficult to be forthright about microagressions and other issues grad students of color deal with at an institution that is not diverse. I think that a powerful counterbalance to those challenges would be to know that there are strong efforts towards increasing diversity in the academy. At the most recent ESA meeting I had to go tuck myself away at the CA capitol rose garden for a while to be with my thoughts: I didn’t realize how important it was to my overall attitude about continuing in academia to see that someone with a similar background to mine had “made it” as professor or head of a non-profit or consulting business. I longed to take someone for out coffee that I could talk to about shared experiences as women of color in American academia. Three days into the meeting, I was still searching for my coffee date. As it is, I don’t see myself in the face of ecology and the implications of that are quite discouraging. I didn’t think this would bother me when I started graduate school, but 5 years later as I contemplate the next step a real sense of effort toward diversity of all kinds in academia matters a lot indeed.
Terry you make some really good points here. I think the problem is much deeper than this tho (not disputing that NSF and NIH etc should put their money where their mouths are). Of the students you teach, how many apply to grad school at all? How many of the most promising ones decide (probably correctly) that Med/Dental/Pharm/Vet or CLS/PT/PA is a better route for them. The problem of diversity in science is a big issue, but so is the “there’s no chance of you having a really successful career” problem. These students, especially the smart ones from a disadvantaged background, recognize the reward system and act accordingly. I’ve seen lots of my best students go into CLS or PA (physician’s assistant, for those not up on the lingo) because in 18-36 months they can get a basically guaranteed middle class income (mid-60s at least for CLS in LA area). Its a bit of a catch-22 – we want to increase diversity in STEM but the economic optimum path for an individual does not run through grad school in biology. The two problems seem to reinforce one another. I have often felt very conflicted about telling students (smart, often minorities) to pursue science as a career, because while I want to make science better and more diverse, I’m not at all sure it’s good advice for those individuals.
Of course there are deeper problems that cause students from disadvantaged backgrounds to apply for grad school and graduate fellowships in lower numbers.
I was frustrated this year by the reviews a non-traditional student received. The student received two glowing “excellent” reviews, the third “very good” review noted only that the student had a “very good academic record which is not competitive with respect to the applicant pool”. Given that the student has 4.0 GPA I can only read this to mean that non-tradiaitonal student paths are not competitive at NSF.
It sounds like one possible route would be to build “bridge” programs. Here’s a nice one I know of between UCSD and HBCUs: http://morehousebridge.ucsd.edu/about-the-program/
Students do paid summer research at UCSD, get coaching on grad admissions and the NSF GRFP. Lots of other bonuses, too, like waived admission fees to UCSD. However I don’t know how well this scales to CSU size rather than small HBCU size.
I’m at a small primarily undergraduate university with just such a masters program — in the three years I’ve been here we’ve graduated five students, all female, none white, and all but one are now in PhD programs. We’re small, but we make a depressingly significant impact on the diversity of PhD students in our physical science field. I wanted to highlight what one commenter said above: our students who graduate with masters degrees are NOT eligible to apply for the GRFP, even though they enter a PhD program at the same first-year level as other students and spend just as long working towards the PhD. Bridge programs are successful and growing, but we’ve got to stop hamstringing the students who go through these programs — it’s a serious blind spot at NSF.
I think what we are seeing is that NSF reviewers have very little on which to distinguish between students. They have little time and little information and in such an environment it is not surprising that “quality of undergraduate insitution” becomes one of the discriminating criteria. This was explicitly stated on the reviews of one of my student applicant who had completed an undergraduate degree in 3 years with a 4.0 average at Towson State (MD). The applicant received a rating of ‘good’ on academic record with the comment that the undergraduate institution was not good.
The NSF fellowships have become, in fact, a lottery. I think it time that NSF recognize reality and use the reviews to identify those in the top 25% and then randomly draw the 2000 winners. That is what they are now doing, but they call it reviewing.
I just cant get behind these bridge programs.. How can we ask already disadvantaged students to do extra work (a masters) that itself may disadvantage them in other ways (e.g. no GRFP). We have to tackle the problem at the source, which is harder but fundamentally required if we are ever to solve this.
Like I said above, blind reviewers to institution as a start. This will not fix, does not deal with the root of the problem, but harms no one and deals with some of the inequity.
I think this is an important discussion and I’d like to offer some practical advise. I am writing as someone who attended both a UC and a CSU school and received an NSF GRF, and who has been a member of NSF panels. Applications from disadvantaged schools and backgrounds should make this known in their applications, and explicitly address the way that a GRF will help their ability to overcome these disadvantages, and increase diversity in STEM, relevant to their circumstances. Proposals are reviewed based on their intellectual merit and upon their broader impacts. Here is how the NSF materials describe broader impacts: “Panelists may consider the following with respect to the Broader Impacts Criterion: the potential of the applicant for future broader impacts as indicated by personal experiences, professional experiences, educational experiences and future plans.” Certainly the author is aware of the broader impacts criteria, having had success with one of his applicants. The broader impacts criteria carries real weight in the evaluation of proposals. In my experience, I think the NSF is a force of positive change, but certainly more needs to be done. In my opinion, outreach to high school students, where students gain opportunities to engage in real research, should be prioritized. The NSF has programs such as the “Research Assistantships for High School Students”, which is designed to “help ensure a diverse pool of future students, faculty and researchers.” http://www.nsf.gov/pubs/2012/nsf12078/nsf12078.jsp.
Just some more context: for all NSF awards (not just GRFP — in fact, I don’t think the GRFP awards show up there, but not sure), CSU (all campuses) got $14.2M from NSF in 2014 for 39 awards. Harvard got $72.4M for 177 awards. (Info from http://dellweb.bfa.nsf.gov/AwdLst2/default.asp ).
This is relevant for the denominator (to get at what fraction of proposals are approved). If we assume that that every undergrad has an equal chance of submitting an award, the rates are (37 / 392951) / (37 / 6700): A CSU undergrad has 1.7% chance of getting an award as a Harvard undergrad does. If we assume that the pool of applicants is proportional to the number of NSF awards an institution gets (maybe every grant represents a research active lab generating one undergrad student per year) the ratio is (37 / 39) / (37 / 177): A CSU undergrad coming from an NSF-supported lab is 4.5 times likelier to get a GRFP than a Harvard undergrad coming from an NSF-supported lab, under this ballpark estimate. Different assumptions, different conclusion. We could also look at the number of science majors, the number of science majors going into PhD programs, etc., to get different denominators.
I imagine the truth is somewhere between 1.7% and 4500%, and it is likely far below 100%, but it might be useful to get a good estimate of the number of proposals to get this (email NSF Bio for the data, perhaps?).
Worrying about lack of opportunities for underrepresented groups is critically important, as well as thinking about the best ways to use limited funds to address this. And I am sure there are many biases we haven’t yet fixed. However, I worry this discussion has gone too far ahead of the data until we know the ratio of proposals submitted. I do note, though, that there may be, and almost certainly are, issues that lead to barriers to submitting a proposal, too, but it’s worth figuring out if the bottleneck is there rather than at the NSF review stage so we know which solutions to try first.
Matt: I think there’s a difference between bridge programs and doing a Masters as a stepping stone. The program I linked above, for instance, is a paid undergraduate research experience, then uses that experience to link students to mentors who help them with the NSF GRFP.
Institutional bias clearly exists throughout NSF, and I don’t know how to get rid of it. I am a professor at a R2 university, and recently at a conference, a colleague from Big Name R1 Institution blatantly told me, “I serve on NSF grant review panels frequently, and when it comes down to it, if there are equally meritorious proposals on the edge of funding, my panels will always recommend giving the award to the one from the R1 institution over those from an R2 or minority serving university: we figure the R1 institutes will have a better infrastructure for research, the PI will have more time and effort to give to the project (vs. having a heavy teaching commitment etc.), and thus the project will have more of a chance of succeeding. And besides, if the PI was really good, why aren’t they at a R1 university?” Now, of course, program managers, who ultimately make the funding decisions, should (and will) see through a runaway panel like that with biases… but I’m left wondering how often panel institutional bias like this will slip through? I suspect that something similar may happen from time to time with graduate fellowship applications as well.
I think beyond institutional bias, this just reflects a culmination of the many of the socio-economic advantages a Harvard undergrad has had vs a CSU-DH undergrad has had. A Harvard undergraduate has been trained since birth (assuming the average Harvard undergraduate comes from an upper / middle class white pedigree) to navigate the system that is in place. They know how to find mentors, how to speak with mentors and send all the linguistic signals that say “I’m from a certain educated class” (as an example). They understand all the subtle socio-cultural contexts to navigate the system before them. The result is far beyond just improving access to STEM, it’s improving access opportunities and recognizing the hidden biases that prevent that access across many fields.
As others have noted, one thing about the NSF GRFP is that you can’t get it if you have a master’s, and you can’t get it if you’re still in a master’s program at the time of application but have been doing it part-time (because of the way they count credit limits). I couldn’t try for one for my PhD because I did a part-time MS while working full-time. I suspect that this disproportionately disadvantages people who were already disadvantaged – someone in a combined BS/MS at an elite school who wants to go on for a PhD afterward can apply during their MS, but someone working their way through an MS can’t. And there seems to be a general bias against anyone who takes a non-traditional path. The GRFP isn’t the only fellowship program to have this problem – the DOD NDSEG, the Hertz, the DOE CSGF, all have it. I vaguely remember the Ford (which also gives preference to underrepresented minority students) being a bit better.
I was lucky that one of the schools I was applying to – the one I’m at now – had a shiny new NSF IGERT that was relevant to my interests, and offered me a fellowship through that.
Rheophile: “It sounds like one possible route would be to build ‘bridge’ programs.”
Possible problems with bridge programs:
They’re seen as the “kiddie table” in agencies like NSF, and are not taken seriously (see Optiplex’s comment above)
The “partnership” mostly consists of the “bridging to” institution saying, “Here’s what will happen. You send us your minority students!” That institution gets the students’ “bridging” research work, and get good karma for trying to help the “little guys.” The “bridging from” institution gets… what, exactly?
Not saying bridge programs can’t work – but you need a very good working relationship with a lot of trust to make sure that everyone benefits from the arrangement.
Zen: Interesting, I hadn’t thought of it that way. What benefits would you be looking for on the regional/PUI side? I was mostly thinking from a student perspective, where it seems like a pretty good deal.
Rheophile: There are several possibilities. For example, support for student at their institution, not just when they are at the “bridging to” institution. Access to technical resources, like core facilities, etc. But the best reason of all is genuine research collaborations between faculty who are mentoring students. Not all institutions will have the number of shared interests needed to have that work.
Is that excel spreadsheet available for people to have a look at? It would save others (like me :)) a lot of time screen scraping.
It’s downloadable from the link to fastlane up near the top of the post. You can just select all and sort for baccalaureate institutions.
https://www.fastlane.nsf.gov/grfp/AwardeeList.do?method=loadAwardeeList
Bottom if screen -> “export options : Excel”.
(Thanks Emilio for the full service linkage.)
This post and all of these problems are excellent and in general I couldn’t agree more. Can I, however, put in a good word for us extremely privileged Prestigious University graduates? The student bodies are a lot more diverse in terms of socioeconomics than is sometimes assumed, and not to be the #notallmen person, but telling someone who overcame as much as some of my peers did pre-college (or during college, or post college) that they don’t deserve their success can also be damaging. Zero-sum games like NSF fellowships are really, really hard…
All of these *comments!
Sorry: apparently a Prestigious University degree does not confer the ability to proof-read! :-)
While it’s true that students who already have a masters degree (and apparently those completing a masters part-time?) cannot apply for a GRFP, everyone completing graduate work was eligible at one time. It is possible to apply 1) during the final year of undergraduate study, 2) during the first year of graduate study, and 3) typically during the second year of graduate study, whether in a masters or PhD program. The problem is that very few masters students apply, possibly because they don’t know about the opportunity or don’t know that they want to pursue a PhD until after their period of eligibility has expired. The gripe that students with masters degrees are ineligible is as old as the day is long, and NSF does not seem to be changing their policy. Maybe it’s time for those of us who serve as mentors and role models of masters students to inform them of the opportunity and encourage them to apply while they still can!
Back when I applied for the GRFP*, they asked which state you graduated high school in. Rumor had it (http://forum.thegradcafe.com/topic/15807-nsf-grfp-2010-2011/) that a certain number of fellowships were allocated per state (or maybe per capita?) because balanced state representation was government-mandated. Anyway, I’m guessing that the majority of CSU undergrads graduated HS in California, a large state that probably turns out more scientists per capita than most other states, so the competition is stiffer for them. Harvard, on the other hand, has a much more geographically diverse undergraduate student body. This might be just one more disadvantage forcing CSU students to pedal a little harder into the wind.
*Admittedly, my information is out of date. I have no idea if this question is still part of the application.
Anonymous: Yes, all grad students are able to apply at some point in their studies. The master’s and doctoral students are all in the same competition, because there is only ONE GRFP, and not one for master’s students and one for doctoral students. A key question is whether master’s students’ applications are going to be competitive with doctoral students’ applications.
The NSF awards site (linked to above) does not specify whether the program they are admitted to is a master’s or doctoral program. I don’t know if there are any data on how many master’s students get GRFPs compared to doctoral students.
My suspicion, though, is master’s students’ applications are going to have an instant mark against them.
I’ve had multiple conversations with NSF staff about master’s programs, because I run the biology master’s program at my institution. Repeatedly, the message I have gotten is that the NSF is NOT interested in supporting master’s students. They want to promote people getting terminal degrees.
If master’s students don’t have a fair shot at a GFRP, it’s a waste of time for the student to send in the application, and it’s a waste of time for the faculty to go around chasing all those students to have them get their applications in.
I’ve recently served on a GFRP panel. Without giving specifics, I can say that on my panel, many of the above charges leveled at panels simply did not appear to be true. In our discussions, small institutions and underrepresented status worked for many applicants as long as the applicants articulated how those experiences work toward a broader impact and as long as they showed potential to succeed as researchers.
The unfortunate thing is simply the sheer number of excellent applications resulted in several proposals with excellent ratings ending up outside the award pool. In such an environment, small things could make a difference — students with high GPAs were passed over if a letter raised concerns about an applicant’s ability to succeed outside the classroom.
The process is not perfect, there are undoubtedly structural, institutional, and other unconscious biases that creep in in subtle ways, and we should be discussing these issues. But I don’t think the root causes of many issues of representation and pedigree reside primarily in the panels. Student writing skills need attention, recommendation letter writing skills are atrocious, the over subscription rates are high, many students may not be aware of the opportunity, and rules regarding who is eligible are somewhat confusing. But our panel made concerted efforts to identify and reward disadvantaged applicants with high potential.
I was just looking at the numbers. I’m not saying the panelists aren’t working for inclusion. I’m sure that they are! The bottom line is that students from campuses like mine get screwed. But that process started long before they didn’t apply for a GRFP.
Terry, you don’t actually know from this data that your students are at a disadvantage within NSF. You’re comparing 37 / #CSU applicants and 37 / #Harvard applicants, and assuming that #CSU applicants >> #Harvard applicants. That assumption is true if # applicants is proportional to undergraduate student body size. Since many people getting funding are first year grad students, the size of that population might be a more relevant measure. If # applicants is proportional to # NSF grants instead, then it’s not true: as I mentioned above, if the denominator is assumed to be that, CSU applicants have a 4.5-fold advantage over Harvard applicants. If every grant to CSU generated four times as many applicants as a grant to Harvard, then CSU students would still have an advantage.
Why does this matter? It matters if you want to fix the issue of improving diversity in science. I haven’t served on a GRFP panel yet, but the anecdotal experiences above and reported elsewhere suggest that measures are taken to reduce bias against underrepresented groups at NSF, including the broader impacts criterion. Based on the data of 37 awards at two different institutions, with an unknown denominator, it could be that it’s completely unbiased within NSF and the bias occurs at the stage of deciding to submit proposals. In that case, the solution might be more funds for undergrad student training, more grants to labs in primarily undergrad serving institutions, and so forth. Criticizing the NSF panel would have no effect if NSF isn’t the problem. Or it could be that the issue is within NSF: CSU sends in 58 times the number of proposals as Harvard does (using the undergrad pop numbers you presented above) and they get funded at 1.5% of the rate. In that case, money going to undergrad training might be better used in training panelists on how to avoid bias.
As in most things in science, it’s probably not either/or but rather multiple factors. And I have no doubt that your students do face barriers that they shouldn’t, and we should try to fix those. But I think it makes sense to get the data to figure out where the bias is, and fix the problem there, rather than assume based only on the knowledge of the numerator that the resulting fractions clearly show that the “deck is stacked against them” by NSF.
Terry, they may be getting screwed but I’m not convinced it’s by NSF…I think it’s by the issues that they deal with up to the submission of the application, assuming they apply at all. Anecdotes and gut feelings aside, reviewers of proposals evaluate what is in front of them. These evaluations are influenced by their biases, but I think that panelists would like to do as much possible to help diversify the stem pool. Unfortunately, if the file isn’t compelling, they can only go so far.
And that’s the key – the application is what is being evaluated, not the student per se (although I know it feels that way to the student). Put another way, stellar students can put together crappy applications, and one can make up for middling grades or low test scores if the other parts of the application shine. I bet a GRFP Application Swat Team working with applicants at CSUs that fall into that “unrealized potential” category could increase the percentage of successful submissions immediately by helping with the packaging ; students at elite schools either have those SWAT teams in place or don’t need them by virtue of a long history of access to mentorship and opportunities.
And finally, I would really like to know what the denominator is – how many students apply from different schools? its that ratio that reveals the extent of the disparity. Can you get that number? Can NSF provide the number of applicants from different universities? I bet they would, and in fact may already have done these analyses.
I agree with you entirely.
I didn’t refresh the page after starting to write my comment above, so I missed Emilio’s comment and Terry’s reply, but +1 for them both.
Brian, here is something you wrote: “You’re comparing 37 / #CSU applicants and 37 / #Harvard applicants, and assuming that #CSU applicants >> #Harvard applicants. ”
That’s entirely, utterly, false. If you read the post, you’ll see this from me: “While deserving people got graduate fellowships this year, many deserving people did not get fellowships. Moreover, many deserving people didn’t apply for fellowships. Many deserving people didn’t even know that NSF has a graduate fellowship program. Many deserving people don’t have access to well-supported faculty members that can provide individualized research experiences.”
I said the deck is stacked against my students. I didn’t say that NSF has stacked the deck against my students. Clearly, what is happening at NSF is a symptom of a much greater problem. NSF is a part of a problem because they are not an adequately proactive part of the solution.
I’m loving the fact that the discussion is focusing on the problem that have led to this huge inequity. And the fact that students at the CSU have much bigger obstacles than students at Ivies. That’s just an obvious fact, as I said. This is no surprise, as I said. This doesn’t change the my main point:
When it comes to the probability of getting an NSF fellowship, an incoming CSU freshman is screwed, compared to an incoming Harvard freshman. And NSF can do more about this if it wants.
Emilio: “The application is what is being evaluated.”
Yes, but… people on Twitter say that reviewers sometimes criticize the institution the student is at. Enough people reporting that that we should not just wave those away.
If the reviewer sees the student’s university as a problem, NO amount of application polishing can fix that.
And I think that’s wrong. No student should get a demerit on their application because of the institution they are at.
Several commenters in this thread, and Terry, have stressed that the GRFP panel reviewers are operating in good faith and trying to increase diversity. But that those institutional criticisms have gone back to applicants suggests that NSF is NOT adequately addressing biases for or against particular institutions.
While I’m here, I’ll also link out to a related post by John Hawks for those following the conversation: http://johnhawks.net/weblog/topics/metascience/grants/nsf-graduate-fellowships-underrepresented-2015.html
Thanks for your reply, Terry. I agree that it’s great that folks are talking about solutions.
You do say that part of the issue might be lack of applications. However, the rest of the thrust of this post, as well as discussion on twitter, has been the problem is within NSF. For example, another quote from your post:
“NSF isn’t going to fix this problem by saying, “We just funded the best applicants and we can’t help it if most of them came from elite institutions!” I don’t know if that’s what they’re saying, but I can’t imagine how else they could rationalize that Harvard undergrads get the number of GRFPs as all of the California State University undergrads.”
And
“NSF Graduate Fellowships are definitely not a part of the solution. If the Graduate Research Fellowships aren’t a genuine part of the solution, than where the heck can we expect a solution to come from within the NSF?”
And in your reply, the focus is still on NSF: “NSF can do more about this if it wants.” Anyone could do more, of course, but, depending on the problem location, it could be the important fix is at CSU, or at a SACNAS workshop, or somewhere: focusing on NSF as the problem is premature. It could be the program is great for those CSU students who apply, it’s just that not enough of them do: in that case, the GRFPs could be part of the solution (let students have time to do research and get paid a decent wage), and the problem is increasing awareness at CSU and other institutions.
As an example of ways to address the problem (and you may have other solutions as examples — I’m not saying this is the best solution), take my department. We’re an EEB department with ~24 faculty and ~50 students. All our grad students take a first year course in core knowledge in ecology and evolution. In the first semester, we usually (this varies, partly b/c it’s team taught) have a long term project for them to write a GRFP-type proposal. They start working on background for their project well in advance of the deadline, and the proposal itself goes through review by their faculty advisor(s) as well as by several of their peers in the course as well as by the faculty instructor(s). This ensures that the final proposal is understandable to a fairly wide audience. Within our fields, we have had good success relative to other institutions: see this post http://eeb.bio.utk.edu/2013/04/11/nsf-graduate-fellowships-updated-again/ for a year (2013) where our dept. had 3 awards and 3 honorable mentions, putting us 9th that year in ecology and systematics; this year, we got 2 awards in ecology or evolution (no honorable mentions), making us tied for 13th in the country in those areas (interestingly, last year we didn’t incorporate the training into the course to the same degree). Our department, which is one of three biology departments on campus and several science departments, got 2/3 of the GRFP awards to our university system this year. There are many reasons for our students’ success (and most of them come from the students themselves), and this course didn’t contribute to students who were entering with a GRFP, but this sort of intense training may contribute to success, and it doesn’t depend directly on NSF taking an action.
I just think it’s important to keep the focus on finding solutions, and realizing that the issue and solutions can be at many levels, rather than keeping the focus on NSF and how this program is not a solution — at least until we have better evidence to support that view.
Brian, I haven’t ever said or claimed that, as you put it “the problem is in NSF.”
Yes, I say that NSF graduate fellowships are part of the problem. It would be folly to claim that NSF is primarily at fault for disparities between poorly funded state universities and Harvard. That’s a silly strawman.
The post focuses on the role of NSF as a part of the problem because that’s the title and the topic of the post! Yes, they are a part of the disparity in awards because they distribute the awards that way, by choice. If they wanted to focus their funds more on the disadvantaged, that’s something that can do, and they are not.
Please make whatever arguments you wish, but please stay away from claiming that I’ve made an argument that I’ve never made.
I’d say NSF GRFPs are part of the solution. I dusted off R and did a few analyses of the GRFP from 2001-2015. For baccalaureate institutions, the trend for Harvard, Stanford, MIT, and Princeton are significantly negative over time. For Rice and Yale there is no significant correlation, and for UC Irvine, UC Riverside, and CalState Long Beach, they are significantly positive. I got virtually the same results for current institution with the change the Long Beach stayed positive but nonsignificant and Rice became positive and significant.
Guess which institutions have the highest proportion of first generation college students? The public ones, obviously. UCR and UCI each have almost 60%. The private ones are often down near 10% (it is hard to find these numbers quickly, so more complexity may arrive on further inspection).
I didn’t do any nonparametric stuff (should have) and it is hard to test names over time (they change format so much from year to year), so this is the most I will do off the cuff. But I think there is pretty good evidence that some of the complaints you make neglect the degree to which NSF is already doing what you suggest.
Another thing to note is that the expected disparity between baccalaureate institutions is actually probably pretty big, even if there is no bias from NSF. Remember, yes, Cal State is a top notch system, but it does have a mission for the mass education of people of all educational aspirations in California. So, sure, while it will send an important proportion of students to graduate school, it also caters to a population who aren’t aiming for PhDs, and we shouldn’t fault them for it. Serving that population is incredibly important! On the other hand, places like Harvard send far more of its students to graduate school proportionately. The ~58-fold undergrad enrollment difference between all of CalState and Harvard will likely be largely offset by the relative proportion of students from those schools who plan to attend graduate school.
Has it occurred to you that these educational aspirations are a part of the problem and not just something to evaluate with a regression analysis?
Zen, I think you did some selective editing. Yes, I said “…reviewers of proposals evaluate what is in front of them”. But the next sentence begins with “these evaluations are influenced by their biases…” I’m unclear as to ehat I write negates the idea that some people might hold an applicant’s institution against them. I am sure such biases exist, and as you point out people have tweeted a few reviews where they got feedback consistent with these biases. But some biased reviews means “there are some biased reviewers”. This dors not mean “this kind of bias is pervasive among reviewers.”, for which there is no data do decide one way or the other. Without data all this is speculation, so in that spirit I’ll bet:
of GRFs resulting from SWAT team’s efforts > # lost because of reviewers biased against CSU.
Any takers?
“Has it occurred to you that these educational aspirations are a part of the problem”
Yes. I’m not very clear on how NSF’s GRFP policy would impact the aspirations of students who have not opted to apply to graduate school. In any event, as I showed, despite challenges, NSF has done a good job of spreading their support to address precisely the concerns you raised. So, while I don’t agree that this is as big a problem as you seem to think, it appears to be becoming a smaller and smaller as time passes.
Also, it is a bit narcissistic for us to tell students that they ought to aspire to the same things we do. As a first generation college student myself, and certainly a first generation PhD, I find it rather presumptuous to suggest a lack of aspiration to become a basic science researcher in a PhD program is a “problem”. My whole family and high school peer group (many of whom are first generation college graduates themselves) would be rather critical about that view.
Look, I’m thoroughly enthusiastic about sharing my love of research and education with undergraduates. And I encourage them to learn a bit about the research path so they can consider that along with their other career ideas. But I hesitate to presume that they should value it as highly as I did and do.
Can we all agree on the following?
a) It’s a damn shame that there are systemic inequities in higher education.
b) NSF GRFP panels do the job that they are charged with quite well.
c) NSF has a number of programs, and dedicates a big part of its mission, to fixing these inequities.
d) The distribution of NSF graduate fellowships can definitely be skewed more towards fixing these inequities than they are at the moment.
If you disagree with a, then you aren’t really equipped for this conversation, and you might as well move on.
If you disagree with b, then I’d really like to hear how and why, because nearly all of my experience with NSF has been very professional and well done, even when I get that raw end of the stick. (There are biases of reviewers that has emerged, and that’s a huge problem, and a part of the systemic biases, but I personally am giving program directors the benefit of the doubt.)
If you disagree with c, then you might want to check out the whole “broader effects” concept and the number of grant programs designed for EPSCOR and URM and HBCU and MSI and all that. I don’t think it’s nearly enough, but it’s still a big priority of NSF.
If you disagree with d, then that’s simply irrational. That’s just a straight-up fact.
This post is not an investigative piece of reporting. It’s a blog post reporting an obvious fact that is surprising nobody. If anybody is surprised about the fact that Harvard STEM majors are a lot more likely to get a GRFP than a CSU STEM major, then they’ve been living under a rock.
I just wanted to share this obvious fact and explain how it affects me, my lab, my daily life, and most importantly my students and the future of science.
Because NSF isn’t fixing these inequities at an adequate rate — in my opinion — then they’re a part of the problem. I understand that’s arguable, and that’s an argument I’ve made. But when arguing this concept, let’s please build arguments without assuming that I — or any other commenter — have claimed things that we have not claimed, or impute assumptions into others that are not able to stand up to logical scrutiny.
NSF can (I hope) allocate fellowships as it wishes. And most of them are going to Ivies, big R1s, and SLACs. That’s a fact. You can’t argue that fact.
If they wanted to fund more people from different universities, they definitely could do so, and they are not. If they wanted to recruit more applications from other institutions, they would definitely get more if the effort was put in, especially if they knew that the evolution criteria would minimize the systemic biases that are currently in there.
The panelists are following the evaluation scheme with which they’ve been charged. I understand it’s uncomfortable for some of us to recognize that there’s a whole system that maintains the marginalization of underfunded public universities on a day to day basis. If you’re a regular here on this site, then you should be used to it, because that’s the reality of my job, and I bring it up all the time. If you swapped jobs with me for a few years, I imagine you might start to feel like I do.
There is a huge number of students in public universities who are being left behind. And a big part of that is because they’re not applying. If they were funded more, application rates would go up, that’s for sure. This post was just about the fact that very few of them are getting graduate fellowships. As a result of historic and systematic biases that NSF is addressing with partial measures that — in my opinion — don’t go far enough.
The trend is toward more inclusion, but let’s face the fact that many of the people who get the GRFPs are the ones who really don’t need them, and those who really would get the biggest boost, aren’t getting them. Of course, recipients benefit, even those who don’t need it, have an opportunity for even more excellence. The thing is, the way we traditionally measure excellence, students from disadvantaged backgrounds are almost always going to score lower, even if they have equal prospects for a tremendous career in science. If we want to diversify STEM and be inclusive, then we should put our money where our mouths are. NSF is doing this a bunch of ways, but with the GRFPs, it looks like they’re doing it only in partial measure and they could be doing a lot, LOT more to build the future scientific community we want. If that’s what the mission is. That would be a good discussion. Let’s get back to that one.
Emlios: “Yes, I said “…reviewers of proposals evaluate what is in front of them”. But the next sentence begins with “these evaluations are influenced by their biases…” ”
Ah, fair comment. My apologies.
For some reason, I had it in mind that you were talking about unconscious biases. I guess I gravitated to that assumption because most people are arguing pretty strongly that panelists are operating in good faith – which is probably the case generally,
But that people were actually writing these criticisms of institutions down in reviews meant that the reviewers were absolutely conscious of that particular bias, And whether rare or pervasive, that kind of invalid critique shouldn’t be making into the reviews that go back to applicants.
When talking about how students are left behind in public university education, I think encouraging more students to pursue academic research is not our highest priority. Especially considering funding for doing that research is flat, funding rates are the lowest in history, and when the competition for postdocs and faculty positions are fiercer than it has been in my lifetime. I want science to grow, but we should definitely tell students that they might expect to compete with 20 others for a postdoc position and 300 others for a faculty position.
Despite these challenges, NSF has doubled the total number of GRFs since 2008. So, when you say “If they were funded more, application rates would go up, that’s for sure”, well, NSF has more than doubled the number of opportunities that are given to the public schools I listed above (again, due to the dirty nature of the data, I can’t say much more without systematic hand coding of data; suffice it to say, that’s not in the cards). So, the relative rate has increased (strong positive correlations with time) and the absolute number of GRFs has doubled. That’s an incredibly strong commitment. It is rare that I’ve seen such a progressive change by a government bureaucracy in such a short time. I know we disagree, but I’m actually surprised and impressed with NSF. I didn’t know that before today.
Regarding aspirations and systematic structural problems with disadvantaged students, I think perhaps this is beyond the purview of the GRF (though not of NSF more broadly). It is a fairly established principle that erasing inequities is more and more expensive as students grow older. The best investment possible is comprehensive head start for example. But we need not go that early into the educational history to improve college STEM education. How about California cut the prison budget and shift some of that to hiring more faculty at CalState and converting adjuncts to faculty with security of employment and benefits? How about more scholarships for college? Maybe NSF could create a science research internship program where students could get paid to learn about science careers and how to write fellowship applications? Or maybe that’s something the state should do. Certainly broader impacts already incentivize this behavior.
Anyway, the spouse wants the laptop, so I’m going to have to abruptly stop mid thought.
It was hard not immediately dismiss your take on things as an oversimplification, but I also could not help but think the same when I scrolled through the list of recipients. In any case, I’m a first-generation college student (a metric the NSF is not even interested in), I received this GRFP this year, and my pivotal recruitment into STEM happened way before the GRFP.
Here’s a little anecdote:
In 2005, I attended NJ Governor’s School on the Environment. It was a month-long immersive, summer science program at Richard Stockton College, and it did not cost my family a penny. (Sadly, the program has since been cut.) I once heard someone say that gov school was a “golden ticket to the ivies,” and it certainly felt like it when I got into Cornell. While I was a decent student who came from a good-enough school, many incoming students started at Cornell with more AP credits than my high school even offered. I had never thought of myself at a disadvantage before (I still don’t like to), but it was undeniable that an overwhelming number of my fellow students had life experiences at the ripe age of 18 that only could have been bought by their parents and zip codes.
Now fast-forward to my time at Cornell. I started working in a lab and was able to spend three summers at Cornell working on my undergraduate research project. An honors thesis and publication came out of it, and I was paid to do it. Cornell work-study covered half my wages and lab money covered the other half. Cornell also changed its financial aid program such that I was no longer paying any tuition after my first year (http://www.bloomberg.com/news/articles/2015-04-01/ten-elite-schools-where-middle-class-kids-don-t-pay-tuition). Because I did not graduate with a mountain of debt and gained relevant research experience, I was competitive applicant for graduate school and I could afford the opportunity cost (lost income) of going to graduate school. Once in graduate school, I could spend the time and receive the feedback needed to craft a successful GRFP application.
Receiving a GRFP would have been impossible without the opportunities I was afforded, and while I am utterly thankful for and humbled by receiving a GRFP, I know many others have not had the system work in their favor. (I also cannot ignore the support I received from my family, which is a different form of privilege. We’d give each other our last dollar.)
I’ve often thought these fellowships need to be reviewed blind with the applicants and mentors names AND institution removed. I applied for several NIH fellowships when I was a postdoc and without fail I would always get at least one unfavourable comment regarding the state university I was working at; often expressing disbelief at the facilities I “claimed” to have access to. Obviously one anecdote does not data make, but for these types of fellowship I see no reason why the reviewer needs to know the identity of the people they are judging. I suspect that should reviewers no longer find themselves knowingly comparing elite institutions against regional colleges that the funds would redistribute themselves in very interesting ways.
Did anyone ever consider the possibility that Harvard students are smarter, harder working, and have many other qualities that make them likely to contribute more to science that students at CSUDH? What would an acceptable ratio of success rates for Harvard vs CSUDH students be? It seems unfathomable that it could or should be 1:1. Although diversity is important, for something like this it has to be balanced with merit.
Please, crimson, why don’t you flesh out this argument some more, about how a few Harvard students are more likely to have as much merit as the top students in the entire CSU system?
Harvard clearly gets the best of the best students applying to college every year. The CSUs clearly do not, even in the state of California. So again, I think it seems pretty unreasonable to expect a 1:1 ratio of success rates among Harvard and CSU undergrads. Or is that unreasonable? Is 1:1 the ratio you’d like to see? If not, what would be acceptable? Comparing these numbers without more context seems like a very knee-jerk thing to do. Who knows, maybe all else equal these numbers are actually surprisingly good for CSU students. Can you refute that?
Your first sentence is so foolish it sounds more like a line from The Onion. Harvard getting the best students? Hahahahaha. That’s just supremely naive and inexperienced. It’s a conceit that some hold to tightly, I am sure.
Way to deflect and skirt the real point. As other commenters have pointed out, one can only guess about the denominators. And in general comparing two numbers without more information is extremely speculative.
It’s not a deflection. It’s an assessment of your argument, which is predicated on the notation that that the best of the best students are the ones that apply to Harvard.
I didn’t see any other argument there, other than asking me about ratios. You were asking me about whether I would like some kind of ratio of success rates. That’s patently absurd, and many (clearly not all) of the nation’s best students are ones that never even thought of going away from home to go to college in the Ivy League.
Other commenters were wondering about the number of applicants, sure. So do I. But what NSF is focused on is outcomes. Let’s take two extreme scenarios:
a. Way way more Harvard applicants than CSU applicants. All of the CSU applicants get funded, and few of the Harvard applicants get funded. That’s seriously messed up, because that low selectivity for the CSU applicants can’t be good for long-term research prospects. If this was the case, then this is something that NSF can and should address.
b. Way way way more CSU applicants than Harvard applicants. The Harvard applicants get picked at a much higher rate than CSU applicants. While some might harbor the conceit that this reflects higher quality of Harvard applicants, what this also indicates is either a bias against the CSU students and/or a lack of preparation of these students to prepare applications against the Harvard students. This also indicates a messed up scenario that NSF can and should specifically address.
It’s outcomes that matter, which is who gets them. If the well-heeled are the ones that are applying, and the ones who get it, that’s a problem. (Yeah, lower-income families get free tuition at Harvard. But that turns out to be a very small percentage of enrolled students. Shocker, I know.)
The AAUW fellowships just got announced today (which I applied to and did not receive). Perhaps it was because I recently heard your views on the NSF grad fellowship, but I couldn’t help noticing the list of Universities of the awardees seemed to be heavy on the ‘well known/prestigous’ side.
I participated in an NIH PREP (post-bacalurate-reserch-education-programs, which focus on science-minorities: http://www.nigms.nih.gov/training/prep/Pages/default.aspx) program. It’s not directed at current PhD students, but rather at getting more people into PhD programs. It’s useful, however, because it gives students an extra year to prepare to compete with students from more privileged backgrounds, depending on the focus of the program.
My program also provided help with PhD applications and writing skills. I think differences in application writing skills may explain a lot of the CSU/ Harvard disparity, just because to get into Harvard you have to be good at applications. Also, if it works like my small private college, you get extra writing help from the moment you arrive and workshops on applications and grants in your senior year. In the PREP program none of us applied for the NSF grant you get before you enter your program. However, we had a workshop on the one you can apply to, while your in a program.
I think these programs are awesome and I’m sad to see that the budget for them keeps getting slashed. I came from an elite school, but one that was very small and struggling to adapt to a the increase in students with disabilities since the ADA. Students had to help the school figure that out and it effected the first two years of my college and I was weak on research experience. In PREP I was also at an elite school, but I was surrounded by other students, who were adapting to an elite institution that wasn’t necessarily designed with us in mind. Even though we all dealt with different issues, our experiences were similar enough that we could share strategies and support each other.
The program I was in no longer exists and before I started they were cut from two year programs to one year programs. Programs are evaluated based on GRE scores (I ended up not taking the GRE, because getting re-tested for accommodations was too expensive) and the % of students admitted to biomedical programs misses their real potential. Only 3 out of the 7 of us went on to elite PhD programs, but I went to a M. Sc program in France. Two of the others went on to paid research assistantships and the last person took a year off and is now in a public health program.