When the National Science Foundation introduced the required “Broader Impacts” criterion, it took more than a little bit of explaining at the outset.
Several years later, most of us understand what a “broader impact” is: In some shape or form, the funded project affects society beyond the scientific findings. There are a lot of ways to approach broader impacts. How do we go about deciding which way to fulfill the broader impacts requirement?
Earlier this year, Nadkarni and Stasch answered this question quantitatively, by evaluating the broader impacts included in nine years of funded proposals within the Ecosystem Studies program. There were some interesting finds, but there is one that I want to single out in particular.
Only 11% of the broader impacts in these proposals specifically targeted groups underrepresented in the sciences.
That’s right, only 11% of the proposals had broader impacts targeting underrepresented groups.
When I think “broader impacts,” I first, and foremost, think of providing training and mentorship opportunities to students from underrepresented groups. I also think of outreach efforts targeting underrepresented populations.
That seems to be a relatively rare priority.
It doesn’t seem to be a big stretch to say that one of the major factors imperiling the future of scientific progress in the USA is that massive sections of our population – and the ones that are growing more quickly – are not interested in, or prepared for, careers in science. If you read every other piece of policy paperwork about science education, you’ll see that the country needs to open the pathway for careers in science to Latino and African-American students. It matters, big time.
But nobody’s doing it in their broader impacts. Doesn’t that strike you as odd?
There are so many possible reasons for this phenomenon, and I don’t want to speculate ad nauseum. Here’s one possibility, though: when people think “broader impacts” they actually do first think about targeting “underrepresented groups.” However, they don’t have a simple or effective route to do so.
How do you reach students from disadvantaged and underrepresented groups? You start with students who are in disadvantaged and underrepresented institutions. Which means that the people who are getting all of these grants funded to implement broader impacts, if not at a disadvantaged institution, should start reaching out.
Are you one of those who haven’t included underrepresented groups in your broader impacts? If so, could you leave a comment about what kinds of things could smooth the path? What do you think that NSF, and we as a community, could do to help researchers at institutions with lots of NSF grants (and relatively few disadvantaged underrepresented students) reach out to underrepresented groups?
Reference:
Nalini M Nadkarni and Amy E Stasch 2013. How broad are our broader impacts? An analysis of the National Science Foundation’s Ecosystem Studies Program and the Broader Impacts requirement. Frontiers in Ecology and the Environment 11: 13–19.
I attended an NSF workshop on Broader Impacts, and was surprised to learn that the presenters (maybe not representative of “NSF”) don’t consider LGBTQ persons underrepresented in science.
But, there was a lot of good discussion on Broader Impacts. Some participants mentioned that they didn’t know how to connect with college and high school students, and it was recommended to get in touch with the Education Department in your college/university to get connected with local area high schools and underrepresented colleges.
I would be surprised of LGBT people were underrepresented in science. They might even be proportionately overrepresented. (?) We do need to make a point to be inclusive, but without seeing any numbers, I don’t suspect that we’re failing to recruit gay scientists as we are non-white ethnic scientists.
Annoyingly, I can’t even find numbers for LGBT people in science/engineering. You’d think someone would have studied this.
My social-group anecdata is that trans women are overrepresented in sotware development, but other than that I couldn’t say.
This is interesting, because it contradicts informal chatter I’d heard. I’d heard that “Why yes, we’ll recruit students from under represented minorities” was a very common approach to meeting the broader impacts criterion.
I think the discrepancy is because the paper only examines FUNDED proposals. I think that skews the data. Intellectual merit still rules: it doesn’t matter how good your broader impacts are if you can’t convince people that you can do good basic science. Since most minority serving institutions are not research institutions, I think grant proposals have a hard time competing on the intellectual merit criteria, which skews against the recruitment of under-represented groups.
Regardless, how few abstracts mention broader impacts is also a bit of a stunner.
Zen, I agree that it is very surprising how few mention Broader Impacts. At the workshop I was at, the presenters kept stressing that the NSF considers Broader Impacts as important and ranks it as highly as Intellectual Merit, and I kept thinking to myself, “in what world?”.
I think broader impacts in your NSF package are kind of like teaching in a tenure package at a research university. On paper, you have to be excellent in broader impacts or teaching to get the thumbs up, but far less than excellent will be tolerated as long as the research is top-notch. I’ve never seen someone not get tenure at a research institution even though they were well-recognized as a subpar teacher, and I’ve never heard of a proposal not getting funded because of subpar broader impacts.
As a graduate student, I helped start a kind of speakers bureau that linked graduate students with public school teachers. Teachers visited a website to see what graduate students were participating and whether their research interests/outreach lesson plans could be integrated into their own lesson plans. It wasn’t entirely successful because of the ephemeral nature of grad student availability, because we needed to repeatedly publicize ourselves to the teachers, and because we had no experience in assessing the value of the program. Nevertheless, two of the three classrooms I visited were in underserved/underrepresented communities. They were super fun for both me and the kids (and hopefully useful as well).
That’s very cool, and even working to make the connection happen is valuable. Informal education experiences can have a huge effect on kids, but I think having an effect on teachers requires a bigger commitment (time, money, personal investment).
I haven’t read the paper but would be curious how they identified work targeting underrepresented groups. For instance, we have a BI program that could be considered museum outreach, but if you look at the description, it is really targeted at children of underrepresented groups. So how is it classified? Based on being a panelist, I know this is hardly rare.
They did it based on what was in the abstract – if the abstract didn’t indicate it was targeting underrepresented groups, they might not have included it. However, based on all of the proposals that I’ve reviewed and seen (though I’ve yet to be invited to be on a panel even though I’ve volunteered myself on a couple occasions), if underrepresented groups are a genuine part of the broader impacts it does tend make its way into the abstract. Maybe they are funded at a lower rate than they are proposed?
Terry, do you think one implication of this study is that making NSF-funded investigators do broader impacts is mostly a poor way for those impacts to reach underrepresented groups? As you note, if the PIs who get funded are at different institutions than where the students from underrepresented groups are, that makes broader impacts oriented towards underrepresented groups rather unlikely. Of course, PIs can reach out to other institutions as part of their broader impact work, but in many cases I’m guessing it’s easier for them to find some other way of satisfying the broader impact criterion within their own institution. So maybe if you want to have broader impacts on underrepresented groups, you need some other type of program to do it? (And NSF does have such programs, if I recall correctly, though maybe I’m misremembering). I’m just talking off the top of my head here, not being in the US means I have very little sense of what sort of “broader impacts” get funded, and how those “broader impacts” fit in with NSF’s overall portfolio of programs.
There are a great variety of NSF programs targeting students in underrepresented groups, such as the LS-AMP program, and also there are other programs that are designed for institutions that are filled with these students, such as the CREST program among others.
However, scientists who aren’t in MSI or HSI institutions, which are most research scientists, aren’t targeting the right populations.
NSF is always going to fund top research labs at relatively privileged institutions, which also have a small population of underrepresented students. Of course top research labs need funding. But these labs should also be targeting underrepresented students, and for the most part, they are not doing it with much substance. There are notable exceptions, of course.