The price of a GRFP, part 2

(This post is a continuation of the price of a GRFP, part 1. If you’re not caught up, you can go back and start from the beginning.)

In this next installment about the GRFP I’ll be aiming to answer the second question in my first piece, about the change in NSF demography from before the Dear Colleague Letter announcing the ineligibility of graduate students to apply twice.

How has this policy change impacted diversity?

One simple way to look at this is to start with the applicants being awarded who come from prestigious undergraduate institutions. As we know from the last post, the # of awards to these institutions is already fairly lopsided. Check out this cumulative distribution function of sorts, with the % of total awards on the Y axis, given to N schools on the X axis:

cdf.png

The top 3 schools recieve 10% of awards. The top 10 get about 25% of them. The top 30 get about 50% of all 2000 GRFPs. Once you’re out in the top 100 you’ve given away nearly 3/4 of your awards, with about 600 schools yearly splitting the last 25% amongst themselves.

But how has the share of awards from year to year to those top schools changed?

1. Top 10 schools recieve over 50 more awards than last year.

shorter_top_10_comparison_before_after.png

Here schools are ranked not by selectivity but by the top NSF-awarded schools. (Of course you can tell these two things go hand in hand.) There’s been a big uptick in awards in the first year this policy came into play, and it’s not precedented by change in the previous year.

2. In fact, Top 10 schools recieve quite a few more awards than they have for a while.

Since we have data going back quite a bit, you can extend the plot further back in time:

num_to_top_10_winning_schools.png

Beginning in 2011, there’s actually a persistent decrease in the number of awards going to top 10 schools. This levels off in 2013 around where it remains – until the spike in 2018. Also note that the change in this year is about 2 times larger than any change in previous years (and those were all decreases).

But the claim was that the number of undergraduate applicants would increase, and that they’re more diverse.

These two figures only address undergraduate schools — not the applicants who are still undergraduates. We don’t directly have undergraduate status, but can infer it imperfectly by looking for applicants who (a) list their undergraduate institution as their current school, or (b) list no current school at all. Anyone in the third category, (c) list a different current school than their undergraduate institution, is most likely a graduate student; we assume others are most likely to be undergrads.

This method is imperfect, but we can use it to understand at least some of the changes in undergraduate awardees over this same time.

3. The same # of undergraduates win the GRFP, but more of them come from top 30 schools.

awards_to_undergrads.png

Here, we see that there’s been a decrease over time in the overall number of likely undergraduate awardees — consistent with a trend that might concern the NSF, and prompt them to put in place a policy like this! But interestingly, the number of undergraduate awardees increases in 2016, not 2017 — and the only increase from before the policy (2017) to after (2018) is in the number of those awards going to undergrads from those top schools.

At least this policy stops people from reapplying a million times, right?

I was interested in understanding the landscape of NSF reapplicants for obvious reasons — because this is the applicant pool most significantly decreased by the policy. Now that graduate students can’t apply twice, who does that really curtail from winning the award?

I identified probable reapplicants in the dataset by comparing the NSF awardees 2012-2018 with the honorable mentions from 2011-2017. I then split these up by undergraduate school of origin.

4. Reapplicants from the least selective schools are most harmed.

reapplicant_change

There’s a precipitous drop of about 60% in reapplicants winning from 2017 to 2018 — indicating that indeed they’re dropping out of the pool! But when you compare reapplicants from top and middling school backgrounds on the left, their 60% award drop pales in comparison to the 80% drop for those from the least selective schools.

There are a few more analyses I want to do, but for now, these are the main takeaways: the new policy is definitely impactful. But it seems to have result in a precipitous increase in awards to top schools. Does that fulfill its mission of expanding diversity? Well, maybe, if the most diverse students at those top schools are getting those awards.

But even for fully funded students, the NSF isn’t the difference between the dream of science and the reality of it. How many times have people called it a medal, or a feather in a cap? Should its mission be to decorate people who have funding or to change the lives of people who don’t?

Last but not least, if you’re curious about any of these things, I’ve put together a shiny app that lets you play with your stats– and see how many GRFP slots there are for you.

What are your odds?

If you’re interested in adding any other sliders, shoot me a comment or tweet at me – I’ve got a lot of data 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s