Implicit Bias and the NFL Draft: Are Teams Under Drafting Non-White Quarterbacks?

The 2021 NFL Draft is next week. And, as is often the case, quarterbacks are expected to be drafted highly and dominate the narrative. In a game with 22 starters (plus kickers and punters!) quarterbacks routinely dominate the top selections of the NFL Draft. And it makes sense. If you land a quality quarterback, you’re already on a path to the playoffs. Quarterback is the most important and illustrious position in all of sports. 

This year, there are five quarterbacks expected to go early: Trevor Lawrence (the presumptive first overall pick), Zack Wilson, Mac Jones, Trey Lance, and Justin Fields. Each of these prospects is being dissected by teams, scouts, and NFL analysis. Every weakness exhibited by these players are being scrutinized on a microscopic level. Naturally, with such a microscopic view and through which one must split hairs to make a decision about whom to pic, those flaws will be examined. Zack Wilson’s poise under pressure, Mac Jones’ athleticism, and, in the case of Justin Fields, concerns about “work ethic.” 

Justin Fields is Black. 

I was listening to Bill Barnwell’s NFL Podcast, The Bill Barnwell Show today; he and guest Nate Tice were discussing the aforementioned quarterbacks. They acknowledged the coded criticisms of Black quarterbacks entering the NFL Draft. Descriptors such as “poor work ethic” or “poor leadership” or “slow decision making” are applied to non-white prospects more than white prospects. It is assumed that non-white quarterbacks succeeded on their athleticism alone and not their mental ability. It’s why Lamar Jackson fell to the end of the first round despite eye popping numbers at Louisville, where he won the Heisman Trophy. Despite winning the Heisman Trophy at quarterback, some NFL scouts, general managers, and TV personalities suggested he switch to wide receiver because professional quarterbacking would be too challenging. Jackson fell to Baltimore at the 32nd overall selection. He was the fifth quarterback taken in that draft.

Jackson went on to win the league’s Most Valuable Player award in 2019.  That year he accumulated 36 passing touchdowns to go along with seven rushing touchdowns. He posted a 113.3 QB rating, a convoluted formula that accounts for passing statistics (and, notably only throwing statistics). The average rating of an MVP quarterback is 107.8. In other words, Jackson can throw. He is considered one of the best long term prospects in the league.

Essentially every team in the league had a chance to select Lamar Jackson, who is better than almost any other quarterback in the league. It was clear that some amount of implicit bias played into the pre-draft criticism of Lamar Jackson. Or maybe the scouts were just wrong on that one. Maybe this was a one-off mistake by 31 NFL teams. 

I began to wonder: how might implicit bias affect Black NFL quarterback process? And how would that show up in the data?

Methods

I want to stress these are blog post methods, not peer reviewed journal methods.

I looked at every quarterback drafted going back to 2010 and where they were selected. There are roughly 250 draft picks every year in which the team selects a player and they then have the rights to offer them a contract. I disregarded last year’s quarterback draft selections as it’s much too early in their careers to draw any meaningful conclusions. Quarterbacks have been selected at #1 overall, all the way to 253rd since 2010. Ninety-three of them were white; twenty-six were non-white. I didn’t know what to do with Terrelle Pryor, as he was selected in something called the Supplemental Draft, so I tossed him out of the analysis.

If teams are missing out on QB talent due to implicit bias, there ought to be a mismatch in where non-white quarterbacks were selected and how they actually performed, when compared to white quarterbacks. If teams are persistently underdrafting non-white QBs and overdrafting white QBs, that would show up in their performance relative to where they were drafted. So how do we measure overall value?

I used Pro Football Reference’s “Approximate Value.” If you’d like to read what’s behind Approximate Value (AV), be my guest. For this post’s purposes all you need to know is this: AV is a metric that tells you how good someone has been over the course of his career. A high AV is good; less AV is not-as-good. It’s not a perfect statistic, but it’s pretty good. It’s also a cumulative statistic (rather than an average), so it gets bigger the longer you play. It is going to penalize the more recent QBs in this analysis. Lamar Jackson was drafted in 2018 and has accumulated 50 AV, despite his career just getting started. By the end of it he’ll have a lot more. I thought about developing some sort of model to project how many more AVs recently drafted players, or perhaps creating a yearly average of AV but, again, these are blog post methods. So, apologies to Baker Mayfield (aka the Winter Soldier) but this method is going to favor players with an already-longer career. 

I also used this Expected AV draft value chart, from Pro Football Reference. This chart demonstrates the average AV of the players selected at that position in the NFL draft. As you might expect, players drafted higher average more AV than players drafted lower. It’s quite a steep curve at the beginning, illustrating that early picks (1-40) are much better than the rest of the selections. Expected AV after selection 100 is essentially nil. The chart is a bit on the old side (2012). But the original author of that chart updated it a few years later and it hadn’t changed much. So I just used the 2012 numbers as a baseline of expected performance. Now we can analyze if a player’s actual AV was higher and lower than their expected performance. And if we see a preponderance of non-white quarterbacks underdrafted, it might be because they’re evaluating them through biased lenses. 

And pity them! A poor allocation of resources (money and draft picks) is a terrible disadvantage in the NFL, particularly when it comes to QB. Not only is under drafting non-white QBs racist, it’s also bad team management.

Results

Remember: these are blog results. We’re using Excel here, not SPSS or R. 

Here is a scatterplot of all QBs drafted from 2011-2019 according to which selection they were (x-axis) and their accumulated AV (y-axis). The blue dots represent white QBs, the orange dots represent non-white QBs. The red line is the expected AV based on where the player was selected. The player with the most AV over the past 10 years is Russell Wilson. He was drafted 147th overall. He’s non-white. The second highest AV was achieved by Cam Newton. He was drafted first overall. He is also non-white. Newton is a great example of someone who was highly regarded (though, the same racially coded criticisms followed him pre-draft) and performed up to and over expectations. Wilson is an example of a player who greatly surpassed expectations, considering how late he was drafted. Another fun point is Trevor Simien who greatly outperformed the expected value of a 250th overall pick. 

What do you notice? What do you wonder?

After looking at that graph I noticed that it looked like more orange dots were above the expected value than blue dots. I also noticed a heck of a lot of blue dots, particularly at draft position #151 and higher. I expected there to be more white QBs drafted than non-white, but I didn’t expect it to have such a higher distribution in the later rounds of the draft. 

In fact, I went on to chart at what point were white and non-white QBs drafted. It turns out that non-white QBs tend to be drafted higher than white QBs. There are comparatively a whole lot of white QBs drafted from 140 to 200 compared to non-white QBs.

Lastly, I calculated and charged the difference in actual AV with expected AV. Below is a box-and-whisker plot that shows the over/underperformance of all QBs. The median white QB under-performed his draft slot by -0.3. The median non-white QB showed a difference of 13.45. This means that non-white QBs outperform their draft slot considerably, while white QBs were slightly underperforming their draft slot. Sixteen non-white QBs outperformed their draft slot, compared to 9 who underperformed. Meanwhile 32 white QBs overperformed their draft slot while 61 underperformed. According to performance, non-white QBs are not being drafted as highly as they should be. 

To recap, the data show the following:

(1) Fewer non-white QBs are being drafted.
(2) Non-white QBs are drafted at higher selections.
(3) Non-white QBs outperform their draft slot.

Discussion

There are limitations to this study, I mean, blog post. 

The first limitation is this: I don’t know anything about college football. I don’t know what the non-white QB situation looks like in the amateur ranks. Are 20% of quarterbacks in college non-white? 40%? 70% 1%? I have no idea. And college football is the talent pipeline that prepares these would-be QBs. This is to say, I don’t know if the problem is with the NFL or college football. I suspect it’s endemic to both, as well as high school. But I already forgot who won the 2021 NCAA Championship and that was like a couple months ago. 

Another limitation is that it disadvantages QBs who began their career more recently. I’m not sure it’d have much of an impact if we spin this scenario forward. Patrick Mahomes, who is non-white, is going to continue to accumulate AV for a long time. As is the aforementioned Lamar Jackson, Baker Mayfield, and University of Wyoming’s own Josh Allen. The study punishes 2019’s first overall pick Kyler Murray by awarding him a negative value-over-average-draft-slot score, despite Murray being highly regarded in only his second year. I don’t see it moving the needle as much for Daniel Jones or Drew Lock. The study doesn’t even bother with last year’s top QBs, such as Joe Burrow, Tua Tagovailoa, or Justin Herbert. 

Perhaps the most significant limitation of this study is that it ignores context. Russell Wilson overperformed his draft slot certainly, in some part, due to the team he was selected to. Some teams are better at developing players than others. Trust me, I know. There’s also an incentive for teams that draft a player highly to make him successful. A player drafted early will get more chances to succeed than a player drafted 170th overall. To account for that, I reran the analysis excluding players selected after 150. It yielded similar results. The median overperformance for non-white QBs grew to 18.75 and the overperformance for white QBs dropped to -1.75.

The final limitation is that I only went back 10 years. This means I missed the most notable QB overperformance in draft history: Tom Brady, who was drafted 199th overall. Also, Drew Brees (32nd overall). Both of these quarterbacks are white. Notable non-white overperformances include Randall Cunningham and Warren Moon, the latter of whom was not drafted at all. If I have the time and motivation, I’ll add a Part 2 post in which I’ll go back even further in the analysis and consider dropping all QBs after pick 150 and see what shakes out. 

With those caveats aside, it’s pretty clear that non-white QBs are being underdrafted. 

Russell Wilson is the prime example. He’s probably a counter-intuitive example. Criticisms of Wilson were that he was not physically gifted enough. He is relatively short for an NFL QB. Instead, he has made it through on his accuracy and decision-making, and leadership, attributes more often reserved for white QBs. The other significant outlier in overperformance is Dak Prescott, who was selected 135th overall. His career is still early and he could easily make his way up on the overperformance list. I’d also like to highlight Tyrod Taylor who is considered a great backup QB for all the non-playing reasons I mentioned early, and is non-white. He was selected 180th overall and has outperformed that slot considerably.

There’s clearly something at play that in the drafting of non-white QBs. Despite being a small number, they are disproportionately drafted higher and and and(!) outperforming that draft position. There are almost no non-white QBs drafted in the backup range. It’s either: they are an undeniable top-talent (but still underdrafted) or they are not draftable. 

The amazing thing is the examples of overperforming non-white QBs are very different archetypes. Russell Wilson is an entirely different player than Cam Newton. Patrick Mahomes is a wildly different player than Lamar Jackson. At this point we’ve seen enough non-white QBs defy any singular mental model formed on implicit bias that there shouldn’t be this issue. Teams that are able to expunge implicit bias from their system will be at a strategic advantage when it comes to the QB position.

This isn’t to say that Justin Fields is being underrated as an NFL draft prospect. Maybe he’ll be a bust like other highly drafted non-white QBs such as Jamarcus Russell (who was outside of the year range of this study) or last year’s disaster Dwayne Haskins, who is probably done in the NFL after accumulating a -14.4 value-over-expected AV after only one season. Maybe criticisms of his “work ethic” or “inability to make a second read” are legit. Maybe they’re not even real criticisms. Sometimes teams will leak criticisms to throw other teams off the scent. Or maybe the criticism is part of a long standing tradition of underdrafting non-white quarterbacks based on false, racist notions about processing speed, accuracy, intelligence, and leadership.

====

I’ve written about the NFL draft and the NBA draft before. I’m 100% sure my interest in sports drafts stem from my Cleveland fanhood, which routinely saw my teams picking at or near the top of the draft, while missing out on the playoffs entirely (this past year notwithstanding). 

If you’re new to this blog, know that I write about math and math education. Here’s my About page.