Measuring NFL Defensive Playmakers

Traditional individual defensive stats don't tell us much. There are tackles, sacks, and turnovers, and that's pretty much it. Recently, I developed "Tackle Factor," a way to make better sense of tackle statistics, at least for the front-seven defenders. It's not perfect, but I think the consensus was that it's a step forward. Still, there's much more that can be done.

Offensive stats are straightforward, but objective defensive stats are problematic. When a running back picks up a 10-yard gain, although other teammates contributed, that's obviously a good play by the ball carrier. And when a running back stumbles at the line for no gain, that's obviously bad. But looking at the same two plays from the other side of the ball is much trickier. A strong safety, say Troy Polamalu, who makes the best play he can by preventing the runner getting past 10 yards, would be be debited for that 10 yard gain. The other four or five defenders who had a chance to make the play sooner, but didn't, aren't mentioned in the play description and wouldn't be docked for the play.

On the other hand, if Polamalu is playing run support, and he reads the play and stuffs the running back at the line, that's certainly to his credit. If only there were a way to credit each defender for plays like this, and at the same time ignore the plays that really should count against his teammates.

A Possible Solution

It might just be pretty simple. All we need to do is add up all the WPA (or EPA) for each play in which the WPA (or EPA) was positive.  "+WPA" and "+EPA" add up the value of every sack, interception, pass defense, forced fumble or recovery, and every tackle or assist that results in a setback for the offense.

What these stats measure is "playmaking" ability. When I watch the NFL Films shows, the sideline audio is usually filled with coaches urging on their players with an emphatic, "Go out there and make a play!" Simply stated, +WPA (and +EPA) puts a number on playmaking.

You might be thinking that only looking at prominently positive plays won't tell us much. +WPA doesn't account for the guy who covers his man like a blanket and never has to make a play. It doesn't account for the nose tackle who holds his ground against the double-team and lets the linebacker make a play. It might reward a gambler, someone who abandons his responsibility chasing the ball. All true. +WPA doesn't account for all the hidden action not reported in the play-by-play, not directly anyway. But does this mean we should ignore it?

The Theory

Think of a player like a corporation. Companies have revenue, which would be +WPA, and they have expenses, which would be -WPA. Revenue minus expenses tells us how much profit or loss a company has, which would be net WPA. Because of the Polamalu problem discussed above, we can't know a defender's expenses from the existing play data, but we can know much of his revenues, his +WPA.

Unlike a company, however, (as Wall Street sadly learned a couple years ago) a player's individual performance from play to play almost certainly follows a normal distribution. Virtually all aspects of human traits and performance are governed by a bell curve, from height to intelligence to athletic feats. There are many instances when a defender plays near his average level of performance, and there are fewer in which he plays either very well or very poorly. The distribution of an athlete's performance is roughly symmetric with respect to his own mean performance level. Nearly all sports statistics are based in some way on the normal distribution, and each player's performance on individual plays is unlikely to be an exception.

We can see and measure a very large part of a defender's performance using +WPA, but his negative performance cannot be captured because of the Polamalu problem. We can infer his overall performance, however, by what we can see and measure. Assuming a roughly normal distribution of performance, an average defender's per-play distribution of WPA would look like this:

Image placeholder title

A below-average defender's distribution might look something like this:

Image placeholder title

And an above-average defender's WPA profile would look like this:

Image placeholder title

There is likely to be a strong correlation between a defender's visible positive impact and his overall net impact. In other words, we should expect better defenders to tend to have both more positive plays and fewer negative plays. This is because of the symmetric nature of the distribution of human performance.

I'm not suggesting every player's performances conform precisely to a bell curve, and I'm not suggesting we can directly calculate the negative side of the curve. What I do claim is that knowing only part of the curve is not such a bad thing, and that measuring defensive players by only their positive impact could tell us far more than we might think.

Not every defender would have the same normal profile. As I mentioned above, there are the "gamblers," players who shoot gaps when they should be reading the play or cornerbacks who jump pass routes when they should stay in position. Certainly, +WPA and +EPA would be biased in favor of these types of players. But if their gambles were really hurting a team, I doubt they'd be given much slack and playing time by their coaches. Only 'winning' gamblers, who are taking smarter risks, would tend to survive long in the NFL.

And I doubt very seriously these players would have a drastically non-normal profile. Even the most reckless gambler can't roll the dice that often. His distribution may be flatter or more skewed than the typical player, with more plays with extreme outcomes, but his overall profile would likely still be roughly symmetric, with more average plays than outliers at either extreme.

Support from Baseball

Baseball stats, which can directly measure +WPA and -WPA, lend support for this theory. Fangraphs.com conveniently breaks out the +WPA, -WPA, and net WPA for each MLB batter. +WPA alone does a very good job of identifying the best hitters. Recent seasons belonging to Alex Rodriguez, Albert Pujols, Prince Fielder, Ryan Howard, Matt Holliday and Manny Ramirez top the list.

For the top 154 batters in each of the last three seasons, the standard deviation of +WPA was 2.15, while the standard deviation of -WPA was 1.28. This suggests that players differ primarily due to their positive impacts, while their negative impacts are relatively alike.

In fact, over the last three MLB seasons, +WPA correlates with net WPA at 0.81, while -WPA correlates with net WPA at only 0.17. In other words, positive performance is the primary driver of overall performance, at least at the elite level. Here is the same point in graphical form. Player +WPA is plotted against his overall net WPA. You can see how tightly they correlate.

Image placeholder title

Although baseball is a very different sport, and player contribution can be measured much more precisely, the principles of athletic performance largely remain the same. And if these principles hold for WPA, they would be true for EPA as well.


Can +WPA really tell us who the best defensive players are? Let's see if the top defenders in +WPA for 2009 make sense. For linebackers, there's Jonathan Vilma, David Harris, LaMarr Woodley, Patrick Willis, Gary Brackett, Ray Lewis, James Harrison, Karlos Dansby, London Fletcher, and Terrell Suggs.

The top defensive ends include: Aaron Schobel, Jared Allen, Andre Carter, Will Smith, Ray Edwards, Trent Cole, Alex Brown, Brett Keisel, Mario Williams, and Dwight Freeney. Guys like Julius Peppers, Justin Tuck, and Robert Mathis are right behind.

For tackles, there's: Kevin Williams, Kyle Williams, Jonathan Babineaux, Darnell Dockett, Justin Smith, Albert Haynesworth, Jay Ratliff, Tommie Harris, Marcus Stroud, Vince Wilfork, and Pat Williams.

Still not convinced? Here are the top cornerbacks: Darelle Revis (who else?), Dominic Rodgers-Cromartie, DeAngelo Hall, Charles Woodson, Asante Samuel, and Tracy Porter. Rhonde Barber and Champ Bailey are not far behind.

And lastly, here are the safeties for 2009: Bernard Pollard, Darren Sharper, Justin Leonard, Brian Dawkins, Adrian Wilson, Roman Harper, and Ed Reed. Care to guess #1 and #2 from 2008? Troy Polamalu and Ed Reed.

+WPA produced an impressive list of players. Just consider the Pro-Bowl appearances belonging to those players. (This might mean Pro Bowl selections ignore negative performance, just like +WPA does. Or it means +WPA is a pretty good stat for defenders. I think it's probably a little of both.)


At best, +WPA and +EPA only tell half the story. But if you read enough stories, halfway through you kind of know how they're going to end. The big point is that the performance we can't see and measure correlates tightly with the performance we can see and measure. We can invent all kinds of scenarios where these stats aren't going to be unfair to this player or that player. And the truth is no objective, quantitative football statistic will ever capture every individual contribution of a player, but +WPA and +EPA are a good start.

+WPA = Playmaker


Popular Video