Next week you can expect Zach Sanders to take over the weekly rankings digest. But this week I’m going to get the ball rolling with a discussion of the model and how it has changed for 2011.
The rankings and prediction model has been redone for 2011. Just like always, it's a team-efficiency logistic regression model. It's based on passing, running, turnover, and penalty efficiency. But now, running is represented by Success Rate (SR) rather than Yards Per Carry (YPC). SR correlates far better than YPC with winning games. YPC is too susceptible to a handful of relatively rare break-away runs. I think of running as a ‘jab’ and passing as a ‘cross’ or ‘uppercut’. The jab is a low-risk punch that doesn't expose your defenses, keeps your opponent off balance and guessing, and keeps him from purely defending against your cross. A good jab is a prerequisite, but the cross is what scores points and wins bouts. SR captures this aspect of the running game.
Running is more than that, of course. It’s essential in short yardage and inside the red zone, and when team has a lead, it burns clock and helps keep the ball out of their opponent’s hands in the 4th quarter. I believe the revised model better reflects the true inner workings of the sport.
There are always new readers each year, so here is a quick and dirty refresher on how the model works. A logistic regression is fed net YPA, run SR, and interception rates on both offense and defense, plus offensive fumble rate. Team penalty rates (penalty yds per play) and home field advantage are also included. These particular aspects are selected because they are predictive of future outcomes, not because they explain past wins. This is a distinction overlooked by most experts and even other stats-oriented sites.
The regression produces the coefficients used in the model. In other words, it tells us how each facet of team performance is best weighted to predict which team will win a game. Each team variable is regressed again to account for how reliable each particular facet is throughout a season. In other words, the facets vary in terms of how consistent they are from game to game. For example, offensive passing efficiency is most consistent, and turnover rates are least consistent.
Turnover rates explain past outcomes very well, but a relatively small part of turnover rates are carried forward. If a team has a very low interception rate of 1.0%, how likely are they to continue the season with few interceptions? Chances are they will remain better than average, but not nearly as low as 1%.
Next, I create a notional team with all league-average statistics. With the regressed values of team efficiency, I use the model to generate the probability each team would win a game against the average team at a neutral site. I call the result Generic Win Probability (GWP). In theory, this should be a team’s long-term ‘true’ winning percentage.
But it’s not complete. Lastly, I take each team’s average opponent GWP, and use it adjust the numbers so that the final GWP accounts for previous strength of schedule.
Generic offensive team efficiency (OGWP) can be estimated by setting each team’s defensive variables to the league average, and re-computing their probability of beating a completely average team. Generic defensive efficiency (DGWP) can be estimated in a similar way.
An explanation of the principles behind the model and an example of how it is calculated can be found here.
Each season, I end up answering the same challenges to the results of the model. So I’ll preemptively address the most common ones.
1. “Your dumb model fails to conform to my intuitive beliefs about how good each team is. And besides, it does not conform to what I’ve been told to think by [major media personality].”
Answer: What can I say? Who is right? All the talking heads that told you for weeks the 13-3 Falcons were the best team in the NFC, or the numbers here that told you that 10-6 Green Bay was actually the much better team? Your intuitive estimates of team strength are far less accurate than you imagine. The thing is, you’ll forget how wrong you were by the end of the season, and re-wire your memory to trick yourself into believing you ‘knew it all along.’ We all do. Want to have a laugh? Go back and look at the expert predictions in the early weeks of 2010, or 2009, or whichever year you can find. This model will have some laughers too, just not nearly as many. Good statistical models have no preconceived biases, are not wowed by spectacular but lucky game-winning plays. They don’t follow the crowd. They don’t believe in streaks, destiny, grudges, or momentum. They don’t chase recent wins.
2. “Your model doesn’t take in to account determination, good coaching, effort, and character.” Answer: Yes it does. To the degree those things show up on the field on Sundays, those things are captured.
3. “How can team X have a higher ranking than team Y if team X’s offense and defense are both ranked behind team Y’s?”
Answer: I can understand this question. It’s unusual, but in some cases the OGWP and DGWP don’t make logical sense when you mentally combine them and compare them to a team’s overall GWP. This is mostly because of penalty rates. The NFL tracks team penalties but does not divide them into offensive and defensive categories, so they count neither toward OGWP nor DGWP. They are, however, included in overall GWP. If you see a team with an unusually high or low GWP compared to their O or D rankings, check out their penalty rate. It’s probably well above or below average. Also, the final results depend on how teams are bunched together. Sometimes a #3 DGWP team is a mile ahead of the #4 team, and sometimes it's just a hair better than the #4 through #9 teams.
4. “How can you possibly have [perennial doormat] Team X ranked ahead of [current media darling] Team Y?”
Answer: Look at the efficiency stats in the second table below. That’s just about all you need to know. Yes, I understand no one else outside of the state of Texas has the Cowboys ranked as the #2 team in the league this week. But are they aware that (despite their injuries) DAL has an 8.3 net YPA? Are they aware that they allowed only a 5.8 net YPA on defense? Both are near the very top of the league. True, there are a couple other teams with as good numbers as these, but how tough has their schedule been?
Example: Why is NO #1 and GB #9 when GB beat NO in week 1?
Answer: Check their numbers. Check their opponent strength. (GB has a relatively poor defensive pass efficiency and has played a somewhat lighter schedule.)
5. “Your model said Team X had a 90% chance of beating Team Y, but they lost! Ha!”
Answer: Yes, that happens…about 10% of the time. And in fact, I’m glad it does. If it didn’t, the model would be under-confident.
6. “Your model doesn’t account for the fact that [undrafted rookie 4th string quarterback] is starting in place of [superstar who just got injured].”
Answer: That’s true. Use the model as a starting point, and adjust on your own. Or, even better, we can insert a reasonable guess as to the rookie’s expected net YPA and interception rate, and recompute. What’s amazing to me is that the model is completely unaware of injuries, and yet still manages to slightly outperform “the market.”
7. "It's easy to pick straight up winners. You don't pick against the spread." Oh, it's easy? Really? You might want to double check that. See #1 above regarding "knew it all along."
8. "Website Z has Team X ranked 10th but you have them ranked 3rd! And their rankings conform to my intuitive expectations!"
Answer: The stats at Website Z are crap. Stop reading it. Go wash your eyes out with rubbing alcohol, and come back here after your sight returns and reread what I wrote about predictive vs. explanatory stats.
Ok, I could go on, but that’s it for now. Here are the first rankings of the 2011 season. Click on the table headers to sort. See the second table below for raw team efficiency stats.
Get the rest of this article over at Advanced NFL Stats.