Advanced NFL Stats: Koko's 2009 Fantasy Report Card

| by

Last August Advanced NFL Stats ventured into the fantasy world with its own player projections. But these projections weren’t what you might expect. Instead of trying to develop a complex and advanced projection system, the Koko system projected fantasy performance using the simplest rules reasonably possible.

The Koko projections, named for George Costanza’s simian nickname from his tenure at Kruger Industrial Smoothing, are what a monkey might guess given the typical regression rates from one season to the next. Koko is a (bad) rip-off of the Marcel baseball projections created by Tom Tango. Instead of competing with other projections, the intention is to establish a minimum baseline of predictive power against which other projections can be measured. Further, Koko tests whether other systems are really worth all the additional analysis, subjective and objective, that goes into them.

Koko looks at the separate regression rates for the various aspects of fantasy performance, such as touchdowns, yards, and turnovers. It estimates future performance based on those rates, on a per-game basis, then sums them for a total projection for each player. For example, TDs regress quicker than yards, so a player that did well because of gobs of TDs is likely to be overrated the following year. And a player who did well because of lots of yards but few TDs may be underrated. The 2009 projections can be found here: QB, RB, WR, TE, K, DEF. Scoring is based on 6 points for a TD, 1 pt for 20 rushing yds, 1 pt for 50 passing yds, -2 for a turnover, and -1 for a sack.

So how did Koko do? Are other systems, with their expert assessments, really any better than a simple regression? I compared Koko to the Yahoo fantasy projections to find out. I chose Yahoo because it’s the most popular, it’s free, and I happen to have its 2009 projections on hand.

Below are the Koko, Yahoo, and actual rankings for QBs in 2009.
.nobrtable br {display: none} table {border-collapse: collapse; border-width: 1px 1px 1px 1px; border-style: solid;} td {padding: 3px;} #colorcol {background-color:#ffffe0;} td.headhover:hover {text-decoration: underline; cursor: pointer; cursor: hand}

Player Act Rank Koko Rank Yahoo Rank
Drew Brees 1 1 3
Peyton Manning 2 3 5
Aaron Rodgers 3 7 2
Tom Brady 4 8 1
Matt Schaub 5 12 7
Philip Rivers 6 6 8
Tony Romo 7 2 12
Ben Roethlisberger 8 20 14
Donovan McNabb 9 9 6
Kurt Warner 10 4 4
Eli Manning 11 13 18
Matt Ryan 12 21 13
Jay Cutler 13 5 9
Carson Palmer 14 11 15
Kyle Orton 15 17 24
Joe Flacco 16 24 20
Jason Campbell 17 23 19
David Garrard 18 15 11
Matt Hasselbeck 19 26 10
Kerry Collins 20 28 25
Matt Cassel 21 14 22
Shaun Hill 22 10 27
Marc Bulger 23 30 28
Trent Edwards 24 25 16
Jake Delhomme 25 22 26
Derek Anderson 26 16 42
Chad Pennington 27 18 17
JaMarcus Russell 28 29 32


I measured the accuracy of each system in two different ways. I measured rank order accuracy by using the Spearman rank correlation coefficient [] for per-game performance. Additionally, I measured the total season accuracy by the correlation of raw total points.

The Spearman rank correlation for Koko was 0.72 and for Yahoo was 0.82.

The season point total correlation for Koko was 0.73 and for Yahoo was 0.79.

So Yahoo edged out Koko, at least for QBs. It’s about what I’d expect. In Koko’s defense, it does not know who switched teams, who added a free-agent WR, or who lost his all-pro starting left tackle to injury or free agency.

For running backs, Koko did not fare so well, but neither did Yahoo. It appears much more difficult to predict RB performance, and the difficulty appears to be related to projecting the number of carries a RB will get, because of sharing the backfield, injuries, rookie performance or other reasons. For RBs, the Spearman rank correlation for Koko was 0.20 and for Yahoo was 0.41.

The season point total correlation for Koko was 0.26 and for Yahoo was 0.56. That’s a big difference. Simpler isn’t always better.

So is there any point to Koko rankings? Sure. For starters, they provide a benchmark for comparing other projections. Second, if there is a big difference between what Koko projects and what another system projects, it’s worth examining why there is such a big difference. Over the next few days, I’ll roll out the 2010 projections.

Notes: I excluded the Vikings' QBs because the 2009 projections were made prior to Favre's return. I did not do comparisons of other positions because the process is labor intensive, and I doubt the results would be much different for receivers.