College Football Doesn’t Need a Better Playoff. It Needs a Better Selection Model.

February 1, 2016

Our final examination of the 2015 College Football Playoff comes on the heels of National Signing Day 2016, when over 1,500 high school seniors will pledge their allegiance to FBS and FCS teams. Ranking high school football players has become its own industry, with at least five national rankings services offering detailed assessments of individual prospects and class composition. “Rankings Fest” might be the best way to describe this phenomenon. Do not worry—we have no intention of coming to this particular party. It is crowded enough.

But it does seem an appropriate way to cap the college football “ranking” season, and a good time to take one final look at our own performance. So in closing the books, we ask “how did our Decision Lens College Football Playoff ranking model do?”

First, a quick primer. We used the Decision Lens prioritization platform to create a decision model to rank the top college football teams against a limited set of measurable criteria: FBS victories, strength of schedule, conference championships, wins over ranked opponents, and losses to non-ranked opponents. We also added one subjective criterion based on voter judgment. Our criteria tree is illustrated below.

Decision Lens College Playoff Rankings
Ranking Criteria Tree

Decision Lens College Playoff Rankings Ranking Criteria Tree

We gathered the data and tallied the results, comparing our findings to what the CFP committee recommended, and—today—what the bowl season results tell us.

To be blunt, our model performed very well. Our final ranking after the conference championship games suggested that Alabama, Clemson, Oklahoma and Stanford each merited a spot in the playoff. The bowl season results largely substantiated our model results, illustrated below.

Decision Lens College Playoff Rankings
All Criteria Equally Weighted
December 18, 2015

Decision Lens College Playoff Rankings All Criteria Equally Weighted

Stanford’s two-loss season made it politically unacceptable for the CFP Committee to select them over one-loss Big Ten champ Michigan State. But Stanford’s dismantling of Iowa in the Rose Bowl was telling. Recall that Iowa effectively played Michigan State to a tie in the Big Ten conference championship game (The Spartans’ late 22-play, 82-yard drive to win the conference title and spoil Iowa’s perfect season should be remembered as one of the most dramatic drives in the history of college football). Factor in Michigan State’s submission to Alabama in the playoff, and these results give a lot of credence to our assertion that Stanford belonged in the top four at the expense of any Big Ten team.

As our model suggested, Alabama reigned supreme. While I was personally a bit skeptical of Clemson, our model held fast and then Clemson delivered two very convincing performances in the playoff, thanks in no small part to DeShaun Watson. Finally, Oklahoma had a somewhat disappointing bowl game, playing poorly in their loss to Clemson. But their body of work, including wins over bowl victors TCU (Oregon) and Baylor (North Carolina) was hard to refute.

Notre Dame was a “bubble” team in our final rankings, but its decisive loss to Ohio State in the Fiesta Bowl brings to mind the adage about horseshoes and hand grenades.

As you can see, the data pretty much speaks for itself without a lot of need for human intervention. Perhaps the CFP committee can save itself some time and energy next season and simply adopt our approach. But we won’t take all the fun out of the ranking exercise. One bit of editorial I would like to add is that each of our model finalists showed one distinct commonality. These great teams also boasted highly impactful players at key positions. It is no coincidence that Derrick Henry (Alabama), Christian McCaffrey (Stanford), Deshaun Watson (Clemson), and Baker Mayfield (Oklahoma) finished in the top four of the Heisman Trophy voting. Causation, or correlation? You decide.

And that is pretty much it. This was great fun. Happy Signing Day! See you next fall!