For data guys like Aike, Village, and anyone else -- would you consider Kenpom a "good" model? My perception is it's a very slow learning rate model. Preseason bias takes a while to shake off and sometimes continues all the way to the NCAAT. Another perception is that too much weight is given to blowouts against bad teams.
My ideal would be 0 weight to any game 50 or more spots under where you are, which means you really couldn't rank teams until league play. I think the MMR system used in competitive video games does it right -- you only move up by playing, and beating, competition your level.
Asking because every year I have one "Kenpom" bracket, and that always flops and does much worse than my "eye test" one. So my perception is the model isn't really good. Could also be the model was intended to do something very different than "predict the score if these two teams played". Would be very interested in any insights.
I look at it as one data point. By midway through the season, it does a pretty good job of predicting the point spread between two teams.
I kind of think it’s early season results are problematic if you want to get the best gauge on reality. But they aren’t unreasonable. I assume that there’s a marketing aspect behind why he does some of the things he does.
The reason I say it’s one data point is that all models will have some bias. You can build 2 models that come out with almost identical overall results, but use slightly different inputs.
Just as an example, maybe one uses free throw shooting as an input, and another uses three point shooting as an input.
Those two models may get similar overall results, but you easily see how a poor three point shooting team may fall too much, and good free throw shooting team may rise too much.
Maybe a third model would use effective field goal percentage and would give you a better all around model. But probably all three models will tell you Michigan is one of the best 2 or 3 teams right now.
My model is optimized for NCAA tournament results, so it tends to do better than Kenpom predicting tournament game winners. But the tournament is so random that his model will almost definitely beat mine occasionally.
Also, how do you define “beats?” If you are asking who came closer on each point spread, that’s one thing. Just picking games is another.
