The problem that XKCD gets at is simply translating/scaling the results; the article is solving a different problem.
Early on in the Netflix Challenge, I was able to get myself (very briefly) a leaderboard score with nothing more than analyzing every user's ratings; re-centering them by their mean, and re-scaling them according to their standard deviation. The by remembering their translations and scales, I could put a globally-predicted score back into their own language.
So just some very basic statistics is sufficient to erase much of the bias toward higher numbers, as well as halo effects and the like.
(I was pretty surprised that Netflix's own algorithm apparently wasn't doing anything this simple)
I was pretty surprised that Netflix's own algorithm apparently wasn't doing anything this simple
Netflix does have really interesting blind spots. They claim to take ratings seriously, to the point of offering a million dollars for the best rating algorithm. Then, as the GP says, they implement the rating algorithm in a way that renders it completely worthless to any household with more than one viewer.
Netflix does offer us a good demonstration of the failings of absolute technocracy, but it leaves the question of how best to rate movies wide open.
Early on in the Netflix Challenge, I was able to get myself (very briefly) a leaderboard score with nothing more than analyzing every user's ratings; re-centering them by their mean, and re-scaling them according to their standard deviation. The by remembering their translations and scales, I could put a globally-predicted score back into their own language.
So just some very basic statistics is sufficient to erase much of the bias toward higher numbers, as well as halo effects and the like.
(I was pretty surprised that Netflix's own algorithm apparently wasn't doing anything this simple)