Some time back I wrote about how algorithms are changing our world, affecting several professions. This is a sequel of sorts to that post: this time, the topic of fascination is algorithms (more to context statistical models) in sports, specifically in prediction of winning teams.
Algorithms captured popular imagination when statistician Nate Silver correctly predicted results of US presidential elections in all 50 states and D.C. He is still probably the only statistician who’s recognized in bars and on streets. Of course, his methodology is secret and nobody is quite sure how his model work. If you’re interested, The Guardian took a decent stab at figuring out how he did it.
During the recent FIFA World Cup in Brazil, Microsoft pulled a Nate-Silver-esque feat of correctly predicting the outcome of every single match from the knock-off stage. Unlike Silver, Microsoft was open about their models, most of which are based on work by renowned statistician and researcher David Rothschild. The best part is that his forecasting mechanism bases the predictions on other forecasting models.
Not as easy as it sounds – the models still need to be continually tweaked on a number of variables like advantage of playing in home country (Brazil) / playing surface / weather. As ‘Inside Microsoft Research’ notes, unlike baseball with it’s abundance of statistics on games, players and playing fields (think Billy Bean from Moneyball) football World Cup is especially difficult. It happens once in four years and playing season is erratic – club football is far more ‘regular’ and popular.
Yet, Microsoft predicted correct outcomes for all 15 matches – a more detailed explanation of Microsoft’s model can be found here. Statistical predictions have definitely turned a new leaf with the constantly-evolving models. They may not be 100% there yet (pun not intended), but with accumulation of more data and more variables accounted for, they are getting closer.
Recently, statistician-du-jour Silver quit his job with New York Times and joined ESPN as editor-in-chief of fivethirtyeight.com. The website covers interesting analytical and predictive articles – about politics, sports and general – based mostly on statistical data available on topic at hand. Interestingly, Silver and fivethirtyeight didn’t quite get it right with FIFA World Cup – they predicted Brazil beating Germany in semi-finals.
And FIFA star predictor Microsoft missed by a wide margin when it came to predicting Indian election results in May. Its research predicted BJP winning 228 seats and Congress winning 87 seats, with the rest distributed among allies and regional parties. BJP’s 280-odd tally turned the prediction on its head.
Then again, you have to admit – Germany beating Brazil 7-1 was a freak occurrence – a fat-tail result in statistic-speak. Or so I would like to believe…out of respect for the beautiful Brazilian Futeball.
As for Indian elections, if seasoned politicos (most of whom were in power for 10 years) failed to gauge the public mood, how could Microsoft?