The advertisements above do not necessarily reflect the views of this blog, its authors, or host.

Superbowl Predictions – Amateurs Beat Pros

This past Sunday night, the New Orleans Saints defeated the Indianapolis Colts in The Superbowl, which is the annual championship game of the National Football League. The final score was 31-17. Year to year, The Superbowl commands one of the largest television audiences (one hundred seven million people watched the whole game, and another estimated fifty million people watched a part of the game.) Likewise, The Superbowl brings out the better (as in placing a wager) in millions of people. For some people, the Superbowl is the only bet they engage in all year.

In American culture, “Superbowl Sunday” has become somewhat of an unofficial holiday celebration. People gather, parties are thrown, lots of food and drink is consumed, and almost everyone has an opinion on the outcome of the game. It’s fun to make your best guess, and then see how close you came to being correct.

But how does an average person’s casual guess stack up to the predictions of “professionals”? How do our guesses compare to the people who immerse themselves in professional football every day? If you are going to place a bet, are you better off following the advice of insightful professionals, or rank amateurs with merely a passing interest?

I came across this blog post from ESPN (Entertainment Sports Network) which is the largest sports television network in the United States. Thirty seven ESPN on-air personalities made their Superbowl predictions of the winning team and the final score. Twenty five of them chose the Colts and twelve chose the Saints, which is a 32.4% accuracy rate. None of those twelve predicted the final score correctly – the closest prediction was Saints 31-24.

Thirty seven professionals is a pretty good sampling from the football coverage community. So by comparison, I wanted to see how thirty seven non-professionals would pick the game. Sunday morning, I posted this on my Facebook page:

“I need your Super Bowl predictions by 6pmEST tonight. Please predict the winning team and final score. Also, ranking on a scale of 1 to 10, please state how sure you are of your prediction. Lastly, please predict something that will wind up being unique to tonight’s SB game compared to SB’s of the past. (Examples, predict the first ever SB drop-kick attempt, or predict the longest pass for a touchdown in a SB.)”

From all the respondents to my Facebook post and from those that sent me personal messages on Facebook, I was able to extract seventeen predictions. I had to scramble Sunday afternoon to come up with twenty more respondents in the form of extended family members, friends via telephone and email, and co-workers replying to my text messages. Once I received the thirty-seventh prediction, I compared the data.

Twenty of my thirty seven respondents chose the Saints, and seventeen chose the Colts. That’s a 54% accuracy rate. None of the twenty predicted the final score correctly, although one person guessed 37-24 (a 13-point margin, so I consider it the closest guess.)

While there is no appeal to the paranormal on the part of the ESPN analysts, there does tend to be an appeal to authority when it comes to people taking advice on how to best craft a guess. If you are going to place a bet, then it would seem logical that you would want to seek the advice of the people who understand the nuances, idiosyncrasies, and understanding of the finer points of the game and the teams involved.

This is the reason I asked everyone to tell me how confident they were in their picks. Although the professionals did not express a confidence level in their predictions, I think it is safe to assume that they would all be relatively high. I set the pro’s confidence bar at 7.5-out-of-10. By comparison, my thirty seven respondents’ average expression of confidence was 6.5-out-of-10 (only 27 out of 37 gave a confidence rating.)

Not unlike the stories about the monkeys who make stock market picks that wind up outperforming the picks of stock market professionals, the same seems to hold true when it comes to guessing football outcomes. The bottom line is that professional prognosticators, armed with knowledge, experience, and confidence in their areas of specialty, perform statistically the same as non-professionals when it comes to predicting the future.

As an aside, none of the unique perditions made by the respondents came to pass. I added that layer of data to see how creative my Facebook respondents could get. There were several unique occurrences in this year’s Superbowl, including the only non fourth quarter on-sides kick in Superbowl history, the most pass completions combined for the starting quarterbacks, and the first place-kicker to make three field goals from distances of forty yards or more.

5 comments to Superbowl Predictions – Amateurs Beat Pros

  • KeithJM

    You have a pretty good sampling of predictions from the two groups, but you only have one event to compare to the predictions. To make it worse, that event has a binary result (A or B) and is at least partly random.

    An equivalent test:

    We will toss one coin, and ask 37 experts and 37 non-experts to predict the outcome. The experts have read this story: http://www.npr.org/templates/story/story.php?storyId=1697475

    They ask some questions about how we’re going to flip the coin and predict heads. The non-experts predict tails. I could make up a coin toss result but it doesn’t really matter. There is a decent chance (though perhaps less than 50%) that the experts will be wrong on this one coin toss.

  • petrucio

    I do agree, a sample size of ONE event couldn’t be more subject to confirmation bias.

    I wonder what article we would be reading if the SB results were different.

    The important sample size here is the number of events being watched, not the number of guessers.

  • stargazer9915

    Give Evan a break. He never claimed that this was a double blind clinical trial. This was the first and only game I watched all year and I picked the Saints with very little info to go on. This was just a casual study, but we can still extract a little information from it. Confirmation bias? I think not. Do not read to much into it.

    I do, however, agree that even the experts do not perform 100% and can even do worse than random guessing at times. Sports and economic predictions are not science and should not be looked at as such. For all we know, Sylvia Brown could have picked New Orleans for the big game, but does that mean psychics exist?

    Take it for what it is, a fun experiment by a football fan, and be happy with it.

  • rsm

    There is a lot of randomness in sports.

    The probably outcome by skill was that the Colts would win, and the need for the ‘pros’ to come up with a narrative to justify it is what makes them sound so certain. The likelihood of the Colts actually winning was probably the equivalent of a weighted coin toss 53-47 or something else ridiculously close. The problem is separating the narrative from the actual underlying skill and luck.

    If any one actually is interested in the different luck/skill components of the various major sports there are a number of journal articles around, but my current go-to for it is the following blog post by Tom Awad (mostly hockey focus, but this was specifically on identifying the actual skill component of wins across the major sports). His math is above my head, but from the ongoing discussions around his work the ‘peer review’ seems to validate his work.

    A couple of relevant links:
    Tom Awad
    On separating out the narrative

  • halincoh

    Thanks rsm – good reference for a sports geek

Leave a Reply