Feb 09 2017

The Super Bowl and Hindsight Bias

Brady SB51Full disclosure – I have been a Patriots fan since in the 1980s. I suffered through a couple long decades of rooting for a mediocre team, including the worst (at the time) Super Bowl defeat at the hands of the Bears. Then along came Belichick and Brady, and it has been a wild ride as a fan.

Super Bowl LI was perhaps the pinnacle – the Patriots came back from a 25 point deficit to tie the game and then win in sudden-death overtime. I feel genuinely bad for Falcons fans, but perhaps worse for those who stopped watching the game in the third quarter because they thought it was over. Those who stayed through to the end were rewarded with historically epic football.

(As an aside, I am a fan simply because it is fun to have a team to root for. Don’t read too much into it.)

What is interesting, from a critical thinking perspective, about the game is the way in which we construct narratives to explain random events, or at least events that have an element of randomness or “luck” involved. At half-time the Falcons were up 21-3 and the discussion among the commentators was all about how well the Falcons were playing and everything the Patriots were doing wrong. The Falcons had “momentum” and the Patriots had to figure out a way to steal this elusive “momentum” back.

Part way through the third quarter the Patriots started scoring, and the Falcons could not sustain a scoring drive. The Patriots ended up scoring 31 unanswered points. Clearly they got their mojo back, and were destined to win the whole time. The narrative then shifted to how Brady was perhaps the best quarterback of all time and Belichick is clearly magic, or perhaps made a deal with the devil. More seriously, some disgruntled fans were convinced the game was “rigged.” How else could you explain such a dramatic reversal of fortune?

But here’s the thing – I don’t think momentum in NFL football really even exists. Momentum is like a winning or losing “streak,” it is an illusion of the drunkards walk, applied only in hindsight to explain what happened. When statisticians examine professional sports they find little evidence for a “hot hands” effect in basketball or any other phenomenon that deviates outcomes from a statistically random pattern. I know there is still some debate about whether or not a tiny effect can be teased from the data, but there is certainly no dramatic effect. If not all of the streaks in professional sports are random, then they are mostly random.

If a basketball player scores on 50% of their shots on the basket, then any given shot has a 50% chance of hitting, regardless of whether the last three shots hit or missed.

Baseball pitchers occasionally pitch a perfect game, without a single batter on the other team getting on base. If you calculate from average batting averages how often this should happen purely from chance, that is how often perfect games occur. There was nothing special about those games. It wasn’t the pitcher’s lucky underwear, or the home field advantage, or anything else. It was just a statistical streak.

I say this of professional sports because at that level the players are the best in the world. They are highly skilled and professional, which means minimizing the effects of variables like not making a strong effort, being distracted, or other psychological effects. Such factors become too small to detect, at least statistically.

In football there are some real factors to consider. Injuries may reduce a team’s effectiveness throughout a game, and the Falcons did lose a few players by the end of the game. There is also a chess-like competition between coaches, with each side making adjustments to counter the other’s strategy, and trying to outwit the other team. Belichick has a reputation for being exceptional at working the game, analyzing the match-up and making adjustments. Brady also deserves credit for never giving up, even when they were down by 25 points and their odds of winning were close to zero.

But a large part of the epic comeback was just chance. In the first half of the game the breaks just went against New England. Two turn-overs resulted in two of the Falcon’s three touchdowns. Little differences, or just a couple of quirky plays, can have a big different on the scoreboard. What’s interesting is that the commentators often base their narrative not on overall play, but on the scoreboard.

While I agree the Falcons were playing better than New England in the first half, that does not fully explain the 28-3 lead they held in the third quarter, which makes the Patriot’s come back seem more amazing. In reality, the Patriots were the same team in the first half, the dice were just not falling their way. The second half was mostly regression to the mean.

The Patriots also got lucky in the fourth quarter. One of their touchdowns was helped by three holding calls against the Falcons that led to Patriot first downs. It is possible that if not for one holding call the Patriots would have lost the game, then the narrative would have been very different. Instead of talking about how Brady was the best quarterback of all time, commentators might be saying that his career is over – because of one holding call.

I also think there is an interesting analogy to the recent presidential election. Hindsight bias causes us to look for explanations and invent narratives to explain quirky or close outcomes. Trump won the electoral vote because of narrow victories in three close states. Just like in football, small differences in total voting had a magnified effect on the “score.” If only 80 thousand votes had gone the other way, Clinton could have won.

But the narrative after the election treated the outcome as inevitable. Everything Trump did “worked,” it’s a sign of the times, and there were numerous explanations for Clinton’s failure. Flip a small percentage of votes and the narrative would also have flipped.

There were very real factors affecting the outcome of the election, but the pitfall is in overinterpreting narrow outcomes as if they were inevitable.

Super Bowl LI was a statistical outlier. Prior to this game, the three Super Bowl games which involved a team winning after being down by 10 points were the greatest comebacks. Now that number has risen to 25 points, and this record will likely stand for a long time. That such a comeback occurred in one game out of 51 (and probably will not be replicated for another 50 games) is consistent with randomness. It is epic because it is rare, but even rare events statistically should happen at a rate proportional to their rareness.

It is OK to have fun reveling in a statistically rare event, especially when the dice roll your way. Just don’t buy into the narratives people invent to impose a comforting order on the chaos.

55 responses so far

55 Responses to “The Super Bowl and Hindsight Bias”

  1. mumadaddon 09 Feb 2017 at 8:39 am

    Is it fair to say that the scorecard is a surrogate outcome for a load of other factors that affect how well the individuals or team performance as a whole, and the statistical regularity of the scorecard is actually reflecting the statistical regularity of the confluence of all these factors?

  2. mumadaddon 09 Feb 2017 at 8:39 am

    Sorry, typo…

    Is it fair to say that the scorecard is a surrogate outcome for a load of other factors that affect how well the individuals or team *perform* as a whole, and the statistical regularity of the scorecard is actually reflecting the statistical regularity of the confluence of all these factors?

  3. RCon 09 Feb 2017 at 9:14 am

    Of course randomness is a huge factor, but I’m not sure I agree that the comeback was almost entirely luck (luck was, of course, a huge factor).

    “Momentum” was a factor here, and by momentum I really mean defensive fatigue caused by multiple long scoring drives. The players were mic’ed up for the game, and most of the audio from late game is essentially just patriots offensive players talking about how the falcons defenders eyes were glassed over.

    Part of this was the fact that the Patriots ran a low variance ball control offense most of the game, while running a high variance defense (that either gave up scores quickly, or got stops). Because of this, the Patriots ended up running 95+ offensive plays while the Falcons ran 46. “Momentum” meant something here because we were talking about a team that had already been on the field for 75 snaps being asked to run 20+ more in quick succession with almost no breaks. That made the statistical reality very different than your typical 4 score margin.

  4. Nareedon 09 Feb 2017 at 9:40 am

    I stopped watching in the first quarter, when I realized I just wasn’t interested in who won or how they did it. I decided to cook my meals for the week instead. But I did take time for the half time show. And that was maybe the best I’ve ever seen.

    Back on topic, I’m curious how does random chance affect blowout games like Dallas vs Buffalo back in the 90s.

    As a Steelers fan, I do recognize without Harrison’s interception of Warner at the end of the first half, Pittsburgh would have likely lost that Super Bowl.

  5. magsolon 09 Feb 2017 at 9:43 am

    I’m not an expert, but I do follow a couple of experts who suggest that some other confounding factors, in addition to underpowered statistical tests, may actually be hiding a true “hot hand” effect:

    https://punkrockor.com/2014/10/09/underpowered-statistical-tests-and-the-myth-of-the-myth-of-the-hot-hand/

    The upshot is that in sports where the defense can “compensate” for streaky players–mainly, basketball and football–there may yet be an effect but it’s drowned out by, for example, the hot player being double-teamed.

    The work doesn’t really seem to make a conclusion either way, but just points out that, based on the methods we’ve used so far, we can’t necessarily rule out the possibility of a hot hand.

  6. TheGorillaon 09 Feb 2017 at 10:52 am

    But this is also a narrative to put order onto the chaos 😉

    I’ve never been impressed by the hot hands research. When something contradicts the experiences of anyone who had ever done anything at a high level they’d better have convincingly isolated the phenomenon, and that’s just not happening

  7. Lane Simonianon 09 Feb 2017 at 11:19 am

    As a sports fan and as someone interested in politics, I find this conversation interesting. I think there are often warning signs that people do not always give sufficient heed to at the time. In regards to the Super Bowl, the Falcons had very few offensive plays (although they were quite proficient in turning those plays into points in the first half). It may have been in part that New England developed a better blocking scheme in the second half, but the Falcons defense was beginning to wear down. Luck plays a role, but many things are not due to chance.

    In regards to politics, in retrospect, Hillary did not pay enough attention to the economic concerns of people in the rust belt. She in a sense allowed Trump to control the narrative through a false populist message. The fact that the economy as a whole had improved tremendously over the last eight years and that with few exceptions many blue collar workers had voted Democratic in the past did not matter. Trump combined an us versus them rhetoric with an I will bring back jobs message to narrowly win the states he needed to take the election.

    As an historian, this is one of my favorite cartoon quotes ever (Bill Watterson, Calvin and Hobbes).

    “History is the fiction that we invent to persuade ourselves that events are knowable and that life has order and direction.”

  8. unitedcats1957on 09 Feb 2017 at 11:49 am

    The latest election isn’t a very good analogy. In your sports example you have two professional teams at the top of their game, and luck most certainly becomes a huge factor. The last US election pitted an experienced professional team of campaign professionals against a team largely consisting of amateurs. Saying they won by luck discounts that. The Clinton team should have won if skill and experience at running elections counts. That they lost isn’t bad luck, they clearly made a number of serious mistakes and miscalculations. There is every reason to analyze how they lost, lest they do it again.

  9. TheGorillaon 09 Feb 2017 at 12:00 pm

    What bothers me most about the election analysis that’s gone on is how much of the focus has been on the numbers — so and so should have campaigned more here, this demographic didn’t turn out like it had in the past, etc etc. The question that actually matters, whether Trump had ended up winning or not, is *what aspects of our culture allowed a radically different phenomenon like Trump to happen?*

    unitedcats is perfectly right when they note the big difference between the two, although I think the analogy still conveys what it’s intended to (if someone isn’t trying to willfully misunderstand *cough hardnose cough*).

  10. Steven Novellaon 09 Feb 2017 at 12:13 pm

    united – I did not say Trump won by chance. I said that hindsight bias leads us to feel as if an outcome was inevitable and way over interpret the “causes.” One aspect of that is that, with the electoral college and football, little differences can have big outcome differences. Clinton won the popular vote by 2.9 million, yet you are analyzing the outcome based upon 80k votes in three swing states.

    Back to football, again, I am not discounting fatigue and other factors. My point is that quirky small events, like an interception or one holding call, have disproportionate effects on the outcome and then we way over interpret.

    Take this thought experiment – two statistically equal teams playing against each other in a perfect computer simulation. There is no psychology, fatigue, or other factors – just their pure statistics. Run that simulation 51 times and I bet you get outliers similar to this game. Therefore – you don’t need to find a reason to explain the outcome. Random chance is enough. That doesn’t mean there weren’t other real factors like fatigue differential, just be really humble and cautious in such conclusions.

  11. Babbyon 09 Feb 2017 at 12:17 pm

    I don’t think the comparison to politics in the article was intended to say that the election outcome was luck. It is referring to assigning numerous narratives after the fact – such as it was inevitable, democrats provided a flawed candidate, Trump spoke to the millions of unheard American’s etc.

    However 80,000 votes in three states is a very small percentage of votes to tip the election in someone’s favour and could have gone the other way with very small adjustments. Then all those narratives reverse – Republicans presented a flawed candidate, Trump alienated too many independents, it was inevitable.

    What caused the Democrats to loose these three states by this small margin? You could point to the Clinton team not campaigning there and relying on a statistical model that said they were winning even though it said the same thing about Bernie Sanders the same year and they lost. Nate Silver cites the Comey letter as a big game changer, or the intelligence agencies assessment of Russian hacking of party emails and influx of fake news stories from the Kremlin. It could be one or a combination of all those smaller, idiosyncratic factors that caused this small 80,000 vote difference. But that’s not the narrative – the narrative was that it was inevitable given a flawed candidate not taking into account the mood of the nation. This may have been the reason it was close – but when the results were that close you certainly can’t call his victory inevitable.

    And you can certainly argue the we are assigning too many narratives to something that could have easily gone the other way based on smaller events. In reference to the Super Bowl, the article didn’t say it was all luck and actually mentioned a number of reasons why they won and other non-luck factors like Coaching strategies.

  12. BBBlueon 09 Feb 2017 at 1:12 pm

    But a large part of the epic come back was just chance.

    And a large part was due to poor play calling on second and 11 from the Pats 23. That was young Shanahan rolling the dice and letting principles rather than probabilities guide his decisions. Not Seattle-passing-at-the-goal-line bad, but bad. As a 49er fan, after that debacle, I have mixed emotions about the new head coach.

  13. Teaseron 09 Feb 2017 at 2:31 pm

    At half time I searched for Team Leading at half superbowl and I found this stat.

    The second statistic we looked at was who was leading at half time. What we found may not surprise you, but if a team is leading at halftime, they go on to win the game almost 85% of the time.

    There are two recent examples that bucked this trend, which came in 2008 and 2010, when neither the Giants nor Saints were leading at halftime of their most recent victories. The only other example in the past 20 seasons is when the Dallas Cowboys overcame a 7-point halftime deficit to beat the Buffalo Bills.

    For those counting at home, there was one halftime tie between the Patriots and Eagles in 2005.

    https://www.sportingcharts.com/articles/nfl/statistics-that-predict-super-bowl-winners.aspx

    Based on this statistical calculation I felt that the Falcons would most likely win. Knowing that there was a %15 chance that the Patriots could win made their comeback that much more exciting. The odds were heavily against them. Therefore they probably cheated!

    The Superbowl is a statistical shangri-la.

  14. Lightnotheaton 09 Feb 2017 at 2:49 pm

    Regarding Trump, I thought Nate Silver at fivethirtyeight pretty much nailed it in his article titled something like “what a difference 2% makes” in which he pointed out that the polls were only slightly off, and that was enough to have pundits talking about a crisis for the democratic party, dems being out of touch with rural whites, etc., instead of talking about an irreversible decline for the republicans because of changing demographics, etc. And with the super bowl, same thing, a few minor things going the other way and the analysis would be totally different.

  15. jt512on 09 Feb 2017 at 3:44 pm

    Steve,

    I agree that we rationalize outcomes with hindsight bias and see patterns where only randomness exists. However, you’re overplaying your hand. Some sequences of events, including in sports, are streakier than they would be if each event in the sequence were independent. For example, in basketball the “hot hand” phenomenon, where a player is more likely to score after a succession of successful shots, has recently been shown to be true. Professional basketball players really do shoot better after a successful shot. Analogous effects probably are true in other sports. After all, athletes aren’t machines.

    I find your post to be uncharacteristically evidence free. For instance, you state that the observed probability of a perfect game in baseball is equal to its theoretical probability. Do you have statistical evidence for that?

    Regarding Super Bowl LI, you state that the observed 25-point comeback, having occurred in one game out of 51, is “consistent with randomness.” That’s just assuming the conclusion. It’s also consistent with alternative hypotheses, including non-independence of successive football plays. You start with a belief that there is no “momentum” in football, and so you conclude that the comeback was a random event.

    In the comments you double down: “Take this thought experiment – two statistically equal teams playing against each other in a perfect computer simulation. There is no psychology, fatigue, or other factors – just their pure statistics. Run that simulation 51 times and I bet you get outliers similar to this game.” I don’t about know you, but I find it difficult to do computer simulations in my head. You’re just guessing at the outcome, or, in other words, assuming your conclusion.

  16. Steven Novellaon 09 Feb 2017 at 5:32 pm

    jt – I am not making this up. This is the results of the research. But here are some links:

    Here is a simulation of baseball games showing that reality matching random distribution of hits:
    http://sabr.org/research/modeling-perfect-games-and-no-hitters-baseball
    “The results were striking. In the 2,000 universes we ran, we found an average of 243 no-hitters, off by less than 4% from the 250 single pitcher no-hitters that actually occurred in 1876–2009.”

    In football the data is also mostly negative:
    ” But, having looked at a lot of NFL data, Moscowitz reaches a sobering conclusion: “There is a much stronger belief in momentum than is warranted by what we see in the data.””

    http://freakonomics.com/2011/11/20/football-freakonomics-is-momentum-a-myth-2/

    In basketball most studies show no hot hands effect. You are cherry picking an outlier that found a slight effect if you analyze the data a certain way. I acknowledged in the article that there was some dissent on this point, but if you look at all the research at best there is a subtle effect.

    I did not assume anything. I took as a premise the vast majority of research that shows no significant hot hands effect. The thought experiment was just that, an illustration of my point, not offered as evidence. The research is there.

  17. jt512on 09 Feb 2017 at 6:36 pm

    Steve,

    I’m glad you actually had evidence to back up your assertions. You didn’t disclose it, so it sounded like you were just assuming your conclusion. And the thought experiment you proposed wasn’t even a thought experiment.

    You are, however, misinterpreting the basketball data. I’m not cherry-picking. Miller and Sanjurjo (2016) proved that all the classic papers on the “hot hand” in basketball (and analogous papers in other fields) were incorrectly analyzed, using a method that is biased toward the null. When the data are correctly analyzed, they show a small hot-hand effect in basketball. It is not that Miller and Sanjurjo analyzed the data a “certain way”; they analyzed the data in a correct way, correcting a methodological error that pervaded the prior literature.

    Yes, the “hot hand” effect is small. Doesn’t matter. It’s real.

  18. Steven Novellaon 09 Feb 2017 at 6:47 pm

    jt – It absolutely does matter that it’s small for the purpose of my discussion. My point was that people overinterpret outcomes. They assume, essentially, that streakiness, momentum, hot hands, etc. are all or mostly real effects that require an explanation rooted in psychology, strategy, etc. Meanwhile the effects are small at best and the appearance of streakiness is mostly the clumpiness of random events.

    There is a massive difference between reality and perception. Whether the reality is a small or zero streakiness effect is irrelevant to my point.

    the proper caveats are in the article: “mostly”, “tiny effect” etc.

  19. Steven Novellaon 09 Feb 2017 at 6:58 pm

    Regarding Miller and Sanjurjo, I am aware of their paper. I wrote about it when it came out. I am still waiting for a response from other statisticians involved in this research. I have not seen a consensus emerge yet.

  20. jt512on 09 Feb 2017 at 7:32 pm

    Steve,

    Yes, I remember when you discussed that paper. In the comments, I posted an explanation of the methodological error that Miller and Sanjurjo discovered and what the correct way to analyze the data is. I don’t know if you ever saw my comment there.

  21. Fair Persuasionon 09 Feb 2017 at 9:06 pm

    Personalities play a big part in football and elections. I see the game changer in the Superbowl game as the character of New England Patriot’s Edelman. He was determined he could change the status of the game by intercepting the football and holding on to it. With respect to elections, the two main political party candidate personalities of Hillary and the Donald, turned off the youngest potential voters, and they did not come to the polls. So the 80,000 who voted one way are not as significant as the lost votes of the youngest generation who stayed away from the polls.

  22. edamameon 09 Feb 2017 at 9:46 pm

    It’s complicated by the fact that the probabilities are constantly changing, so the simulations are going to be gnarly. The Falcons acted like it was the same game in the second half as the first. This is part of what killed them.

    Football is partly about situational awareness. If the Patriots have a 28-20 lead, and are in field-goal range, I guarantee they won’t be snapping the ball with 15 seconds left on the play clock into a shotgun formation. Further, Brady would have been milking the hell out of the clock if the Patriots had a 25 point lead. The Falcons ran the ball 4 times in the second half! Complete meltdown in situational awareness.

    Belichick and the Patriots have success year in year out partly because he constantly thinks about the best approach in extremely specific situations, and drills his players on these details starting with training camp: he throws out random situations, downs, yardage, other details, and just rapid fire quizzes them what they need to do in extremely specific situations:
    https://www.youtube.com/watch?v=SVrIlXDZOF8&feature=youtu.be&t=52s

    The adjustments, what the situation demands, is constantly changing. Belichick is continually making adjustments to what the other team is doing, not just at halftime. Obviously, all good coaches do this. Atlanta really blew it because they didn’t make such adjustments. They kept playing like it was the first quarter. It was freaking madness.

  23. Steven Novellaon 10 Feb 2017 at 10:32 am

    edamame – I acknowledged all that in the article.

    The point remains (and this was the point of my thought experiment) there is a probability to outcome. You will get a two standard deviation or greater outcome 5% of the time by definition with a normal distribution. It will happen by random chance alone. When it does, people will look for a causal explanation of the specific outcome without putting it into the context of the distribution of outcomes.

    When researchers do look at the overall statistics, they find a pretty good random distribution (with debate about whether or not there is a small deviation).

    Further, what you are talking about makes the Patriots a good football team. It affects their chances of winning. It may skew game outcomes in that they perform relatively better later in the game, making “come backs” for them more likely. Ironically, the Patriots tend to be better at playing the statistics. The Falcons clearly failed to do that.

    Gould said it best – statistical outliers like this are luck superimposed on skill. You have to be a Patriots-level football team to create the possibility of this level of comeback happening. Actually happening also requires luck. Again – flip one holding call, and they lose.

  24. cloudskimmeron 10 Feb 2017 at 6:36 pm

    Long ago I read an analysis of major sports which concluded that the outcome of football games is largely chance, since the injury rate means there can’t be enough games played to definitively determine who is the best team. When there is a big difference in the score, yes, that team is clearly the best… on that day, with those team members, but in a close game, like this one, if you could replay it, the Falcons could clearly have won. (no, I can’t provide a reference–it was a long time ago.) Even in baseball, where they can play enough games to rule out chance, when the teams are evenly matched, it could still go either way since neither team is clearly superior.

    For those who say the Falcons were tired… well, I’m sure the Patriots were tired, too. And emotions probably did play a part. The Patriots were able to overcome the disappointment of so many things going wrong for them during the majority of the game, and the Falcons went into overtime with the feeling that they had blown a big lead, and that may have been a factor.

    I am curious about how people decide which team to root for. Does the fact that Tom Brady clearly cheated last year make anyone want to cheer for the other team, or do people feel they must stick with their team, regardless of how abhorrently they behave?

    I wish people would stop talking about “momentum” where it doesn’t exist. Momentum is the product of mass and velocity; emotional momentum is fiction, except in “explaining” things after the fact, which amounts to rationalizing.

    No one has mentioned that in our sexist society, women are at an extreme disadvantage, and I think it played a huge role in why Hillary lost. People–and yes, this includes women–don’t like to see women in positions of authority, which is certainly why the election was so close, even when a highly qualified woman was running against a buffoon. Yes, I am prejudiced. Why do you ask? I am also angry and disappointed that I live in a society that still finds it impossible to treat women as people. We are making progress, but the election showed that we haven’t made nearly enough. Getting back to sports, when players assault women, the fans send death threats… to the women; another sad commentary on our society, or perhaps on the mentality of some sports fans.

  25. edamameon 10 Feb 2017 at 7:33 pm

    cloudskimmer he did not clearly cheat. It seems more probable that the league botched the investigation and it all started because they neglected pv=nrt until it was too late:

    https://mobile.nytimes.com/2016/01/23/sports/football/nfl-ignores-ball-deflation-science-at-new-england-patriots-expense.html?0p19G=c

  26. edamameon 10 Feb 2017 at 8:56 pm

    cloudskimmer
    >>>For those who say the Falcons were tired… well, I’m sure the Patriots were tired, too.

    There is a huge asymmetry between offense and defense. The Patriots played 99 offensive snaps to the Falcons 43. Playing defense in football is much more physically exhausting, as a rule, than playing defense. This is why defenses often end up faking injuries and such to slow down rallying offenses. Those guys were spent by the middle of the fourth quarter.

    So both were tired, but the Falcons defense was wrecked.

    Dr Novella
    Yes I should be more clear. Note I’m not arguing for momentum in the traditional sense (e.g., there was an interception and things shifted right after that). I think that has largely been debunked.

    I appreciate your overall statistical point (curmudgeonly as it is on the hells of such a game), but in this game in particular, you can also point to some obvious (from a football perspective) strategic decisions (as you did mention).

    The suggestion that finding such reasons would amount to confabulation, post-hoc, I find a bit tenuous, especially for such a strategy-heavy game as football, is my point. Disentangling signal from noise, in a process with nonstationarities that are legion, is incredibly tricky.

    So I agree that momentum is dicey in the way it is usually defined by statisticians. But in this game in particular, being physically tired is a real thing, and that had a huge effect on the Falcons defense by the end.

  27. edamameon 10 Feb 2017 at 8:59 pm

    Snap counts were wrong in my previous post: I tried to do that from memory: Patriots had 93 and Atlanta 46.

  28. FuzzyMarmoton 11 Feb 2017 at 11:45 pm

    This seems to sum it up quite well:

    https://xkcd.com/904/

  29. BillyJoe7on 12 Feb 2017 at 1:16 am

    Can we all at least agree, contrary to the perception of most people, that momentum does play a role but that the effect barely rises above the noise? In other words, the momentum to chance ratio is pretty small, but the perception of most people – including those commenting here, as has been amply demonstrated – is the opposite.

    And I say this even though my team had one of the biggest comebacks of all time.

  30. jt512on 12 Feb 2017 at 2:07 am

    BillyJoe7: I think you’ve summed it up quite well.

  31. Steven Novellaon 12 Feb 2017 at 8:45 am

    I would like to point out that the statisticians gave the Falcons something like a 99% chance of winning at their peak in the game. So, we would expect the losing team to come back 1% of the time, or one game in 100. Right now we have this level of comeback in 1 game out of 51, which is a 2% chance, which is pretty darn close to what the statisticians said.

  32. BillyJoe7on 12 Feb 2017 at 12:23 pm

    …but, Steven, my team made one of the biggest comebacks of all time!

    😀

    (Sorry, I’m not intending to make fun of people here. Momentum IS a pretty compelling illusion, and I am more than happy to give it a ride when my team wins)

  33. FuzzyMarmoton 12 Feb 2017 at 2:01 pm

    Dr. Novella- ESPN had the Falcons at a 99.8% win probability at one point, and pro football reference at 99.9%. So that would be 1 in 500 on 1 in 1000 instead of 1 in 100.

    Matching these win probabilities with historical occurrences is a little tautological, because the win probabilities are determined directly from the frequency with which a team in similar circumstances (score, field position, down, distance) has won.

    Aside: I love hearing “momentum is not a factor in football” when few activities are a better demonstration of Newton’s 1st law.

    But seriously, it seems that momentum is a particularly hard thing to define in football, because there are so few independent events to measure. The situation of each play is determined in a complex way by the previous plays. And what constitutes “success” on one play varies tremendously based on the play call.

    Defining momentum on the level of a drive is probably a little better, but is still difficult to parse. Field position, turnovers, score, and strategic objectives make a football drive much less like a Bernoulli trial than a basketball possession or a baseball at-bat.

  34. jt512on 12 Feb 2017 at 3:29 pm

    Steve wrote:

    I would like to point out that the statisticians gave the Falcons something like a 99% chance of winning at their peak in the game. So, we would expect the losing team to come back 1% of the time, or one game in 100. Right now we have this level of comeback in 1 game out of 51, which is a 2% chance, which is pretty darn close to what the statisticians said.

    That reasoning is incorrect; in fact, it’s a conjunction fallacy. Let’s say the “statisticians” were right, and the Falcons had a 99% chance of winning when they had their largest lead—say 25 points. This means that the Patriots had a 1% chance of coming back given that they were 25 points behind. That’s a conditional probability, conditioned on being behind by 25 points. But the 2% statistic you mention is your estimate of an unconditional probability: (A) the probability of a team being behind by 25 and (B|A) winning given that you are 25 points behind. This unconditional product must be less that 1% because it is the product of two probabilities: P(A), the probability of falling behind by at least 25 points, and P(B|A), the probability of winning given that you were 25 points behind.

    To correctly estimate the unconditional probability, we need an estimate of P(A), the probability of getting behind by 25 points in a Super Bowl. Let’s say that happens on average once every 25 games. Then P(A)=.02, P(B|A)=.01 according to the “statisticians.” Thus the unconditional probability, the probability of getting behind by 25 points and then coming back to win is P(A and B) = P(A)P(B|A) = (.02)(.01) = .0002, or 1 in 5000. So we observed a very rare game: the occurrence of a 25 point lead followed by a 26 point comeback. (I’m just guessing P(A); maybe it happens more often—perhaps someone can look up the actual historical probability).

    But this unconditional probability, P(A and B), is irrelevant to your argument, because the statisticians’ 1% is the conditional probability, not the unconditional one. The bottom line is, if the “statisticians” were correct, then we observed the Falcons pull off a 1-in-100 event. The only way to judge empirically if that estimate is reasonable would be to look at the outcomes of a sufficiently large database of Super Bowl games in which a 25-point lead occurred during the game, and we don’t have that large database.

  35. BillyJoe7on 12 Feb 2017 at 3:32 pm

    When the commentators start talking about momentum they are looking at what’s happening on the scoreboard. They then use these other factors to explain what is happening on the scoreboard.

    “the win probabilities are determined directly from the frequency with which a team in similar circumstances ”

    Yes, it seems it’s better to use statistics rather than momentum (or whatever else is happening on the field) to predict the win.

  36. Steven Novellaon 12 Feb 2017 at 8:36 pm

    jt – you are correct. I used the statistics incorrectly. But I don’t think that changes my main point. The problem is, you can’t really do statistic on one game. You have to look at lots of outcomes.

    My point is that while this game was certainly a statistical outlier, games like this should happen from time and time, and they do. We don’t necessarily have to invoke other explanations.

    It’s similar to finding out what someone who has won the lottery twice did to win, because the odds against that happening by chance are so great.

  37. jt512on 12 Feb 2017 at 9:37 pm

    Steve,

    I totally agree with your basic premise. Clearly, rare events happen by chance, and therefore, when we observe a rare event, we don’t necessarily need to look further for an explanation. But sometimes rare events happen by non-chance mechanisms, too. The problem with your Super Bowl example is that we don’t know what the probability of the event would have been, if it were due to chance. And in the absence of knowing that probability, or having a good way to estimate it, I think it is just as mistaken to presume that it was primarily due to chance as it is to presume that it was primarily due to tactics and skill. I think your example of our last election is problematic for the same reason (perhaps even more so).

  38. CKavaon 13 Feb 2017 at 3:21 am

    “I say this of professional sports because at that level the players are the best in the world. They are highly skilled and professional, which means minimizing the effects of variables like not making a strong effort, being distracted, or other psychological effects. Such factors become too small to detect, at least statistically.”

    This seems like a very large claim to make. The fact that the influence of such factors cannot be detected by looking win-loss statistics does not mean that they make an insignificant contribution to individual performances and match outcomes. Assuming that high level athletes are largely immune to psychological effects brought about by unexpected situations or a loss of morale also seems highly problematic. The history of all sports I am familiar with are littered with cases and direct accounts by players that would seem to contradict this. Yes, you can attribute a certain amount of that to post-hoc reasoning but given that we don’t have minute by minute access to players emotional experiences I don’t see good reason to dismiss things like emotional contagion as being irrelevant.

    This doesn’t detract from your broader points that people too readily attribute greater significance to outcomes after they happen, are prone to detect patterns in random variance, and are unduly swayed by outliers, but I still think you are dismissing psychological factors with too little evidence either way.

  39. Steven Novellaon 13 Feb 2017 at 7:08 am

    To clarify – I am not saying that skill does not have an effect on individual games. I also am not saying that all of the factors that people point to about the election did not have an influence.

    My primary point with sports is that we cannot necessarily infer from an individual game what those factors were vs the effects of the random walk. We reach for big psychological explanations when we don’t have to. The evidence shows that psychology plays less of a role in professional sports than we think, at least on average.

    This is not saying that athletes are immune to psychology. I would suspect that any real psychological effects are probably quirky and difficult for the fan to know (maybe they just had a fight with their spouse).

    I do think the fact that we cannot see such effects in the larger data is very telling. To give another example, because people win the lottery twice pretty much exactly as often as would be expected by chance, it is safe to conclude that no one is cheating or using psychic powers to win. It seems you are saying that this does not mean we can know that one specific person who won twice was not psychic, but I think it does (to the limits of the statistics).

    Regarding the election, my point was more limited. It was purely about hindsight bias (not about statistics), the factors that were emphasized were the ones in line with the ultimate binary outcome. Because Trump barely won, everything he did worked and Hillary was a fatally flawed candidate.

  40. RCon 13 Feb 2017 at 9:01 am

    “The evidence shows that psychology plays less of a role in professional sports than we think, at least on average.”

    What evidence though? This is a huge claim to make.

    And we have glaring counterexamples – like pitchers and catchers getting the yips, and not being able to throw the ball.

  41. Steven Novellaon 13 Feb 2017 at 9:24 am

    RC – i linked to some studies above:

    http://sabr.org/research/modeling-perfect-games-and-no-hitters-baseball
    “The results were striking. In the 2,000 universes we ran, we found an average of 243 no-hitters, off by less than 4% from the 250 single pitcher no-hitters that actually occurred in 1876–2009.”

    In football the data is also mostly negative:
    ” But, having looked at a lot of NFL data, Moscowitz reaches a sobering conclusion: “There is a much stronger belief in momentum than is warranted by what we see in the data.””
    http://freakonomics.com/2011/11/20/football-freakonomics-is-momentum-a-myth-2/

  42. tmac57on 13 Feb 2017 at 9:54 am

    Is there any room in this debate for the concept of ‘flow’ or being ‘in the zone’?
    It seems like that if an individual or two on a team managed to become hyper focused due to the intense pressure of the situation, that they might begin to significantly improve their performance long enough to make a difference.
    This use to happen to me when I played foosball a lot in my 20’s. I would enter a sort of trance-like state where my playing would become sort of super-charged, and way beyond my usual abilities. Anecdotal for sure, but it certainly felt very real.

  43. edamameon 13 Feb 2017 at 10:10 am

    Freakonomics cites Moscowitz, which links to a book. Are there any primary studies?

  44. BillyJoe7on 13 Feb 2017 at 10:27 am

    Well I think we need to rise above the level of the personal anecdote.

    If the data says that the effect of momentum barely rises above the noise, then no amount of personal anecdote can change that. You need to demonstrate that the data is wrong or that the studies used to produce it were methodologically flawed in some way.

    I also personally feel that momentum plays a big psychological role in the outcome of sports games but, if the data says otherwise, so much the worse for my feeling.

  45. Steven Novellaon 13 Feb 2017 at 10:32 am

    Here are a couple of reviews that might help:
    http://www.athleticinsight.com/Vol8Iss1/Momentum.htm
    http://www.livescience.com/5120-reality-momentum-sports.html

    Bottom line – there appears to be what psychologists call psychological momentum (PM), but there is no clear evidence that PM leads to outcome differences. It is clear that the more skilled and experienced the athlete the less of an effect PM has on actual performance. There remains debate as to whether the effect of PM diminishes to zero or just a tiny effect at the highest professional level.

  46. Steven Novellaon 13 Feb 2017 at 10:35 am

    Regarding “in the zone” this refers to a real neurological difference in performance. When we first learn a skill we use a lot of conscious cognitive effort. As we train the performance shifts more and more to automatic subconscious processing. Eventually you shoot baskets without thinking, and if you start to think about it too much you actually lose performance.

    So, inexperienced athletes may at times feel as if they are in the zone. Professional athletes, on the other hand, live in the zone. Something may take them temporarily out of the zone by making them self-conscious, but that should be rare in the best athletes.

  47. BillyJoe7on 13 Feb 2017 at 10:49 am

    Well, I wasn’t going to call tmac an inexperienced athlete based simply on his personal anecdote. 😀

  48. edamameon 13 Feb 2017 at 11:08 am

    I’ve seen hot-hand nba and all that before, the classics, but not much on the NFL. In terms of primary studies specifically of the NFL (not reviews), the only one I’ve seen is the one discussed here:
    https://www.wired.com/2012/09/nfl-momentum/
    But I’ve never seen a peer-reviewed one, unless they ended up publishing (they presented at an undergrad conference). This one did convince me against rapid short-term momentum changes as they defined it.

  49. edamameon 13 Feb 2017 at 11:16 am

    Dr Novella I think the distinction between psychological momentum and objective momentum is useful, and something for us to hang our hat on for future discussions. Clearly players experience PM. The Super Bowl was probably the most extreme experience of a shift. I have heard players from both teams talk about it.

    Here is my wager: if objective momentum exists, and we come up with criteria for identifying when it does, we will retroactively find all of its hallmarks in that game (or some of its hallmarks at very high magnitude).

    I keep harping on the NFL, versus NBA, because in the NFL there is such a big asymmetry between offense and defense in terms of tiredness (playing defense is much more tiring than playing offense), so that’s where my money goes in terms of discovering objective momentum shifts. The Patriots had the ball for 40 minutes, and it just wore down the Falcons defense.

    I guess you could ask whether this is “momentum” or just an objective shift in the probability of success? This is what I was, perhaps clumsily, trying to get in my very first line in this thread:
    “It’s complicated by the fact that the probabilities are constantly changing, so the simulations are going to be gnarly. ”

    What is a momentum shift if not a drastic change in the probability of success? This is something that can happen when your offense is only on the field for five seconds in the first 3 quarters. 🙂 That’s why I was harping so much on Atlanta not running the ball. They literally set up a momentum shift. Or whatever you want to call it.

  50. Steven Novellaon 13 Feb 2017 at 11:26 am

    edamame – i get that about Football. Baseball is much more of an individual vs individual, pitcher vs batter, competition that is amenable to clear statistics. (I know there is complexity there as well, but it is probably as good as we get in competitive sports.) Football is complicated by shifting strategy, fatigue, and injuries which change the match up between the two teams potentially throughout the game.

    So there is the potential for very real momentum shifts entirely independent of PM. I suspect that the small momentum seen in the research is probably due to these factors rather than PM.

    I’m willing to accept whatever the research shows. It just appears that any momentum effect is small, perhaps because these other factors tend to even out in most games.

    Believe me, I would just rather believe that my team, the Pats, engineered a remarkable comeback because they are just awesome. We can’t, however, discount the role of randomness, but psychologically that is exactly what we tend to do.

  51. tmac57on 13 Feb 2017 at 11:52 am

    BillyJoe7- In my personal experience as a professional anecdoter, anecdotes are very useful 😉 😉 😉

  52. jt512on 13 Feb 2017 at 8:05 pm

    Steve,

    You wrote:

    To give another example, because people win the lottery twice pretty much exactly as often as would be expected by chance, it is safe to conclude that no one is cheating or using psychic powers to win. It seems you are saying that this does not mean we can know that one specific person who won twice was not psychic, but I think it does (to the limits of the statistics).

    I do not think it is correct to conclude that if a rare event occurs at approximately the frequency expected by chance, that every instance of that event is due to chance. For the example of multiple lottery winners, it is safe to conclude that no one won due to their psychic powers, because no one has psychic powers (or to be a stickler, the prior probability that anyone does is of homeopathic magnitude.)

    On the other hand, I don’t think we can rule out that some multiple lottery winners have used skill to win (or have cheated) just because the number of such multiple winners is approximately (or even exactly) the number predicted by chance. The predicted number of multiple winners is itself a noisy statistic. If the hypothesis “all multiple lottery winners have won by chance” predicts that 10 people will have won the lottery twice, then it is likely that we will have observed 10 ± 5 multiple winners, because the prediction comes with error bars. If, in fact 10 people won the lottery twice, if a few did so by skill (or cheating), we’d never see it due to the uncertainty in the prediction.

    In fact, we can rule in that some multiple lottery winners, such as this Stanford statistician, have won by skill.

  53. BillyJoe7on 14 Feb 2017 at 5:54 am

    I’m not sure if I’d decribe scratchies as a lottery. A lottery is where you buy a numbered ticket and wait for the barrel to spin and spew out your number, or where you pick numbers and wait for a machine to randomly pick the same numbers.

  54. jt512on 14 Feb 2017 at 7:21 am

    Texas Lottery Scratch Tickets”

  55. BillyJoe7on 14 Feb 2017 at 3:15 pm

    I’m not sure if I’d decribe scratchies as a lottery.

Trackback URI | Comments RSS

Leave a Reply

You must be logged in to post a comment.