Which is worse – clickbaity headlines for news articles that are factually correct, but may be playing up a sensational angle, or straight-up misinformation? It depends on what you mean by “worse”. A new study tries to address this information, with some interesting findings.

Misinformation is an increasingly important topic, one with far reaching implications for society. Our individual lives and our society is increasingly run on information. It is a critical resource, and the ability to evaluate and utilize information may be a determining factor in our quality of life. My favorite example remains Steve Jobs, because he is such a stark example. He was one of the richest people on the planet, with every physical resource at his disposal, and was a titan of an information industry. And yet he died prematurely of a potentially curable disease. He chose to delay mainstream treatment in order to pursue “natural” therapies that were ultimately worthless. We cannot know for sure what would have happened if he did not take this course, but his odds of survival would have been better.

At a societal level the most visible impact that our information ecosystems have deals with politics and public health. We are facing a rather dramatic decision regarding the next presidential election in the US, and this will ultimately be determined by how people are accessing and evaluating information. This has always been the case in a democracy, but I think most people alive today have not experienced a divergence of narrative and opinion as intense as we have today.

We also just when through the worst pandemic in a century, which brought into focus every issue dealing with misinformation. How do we deal with it in an age of social media? How do we balance the interests of making sure people get accurate health information so they can make informed choices, and freedom of speech and the value of open debate? There is no one correct answer, we just have to choose our tradeoffs.

But we can do research to at least inform our choices, to know what the tradeoffs are. That is where the new research comes in. Researchers at the MIT Sloan School of Management, publishing in Science, showed thousands of study participants 130 vaccine-related stories. They surveyed the subjects for their demographics, and asked them how the news story affected their intention to get vaccinated against COVID-19. They used this information to estimate the net effect of these vaccine-stories on society.

Their core findings are unsurprising. The more a news article suggested that vaccines could be harmful to a person’s health, the more they discouraged vaccination. Second, the more widespread the news article the more of an impact it had. Again – these are completely unsurprising findings. But then they calculated the net effect that a misleading clickbait headline of a factually correct article in the mainstream media would have vs terrible misinformation spread by social media. They found the clickbait headline would have 46 times a greater negative effect on vaccination rates than the social media misinformation.

The reason for this is because, even though the misleading mainstream article has a much lower negative impact on vaccine attitudes, it was read by far many more people. One mainstream article may be read by 54 million people, while a Facebook post flagged as misinformation may be read by only 0.3% as many people. Part of the reason this may be surprising is because we tend to intuitively underestimate the impact of very large numbers (we are just not wired to deal with such numbers). This comes up in many contexts. In medicine, we tend to intuitively focus on how typical a set of symptoms is for a disease, rather than the base rate (how common) that disease is. In reality, an atypical presentation of a common disease may be much more likely than a more typical presentation of an extremely rare disease.

The same is true with misinformation – we will intuitively give more consideration to the negative impact of an individual article rather than consider the numbers of people who may have read it, and have a hard time grasping the net effect of a small effect times millions of people.

How should we interpret these results? First, it does not mean that we should be ignoring misinformation on social media. In fact, I could argue that this research shows flagging misinformation and marginalizing it works. That was the whole point – to keep the number of people who see the extreme misinformation to a minimum.

But it also means that mainstream outlets have an extreme responsibility to be very careful with their headlines and the way they sensationalize important news. Headline writers are typically not the journalists themselves, and their whole job is to create eye-catching headlines. But responsible mainstream outlets should have editors who review and approve those headlines, and should filter out dangerous clickbait. It doesn’t matter that all the appropriate caveats are deep in the body of the article. The headlines themselves, and the opening paragraphs, need to capture an accurate gist of the story. It takes more work and consideration to create an intriguing headline that is also not misleading, but it’s worth the effort.