Aug 17 2015

Controversial Science Topics on Wikipedia

The press release reads: “On Wikipedia, politically controversial science topics vulnerable to information sabotage.” They could have left off the qualifier, “On Wikipedia” and I think the statement would remain accurate.

But of course they are referring to a specific study, “Content Volatility of Scientific Topics in Wikipedia: A Cautionary Tale,” by Adam Wilson and Gene Likens. Likens co-discovered the acid rain phenomenon in North America and was concerned that the Wikipedia entry on acid rain seems to be edited frequently with misinformation.

Wilson and Likens made a comparison of the last ten years of Wikipedia edits for three politically controversial (but not scientifically controversial) topics: evolution, acid rain, and climate change, and compared these to four non-controversial topics: the standard model in physics, heliocentrism, general relativity, and continental drift. They found that the controversial topics were edited much more frequently than the non-controversial ones.

“For example, over the period we analyzed, the global warming page was edited on average (geometric mean ±SD) 1.9±2.7 times resulting in 110.9±10.3 words changed per day, while the standard model in physics was only edited 0.2±1.4 times resulting in 9.4±5.0 words changed per day.”

This is about a ten-fold increase, a full order of magnitude, from lowest to highest. Evolution had 1.3 average edits per day. Acid rain was only slightly increased at 0.5, with the non-controversial topics ranging from 0.2-0.4.

The study results are a reflection of what is popularly known as the “edit wars” on Wikipedia. Topics ranging from the trivial to the profound are often hotly debated on Wikipedia in the form of edits and counter-edits, as many people try to have the text reflect their particular biases. Information is Beautiful has an infographic on what they consider the “lamest edit wars.”

Wikipedia is acutely aware of the issue. Contrary to what I frequently hear, not anyone can simply edit any Wikipedia any more. For “protected” pages, which usually includes controversial topics, editors have to register and are monitored. Hotly debated topics are especially monitored, and super-editors resolve disputes. Wikipedia employs algorithms to identify editing vandalism.

In other words, serious efforts have been made to employ mechanisms of quality control within the overall paradigm of crowdsourcing. I think that’s a great idea – combine the best of both worlds to create a filtered crowd-sourced reference.

This is still very much a social experiment, but I think we have to declare it as least a partial victory so far given the popularity of Wikipedia. Also, Wikipedia is more accurate than most people would guess. A recent study found that it was 97.5% accurate overall when compared to textbooks as a standard. However, another criterion of quality is completeness, and scientific articles were rated on average only 83.8% complete. Still, that’s not bad for a free online encylopedia.

The biggest issue appears to remain controversial topics. One way to deal with this is to include a section on controversial question within the topic. If you take a look at the Wikipedia page on global warming, for example, you will see a section on scientific and public discourse. This can mollify the edit wars a bit, but still everyone would want the article to reflect their personal views.

This gets back to my opening thesis – the edit wars in Wikipedia are a reflection of a deeper phenomenon – that scientific questions are vulnerable to politically, socially, religiously, emotionally, ideologically charged feelings. This is a human condition –  the edit wars of Wikipedia are just a symptom.

As a science communicator I frequently dig into socially controversial science topics or factual claims. Recently, for example, I tried to find if there was any objective consensus on whether or not raising the minimum wage costs jobs. I think the answer is “it depends,” but my more important discovery was actually the lack of a strong consensus. A discussion of this topic is beyond the scope of this article, so don’t get distracted – my point is it can be extremely challenging to find objective empirical information on a topic that is so political.

The same is true of many such topics, such as GMOs, organic farming, the efficacy of recycling, the effectiveness of gun control laws, and the incidence of rapes on college campuses. There are factual claims at the heart of all of these topics, and it can be difficult to find a strong objective consensus. It’s not necessarily impossible, you just have to wade through a lot of noise. There is the equivalent of the edit wars taking place on the internet in general, and sometimes in the scientific literature itself. Sometimes topics are scientifically, not just politically, controversial.

Wikipedia is merely a reflection of this phenomenon.

What Wikipedia should do and is doing is to impose a process of quality control. The same applies to individuals. In fact, I have often described scientific skepticism as a process. The process includes evaluating source material for quality and objectivity. Taking a look at a variety of source material until you have at least a sense of all sides of a debate. Look for a scientific consensus and determine how strong and how solid it is.

Part of this process is identifying your own biases. Being a skeptic means stepping back from those biases as much as possible. The best way to do this is to not only follow a valid process, but to value that process over the specific conclusions at which it arrives. The process needs to be more important than the conclusion, otherwise the desired conclusion will control the process. We call that motivated reasoning, and people are really good at it.

Wikipedia is actually doing a much better job than most people might guess, but there, of course, is still room for improvement. The experiment continues.

22 responses so far