Wikipedia is now the most common general reference on the internet, and the 7th most popular site overall. It is, in a way, our collective online store of human knowledge.

Wikipedia is also a wiki, which means that anyone can theoretically write and edit articles. Wikipedia has editors, however, and they manage this process, evaluating the authoritativeness and apparent motives of editors. It is not the wild west like it used to be. (We discussed this recently on the SGU – look for the interview with Tim Farley and Susan Gerbic here.)

The question remains, however – how accurate and reliable is the information on Wikipedia? There have been several comparisons of Wikipedia to traditional references, such as the Encyclopedia Brittanica, and Wikipedia compares fairly well to such sources.

How does Wikipedia do when it comes to more technical, and specifically health care, information? This is a critical question as many people use the internet and Wikipedia specifically as a source of medical information, often to guide medical decisions or self-treatment. A recent study set out to test that very question.

The authors picked the top 10 most costly medical conditions. They then assigned two physicians in training to each of the ten article, with instructions to identify every assertion in their assigned article. That means they needed to pull out every statement or implication of fact. They were instructed to then search the published literature to see if the assertion was correct. Other reviewers then compared the results of the two reviewers assigned to each article.

Finally the results were tallied. The authors essentially did a statistical analysis to see if there was a significant discordance between Wikipedia assertions and the results of up-to-date peer-reviewed research. They found a significant discordance in 9 of the 10 article examined.

If you look through table 3 of the article, you will see for each article and each reviewer the number of factual assertions and the number that were discordant with the research. It’s variable, but overall 24% of the assertions were discordant with the research.

That level of error in Wikipedia articles for common medical topics is certainly concerning. Not only patients, but medical students and even physicians admit to using Wikipedia as a source of information.

The authors acknowledge that one of the weaknesses of the study is that they did not verify that the results of literature search were themselves accurate. It is also possible that some of the assertions related to questions that are currently unanswered or controversial. So every discordance does not necessarily mean Wikipedia is wrong, but it is a reasonable approximation.

The study also did not consider errors of omission, only factual statements made within the articles.

The results of this study reinforce my overall attitude toward Wikipedia, reflected in my use of it. For general information that is not highly technical or controversial, Wikipedia is a good first stop. It’s a good way to get a quick overview of a topic and see what all the relevant issues are. I don’t rely on Wikipedia, however, as an ultimate resource. I always attempt to track back to original sources.

For highly technical issues, Wikipedia is variable. If one or more true experts have dedicated some time to an article, it may be excellent – as good as any textbook. If not, then the information can be highly incomplete, superficial, or even wrong.

You also have to be careful with any controversial topics. The Wikipedia editors are doing an excellent job of policing such articles, but still the information is only as reliable as the expertise of the authors, and is subject to all of their biases.

Conclusion

Wikipedia, for good or ill, is now by far the most popular general reference for information. The wiki format is incredibly powerful, and I think we are stuck with it in some form for the foreseeable future.

It is also evolving, and the editors of Wikipedia are dedicated to improving its quality and reliability.

We all share the responsibility for making Wikipedia the best resource possible. If you have an area of expertise, Wikipedia is an excellent way to share that expertise with the world. If you have an ideological axe to grind, don’t grind it in Wikipedia.

For users, Wikipedia is an excellent resource, but not an ultimate authoritative source of information. For important issues, you still need to find original expert references or peer-reviewed sources.