May 29 2014

Medical Information on Wikipedia

Wikipedia is now the most common general reference on the internet, and the 7th most popular site overall. It is, in a way, our collective online store of human knowledge.

Wikipedia is also a wiki, which means that anyone can theoretically write and edit articles. Wikipedia has editors, however, and they manage this process, evaluating the authoritativeness and apparent motives of editors. It is not the wild west like it used to be. (We discussed this recently on the SGU – look for the interview with Tim Farley and Susan Gerbic here.)

The question remains, however – how accurate and reliable is the information on Wikipedia? There have been several comparisons of Wikipedia to traditional references, such as the Encyclopedia Brittanica, and Wikipedia compares fairly well to such sources.

How does Wikipedia do when it comes to more technical, and specifically health care, information? This is a critical question as many people use the internet and Wikipedia specifically as a source of medical information, often to guide medical decisions or self-treatment. A recent study set out to test that very question.

The authors picked the top 10 most costly medical conditions. They then assigned two physicians in training to each of the ten article, with instructions to identify every assertion in their assigned article. That means they needed to pull out every statement or implication of fact. They were instructed to then search the published literature to see if the assertion was correct. Other reviewers then compared the results of the two reviewers assigned to each article.

Finally the results were tallied. The authors essentially did a statistical analysis to see if there was a significant discordance between Wikipedia assertions and the results of up-to-date peer-reviewed research. They found a significant discordance in 9 of the 10 article examined.

If you look through table 3 of the article, you will see for each article and each reviewer the number of factual assertions and the number that were discordant with the research. It’s variable, but overall 24% of the assertions were discordant with the research.

That level of error in Wikipedia articles for common medical topics is certainly concerning. Not only patients, but medical students and even physicians admit to using Wikipedia as a source of information.

The authors acknowledge that one of the weaknesses of the study is that they did not verify that the results of literature search were themselves accurate. It is also possible that some of the assertions related to questions that are currently unanswered or controversial. So every discordance does not necessarily mean Wikipedia is wrong, but it is a reasonable approximation.

The study also did not consider errors of omission, only factual statements made within the articles.

The results of this study reinforce my overall attitude toward Wikipedia, reflected in my use of it. For general information that is not highly technical or controversial, Wikipedia is a good first stop. It’s a good way to get a quick overview of a topic and see what all the relevant issues are. I don’t rely on Wikipedia, however, as an ultimate resource. I always attempt to track back to original sources.

For highly technical issues, Wikipedia is variable. If one or more true experts have dedicated some time to an article, it may be excellent – as good as any textbook. If not, then the information can be highly incomplete, superficial, or even wrong.

You also have to be careful with any controversial topics. The Wikipedia editors are doing an excellent job of policing such articles, but still the information is only as reliable as the expertise of the authors, and is subject to all of their biases.

Conclusion

Wikipedia, for good or ill, is now by far the most popular general reference for information. The wiki format is incredibly powerful, and I think we are stuck with it in some form for the foreseeable future.

It is also evolving, and the editors of Wikipedia are dedicated to improving its quality and reliability.

We all share the responsibility for making Wikipedia the best resource possible. If you have an area of expertise, Wikipedia is an excellent way to share that expertise with the world. If you have an ideological axe to grind, don’t grind it in Wikipedia.

For users, Wikipedia is an excellent resource, but not an ultimate authoritative source of information. For important issues, you still need to find original expert references or peer-reviewed sources.

Share

15 responses so far

15 Responses to “Medical Information on Wikipedia”

  1. MKandeferon 29 May 2014 at 10:17 am

    “They found a significant discordance in 9 of the 10 article examined.”

    Did they fix this discordance? :)

  2. YtterbiJumon 29 May 2014 at 1:17 pm

    It’s interesting that a large percentage of the assertions in the articles were only found by one of the researchers (“dissimilar”) rather than found by both researchers (“similar”). It kind of makes me think that they didn’t define what is an “assertion” well enough, so the researchers got confused.

    I would love to see a follow-up study to find out how many of the discordant assertions have been fixed in the articles since the study was conducted 2 years ago, taking into account how the peer-reviewed scientific literature has changed since then too.

  3. elmer mccurdyon 29 May 2014 at 1:40 pm

    I tend to use it for the references, which often don’t quite say what the wiki articles say they say (and yes, sometimes the references are wrong, obviously). It’s a really well-organized collection of links.

  4. elmer mccurdyon 29 May 2014 at 1:43 pm

    “Talk pages” are also sometimes useful, when a topic is controversial.

  5. ConspicuousCarlon 29 May 2014 at 1:55 pm

    1. The authors of this study are all osteopaths. They might have a motive to “find” errors, since osteopathy is one of the categories with a less-than-favorable Wikipedia article.

    2. The paper doesn’t list the errors, so we can’t review the reviewers’ findings to see if the errors are truly refuted by a fair interpretation of Pubmed results, or if the errors are significant.

  6. Paulzon 29 May 2014 at 5:59 pm

    @MKandefer
    ““They found a significant discordance in 9 of the 10 article examined.”
    Did they fix this discordance? :)

    Hear hear. I hope someone is going back over those articles now!

    Maybe that could be a step for wikipedia – how much spare donation money do they get? They should spend some of it on Technical Expert policing.

  7. Paulzon 29 May 2014 at 6:00 pm

    Wait…

    @Carl

    “1. The authors of this study are all osteopaths. They might have a motive to “find” errors, since osteopathy is one of the categories with a less-than-favorable Wikipedia article.”

    *goes back and checks the original link*

    Well son of a-

    You’re right.
    Yeah, that’s some horse crap right there.

  8. Bill Openthalton 29 May 2014 at 7:28 pm

    Any encyclopaedia contains outdated information, and is limited by the quality of the contributors. Wikipedia is probably more accurate and up-to-date than the best paper encyclopaedia.

  9. inconsciouson 30 May 2014 at 10:38 am

    @Carl & Paulz

    Don’t you think if there was some kind of fishy stuff going on that Dr. Novella would have noted suspicions?

    None of the 10 wikipedia entries were related to Osteopathy, so how could this possibly be relevant.

    Also, note that the article was published in the JAOA – the Journal of the American Osteopathic Association. How could it *possibly* be a surprise that the authors are all DOs?

  10. Steven Novellaon 30 May 2014 at 10:47 am

    Most osteopaths are indistinguishable from regular physicians. The training was aligned decades ago, and most go into the same residencies as MDs. A minority retains some of their bone manipulation pseudoscience, but this does not condemn the entire profession or everything they do.

    The methodology of this study seems sound, with the noted caveats and weaknesses.

    But as always, independent replication is nice.

  11. reedonlyon 30 May 2014 at 1:05 pm

    “We all share the responsibility for making Wikipedia the best resource possible. If you have an area of expertise, Wikipedia is an excellent way to share that expertise with the world. If you have an ideological axe to grind, don’t grind it in Wikipedia.”

    While I agree, the fundamental flaw is the idealogue’s version of the Dunning-Kruger Effect, wherein incompetent people may be so incompetent as to not know they are incompetent. Some people are so ideological that they don’t know they are ideologues – they just assume their worldview is correct, and therefore, they are experts, which gives them the confidence to impose their views on others.

    The other manifestation of Dunning-Kruger also holds here. In my discussions with college faculty about the accuracy, and thus appropriateness, of Wikipedia in a college classroom, several said they had personally found errors in Wikipedia articles. When I then challenged them as to whether they had made efforts to correct those errors, the general response was “I’m not expert enough.”

  12. ConspicuousCarlon 30 May 2014 at 1:34 pm

    > inconsciouson 30 May 2014 at 10:38 am
    > @Carl & Paulz
    >
    > Don’t you think if there was some kind of fishy stuff
    > going on that Dr. Novella would have noted suspicions?

    That’s kind of a dumb question. Obviously I think it’s fishy and Novella didn’t mention it.

    > None of the 10 wikipedia entries were related to
    > Osteopathy, so how could this possibly be relevant.

    How incredibly naive.

    > Also, note that the article was published in the JAOA – the Journal
    > of the American Osteopathic Association. How could it *possibly*
    > be a surprise that the authors are all DOs?

    It’s not a surprise, it’s a fact, though it wasn’t mentioned in this blog post so it might have been a surprise to anyone who assumed Novella would have mentioned anything unusual.

  13. ConspicuousCarlon 30 May 2014 at 1:51 pm

    > Steven Novellaon 30 May 2014 at 10:47 am
    >
    > Most osteopaths are indistinguishable from regular physicians.
    > The training was aligned decades ago,

    And yet this large osteopath group still have BS like this on their website?
    http://www.osteopathic.org/osteopathic-health/about-dos/about-osteopathic-medicine/Pages/default.aspx

    doctors of osteopathic medicine (DOs) practice a “whole person” approach to health care. Instead of just treating your specific symptoms, osteopathic physicians concentrate on treating you as a whole.

    Are they saying MDs don’t know how to treat the whole person?

    Or how about this one…

    Osteopathic physicians understand how all the body’s systems are interconnected

    Compared to who? Do MDs not understand how the body’s systems are interconnected? They seem to think this is a defining point of osteopathy. Surely this wording rings a few bells so far.

    They may receive a full medical education, but it sounds like they like to think their profession has some fundamental advantage as well.

    Here’s one close to me (first google result), which sounds as nutty as a chiropractor:
    http://austinofm.com/osteopathy/
    (my bold)

    Osteopathy is a method of practicing medicine that incorporates the philosophy of treating the body systems as a single functional unit and manual medicine (Osteopathic Manipulative Medicine) to precisely adjust the body’s tissues (muscle, fascia, bone, etc.) in order to restore normal function to these tissues.

    Obviously not all osteopaths are nuts, and I am aware that osteopathic schools study real medicine, but the fact that this paper was the work of osteopaths should at least be noted. Since the claimed errors aren’t provided for us to confirm, one might want to do the tedious work of looking at each individual involved before taking their word about mystery errors.

  14. inconsciouson 30 May 2014 at 4:08 pm

    @Carl

    Your points about the AOA’s and other osteopathic websites’ propaganda are 100% spot on. They’re ploys to make the profession sound “special.”

    I’m actually a DO/PhD student and I have a *huge* chip on my shoulder about these very things. Personally I think that two two degrees (MD and DO) should merge. There’s really no need to have two paths to physician-hood, IMO. Hopefully the recent decision to merge graduate medical education between the ACGME and AOA is the first major step in that direction.

    However, as Dr. Novella mentioned, the study in discussion here, though it’s wanting with respect to a few methodological details, doesn’t reek of bias – particularly ‘osteopathic’ bias. As such, it’s not really relevant to bring up the fact that the authors are all DOs. The vast majority of DOs just practice medicine and ignore the propaganda spouted by the AOA – just like the vast majority of MDs don’t practice quack medicine either. Thus your original post’s implication was rather insulting.

  15. SimonWon 07 Jun 2014 at 3:56 pm

    Does the study demonstrate what people are concluding?

    I haven’t read it, but the coverage suggests there is no control.

    Thus all it demonstrates is that wikipedia disagrees with peer reviewed medical literature on some points. And is being used to suggest Wikipedia is wrong where it disagrees, which isn’t the conclusion I’d immediately draw from that.

    Since science self corrects, some peer reviewed medical literature must disagree with other peer reviewed medical literature, so if Wikipedia disagreeing more or less often?

    To reach a useful conclusion, studies must compare with something you would use as an alternative. e.g. Medical Textbook, or Continuing Professional Education journals, or peer reviewed papers.

    Otherwise the study says nothing useful and is just reinforcing Steve’s prejudices.

    Declaration of interest – my GP is part of the project for medical articles to be edited by qualified medical professionals.

Trackback URI | Comments RSS

Leave a Reply

You must be logged in to post a comment.