Nov 21 2017

How Wikipedia Tackles Fringe Nonsense

Published by under Uncategorized
Comments: 23

wikipedia2Wikipedia is an interesting experiment in amateur crowdsourcing of information. I think it is a massively successful experiment, but it faces specific challenges. This page on Wikipedia discusses their approach to what information they allow to remain in their pages. They have a number of policies and practices that are meant to act as a quality control filter.

In my opinion they have settled upon a reasonable approach that might even be used as a model in other contexts. They begin with a completely open model – anyone can become a contributor and add information to Wikipedia. This is the crowdsourcing angle – many hands make light work. There are currently approximately 5,505,947 articles on the English language version of wikipedia. Wikipedia was founded on January 15, 2001, so that is almost 17 years. It is hard to imaging creating a reference with that much information by any other method in that much time.

So the wiki model is ideal for quantity, but what about quality. From the beginning there were concerns about the quality of the information – if anyone can post information, how can we know how accurate it is? A 2008 study comparing Wikipedia to other references for historical articles found Wikipedia to have an 80% accuracy rating, compared to 95-96% for other references. However, a 2005 Nature study of science entries found that Wikipedia was almost as good as Brittanica online – with no differences in major errors, and with an average of 4 errors total per article for Wikipedia, and 3 per article for Brittanica.

There haven’t been many studies since then, but a 2012 small follow up study found no significant difference between Wikipedia and other sources. Wikipedia has tightened its editorial policies over that time, so the improvement makes sense.

What are Wikipedia’s quality control policies? They require information to have independent verifiable references. Further, they maintain a stance of neutrality toward controversies. They see this as appropriate for a standard reference. It is not their job to debate information or to give a soapbox to every fringe view, but to be a general reference for consensus scholarship.

These filters are absolutely essential for any open source project like this. One primary reason, as they have learned first hand, is that enthusiasm is not always proportional to quality. In other words, advocates of fringe ideas are likely to have a great deal of energy in promoting their views. They feel that they are in a beleaguered minority and have to promote their view against the tide of mainstream opinion. Wikipedia, without filters, is set up to give disproportionate attention to a vocal minority. The filters are necessary to make sure the information in Wikipedia reflects the balance of information and opinion, not just the enthusiasm of its advocates.

Wikipedia admits, however, that its filters are not currently adequate:

This maneuvering and filibustering is soon likely to exhaust the patience of any reasonable person who naturally prefers not to reason with the unreasonable, and who, unlike the advocate, has no special interest or passion other than striving to maintain neutrality. Additionally, by continually engaging fringe advocates in endless argument, you run the risk of turning Wikipedia into a battleground or a debating society. At the present time, Wikipedia does not have an effective means to address superficially polite but tendentious, long-term, fringe advocacy. Some contend that this is a main flaw of Wikipedia; that unlike conventional encyclopedias, fanatics (no matter how amateur or idiotic) can always get their way if they stay around long enough and make enough edits and reversions. [3] In this sense, Wikipedia’s ‘commitment to amateurism’ does not always work for the best interests of the project.

What I find particularly interesting is that the Wikipedia experience is a microcosm of a free society in general, and not just with social media. In a free and democratic society everyone has a voice, and everyone is equal. However, information is not equal. There can be dramatic differences in quality of scholarship, and those differences matter.

Shouldn’t the free marketplace of ideas sort it all out, though? In my opinion, yes and no. A marketplace is not a neutral void, it is a system with its own rules and forces at work. The feedback mechanisms will tend to lead toward certain outcomes, but those outcomes are not necessarily optimized for the general good of society.

For example, within a marketplace subjective value is placed on certain things, qualities, or services. The relative value of the various components of the marketplace will have a large effect on outcomes, which may have very negative long term unintended consequences.

That is essentially what the Wikipedia editors are saying – the open system by itself favors enthusiasm over quality, and that may not optimally achieve the goals of creating the best general information reference. The Wikipedia editors have therefore come to the same conclusion with their experiment that society has come to with the centuries long experiment with capitalism – the free market is powerful and should be leveraged, but we need to monitor the effects of market forces and tweak the rules to mitigate unintended consequences.

To take the broadest view – systems can have either top-down or bottom-up processes. Both have their strengths and weaknesses. A pure top down system is slow, cumbersome, and oppressive. It is well-controlled, but will get there slowly. A pure bottom-up process is chaotic and easy to exploit. It is fast, but you can’t really control where it’s going.

As with so many things in this world, the best option seems to be a balance between the two, and that is what Wikipedia rapidly evolved towards (and is still evolving). They have successfully leveraged the power of a bottom-up crowdsourced system. But they quickly found they had to add some top-down editorial filters to keep Wikipedia from becoming a constant brawling mess.

Another comparison worth making is the approach to information in Wikipedia and our educational system. Wikipedia correctly perceives their roles as documented the consensus of human scholarship. What things can we claim to know with reasonable confidence, based on some operational rules of transparency and independence? Wikipedia is not the place to debate new or fringe ideas, but to simply reflect those ideas which have already gained a sufficient acknowledgement among appropriate scholars.

That is the exact approach I think public schools should take. They exist to teach consensus scholarship, not give equal time to every fringe idea. Where schools differ is that they are not just a reference of information for students, but also exist to teach students how to think, how to study, and how to be scholars. For that reason students may need to be exposed to fringe ideas, to teach about them and how to evaluate them, but not to teach them as facts or as legitimate alternatives to mainstream scholarship.

For those who espouse fringe ideas, if they want to promote them and to elevate their fringe ideas into the mainstream, there are places to do that. They have to do the hard work of scholarship and convince academics of the value of their ideas. That is a different marketplace of ideas with its own rules.

What happens, however, is that fringe ideas fail in the marketplace of academia and science, and some proponents of those failed ideas then try to create a second life in the public marketplace which has different and often more permissive rules. They try to game Wikipedia, journalists, or the educational system, or they simply market their ideas in books or online.

That is why there needs to be additional rules for good journalism, or for projects like Wikipedia in order to prevent such end-runs around proper scholarship.

23 responses so far

23 thoughts on “How Wikipedia Tackles Fringe Nonsense”

  1. Nidwin says:

    “Dr Novella:That is why there needs to be additional rules for good journalism, or for projects like Wikipedia in order to prevent such end-runs around proper scholarship.”

    I’m not convinced that additional rules is going to resolve anything.

    AVEN has very strict rules and a clear “code of behavour” this doesn’t exclude a lot of weird, innapropriate or completely wrong stuff being around there about the main subject. It’s actually through the nonsense that we were and are able to find better answers about the main subject.

    We need those alternative garbage medias because that’s how we can evaluate the work that still needs to be done. Filtering with additional rules will just hide issues that will resurface tenfold wise in the alternative garbage media.

  2. edwardBe says:

    Nidwin, I think Steve covers your objection in his discussion of public schools, “For that reason students may need to be exposed to fringe ideas, to teach about them and how to evaluate them, but not to teach them as facts or as legitimate alternatives to mainstream scholarship.” This applies equally to journalists, but journalists usually indulge in “presenting both sides” (false balance) instead of pointing out that the fringe idea is not a part of the scientific consensus.

  3. daedalus2u says:

    Free markets can only work if there are no unpriced externalities. If there are unpriced externalities, then you don’t have a “free market”.

    When some people gain disproportionate value (as in profit) from spreading false ideas, that is an unpriced externality. Those who do not profit cannot spend as much to spread ideas to counter the ideas spread by those who profit.

    This is one of the major flaws of the US media and political system. Who ever has the most money can buy the most media, and put whatever they want on that media, even if it is false. If they can then leverage that false information into political power, and then into economic power, the perpetuation of oligarchy through false advertising is extremely difficult to counter.

    Once the economic power of the oligarchs becomes sufficiently high, there are essentially no peaceful ways to counter it, because the oligarchs will implement non-peaceful but “legal” means (because oligarchs then control what is “legal”) to thwart change.

  4. BillyJoe7 says:

    SN: “the open system by itself favors enthusiasm over quality”

    This draws a parallel to the the American system of voluntary voting. It gives undue representation to fringe political movements. People on the political fringe are always enthusiastic and always vote. If the rest vote only half the time, those fringe ideas get double the political representation.

  5. BillyJoe7 says:

    SN: “public schools…exist to teach consensus scholarship, not give equal time to every fringe idea”

    This has a parallel amongst those of us who want to learn about subjects we never encountered in the educational system. Expose yourself to the consensus of experts on that subject, understand why they conclude what they conclude, be aware of the evidence, the logic, and the reasoning. Then, and only then, expose yourself to the fringe ideas.

    If you go about this the wrong way you end up like the trolls on this blog.

  6. tmac57 says:

    In this dangerous era of digital media where everyone has access to fringe and false information distributed on the cheap, the study of epistemology and skepticism needs to be elevated to greater importance and visibility than ever before.
    I have been increasingly thrown into conflict with people who simply have no clue as to how to ferret out bad information, and who further think that all information is equal and they feel justified in picking and choosing whatever makes them feel self satisfied and righteous.
    It is analogous to binging on junk food, sweets, alcohol and whatever makes you feel good. The negative results are down the road so why worry, and it’s too much trouble to learn about what’s healthy to eat, and it often comes with greater costs, and “who really knows if it’s all that bad…right”. And why not feed and promote it to your friends and family too. They can tag along on your path to self destruction and keep you company and reinforce your belief that you are right, so you never have self doubts about what harm you are causing.
    Caveat Lector! There is a storm brewing and it’s going to be very bad I fear.

  7. RP2006 says:

    Dr. Novella:
    When I saw the title of this article, “How Wikipedia Tackles Fringe Nonsense” I read it with extreme anticipation of it including a discussion of the role Susan Gerbic’s Guerrilla Skepticism on Wikipedia project has in that cause. But when I got to the end, my heart broke; there was no mention of my team at all. How can that be? Tackling fringe nonsense IS a huge part of what we do. And I know you and the team are aware of us. But maybe just not sufficiently. I now feel I missed an opportunity at the Joe Nickell CSICon lunch-time presentation with you and the other SGU men. We talked conspiracy theories and other interesting stuff, which I thoroughly enjoyed, but I now believe that I should have tried to use some of the time I had with your team to impress upon you the worth of what the Guerrilla Skeptics do to make Wikipedia a better science resource. I wish I had a do-over!

  8. davidr says:

    SN: “However, a 2005 Nature study of science entries found that Wikipedia was almost as good as Brittanica online – with no differences in major errors, and with an average of 4 errors total per article for Wikipedia, and 3 per article for Brittanica.”

    What was most telling about that study was the response. Brittanica disputed the findings. Wikipedia fixed the errors found in the study.

  9. Nidwin says:

    @edwardBe

    I understand but there’s a problem with this.
    Who’s going to decide what is fringe science or fake news that should be included in the educational curriculim?
    And what are you going to tag as fringe-pseudo science or fake news and filter it out?

    I agree that there’s a lot of crap out there on the digital social media. But for every ton of garbage there are also a couple of kilos of low hanging fruit that we were able to find. And when you look a bit closer in one ton of garbage you’ll, sometimes, find a kilo of useable information hidden here and there.

    The digital social media, while being submerged with a lot of crap and being the refuge for snake oil sellers, also had and still has a huge positive influence.

    I was positively shocked when I binged (long time I hadn’t watched shows) three amazing science fiction shows from this decade, earlier this Year. Sense 8, Orphan Black and The 100. I’m convinced that without the digital social revolution those three shows wouldn’t have been so LGBTxyz friendly because of human fossils like Dr Egnor able to impose their ridiclulous nonsense.

    More recently the digital highway has been quite efficiently used to out those sexual predators in Hollywood, sports and even politics.

    Filtering and censoring can go both ways, depending who’s in charge, including science/academia. It’s just too risky and without knowing, filtering something that may actually be relevant or even true.

    I was pissed off, take my word for it, when I read what the chruch was spreading out to get their hands on generation Z because they feel our latest generation is slipping away from them en masse. But it’s because of the digital social media that I was able to get my eyes on those written pieces of insanity.

    But, my choice, I take a free digital highway with all the garbage it also produces, without more rules, regulations or filters against those snake oil sellers. The benefits outweight the shitt most of the time in my opinion.

    And for the human fossils, they can just rot in their own hell for my part. Society as a whole will go forward. Even if Today those tards seem to be quite vocal and clearly have influence, they will only be remembered in future history books for what they truly are. Human fossils out of sync of their own time. And as most of us around here, till then I’ll oppose and expose their stupidity whenever appropriate.

  10. hardnose says:

    “Another comparison worth making is the approach to information in Wikipedia and our educational system. Wikipedia correctly perceives their roles as documented the consensus of human scholarship.”

    There is a big problem with this statement. Parents can choose where to send their kids to school, we choose colleges to attend, books and journals to read, etc.

    With Wikipedia we have no choice. It is the biggest and most convenient source of information and people (wrongly) assume it is democratic and unbiased.

    It may seem fair and unbiased to a materialist — people cannot see their own ideological bias.

    But Wikipedia is obviously paternalistic, elitist and biased, to anyone who is not a materialist.

    So your crusade to convert the world to materialism has won a great ally in Wikipedia.

  11. JoeMamma says:

    @hardnose

    Well there’s a supremely broad claim with precisely zero examples citing a specific instance where Wikipedia is inaccurate.

  12. BillyJoe7 says:

    How dare Wikipedia refuse to print my opinion. Materialist cogs.

  13. RP2006 says:

    Dr. Novella:

    As one can see in the screen shot at the top of your article, the Wiki article you referenced, “Why Wikipedia cannot claim the earth is not flat”, has a warning which says:

    “This page is an essay. It contains the advice or opinions of one or more Wikipedia contributors. This page is not one of Wikipedia policies or guidelines, as it has not been thoroughly vetted by the community. Some essays represent widespread norms; others only represent minority viewpoints.”

    It struck me that you had interpreted this article far too broadly, but to be sure, I started a discussion on the Wikipedia Talk page of that Wiki article to get feedback from the involved editors. As it turns out, a person claiming to be the primary writer of that very article gave me this feedback to pass along to you:

    “Novella probably should have explained that what he linked to was an essay that described advice for how to apply one of the encyclopedia’s core policies. This would have avoided the possibility of his readers confusing a mere essay for actual policy. E.g. he quotes a paragraph that I (an editor) largely wrote, but represents it as an opinion of the encyclopedia as a whole: ‘Wikipedia admits, however, that its filters are not currently adequate’ ”

    BTW, this is the article that I believe you should have read and referenced on this particular topic: https://en.wikipedia.org/wiki/Wikipedia:Fringe_theories

    So, if you plan on writing any other Wikipedia related articles in the future, feel free to contact me (or Susan Gerbic) if you want advice on where to find the pertinent Wikipedia policy statements. Thanks!

  14. chikoppi says:

    [hardnose] But Wikipedia is obviously paternalistic, elitist and biased, to anyone who is not a materialist.

    Curious that the flat-earthers make near-identical protests. Why aren’t their “facts” just as good as anyone else’s?

  15. Scyntizt says:

    What do you think of Wiki Tribune? It’s New project by Jimmy Wales for news/journalism where everybody can contribute, sort of like Wikipedia.
    http://www.wikitribune.com

    Would be great if you could talk a bit about this on SGU.

  16. bachfiend says:

    Hardnose,

    ‘With Wikipedia we have no choice. It is the biggest and most convenient source of information and people (wrongly) assume it is democratic and unbiased.’

    Well, why don’t you submit a page on the Intelligent Universe or the Conscious Universe?

    You need to include a discussion as to what they are, the implications, and provide evidence and references to the published literature, whether in books, journals, the Internet or newspapers/news magazines.

    Otherwise they’d be very short pages. You can repeat ‘I think it’s so, because it makes me feel good’ and ‘we don’t know everything, so we don’t know anything’ only a limited number of times.

  17. Steven Novella says:

    RP – Thanks for the clarification. The article did link to the specific core policies discussed, but your clarification is important.

    Regarding Geurilla Skepticism on Wikipedia, sorry for the oversight. It wasn’t really the point of this essay, which was more about the bigger issue of balance between openness and standards. But it’s worth pointing out. Thanks.

  18. Rogue Medic says:

    hardnose,

    Your desire for an Allahpedia is admirably irrational, but you need to show that your claims are, in some way, real. You have not.

    Your rants against science do provide an opportunity for reasonable people to expose the blatant flaws in your attempts at logic (that other part of the materialism you oppose).

    You need evidence and logic.

    Evidence and logic are essential to science, which you label as materialism.

    Without evidence, your claims are immaterial.

    Without logic, your claims are irrational.

    Try reality, for a change. Reality really is real.

    .

  19. BillyJoe7 says:

    Rogue Medic,

    He doesn’t even understand science.
    He thinks his non evidence based wild speculations constitute a theory.
    He thinks science is about proofs.

    He’s been loitering around here for ten years and still hasn’t come to grips with the basics.

  20. hardnose says:

    “Your rants against science”

    I have NEVER ranted against science. I am here to defend science against ideology.

  21. bachfiend says:

    Hardnose,

    ‘I have NEVER ranted against science. I am here to defend science against ideology.’

    I’ll correct your statement for you.

    ‘I ALWAYS rant against science that looks for evidence and plausible mechanisms, if it conflicts with my worldview that the Universe is both conscious and intelligent. I am here to defend pseudo-science that regards evidence and plausible mechanisms as being unnecessary against the worldview that they are vitally important.’

    You still don’t know the difference between ‘ideology’ and ‘worldview’.

  22. BillyJoe7 says:

    He is so ignorant about science that he doesn’t even understand that he is ranting against science – and for pseudoscience and anti-science – every time he opens his mouth.
    Ten years of mummified ignorance.

  23. Rogue Medic says:

    BillyJoe7,

    The IgNobel Award ceremony included a Dunning-Kruger song this year.

    Perhaps we could get a Miss Sweetie Poo bot to tell hardnose, Dana Ullman, Jenny McCarthy, Deepak Chopra, Michael Egnor, and their anti-science ilk, Please stop. I’m bored.

    Or a simple citation needed response to their claims that voodoo works.

    https://www.sciencefriday.com/segments/from-cat-rheology-to-operatic-incompetence/

    P.S. hardnose, Here is your random fictional Deepak Chopra quote:

    “God depends on irrational balance” _

    http://wisdomofchopra.com/

    .

Leave a Reply