Wikipedia is an interesting experiment in amateur crowdsourcing of information. I think it is a massively successful experiment, but it faces specific challenges. This page on Wikipedia discusses their approach to what information they allow to remain in their pages. They have a number of policies and practices that are meant to act as a quality control filter.

In my opinion they have settled upon a reasonable approach that might even be used as a model in other contexts. They begin with a completely open model – anyone can become a contributor and add information to Wikipedia. This is the crowdsourcing angle – many hands make light work. There are currently approximately 5,505,947 articles on the English language version of wikipedia. Wikipedia was founded on January 15, 2001, so that is almost 17 years. It is hard to imaging creating a reference with that much information by any other method in that much time.

So the wiki model is ideal for quantity, but what about quality. From the beginning there were concerns about the quality of the information – if anyone can post information, how can we know how accurate it is? A 2008 study comparing Wikipedia to other references for historical articles found Wikipedia to have an 80% accuracy rating, compared to 95-96% for other references. However, a 2005 Nature study of science entries found that Wikipedia was almost as good as Brittanica online – with no differences in major errors, and with an average of 4 errors total per article for Wikipedia, and 3 per article for Brittanica.

There haven’t been many studies since then, but a 2012 small follow up study found no significant difference between Wikipedia and other sources. Wikipedia has tightened its editorial policies over that time, so the improvement makes sense.

What are Wikipedia’s quality control policies? They require information to have independent verifiable references. Further, they maintain a stance of neutrality toward controversies. They see this as appropriate for a standard reference. It is not their job to debate information or to give a soapbox to every fringe view, but to be a general reference for consensus scholarship.

These filters are absolutely essential for any open source project like this. One primary reason, as they have learned first hand, is that enthusiasm is not always proportional to quality. In other words, advocates of fringe ideas are likely to have a great deal of energy in promoting their views. They feel that they are in a beleaguered minority and have to promote their view against the tide of mainstream opinion. Wikipedia, without filters, is set up to give disproportionate attention to a vocal minority. The filters are necessary to make sure the information in Wikipedia reflects the balance of information and opinion, not just the enthusiasm of its advocates.

Wikipedia admits, however, that its filters are not currently adequate:

This maneuvering and filibustering is soon likely to exhaust the patience of any reasonable person who naturally prefers not to reason with the unreasonable, and who, unlike the advocate, has no special interest or passion other than striving to maintain neutrality. Additionally, by continually engaging fringe advocates in endless argument, you run the risk of turning Wikipedia into a battleground or a debating society. At the present time, Wikipedia does not have an effective means to address superficially polite but tendentious, long-term, fringe advocacy. Some contend that this is a main flaw of Wikipedia; that unlike conventional encyclopedias, fanatics (no matter how amateur or idiotic) can always get their way if they stay around long enough and make enough edits and reversions. [3] In this sense, Wikipedia’s ‘commitment to amateurism’ does not always work for the best interests of the project.

What I find particularly interesting is that the Wikipedia experience is a microcosm of a free society in general, and not just with social media. In a free and democratic society everyone has a voice, and everyone is equal. However, information is not equal. There can be dramatic differences in quality of scholarship, and those differences matter.

Shouldn’t the free marketplace of ideas sort it all out, though? In my opinion, yes and no. A marketplace is not a neutral void, it is a system with its own rules and forces at work. The feedback mechanisms will tend to lead toward certain outcomes, but those outcomes are not necessarily optimized for the general good of society.

For example, within a marketplace subjective value is placed on certain things, qualities, or services. The relative value of the various components of the marketplace will have a large effect on outcomes, which may have very negative long term unintended consequences.

That is essentially what the Wikipedia editors are saying – the open system by itself favors enthusiasm over quality, and that may not optimally achieve the goals of creating the best general information reference. The Wikipedia editors have therefore come to the same conclusion with their experiment that society has come to with the centuries long experiment with capitalism – the free market is powerful and should be leveraged, but we need to monitor the effects of market forces and tweak the rules to mitigate unintended consequences.

To take the broadest view – systems can have either top-down or bottom-up processes. Both have their strengths and weaknesses. A pure top down system is slow, cumbersome, and oppressive. It is well-controlled, but will get there slowly. A pure bottom-up process is chaotic and easy to exploit. It is fast, but you can’t really control where it’s going.

As with so many things in this world, the best option seems to be a balance between the two, and that is what Wikipedia rapidly evolved towards (and is still evolving). They have successfully leveraged the power of a bottom-up crowdsourced system. But they quickly found they had to add some top-down editorial filters to keep Wikipedia from becoming a constant brawling mess.

Another comparison worth making is the approach to information in Wikipedia and our educational system. Wikipedia correctly perceives their roles as documented the consensus of human scholarship. What things can we claim to know with reasonable confidence, based on some operational rules of transparency and independence? Wikipedia is not the place to debate new or fringe ideas, but to simply reflect those ideas which have already gained a sufficient acknowledgement among appropriate scholars.

That is the exact approach I think public schools should take. They exist to teach consensus scholarship, not give equal time to every fringe idea. Where schools differ is that they are not just a reference of information for students, but also exist to teach students how to think, how to study, and how to be scholars. For that reason students may need to be exposed to fringe ideas, to teach about them and how to evaluate them, but not to teach them as facts or as legitimate alternatives to mainstream scholarship.

For those who espouse fringe ideas, if they want to promote them and to elevate their fringe ideas into the mainstream, there are places to do that. They have to do the hard work of scholarship and convince academics of the value of their ideas. That is a different marketplace of ideas with its own rules.

What happens, however, is that fringe ideas fail in the marketplace of academia and science, and some proponents of those failed ideas then try to create a second life in the public marketplace which has different and often more permissive rules. They try to game Wikipedia, journalists, or the educational system, or they simply market their ideas in books or online.

That is why there needs to be additional rules for good journalism, or for projects like Wikipedia in order to prevent such end-runs around proper scholarship.