Mar 25 2014

Standards of Evidence – Wikipedia Edition

There are many public intellectual debates occurring over scientific and skeptical issues – the place of creationism vs evolution in public science classes, the including of alternative medicine in academic curricula, the validity of debate on global warming, etc.

Many of these issues, while important, are proxy issues for a deeper cultural conflict – the role of standards in the intellectual, academic, and scientific spheres.

Scientific skeptics (whether they go by that label or not) generally take the position that there should be fair and reasonable standards by which to evaluate any factual claim or intellectual position. We need a process to ensure that our collective thinking is logically valid, balanced in it judgments, and properly accounts for all available evidence. With regard to empirical claims, we call this process science, but these virtues are generic to any intellectual endeavor.

Therefore, what gets taught is public science classrooms should meet some reasonable standard to be consider part of mainstream science. The standard of care in medicine should follow some evaluation of evidence and outcome. Peer-reviewed scientific or academic journals need to have a standard of quality and transparency. Licensed professionals should be required to meet some standard of professionalism, knowledge, and ability.

Underlying all of these issues is the more fundamental issue of standards.

There are those, however, who do not want standards. It is not by coincidence that these are the people who do not or cannot meet existing standards. They feel it is better to whine about the standards than to improve the quality of their claims and arguments.

They typically will characterize quality standards as oppressive, closed minded, protectionist, and elitist. As they continue down that rabbit hole, they further claim that standards are really just part of a conspiracy by those in power to protect their privilege or world view.

I am, in fact, happy to see this basic struggle apply to Wikipedia, because it means that Wikipedia has sufficient standards in place to at least annoy some cranks enough to complain about it.

A petition on Change.org, in fact, asks Wikipedia founder, Jimmy Wales, to be more open to fringe claims about “holistic” medicine. They hit all the red flags about interpreting standards as a conspiracy, and name skeptics as the primary perpetrators of this conspiracy.

You can read the entire text, and some more commentary on the issue, at Respectful Insolence.

Here is Jimmy Wales response:

No, you have to be kidding me. Every single person who signed this petition needs to go back to check their premises and think harder about what it means to be honest, factual, truthful.
Wikipedia’s policies around this kind of thing are exactly spot-on and correct. If you can get your work published in respectable scientific journals – that is to say, if you can produce evidence through replicable scientific experiments, then Wikipedia will cover it appropriately.
What we won’t do is pretend that the work of lunatic charlatans is the equivalent of “true scientific discourse”. It isn’t.

Nicely done. Wales is essentially saying, we have standards. Deal with it.

As Orac points out, however, you have to be careful about using publication in peer-reviewed journals as a dominant criterion. The infiltration of nonsense into academia is such (combined with the proliferation of low-quality journals) that this is an insufficient criterion.

But the broader point is well made. There are standards of what is consider a legitimate claim. If you can’t meet that standard, stop whining about it, and certain don’t expect Wikipedia to lower their editorial standards just to let in your particular brand of nonsense.

This one definitely goes in the win column.

Share

20 responses so far

20 Responses to “Standards of Evidence – Wikipedia Edition”

  1. killerbeeeon 25 Mar 2014 at 3:51 pm

    It’s a shame blogs aren’t peer reviewed.

  2. pdeboeron 25 Mar 2014 at 4:56 pm

    @Killerbeee What do you call this?

    @Steve

    I think nonsense in academia is covered in Wales Statement “Wikipedia will cover it appropriately.” As you say about the media, facts should be covered in proportion to their validity(Paraphrasing horribly…). This would be appropriate coverage, and I feel Wikipedia generally follows that rule.

  3. tmac57on 25 Mar 2014 at 9:11 pm

    Just like the proponents of alternative medicine seek to undermine the public’s confidence in medicine,there is also a move afoot to promote what I would characterize as alternative reality.

    That is the world view where pretty much any assertion can be made ,totally fact free,then repeated ad nauseum in the echo-chamber of self-selected alternative “news” sites,email,and social media.
    And when challenged to produce evidence,or countered by fact checking the proponents fall back on essentially a defense of “I don’t trust any fact checkers or MSM”.
    This insular segment of society are self congratulatory for their belief that they are the enlightened, while everyone else are mere sheep.Yet they never question or explore the extraordinary nature of their own claims, while poo pooing the much more plausible ideas that best fit the known evidence.

    More power to Jimmy Wales and Wikipedia for sticking to their guns in the pursuit of a more rational source for understanding our world.

  4. grabulaon 25 Mar 2014 at 10:15 pm

    @tmac – isn’t that basically the post modernist movement? People seem to think their losing their freedoms if they can’t define their own reality. I see it a lot in my area (boulder, colorado) and as of late on forums and comment threads like these. We seem to be having a hard time differentiating between living the lifestyle we want to live and not forcing that lifestyle on others or accepting that maybe our reality doesn’t jibe with actual reality.

    I read Orac’s blog on this yesterday and then jumped to this blog: http://scienceblogs.com/evolutionblog/2014/03/18/coyne-and-moran-on-teaching-id/ In the comments section there are people who seem to think it’s necessary to teach ID for example in a science class so you can understand both sides of the “controversy” as if there is one. Their points being that ultimately people should be denied nothing.

    I wonder if this is associated with the idea that the last few years that it’s not ok to let your kids fail, and slap a bumper sticker on your car proudly stating that your kid almost made the honor roll once, and is on the C team of his local high school soccer club.

  5. Will Nitschkeon 26 Mar 2014 at 1:25 am

    @ Steven Novella

    I don’t disagree with the argument that what gets taught in a class room should meet a reasonable standard. However, you seem to be conflating that with what people read in Wikipedia…? Has Wikipedia now become the standard for classroom teaching? Who decided that? Perhaps children may enhance their critical thinking skills by discovering that nonsense exists in Wikipedia?

    However, outside the class room, scientific understanding and research is in a very different class from, say, medical or engineering practice. Parts of the scientific endeavour are by their very definition on the frontiers of our understanding of the natural world. These activities cannot be subject to formal ‘standards’. Who decides the standards? How do we decide that those who have decided the standards have selected the correct standards? Are we going to vote on it? Or hope for the best?

    Feel free to ban me if you find any of these criticisms or general observations uncomfortable. ;-)

  6. grabulaon 26 Mar 2014 at 1:54 am

    @WillN

    “However, you seem to be conflating that with what people read in Wikipedia…? ”

    You’re inferring that. It’s important to those of us who appreciate the truth, that common sources of knowledge, like wikipedia are maintained to a reasonable and rational standard.

    “Parts of the scientific endeavour are by their very definition on the frontiers of our understanding of the natural world. These activities cannot be subject to formal ‘standards’. Who decides the standards? ”

    This is complete garbage. For one thing, the woo peoples make all sorts of claims that what they do can’t be understood by science, which is a huge misconception of what science actually is. They like to claim they are pushing the boundaries of our knowledge but can’t back up one bit of their claims with any sort of verifiable evidence.

    As for formal standards, you absolutely need formal standards in order to determine the validity of a thing or a theory. Formal standards allow experiments to be replicated and evidence to be got through different avenues of approach.

    But this is all so very academic Will. I keep waiting for the big reveal, thinking you motivation might be one other than just to harass Dr. Novella. Being from Australia I hypothesize that you might be in some intimate way related to the old AVN and are out on the attack under some directive (ooooh spooky conspiracy theory) or as is more likely, have had too many drinks and have decided he will be the object of your drunken rages. The latter really does explain your lack of reading comprehension and your inability to reply coherently to even the slightest challenge you receive.

  7. Will Nitschkeon 26 Mar 2014 at 2:08 am

    @ Steven Novella

    A question: when you write “there are many public intellectual debates” and then call for standards are you suggesting that the public intellectual debates be subject to standards? Or would they continue to be exempt from standards? If debates are to be subject to standards, who decides the standards, and doesn’t this imply that certain public intellectual debates that do not meet these ‘standards’ would not be permitted?

  8. BillyJoe7on 26 Mar 2014 at 7:19 am

    WN,

    “If debates are to be subject to standards, who decides the standards”

    Wrong question.
    Right question: If debates are to be subject to standards, what are these standards?
    Answer: Logic, rationality, and scientifically derived evidence.

    Please read the article.

  9. Steven Novellaon 26 Mar 2014 at 7:21 am

    Will – your inferences are all incorrect.

    I never wrote anything that would imply restrictions on public discourse, or that Wikipedia is the standard for classroom teaching. As is often the case, your “questions” have all be answered already in the text.

    This is the paragraph you should read again:

    “Therefore, what gets taught is public science classrooms should meet some reasonable standard to be consider part of mainstream science. The standard of care in medicine should follow some evaluation of evidence and outcome. Peer-reviewed scientific or academic journals need to have a standard of quality and transparency. Licensed professionals should be required to meet some standard of professionalism, knowledge, and ability.”

    Perhaps you got stuck on the word, “therefore.” I gave these specific examples to illustrate what conclusions flow from my premises – that specific institutions of learning and professionalism should be held to specific standards that are appropriate to them.

    Wikipedia wants to be a reliable resource of information. Good for them. This requires standards. There are certain generic intellectual virtues that should be part of any such standards – valid logic, verifying facts, sourcing claims, etc. The wheel does not have to be reinvented. Wikipedia’s challenge was adapting these virtues to the wiki format. They are doing a pretty good job. There is always going to be someone on the losing side of quality control, and they will typically whine about the standards. That’s just human nature. Let them whine.

  10. RickKon 26 Mar 2014 at 9:12 am

    Will,

    What standards do you use to differentiate between “frontiers of science” and “complete bollocks”?

  11. steve12on 26 Mar 2014 at 11:48 am

    People who say outrageous / stupid things for attention always generate interest at first, and then become BORING.

    If Will hadn’t crossed over before, he certainly has now.

  12. Bruceon 26 Mar 2014 at 4:09 pm

    steve12,

    I think you hit the nail on the head. The best way to get someone to respond to you directly is to disagree with them, agreeing is not only conversation ender (read: “you are right!”; “yes, I am, thanks!”) but is also seen as a weak response as it somehow implies you have no opinion of your own. It is much harder to agree with the consensus and keep the discussion going and draw attention to yourself than it is to disagree and then sit back and watch the direct responses to your comments come washing in.

    Will has crossed over very much now (quite a while ago actually) and is not interesting in any way as he seems to have abandoned all attempts at actually reading and comprehending the original blog anymore and is showing that all he is interested in is getting a response by quite blatantly trying to dig at Steven directly.

  13. Will Nitschkeon 26 Mar 2014 at 5:30 pm

    @ Steven Novella

    “Will – your inferences are all incorrect.”

    If you read my post properly, rather than assume a nefarious agenda, you would see that my ‘inferences’ were posed as questions. I was not actually able to ‘infer ‘what your position was, as, with many of your posts, you tend to list the issues but then hesitate in taking a position, yet seem to hint that you have one.

    “Wikipedia wants to be a reliable resource of information. Good for them. This requires standards. There are certain generic intellectual virtues that should be part of any such standards – valid logic, verifying facts, sourcing claims, etc. The wheel does not have to be reinvented. Wikipedia’s challenge was adapting these virtues to the wiki format. They are doing a pretty good job. There is always going to be someone on the losing side of quality control, and they will typically whine about the standards. That’s just human nature. Let them whine.”

    Unfortunately the contradiction is that, in theory, anyone can edit or add to Wikipedia. Hence it either has to apply a standard or drop the pretence that the material it presents is ‘open source’ as it is not possible to do both.

    Of course, as usual, you ignored my pertinent questions and simply went off (again) on some minor sideline I’m assuming you felt you could deal with. Who makes the standards for general discourse, or how does Wikipedia intend to apply them? What are you hinting at here? Should such standards be applied to public intellectual debates? If not, why mention them in the opening remarks of your post?

    Don’t be scared to try to answer some tough questions on your positions. I don’t bite, or even if I do, in the grand scheme of things this exchange is not all that important. What you do in your practice, and what I do in my business, is far more important.

  14. Will Nitschkeon 26 Mar 2014 at 9:27 pm

    @ Steven Novella

    Also Steven, since you wrote that the debates on the topic of global warming have no validity (whatever that intentional vagueness means?) and since you don’t accept the existence of a ‘pause’ would you care to comment on this video -

    http://www.youtube.com/watch?feature=player_embedded&v=PxYeBqG6H58

    I would be interested in your reaction and if you have anything substantive to say, other than label or name call, or object to the “tone”.

    Thank you.

  15. steve12on 26 Mar 2014 at 11:31 pm

    Bruce

    Just wanna say that I agree. You’re right – nowhere to go with this!

    I will say this – there are people who ask interesting questions about why the consensus is what it is and foster discussion. Will’s just a child yelling.

  16. grabulaon 27 Mar 2014 at 1:02 am

    “Unfortunately the contradiction is that, in theory, anyone can edit or add to Wikipedia. Hence it either has to apply a standard or drop the pretence that the material it presents is ‘open source’ as it is not possible to do both.”

    They have a standard. The whole point of the article above is that they have standards. Man, you really are just trolling aren’t you.

  17. Bruceon 27 Mar 2014 at 6:14 am

    Steve12:

    “I will say this – there are people who ask interesting questions about why the consensus is what it is and foster discussion”

    Yup, you are right.

    Not much more to say really…

  18. BillyJoe7on 27 Mar 2014 at 7:11 am

    Goddamn, now our little kangaroo wants to hop into climate change.

  19. darrenyorstonon 28 Mar 2014 at 6:26 am

    Hello Steve,

    Long time fan, first time poster!!

    Very interesting topic. I thought you might find this story interesting as it is on the same topic:

    https://theconversation.com/navigating-the-online-information-maze-should-students-trust-Wikipedia-24559

    Instead of attacking the medium of distribution we need to apply some basic thinking skills to evaluate information. Its a shame that many of our academics seem to have missed the need to learn critical thinking skills and are stuck in the age of parchment. Civilisation is changing and we need to keep up with our method.

    Thanks for your insights!

  20. SimonWon 01 Apr 2014 at 1:47 am

    “Unfortunately the contradiction is that, in theory, anyone can edit or add to Wikipedia. Hence it either has to apply a standard or drop the pretence that the material it presents is ‘open source’ as it is not possible to do both.”

    Wikipedia claim to be “free” not “open source”. Free doesn’t imply no standards. Anyone can edit wikipedia, your edits will likely be rejected if they are not good. Free means you can create your own copy of wikipedia, and allow your own edits, just don’t be surprised if no one is interested in reading it. Exactly the same process happens in free software.

Trackback URI | Comments RSS

Leave a Reply

You must be logged in to post a comment.