Apr 06 2015

The Google University Effect

I remain endlessly fascinated with the incredible social experiment we have all been living through over the last decade (and I can say, if you are reading this, you are part of the experiment). The internet and social media have changed the way we access information and communicate. The traditional top-down systems of information and opinion dispersion are eroding, being replaced by a largely bottom-up free-for-all.

I think we’re still figuring out all the consequences of these changes, both intended and unintended. One effect that has been casually observed is that many people believe they have expertise they do not have because they have been able to do “research” online. The democratization of information has led to a false sense of democratization of expertise.

While free access to information is great, there is no systematic way in which the public is taught how to use this information to maximal benefit, and avoid the most common pitfalls. Schools are generally behind the curve in terms of teaching students how to manage their online information access. Most adults were done with their formal education before the wave of social media.

The result is the “Jenny McCarthy Effect.” She is a celebrity who feels that she can substitute her own non-expert opinion for the strong consensus of expert opinion on the safety and effectiveness of vaccines because she “did her own research.” She is an obvious example of how searching for information online can give someone a false confidence in an unscientific opinion, illustrating the fact that relying on “Google University” can be extremely misleading. There are some specific pitfalls at work here.

The first pitfall is the subject of a recently published series of experiments by Matthew Fisher, a doctoral student in cognitive psychology at Yale University. He looked specifically at the effect of searching online for information and confidence in one’s knowledge on that topic. Of course it makes sense that if we search for and read information on a topic this will increase our confidence in our knowledge about that topic. Fisher, however, tried to control for as many variables as he could to see if there was an independent effect of just searching, regardless of how it affected our actual knowledge.

He found that people had higher confidence in their knowledge even when they searched for a subject vs being taken there directly, when the topic had no relevant information online, when their searches were filtered for relevant information, and when they read the information online vs in print. So even when the actual information was controlled for, the act of searching online itself seemed to raise confidence in one’s knowledge.

These types of experiments are obviously complex and we’ll need to see this replicated from different angles, but so far it does seem that having access to a vast store of knowledge that one can sift through raises our assessment of our own knowledge (beyond access to relevant information itself).

It seems to me that there are likely other effects at work as well – chief among them is confirmation bias on steroids. Searching online gives us the opportunity to mine a vast amount of information and select (even if unconsciously) that information which confirms what we already believe or want to believe. Search online for information about vaccines and you can find plenty of information that supports the safety and effectiveness of vaccines, and plenty of information that vilifies vaccines. Choose any controversial topic and the results will be the same.

Confirmation bias is powerful and dangerous specifically because it creates the illusion that the data supports our beliefs, because we are unaware of the degree to which we have filtered and biased that information. The internet is a setup for confirmation bias.

The extreme version of this phenomenon is what we call “echochambers.” Filtering information can be formalized into online communities where only one perspective is expressed, and information that supports that perspective is shared, while opposing information is filtered out or directly contradicted. This is a pervasive effect, and is true of scientific and skeptical sites as well as pseudoscientific ones.

Another potential problem is the confusion of knowledge with expertise. This is often what leads to cranks – people who may be very smart and have a great deal of factual knowledge, but come to absurd conclusions in which they have high confidence. One problem with cranks is that they do not properly engage with the relevant intellectual community.

It is critically important to engage with the community, especially in highly complex and technical areas of knowledge. It can be very difficult for any individual to see a complex issue from every angle, and to consider all perspectives. Left alone we will tend to create a neat narrative, and become increasingly convinced in the truth of that narrative. Engaging with the community will tend to challenge that narrative, leading to a deeper and more nuanced understanding of the topic. This is the heart of true expertise.

Studying a subject alone by searching online can be a crank factory – giving factual knowledge without really engaging with the ideas. Then the echochamber effect can give the illusion of engaging, but only with a biased community rather than the broader community. The result are people who falsely believe they have sufficient knowledge in areas they do not truly understand. The Dunning Kruger effect kicks in as well, and they likely do not appreciate the gulf between their Google University understanding of a topic and the depth of understanding of true experts.


The internet may be creating an army of overconfident pseudoexperts. There are a number of fixes to this problem on the individual level:

– Be humble. Do not think a little knowledge makes you an expert. Respect the opinions of actual experts. (You don’t have to agree, but at least take them very seriously.)

– Understand the inherent advantage of a consensus of expert opinion over the opinions of any individual.

– When searching online, go out of your way to search for information which goes against your current belief or conclusion. Try to find what both or all sides are saying, and reserve your personal judgement until you think you have heard all sides.

– Understand how online searching is a setup for confirmation bias. Google itself can bias the results of your search. You can turn this feature off.

– Understand that, in addition to confirmation bias, there is organized bias on the internet – echochambers, astroturf campaigns, and deliberately biased ideological information. Be on the lookout for false information, and carefully vet a source before you rely upon it.

– As always, there is no substitute for skepticism and critical thinking.

20 responses so far

20 thoughts on “The Google University Effect”

  1. Lukas Xavier says:

    Be humble. Do not think a little knowledge makes you an expert.

    Unfortunately, there’s a tendency (at least in some people) to believe statements, not in accordance with their factual accuracy, but according to how confidently they’re asserted. If a person seems really sure of themselves, they must be right. This creates a bias in favor of people who are not humble and against people who admit honest doubt.

    An example of this is when scientists admit limits or flaws in their research. Within science, this is an admirable trait; you should never overreach what your evidence can support; your conclusions are always tentative and open to revision.

    However, certain pseudoscience cranks will then go on to use those caveats as a way to undermine the legitimate scientific conclusions, or to open up loop holes for their favorite woo. And their audience will accept those claims, because the peddlers of woo typically are extremely confident in their conclusions.

    Our only real advantage is the truth and, strong as that is, it doesn’t help if people just plain don’t care.

  2. GWD says:

    Good article! Now I am wondering when Google will start making out printable diplomas and if they will charge for them.

  3. Damlowet says:

    100% Agree with you Lukas, I was literally having this conversation over my Old’s dinner table yesterday for Easter dinner.
    I would add to that also, in that the reason that people are persuaded by the 100% confidence of the cranks, is that the scientific method is not understood. If it were understood for what it is and what it indicates, people would realize that there can never be a 100% assertion to anything.

  4. petrucio says:

    I find it tremendously useful to show people around me the “Mount Stupid” comic (http://www.smbc-comics.com/?id=2475). It may sound like just a joke at first, but it easily distills a lot of what you discussed in this post, and people really see that there’s a lot of truth to it.

    Then when something comes up and someone goes “I’ve seen online that so and so”, you can just comeback with “Beware the Mount Stupid”, to make them really stop in their tracks and think deeper. I guess your mileage will vary, but it has worked with me.

  5. BetaclampDan says:

    I’ve noticed the points this article illustrates splashed across social media. It always makes me die a little inside when I see yet another article from NaturalNews, but belief is a powerful thing and even when the information is so preposterous (as it often is on prior mentioned lunatic website) if you have a distrust in science or the medical establishment it plays right into your ideology and therefore gets accepted.

    The problem is that no reasoning seems to make much difference with the die-hard credulous, and so the only ones that we might be able to reach are those who could go either way, but being that the internet misinformation probably outnumbers the real information this is a difficult accomplisment.
    I think schools have a lot to teach on information gathering, and lessons in critical thinking but I will not be holding my breath, (unless I happen to see one more inane link to NaturalNews in which case I think I might hold my breath until I pass out).

    Another great blog post.

  6. John Danley says:

    I used to think that Sturgeon’s law was for fishermen. I know better now.

  7. Factoidjunkie says:

    I agree that all these things regularly happen. I also submit some of them happen even when a person works diligently to use techniques to overcome or at least massively subdue the effects.

    Two additional thoughts, though. First, even universities, steeped in the understanding of method and the need for critical thinking will emerge with major biases. I’ve been educating myself on the history of economics and this subject makes clear that bias is nearly always at work. There are even different “schools,” of thought, both figuratively and literally.

    This same phenomenon applies to physics, though, as well. There may be greater evidence for string theory than quantum loop theory, but that doesn’t lessen the disagreement at times. Sean Carroll just posted a debate he attended in which two individuals with different views of “time” defended the other’s view (talk about a cool way to inspect your personal bias).

    The second thought concerns an individual pursuing a goal of continued education. Online is a huge resource that can massively help someone learn about new areas. And if that person works with the knowledge that confirmation bias is a chronic problem to control, then a person can be reflectively self-educating.

  8. evhantheinfidel says:

    Over-confidence is actually endorsed in my high school. The starkest example of this was in my US history class, when we were assigned a paper wherein we were supposed to examine various examples of treason throughout US history (see Rosenbergs) and determine the psychological underpinnings of these betrayals. I did the assignment within the bounds of the given rules, but expressed my lack of expertise, and my thoughts that it was impossible to really tell given the small sample size and the large amount of potentially confounding factors surrounding the situations, and I was counted off for expressing too little confidence both in my writing and my eventual presentation of the paper. This wasn’t even laid out in the rubrics we got, it was just determined to be “bad presenting”. The teacher very clearly laid out that this was why I was counted off (I got a C instead of an A), and refused to budge even after I talked to him in what I thought was an extremely reasonable way. Most cases weren’t this bad, but this serves as a fairly accurate representation of the general expectations in my high school.

    Now, I am NOT saying that this is what high schools are generally like in the US at large, but if they are, I think that’s quite problematic. High school should at least be the beginning of the tearing down of people’s over-confidence, which will continue for a bit in tertiary education before being built up properly and specifically.

  9. Pete A says:

    I seem to suffer from the opposite of the Dunning–Kruger effect: I’ve often been told that I’m far more knowledgable than I think I am and that I should stop putting myself down. However, my general knowledge and overall competence are well below average so I’m not putting myself down, I’m just being honest. It seems odd to me that my honesty and humility are frequently viewed as a psychological fault in dire need of correction. A frequently made remark is that I suffer from low self-esteem. So what? The importance of obtaining a high level of self-esteem has been debunked and its dangers highlighted — this article by Will Storr is fascinating:

    My question is: How does one determine if they have expertise in a particular subject or if they might even be an expert in that subject? I’m thinking of a few things I achieved during my career, e.g. finding a solution to an engineering problem that was considered by experts in the field to be unsolvable. I don’t think that makes me an/the expert, it just demonstrates that my peers were not as expert as they thought they were!

    The biggest problem I’ve always had with Google and other search engines is that the results are ordered by popularity, not by factual accuracy. Similarly, the biggest problem with the information content of the World Wide Web is that over 99% of it varies between being slightly inaccurate to abjectly false. For the vast majority of people, trying to find the truth about an evidence- or science-based subject is like looking for a needle buried in one of many thousands of haystacks. Nobody is going to read through all of the multiple thousands of search engine results and apply critical thinking to each and every article.

    Learning how to seek sources of accurate information, and how best to avoid fuelling one’s confirmation bias, takes an extraordinary level of due diligence, which is a skill that is neither innate nor widely taught. The other thing I need to keep firmly in mind is that most people much prefer being popular than being factually correct — the two are mutually exclusive in most social settings!

  10. WyseMD says:

    “When searching online, go out of your way to search for information which goes against your current belief or conclusion. Try to find what both or all sides are saying, and reserve your personal judgement until you think you have heard all sides.”

    I think this is what separates the true critical thinker from the sheeple. Sometimes it takes a lot of effort so suppress one’s own confirmation bias.

    As a doctor I often see medical things on the internet or the TV news that I know are false or grossly distorted but that the lay person would take as truth. It makes me wonder how many things I see and hear that I am not an expert in that I blindly take as truth. I do not have the time to do in depth research on everything. I find the NeuroLogica blog is a good place to start. Thanks Steve.

    Looking forward to the NECSS this weekend.

  11. elmer mccurdy says:

    I lost 20 years of my life to doctors spewing pure crap in my face, always with utmost confidence. That arrogant sociopath Itamura even once had the gall to tell me, “Stop reading [Elmer]”

    Finally took Dr. Kharrazi 2 seconds to finally notice, using a highly innovative modern instrument called his “eyes” that my pec tendon was completely torn, and the muscle was severely atrophied and retracted 5 cm from the shoulder. So I finally, 20 years two late, got a new tendon, and they cut through the same scar that had been used by Itamura, Johnson, etc. etc. etc.

    Conclusion: don’t be fooled by your doctor’s reflexively confident manner. The chances are very very high that he’s completely full of shit. And remember, he doesn’t give a fuck about your welfare.

  12. Elmer – I am sorry you had a bad experience. However, you are making the hasty generalization logical fallacy. You have no basis to believe that your one experience is representative or typical. You are clearly angry, and I get that, but that is not a justification for making sweeping negative statements about all doctors.

  13. elmer mccurdy says:

    It’s a generalization from a pretty large sample.

  14. elmer mccurdy says:

    I did not have “a bad experience.” I had a long series of consistently bad experiences over a 20 year period.

  15. jsterritt says:

    @elmer mccurdy

    “The chances are very very high that he’s completely full of shit.”

    You indict two ortho surgeons for being arrogant and wrong and one for being a veritable miracle worker. This is not a broad sample (you can tell by the absence of a curve: your docs are either monsters or saints). You cannot characterize your experience as “a pretty large sample” when it is a single patient outcome, nor extrapolate from it the generalizations you make. That’s the point of the OP: expertise is not easily acquired.

    I’m sorry about your sh!tty and lengthy health issue. I’m glad to hear it has improved and that you’ve found a doc who helped.

  16. andrewtoronto says:

    The writer should replace “Respect the opinions of actual experts” with “respect the results of properly ascertained scientific evidence”. Anyone can claim to be an “expert. The title is meaningless. “experts” are a dime a dozen and often disagree. the word “expert” needs to be shot.

    history is full of “consensus of expert opinion”. that didn’t make the opinion correct. An opinion’s validity is proportional to how well it is backed up by logical/scientific evidence.

    “beware the expert opinion” -carl sagan.

    …”Conclusion: don’t be fooled by your doctor’s reflexively confident manner. The chances are very very high that he’s completely full of shit. And remember, he doesn’t give a fuck about your welfare.”

    hyperbole. there are bad doctors of course. my wife is a doctor. a good one. she helps people and saves lives. she understands firsthand how a little knowledge is a dangerous thing. everyday she sees patients who think they know best because they read a little bit on the net. sometimes they come in with their problem correctly self-diagnosed. most of the time not. the reason is that the internet is a terrible place to get medical advice. you tend to get worst case scenarios and an influx of info that is impossible for the scientifically illiterate to parse. the internet is turning people into hypochondriacs. i have a high degree of layman’s medical and scientific knowledge. but i dont pretend to have a fraction of medical knowledge that is inside my wifes head.

    scientific illiteracy is perhaps a greater danger now and in the future than old-fashioned illiteracy.


  17. RC says:

    I think elmer is a bit overboard, but there’s definitely a problem in this country with doctors – I’m not sure if it’s overconfidence, or seeing too many “cry wolf” type patients, or insurance not paying for ‘unnecessary’ diagnostics, or what, but getting a doctor to actually listen to you, or actually examine something is becoming exceedingly difficult.

    My wife has a back problem – a broken vertebrae and bulging disks (due to a work injury). It took seeing several doctors before one would even give her an xray. (this was at Johns Hopkins – so not some podunk hospital)

    My experience at this point is you either have to go in with a suspected diagnosis and badger doctors into doing the diagnostic work, or you’re going to get a prescription for 800mg ibuprofen (or whatever the treatment analog is for your symptoms) and told to take it easy. Their focus seems to be on getting out of the room as quickly as possible.

  18. The sooner Google introduces a indicator that ranks the “quackery” of each results the better.

  19. rbaker says:

    While those studies are certainly interesting, think it’s worth pointing out that I’d be shocked to find out that if you could control for learning some information in the context of reading a book vs. reading a book assigned as part of a university program, that there wasn’t some bias there as well.

    The question that really needs to be answered would be the amount of bias relative to the comparative average accuracy of each learning method.

  20. BillyJoe7 says:

    I am reluctant to take elmer’s story at face value.

    I find it hard to believe that his GP missed a diagnosis of a ruptured pectoral tendon. And that he continued to miss that diagnosis for twenty years! I find it even harder to believe that a series of specialists (“Itamura, Johnson, etc. etc. etc.”) also missed the diagnosis and continued to miss the diagnosis for twenty years. Even after operating on him on many occasions (‘they cut through the same scar that had been used by Itamura, Johnson, etc. etc. etc.”)! That, frankly, is unbelievable. He probably also attended a variety of allied health professionals who apparently also missed the diagnosis. Finally he found one who diagnosed him correctly.

    A more likely scenario is that he suffered an injury to his pectoral muscle that led to atrophy of that muscle and its tendon over the next twenty years. Eventually the atrophied tendon ruptured completely as a result of some minor injury. The next specialist he saw then correctly diagnosed the rupture, which had not previously been present.

    I think elmer should inform his previous specialists – two of whom he has named and shamed – and give them a right of reply.

    How about it elmer?

Leave a Reply