Feb 04 2021

Is Dunning-Kruger a Statistical Artifact?

Published by under Neuroscience

The short answer to the headline question is – not really, but it’s complicated.

The Dunning-Kruger effect, which I have written about several times before, was first published in 1999 by the named psychologists. The basic effect is this – if you graph self-perception of knowledge in a specific domain and performance on an objective test of that knowledge, there is a typical graph of the relationship between these two things. Specifically, the less people know, the more they overestimate their knowledge. They still rate themselves lower than people who know more, but the gap between perception and reality grows. Further, at the high end (the top quartile) people actually underestimate their relative knowledge, probably because they overestimate average knowledge in the public. And everyone thinks they are above average.

This effect is extremely robust and has been replicated many times in many contexts. As the authors have emphasized before – the DK effect is not about stupid people, it is about everybody. It is not about intelligence, but knowledge.

There is also a distinct effect some are calling a super-DK effect in which in specific knowledge areas, like genetic engineering, the people who low the least think they know the most. This is not just about knowledge, but about misinformation. If people are actively misinformed they will have the illusion of knowledge.

The DK effect has been a cornerstone of science communication and our understanding of how to improve knowledge in the last two decades. However, a recent study calls the basic effect into question – The Dunning-Kruger effect is (mostly) a statistical artefact: Valid approaches to testing the hypothesis with individual differences data. The study essentially showed you can reproduce the DK graph using a randomly generated set of data. How is this possible?

Continue Reading »

No responses yet

Aug 13 2018

Dunning Kruger Effect and Anti-Vaccine Attitudes

One of the persistent themes of this blog is that expertise matters. This is not to say the experts are always right (sometimes they disagree with each-other), and there is also a range of expertise, and different kinds of experts can have different biases and blind spots. But all things considered, someone who has formal expertise on a specific topic is likely to know much more about that topic than someone who has read about it on the internet.

Further, most people underestimate the amount of knowledge that exists on a topic, and therefore the vast gulf of knowledge that exists between them and the experts. In fact, the more someone knows about a topic the more they understand how much is known, and the more humble they tend to be with respect to their own knowledge. The flip side of this – people who know little tend to overestimate their relative knowledge – is an established psychological phenomenon known as the Dunning-Kruger effect.

Operationally Dunning and Kruger found in their study that the lower someone performed on a test of knowledge, the greater the gap between their perceived knowledge and performance and their actual performance. At around the 80th percentile and above, people tend to underestimate their relative knowledge. Below that point they tend to increasingly overestimate it, and everyone thinks they are above 50%.

Continue Reading »

No responses yet

Nov 06 2014

Lessons from Dunning-Kruger

In 1999 psychologist David Dunning and his graduate student Justin Kruger published a paper in which they describe what has come to be known (appropriately) as the Dunning-Kruger effect.  In a recent article discussing his now famous paper, Dunning summarizes the effect as:

“…incompetent people do not recognize—scratch that, cannot recognize—just how incompetent they are,”

He further explains:

“What’s curious is that, in many cases, incompetence does not leave people disoriented, perplexed, or cautious. Instead, the incompetent are often blessed with an inappropriate confidence, buoyed by something that feels to them like knowledge.”

As you can see in the chart above, the most competent individuals tend to underestimate their relative ability a little, but for most people (the bottom 75%) they increasingly overestimate their ability, and everyone thinks they are above average. I sometimes hear the effect incorrectly described as, “the more incompetent you are, the more knowledgeable you think you are.” As you can see, self-estimates do decrease with decreasing knowledge, but the gap between performance and self-assessment increase as you decrease in performance.

Continue Reading »

146 responses so far