Archive for February, 2021

Feb 04 2021

Is Dunning-Kruger a Statistical Artifact?

Published by under Neuroscience

The short answer to the headline question is – not really, but it’s complicated.

The Dunning-Kruger effect, which I have written about several times before, was first published in 1999 by the named psychologists. The basic effect is this – if you graph self-perception of knowledge in a specific domain and performance on an objective test of that knowledge, there is a typical graph of the relationship between these two things. Specifically, the less people know, the more they overestimate their knowledge. They still rate themselves lower than people who know more, but the gap between perception and reality grows. Further, at the high end (the top quartile) people actually underestimate their relative knowledge, probably because they overestimate average knowledge in the public. And everyone thinks they are above average.

This effect is extremely robust and has been replicated many times in many contexts. As the authors have emphasized before – the DK effect is not about stupid people, it is about everybody. It is not about intelligence, but knowledge.

There is also a distinct effect some are calling a super-DK effect in which in specific knowledge areas, like genetic engineering, the people who low the least think they know the most. This is not just about knowledge, but about misinformation. If people are actively misinformed they will have the illusion of knowledge.

The DK effect has been a cornerstone of science communication and our understanding of how to improve knowledge in the last two decades. However, a recent study calls the basic effect into question – The Dunning-Kruger effect is (mostly) a statistical artefact: Valid approaches to testing the hypothesis with individual differences data. The study essentially showed you can reproduce the DK graph using a randomly generated set of data. How is this possible?

Continue Reading »

No responses yet

Feb 02 2021

Multiverse Revisited

Published by under Logic/Philosophy

A couple weeks ago I wrote about the logical implications of the multiverse as a potential solution to the so-called fine-tuning problem. I was responding to a paper discussed by a philosopher  (Goff) claiming that the multiverse hypothesis is not a valid argument but rather based on a logical fallacy, the inverse gambler’s fallacy. I won’t repeat the entire discussion here, just read the original article. I am writing this follow up because the original article garnered a great deal of discussion. I also presented the issue on the SGU, triggering a flood of e-mail responses. Clearly I need to take another bite at this apple.

I do think the discussions have clarified my thinking, although they have not changed my position. I am still willing to change – statistics can be very counterintuitive and can hinge on seemingly unimportant details. The Monty Hall problem is a classic example, which some call a “statistical illusion”. One where I was tripped up previously deals with gambling, this time the regular gambler’s fallacy. A number of sources will claim that casinos win largely because players go bust, and end their betting as a loser, but the house never busts. So there is an “absorption wall” at one end, but not the other. Players can keep playing if they win, long enough to lose again in some cases, but have to stop if they lose too much. While this seems to make sense, it is wrong. The house wins entirely because the odds are in their favor, and the loser absorption wall has no effect on this outcome. This is because players are just as likely to win or lose after they go bust, so going bust does not prevent them from winning more than it prevents them from losing even more.

Continue Reading »

No responses yet

Feb 01 2021

Protein Switches and COVID Testing

Researchers report in Nature the development of a new technique for designing protein switches that can be used as biosensors. The recent development of the technology to design specific protein switches is an underreported story, in my opinion, and represents a technology with incredible possibilities. Reporting on the recent study emphasizes one possible application – development of a new rapid test for SARS-CoV-2. It is understandable why this would garner the most interest – but the underlying technology is perhaps a bigger science news story.

A protein switch is simply a protein that can change its 3-dimensional configuration in response to binding with something, such as another protein or a hormone or some biological signal. When a protein changes its configuration, it changes its function. This can turn a function of the protein on or off, open or close a pore or channel, or alter its activity. Protein switches are a basic component of biological function as they allow for the sensing of internal biological states and reaction to those states by altered cellular function.

It was only in 2019, less than two years ago, that scientists reported the design and creation of the first completely artificial protein switch. Again, this story did not make a huge splash, but looking back this may have been as momentous as the development of CRISPR as a tool for genetic engineering. It’s hard to tell how much of a long term impact it will have – but just as CRISPR (and related tools of genetic engineering) gives us unprecedented control over a fundamental aspect of biology (genetics), protein switches also potentially give us a similar level of control, arguably more direct.

Continue Reading »

No responses yet

« Prev