Search Results for "bias"

Oct 20 2020

Daryl Bem, Psi Research, and Fixing Science

In 2011 Daryl Bem published a series of ten studies which he claimed demonstrated psi phenomena – that people could “feel the future”. He took standard psychological study methods and simply reversed the order of events, so that the effect was measured prior to the stimulus. Bem claimed to find significant results – therefore psi is real. Skeptics and psychologists were not impressed, for various reasons. At the time, I wrote this:

Perhaps the best thing to come out of Bem’s research is an editorial to be printed with the studies – Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi by Eric Jan Wagenmakers, Ruud Wetzels, Denny Borsboom, & Han van der Maas from the University of Amsterdam. I urge you to read this paper in its entirety, and I am definitely adding this to my filing cabinet of seminal papers. They hit the nail absolutely on the head with their analysis.

Their primary point is this – when research finds positive results for an apparently impossible phenomenon, this is probably not telling us something new about the universe, but rather is probably telling us something very important about the limitations of our research methods

I interviewed Wagenmakers for the SGU, and he added some juicy tidbits. For example, Bem had previously authored a chapter in a textbook on research methodology in which he essentially advocated for p-hacking. This refers to a set of bad research methods that gives the researchers enough wiggle room to fudge the results, enough to make negative data seem statistically significant. This could be as seemingly innocent as deciding when to stop collecting data after you have already peeked at some of the results.

Richard Wiseman, who was one of the first psychologists to try to replicate Bem’s research and came up with negative results, recently published a paper discussing this very issue. In his blog post about the article he credits Bem’s research with being a significant motivator for improving research rigor in psychology:

Several researchers noted that the criticisms aimed at Bem’s work also applied to many studies from mainstream psychology. Many of the problems surrounded researchers changing their statistics and hypotheses after they had looked at their data, and so commentators urged researchers to submit a detailed description of their plans prior to running their studies. In 2013, psychologist Chris Chambers played a key role in getting the academic journal Cortex to adopt the procedure (known as a Registered Report), and many other journals quickly followed suit.

Continue Reading »

No responses yet

Oct 12 2020

Pre-Bunking

Psychologists in the UK have created a game that pre-debunks (or “pre-bunks”) COVID-19 conspiracy theories. The game is based on research that shows it can be more effective to give people information about how to identify conspiracy theories or misinformation before they are exposed to it. This is a fantastic idea, and I love the fact that this is being done in coordination with research to show if it is effective.

The current game is called Go Viral. It puts the player in the role of someone spreading conspiracy theories about the pandemic, and their goal is to make the misinformation go as viral as possible. This way the players learn the deceptive tactics of those who spread such misinformation by doing it themselves. This tactic reminds me of magicians who are skeptics. They have learned the techniques of deception, and have experienced how easy it can be to deceive people. Stage magic is essentially the practical art of misdirection, that exploits many of the weaknesses in our ability to perceive and construct an experience of what is happening. This puts magicians into a perfect position to detect deceptive practices on the part of others.

James Randi, for example, made a career out of exactly this. He has caught faith healers, for example, using standard mentalist tricks to deceive their audience. One example is the one-ahead trick. You have everyone fill out a “prayer card” with their basic information and what they want to pray for. All these cards are placed in envelopes and are then placed in a bowl, and the preacher draws them one by one “predicting” what each one will say prior to opening the envelope and “revealing” that they were correct. The audience is flabbergasted as the preacher, by seemingly divine means, knows all about them. However, the preacher is simply stating what they just read on the previous card. If you are a magician, this technique is easy to detect – and now you can detect it much easier because I just told you about it.

Continue Reading »

No responses yet

Oct 02 2020

Speaking of Venus

Published by under Astronomy

I recently discussed the exciting news of the discovery of phosphine gas in the clouds of Venus. This is exciting because phosphine is a potential marker of life. It should not exist on a small rocky world, and there is no known abiotic source on Venus. Microbes in the clouds above Venus are a plausible source, although scientists are careful to point out this is not proof of life, just a possibility. This finding has renewed interest in exploring our nearest neighbor, and even prior to this discovery NASA was planning another probe to Venus. This probe will likely have its mission altered to follow up on the phosphine discovery.

Venus is also interesting because it likely had a complex history over the last several billion years. A recent computer simulation, in fact, indicates that it may have been hospitable to life on the surface more than a billion years ago. Now the surface of Venus is hot enough to melt lead, making it the hottest planet in the solar system – even hotter than Mercury, which is closer. This is due to the extreme greenhouse effect from its mostly carbon dioxide atmosphere. The clouds above Venus are largely sulphuric acid. But a billion years ago it may have been more similar to Earth.

The dramatic change in Venus, in turn, may be tied to Jupiter. This is further connected to what we are learning about how stellar systems typically evolve, by observing exoplanetary systems. We now have many more data points than just our own solar system. We have confirmed over 4,000 exoplanets to date, with thousands more detected and awaiting confirmation. We can usually tell the mass, volume, and distance from the parent star, and so can construct a basic diagram of each system. However, depending on the method of detection used, we do not typically find every planet in an exosystem. Methods are generally biased towards larger and closer worlds.

One of the things we have discovered is that some planets are so-called hot Jupiters – they are gas giants orbiting very close to their star. About 10% of expolanets are hot Jupiters, and about 1% of systems have at least one hot Jupiter. Again – our detection systems more easily discover large and close planets, so these ratios may not represent reality and may be an overestimation. But a the very least, hot Jupiters are common throughout the galaxy.

Continue Reading »

No responses yet

Aug 18 2020

How Algorithms Affect Your Life

This is one of those things that futurists did not predict at all, but now seems obvious and unavoidable – the degree to which computer algorithms affect your life. It’s always hard to make negative statements, and they have to be qualified – but I am not aware of any pre-2000 science fiction or futurism that even discussed the role of social media algorithms or other informational algorithms on society and culture (as always, let me know if I’m missing something). But in a very short period of time they have become a major challenge for many societies. It also is now easy to imagine how computer algorithms will be a dominant topic in the future. People will likely debate their role, who controls them and who should control them, and what regulations, if any, should be put in place.

The worse outcome is if this doesn’t happen, meaning that people are not aware of the role of algorithms in their life and who controls them. That is essentially what is happening in China and other authoritarian nations. Social media algorithms are an authoritarian’s dream – they give them incredible power to control what people see, what information they get exposed to, and to some extent what they think. This is 1984 on steroids. Orwell imagined that in order to control what and how people think authoritarians would control language (double-plus good). Constrain language and you constrain thought. That was an interesting idea pre-web and pre-social media. Now computer algorithms can control the flow of information, and by extension what people know and think, seamlessly, invisibly, and powerfully to a scary degree.

Even in open democratic societies, however, the invisible hand of computer algorithms can wreak havoc. Social scientists studying this phenomenon are increasing sounding warning bells. A recent example is an anti-extremist group in the UK who now are warning, according to their research, that Facebook algorithms are actively promoting holocaust denial and other conspiracy theories. They found, unsurprisingly, that visitors to Facebook pages that deny the holocaust were then referred to other pages that also deny the holocaust. This in turn leads to other conspiracies that also refer to still other conspiracy content, and down the rabbit hole you go.

Continue Reading »

No responses yet

Aug 03 2020

Do Your Own Research?

A recent commentary on Forbes advises: You Must Not ‘Do Your Own Research’ When It Comes To Science. I agree  with everything the author, Ethan Siegel, says in the piece. It was a good start – but did not go far enough. For example, he did not really reach any conclusion about what people should actually do, beyond “listen to the experts.” OK – how, exactly, do we do that? This is not a criticism (I have written similar articles before) but an observation: after trying to communicate these same skeptical themes for decades and getting thousands of questions from the public, I have realized that it is perhaps not so obvious what it means to listen to the experts.

First let me amplify what Siegel gets right, although I may reframe it a bit. He correctly describes the typical process that people use when evaluating new information, although does not name it – confirmation bias. His summary is as good as any:

  • formulating an initial opinion the first time we hear about something,
  • evaluating everything we encounter after that through that lens of our gut instinct,
  • finding reasons to think positively about the portions of the narrative that support or justify our initial opinion,
  • and finding reasons to discount or otherwise dismiss the portions that detract from it.

Continue Reading »

No responses yet

Jun 19 2020

News vs Commentary

The line between news and commentary has arguably become more blurred in recent decades. This has implications for libel law, which also reflects the shifting media landscape. A recent lawsuit involving Tucker Carlson illustrates the problem.

Carlson is being sued for defamation by Karen McDougal for a segment in which she claims Carlson accused her of extortion.  She is one of two women that we know of who were paid off to remain silent about affairs with Trump. Here is the money quote from Carlson:

“Two women approached Donald Trump and threatened to ruin his career and humiliate his family if he doesn’t give them money. Now that sounds like a classic case of extortion.”

For background, libel cases are hard to prove in the US. You need to demonstrate that statements were made in public that are claims to facts, that are factually wrong, where the person making the statement knew they were wrong or had a disregard for the truth, that there was malice of intent, and that actual harm resulted. For some statements you don’t have to prove harm, they are “libel per se,” such as accusing someone of pedophilia. The harm is taken for granted. If the target of the alleged defamation is a public figure, then the burden of proof is even higher.

At issue here are whether Carlson’s statements were presented as facts or opinion. Opinion is completely protected free speech, and cannot be defamatory legally. The first part of Carlson’s statement above is stated as simple fact. The second part (“that sounds like”) seems to be his analysis or opinion. Forgetting the other aspects of the defamation standard for now, this question seems to be the crux of the case. Was Carlson making a factual claim he knew to be untrue, or without concern for whether or not it was true? The defamation standard requires more than just being wrong.

Continue Reading »

No responses yet

Jun 18 2020

Intelligent Life in the Galaxy

Published by under Astronomy

The headlines (taken from the press release) read: “New light shed on intelligent life existing across the galaxy.” But here’s the thing – I don’t think the referenced study does that at all. So what are they talking about?

The study uses their own version of the Drake Equation, which is a way of calculating how many spacefaring civilizations there are likely to be in the universe. The equation itself is correct – you consider the number of stars, the subset of those with planets in the habitable zone, the number of those who develop life, then intelligent life, then technology and multiply all that by the average lifespan of such civilizations. The equation works, as far as it goes, it’s just not terribly useful. The reason is that we don’t know the values of any of the variables. We can guess some of them, those dealing with stuff we can see, like how many planets are out there, but we essentially have no idea about any of the variables dealing with life.

The reason we have no idea is basic scientific logic – because we have one data point, Earth. Remember when we encountered the first interstellar object? That one encounter left us with no practical way to calculate how common such objects were. It could have been a one-off extremely unlikely event. But as soon as we encountered a second interstellar object, we had a rough idea how common they were. We had something to calculate.

You just can’t extrapolate from one data point. We may be the only life in the entire universe, or the universe might be teeming with life – both ends of the spectrum are consistent with our one known data point. We have no idea how common life is, how common intelligent life is, or technological civilizations, or how long they survive on average. None – really. So any numbers we put in are just wild guesses, and the errors on those wild guesses multiply.

Continue Reading »

No responses yet

Jun 16 2020

The Stats on Police Killings

Published by under Culture and Society

During the current national attention being paid to police practice and inequities of police killing of African Americans it is important to put the data that we have into as much context as possible, in order to understand the phenomenon and make sure that our efforts to improve the situation are properly targeted. Unfortunately, the data are complex, which makes it easy to see what one wants to see. I will try to break down the research as objectively as I can, although it is likely that perception of bias will also depend on perspective.

We can start with the most basic numbers:

Risk is highest for black men, who (at current levels of risk) face about a 1 in 1,000 chance of being killed by police over the life course. The average lifetime odds of being killed by police are about 1 in 2,000 for men and about 1 in 33,000 for women. Risk peaks between the ages of 20 y and 35 y for all groups. For young men of color, police use of force is among the leading causes of death.

This same data, from an August 2019 study, also shows that the overall risk of death at the hands of the police is about 2.5 times greater for black Americans than white Americans. This is an often cited figure, and it is salient, but there are additional layers here. Let’s break down the police use of force into non-lethal force, lethal force against armed citizens, and lethal force against unarmed citizens. Some studies focus on “shootings” rather than all uses of lethal force, mostly because that is how databases are often set up, but that can also miss important cases.

There is strong evidence that police use of non-lethal force is greater against black individuals than white. This holds up across a broad range of activities, such as drawing a weapon, using a baton, handcuffing, and using a taser. This difference is not explained by factors other than race. (The study authors controlled for other causes, and race emerged as an independent variable predicting use of force.)  This is where police education is likely to be most effective, because it does seem to be a factor of police behavior.

Continue Reading »

No responses yet

Jun 09 2020

Perhaps More Than Ever – Truth Matters

Published by under Culture and Society

The following quote from a recent address to graduating student resonated with me:

“What’s become clear is that social media can also be a tool to spread conflict, divisions, and falsehoods, to bully people and promote hate,” he said. “Too often, it shuts us off from each other instead of bringing us together, partly because it gives us the ability to select our own realities, independent of facts or science or logic or common sense. We start reading only news and opinions that reinforce our own biases. We start cancelling everything else out. We let opinion masquerade as fact, and we treat even the wildest conspiracy theories as worthy of consideration.”

The speaker advises students to, “Use all that critical thinking you’ve developed from your education to help promote the truth.” I agree, although honestly I think students need to learn much more critical thinking than is typically the case. These words could have been spoken by any skeptic or science communicator, and is a core message of the skeptical movement. We need scientific literacy, deep understanding of critical thinking and how to apply it every day, and media literacy. But these words were spoken recently by former president Barack Obama. Don’t leg your political opinion of him, if they are negative, color your perception of these words. Let them speak for themselves.

That is actually the point I want to make in this post. Humans are tribal by nature. We now know from years of psychological study that we tend to plant our flag with one group, one ideology, one narrative – and then defend it at all costs. The more we identify with a position, or see it as a marker of our group, and the more we do, the greater our motivated reasoning. For things we don’t care about, or do not identify with, we tend to revert to a fairly rational approach – listening to new evidence and incorporating into our view. So we have the capacity to be rational, when our identity does not get in the way.

Continue Reading »

No responses yet

Jun 02 2020

Journalism Without Skepticism

A recent interview published in Scientific American is a good case study in what can happen when you have journalism without skepticism.  By skepticism I mean a working knowledge of the discipline of scientific skepticism, which combines our current understanding of the philosophy of science, the nature of pseudoscience, critical thinking, mechanisms of self-deception, deliberate deception, and specific knowledge about individual pseudoscientific and paranormal topics.

The interview was conducted by John Horgan, who I have trashed in the past for criticizing skepticism while demonstrating an almost complete ignorance of it. The subject of the interview was Leslie Kean, a journalist who has written a book on UFOs and another on life after death. Doing a deep dive into these two issues is beyond this one article, and they have already been covered at length here and elsewhere. I want to focus on what the interview itself reveals.

Kean appears to take a solid journalistic approach to these issues, but there is a massive hole in her approach. She does not seem to be aware that there is already a thorough investigation into these questions, showing convincingly in my opinion that they are not genuine. She ignores it because she thinks she already understands it, when she doesn’t – so she is missing the skeptical take on these issues. She is dismissive of skeptics as deniers and as closed-minded. She then goes on to make rookie mistakes, that any well-informed skeptic could have pointed out to her. The result is a repetition of long debunked fallacious arguments, but with a patina of serious journalism.

Continue Reading »

No responses yet

« Prev - Next »