May 02 2014

Framing Bias

Let’s say you need a surgical procedure and the surgeon tells you there is a 98% survival rate with the procedure. How would you feel about that? What if she told you there was a 2% mortality rate? Would you feel the same way? Probably not, according to years of psychological research.

This is known as framing bias, just one more of the many ways in which our brains are biased in the way we evaluate information. The two scenarios above are identical, but statistically people will make different decisions based upon how the information is framed. We generally respond better to positively framed information (98% survival) than to negatively framed information (2% mortality).

The framing effect is often exploited by those who are deliberately trying to manipulate our reactions. Politicians, for example, can talk about employment rates or unemployment rates. Events can give you an early-bird discount or a late registration penalty. Products can have 4% fat or be 96% fat free.

Framing is another way in which we construct our picture of realty, by deciding what information is important.

A recent example of a framing study (abstract here), which prompted this post, looked at decision about what kind of person subjects would be likely to date. They compared men and women on various features – kindness, earning potential, attractiveness, ambition, and intelligence. They presented information about potential partners either with a positive or negative frame – “7/10 people who know this person think this person is kind,” vs “3/10 people who know this person think this person is not kind.”

Not surprisingly, the subjects responded better to the positively than negatively framed information. However, I have a serious problem with this study. In order to isolate a framing effect, the positively and negatively framed information has to be identical (98% survival is exactly the same thing as 2% mortality). The type of framing used in this study is not identical.

For example, if  30% of people who know John think John is not kind, that does not necessarily mean that 70% think he is kind. It’s possible that some or all of the other 70% have no particular opinion about John’s level of kindness.

In other words, the personality and physical characteristics used in this study are not strictly binary, like being alive or dead. For all of them there is a middle ground. By saying that John is not kind, that could be interpreted as John being a jerk to a sufficient degree to characterize him as not kind, and to communicate that fact to others. I certainly can think of many people that I would not characterize as either particularly kind or unkind.

Perhaps the study could be repeated presenting survey results in a Likert scale fashion (1- Strongly disagree, 2- Disagree, 3- Neither agree nor disagree, 4- Agree, 5- Strongly agree). This way the results could be framed with mirror image results and be actually identical.

Despite the shortcomings of this one study, framing is a robust psychological phenomenon that has been documented many times. It is a very common bias and it’s useful to be aware of it.

Whenever information is presented to you, it’s a good habit to remove the framing from the information and restate the results from as many different perspectives as possible, being sure to include all possible relevant information. That’s a good way to dissect what is really going on, make more meaningful comparisons, and protect yourself not only from bias, but from deliberate manipulation by others.

This process would also force you to think about what the other alternatives are – for example, does everyone who doesn’t think John is not kind think he is kind? Surveys in general fall prey to this sort of bias or hidden assumptions. It’s a good idea to be skeptical of surveys, how the questions were framed, and what the responses actually mean. Surveys are very tricky things, and most of the ones I read have some sort of implicit assumption or bias hiding in the questions.

Add framing bias to the list of ways in which your thinking is flawed, and like most biases you can compensate for it with a little metacognition.

Share

8 responses so far

8 Responses to “Framing Bias”

  1. Jerry in Coloradoon 02 May 2014 at 10:18 am

    When we were in Indonesia years ago, we stopped at a mountaintop stall to get some refreshments. They had cold cans of Coke for 5000 rupiah (about 40 cents). I overheard a fellow traveler comment ” I would not pay 5000 anything for a can of Coke.” Clearly framing bias, I would say.

  2. steven johnsonon 02 May 2014 at 11:14 am

    Have there been very many studies confirming the effectiveness of metacognition?

    Are the methods for compensating cognitive biases specific to the type of bias? For example, is the best way to compensate for framing bias the practice of routinely reframing all controversial questions?

    Or is it equally effective to, more or less, run down the list of cognitive biases, asking if any might be a problem in a given problem?

  3. SteveAon 02 May 2014 at 12:11 pm

    “For example, if 30% of people who know John think John is not kind, that does not necessarily mean that 70% think he is kind. It’s possible that some or all of the other 70% have no particular opinion about John’s level of kindness.”

    It doesn’t necessarily mean it, but if I was a participant in the study I’d assume that all the people questioned were in a position to have a view on John’s kindness one way or another and that it was a yes/no proposition.

    Not to say that it’s good reasoning, but I think it’s the conclusion most people would jump to.

  4. Steven Novellaon 02 May 2014 at 3:04 pm

    Steve – I disagree. That is precisely why the Likert scale is a standard with this sort of study/survey. We can’t say if “most” people would assume not unkind is the same as kind. Because many surveys are not binary forced choice, people might assume there is a middle option.

    In any case, the uncertainty (even with “most”) affects the results, and we can’t ascribe it purely to framing bias.

  5. Bruceon 02 May 2014 at 3:18 pm

    This is a topic that has fascinated me since university and the part about how the question is asked can sometimes reveal more about what the surveyor is trying to find out than they sometimes think.

    I would be very wary of over-analysing survey data though, in trying to look at data from too many perspectives you might end up unconsciously (cherry) picking up on an outlier that is not actually significant.

    I suspect, though, that I am telling grandma to suck eggs in this regard.

    In any case, great post, I love all aspect of human surveys and the potential pitfalls from writing of the survey questions to interpreting the results.

  6. practiCal fMRIon 03 May 2014 at 12:01 am

    Steven, would you mind if I pasted a link to your post on PubPeer?

    https://pubpeer.com/publications/D8A7AD41F1D5D27E6F233F89F799ED

    Post-pub peer review via blogs is an important part of the new world of science.

  7. Terry Hewitton 06 May 2014 at 12:34 am

    One of my favorite examples of framing:

    First way to frame the question:

    “Father, is it OK to smoke while I pray?”

    Answer:

    “My dear son, when you pray you should devote all of your attention to God. Smoking while praying would be disrespectful. Surely you can wait until after you have prayed before having a cigarette.”

    Second way to frame the question:

    “Father, is it OK to pray while I am smoking?”

    Answer

    “Of course you can my child, you should pray as often as you can, no matter what you are doing”

  8. Maximilianon 10 May 2014 at 5:21 pm

    As I was reading the study in this post, before Steven Novella said “I have a serious problem with this study” I was thinking the same thing. The framed information is definitely not identical. If I were in this study I would have assumed that the other 3/10 people just didn’t think that the person (John) was particularly kind. Not kind enough to mention, but NOT that he was unkind. I would definitely have thought of the middle option.

Trackback URI | Comments RSS

Leave a Reply

You must be logged in to post a comment.