Sep 13 2012

Who’s To Blame for Bad Science News Reporting

The work of a science blogger is largely comprised of correcting and criticizing bad science news reporting. I therefore had a love-hate relationship with horrible science reporting – I simultaneously am annoyed and disgusted at the spread of misinformation by professionals who should know better, but delighted by the excellent blog fodder. In fact I see the role of the science blogger as largely filling the gap between scientists and journalists. We tend to be journalists with a strong science background, or scientists who have developed their skill at writing and communicating to the public.

We spend a lot of time lambasting journalists for not doing their job, but in reality there are often three stages and parties involved in the creation of science news: the scientists themselves and the writing in the original paper, the press officers and the press releases they write, and then the journalists and their news articles. In each step of this process their is the potential to distort and exaggerate the findings of the study.

A new study published in PLOS Medicine examines the incidence of “spin” in medical news reporting (exaggerating the clinical effects or implications seen in the study being reported) and the factors that contribute to that spin. What they found is that the single biggest factor predicting the presence of spin in health news reporting is the presence of spin in the abstract of the article written by the scientists themselves.  The authors conclude:

Our results highlight a tendency for press releases and the associated media coverage of RCTs to place emphasis on the beneficial effects of experimental treatments. This tendency is probably related to the presence of “spin” in conclusions of the scientific article’s abstract. This tendency, in conjunction with other well-known biases such as publication bias, selective reporting of outcomes, and lack of external validity, may be responsible for an important gap between the public perception of the beneficial effect and the real effect of the treatment studied.

We appear to have a synergy in positive biases – researchers are biased toward finding positive results, then emphasize their positive results in their article, journal editors are more likely to publish positive results and then send out press releases touting positive results, and finally reporters like to emphasize and sensationalize positive results. The end result of all of this positive bias is creating the false impression in the public – and even among medical professionals – that a treatment is effective, even when it may not be. (See, for example, my recent analysis of an acupuncture meta-analysis at Science-Based Medicine.)

The following chart tells the story of how much spin the researchers found when reviewing science press releases and reporting:

I am not surprised that this chain reaction of sensationalizing study results begins with the researchers themselves. Scientists have a responsibility not to overcall their own results when interpreting their data – but we know there is a tendency to do so. The process of peer-review is supposed to be a check on this tendency, and it is, but it’s imperfect and “spin” often slips through the cracks.

Scientists also need to spend more time and energy learning how to communicate with the media and the public about their results. Even when they are reserved in their official conclusions, they often oversell their results when speaking to the media, or allow press officers or journalists to goad them into making sensational statements about their research. Further, when a press officer adds spin, distorts or exaggerates the research findings, the scientists should make an effort to correct this.

In other words – scientists need to take a certain amount of responsibility for how their own research is presented to the public at every step of the process.

All of this does not let the press officers and science journalists off the hook. It is the job of the press officer to garner attention for the research published in a particular journal or by scientists in their institution, but they still have a responsibility to get the bottom line correct. Science journalists, however, have a responsibility to tell the real story, to talk to scientists who were not part of the study and get their opinion, to look for spin and point it out or correct it, and put the study results into some context.

What we often see, however, is journalists who republish press releases with little or no change, or simply taking the press release at face value with no investigative journalism at all. In short – they often do not do their jobs.

Obviously there are good science journalists out there who are doing their job, but they appear to be in the minority. The result is a massive bias toward sensationalized positive results. When reporting on medical findings this has immediate practical implications and people may often make medical decisions or seek specific treatments based on this reporting.

Conclusion

Bad science news reporting is a serious problem, as this recent study points out. This data also suggests that scientists themselves do have a significant influence over the ultimate reporting of their research, starting with how they interpret their own findings in the technical paper. Often it seems that scientists are not aware that they are writing to a broader public. Most of the time research papers will be read by relatively few people in a narrow field of research, and scientists are surprised and unprepared when their study gets picked up by the mainstream media.

An obvious solution is to give scientists more training in how to write conclusions that are responsible and do not lend themselves to hype, but also in how to deal with the media. There are efforts to do just that, but clearly we have a long way to go.

Share

6 responses so far

6 Responses to “Who’s To Blame for Bad Science News Reporting”

  1. MWSlettenon 13 Sep 2012 at 9:57 am

    Interesting (and often funny) discussion of the way neuroscience is often misrepresented by the media.

    http://www.newstatesman.com/culture/books/2012/09/your-brain-pseudoscience

  2. Oracon 13 Sep 2012 at 10:56 am

    It often begins with the press release:

    http://www.sciencebasedmedicine.org/index.php/related-by-coincidence-only-journal-press-releases-versus-journal-articles/

  3. Oracon 13 Sep 2012 at 10:58 am

    Speaking of good science journalists, sadly we’ve lost one of the best. Trine Tsouderos recently announced that she’s leaving the Chicago Tribune to take a job that doesn’t involve science journalism. It’s a huge loss.

  4. SARAon 13 Sep 2012 at 3:19 pm

    Human Nature being what it is, I don’t suppose we will ever completely eradicate bad science reporting.

    I saw an article a while ago about a drug that might eliminate baldness. I clicked on the link to the study and found that it was being developed for some disease (sorry I don’t remember which) but it had a curious side effect of growing more hair in a certain percentage of the patients.

    The side effect was not prominently listed in the abstract. The abstract was about the drug’s effectiveness with the disease. But the article left it’s intended purpose to one line at the end of the article.

    Science Journalists also look for the hook. A cure for baldness is a hook.

  5. pseudonymoniaeon 13 Sep 2012 at 10:23 pm

    A point you might want to include Dr Novella is that there tend to be rather perversE incentives at each and every level of the process to obfuscate, prevaricate and ultimately fabricate simpler and ostensibly more interesting explanations for the results of many scientific findings. Clearly the incentives to correct the explanatory failings of others also exist, by generally only for individuals who are more talented, conscientious or both (at transmiting knowledge) than the average member of these chains of misinformation.

  6. ccbowerson 14 Sep 2012 at 2:41 pm

    “I simultaneously am annoyed and disgusted at the spread of misinformation by professionals who should know better, but delighted by the excellent blog fodder.”

    I see this as problematic in that it increases the gap of knowledge amongst the general public even further. The majority of the general public are passive learners of science and will disporportionately hear more misinformation as general science reporting becomes more sensationalized. Bloggers, on the other hand tend to reach people who seek out information (therefore less likely to be passive learners), and this group will also be more knowledgeable on average.

    If these apparent trends continue, it will result in the most knowledgeable becoming even more so (and more sophisticated about the bad reporting), and the least knowledgeable becoming more confused (confused at best, misinformed at worst). I don’t see blogging as an antidote to the problem until their visibility approaches that of mainstream media. Right now it is largely a self selected bunch being exposed to the good stuff.

Trackback URI | Comments RSS

Leave a Reply

You must be logged in to post a comment.