Jun 10 2013

Science Journalism

I recently got into a small kerfuffle with a journalist, actually a sports writer who decided to dabble in science journalism. The exchange started at science-based medicine when I wrote a piece critical of the claims being made for a new device called the GyroStim, which is being offered as a treatment for brain injury.

In this article I linked to a piece in the popular press about the treatment, in the Denver Post by a sports writer, Adrian Dater. Dater thought I was being unfair in my criticism of his piece, and so wrote a response on his blog.  The exchange and the comments have exposed many of the problems with journalism in general and science journalism in particular, that I would like to explore further here.

First I have to say that there are many excellent journalists and science journalists out there. I am not implying that that there are no good journalists. I do find, however, that the baseline quality of science journalism is lacking and, if anything, getting worse. Part of the problem is the evaporating infrastructure for full-time journalists. Many outlets no longer maintain specialist journalists, and use generalists (including editors) to cover science news stories.

What follows can be seen as a quick primer, or at least a list of helpful suggestions, to journalists who wish to cover science topics. I will primarily use examples from the recent exchange over the GyroStim.

Think About the Narrative

Almost all news stories have a clear narrative. The facts of the story are presented in a way to create a meaningful story. Even if all the facts are individually correct, the choice of which facts to present, in what balance, and in which order affect the bottom line impression left by the article. The choice of headline is also important, and I know for big news outlets the article author is often not the headline writer, but that doesn’t mean the headlines don’t matter also – they tend to frame the article.

For example, Dater defended his article by claiming that he got all the facts right, that he included “balance” (more on that below) by indicating the Gyrostim is not FDA approved and quoting a doctor saying more evidence is needed, and that he did not directly endorse the treatment.

My criticism, however, was based on the narrative that he blatantly created. I find it interesting that he seems to be unaware of this narrative or its effects.

The story was framed as a touching account of a father who is an engineer who decided to build a machine to cure his daughter of cerebral palsy. Right out of the gate the reader is rooting for this machine to work. The pull-out quotes include, “Spinning stimulates the brain,” and “Miracles almost every day.” He also ended with a hopeful anecdote:

“The machine is amazing, it really is,” said Hishon, who played nine games for the Lake Erie Monsters this season after nearly two years of concussion symptoms. “It just seemed to wake something up in my brain. I can’t explain it, but it definitely worked wonders with me.”

The fact that there is some token skepticism tucked in the middle is just part of the narrative – every story needs a villain, right?  Dater may not have intended the skeptics to be the villain of his narrative, but that is the role he assigned them. On the one hand you have a loving father, hope, the device is being researched, you have excited practitioners, and many patients who are thrilled with their results, and on the other side some talking-head canned skepticism – “more evidence is needed, not FDA approved, blah, blah.”

It seems to me that many journalists don’t even think about the narrative – it just emerges as a default story format. Start with a human interest angle to draw in the reader, then just report what both sides are saying, be sure to include plenty of anecdotes, and then end on a hopeful note.

What many non-science journalists don’t seem to get is that this is not a proper narrative for a science story. Further – all journalists need to decide what their narrative is before writing the story (although hopefully after they have researched it – often journalists decide on their narrative first then just backfill the facts and anecdotes).

Here are some other narratives that journalists covering science stories might consider:

- The allure and harm of false hope, and the exploitation of false hope by dubious practitioners and companies.

- A cautionary tale about getting excited prematurely by some newfangled treatment before it is adequately tested, given that most new treatments do not pan out.

- Is new and high tech always better – does this machine that costs tens of thousands of dollars work better than a $20 device that has a similar function?

Put the Story Into Context

The most challenging part of science reporting is putting a new story into a deeper scientific context. This requires background research and talking to a variety of experts – and asking the right questions and really listening to what they say. This context includes:

- What is the plausibility of the new claim? Does it confirm or contradict what is currently believed to be true?

- Does the new device, treatment, product resemble anything that has come before? Is it truly new, or just a rebranding of an old concept – and if the latter, how have previous incarnations fared?

- What is the current consensus, if any, on this new claim? Is it truly controversial, or very one-sided with the majority of scientists taking one position and only a few outliers disagreeing with the consensus?

- What are the credentials and backgrounds of the experts on which you are relying. Is their degree generally recognized as valid? Do they have a history of making other dubious or controversial claims? Do they have a history of fraud?

- Overall, how does the new discovery, claim, treatment, etc. fit into existing evidence and scientific theories?

- What are the implications of all this for the current stance one should have toward the claim – should it be considered experimental, should it be taught in public science classes, should it be legal, etc. ?

- What steps are needed in the future? What questions need to be resolved?

Adding the above context is exactly what we do at Science-Based Medicine, and at many other “skeptical” blogs. Good science journalists also do this. This is the real story, not the fluff narrative that is better suited to covering the local dog show.

False Balance and Token Skepticism

The balance of the article should generally reflect that balance of scientific acceptance.  If 95% of the scientific community accepts one consensus, then that is what the bulk of the article should reflect. If you feel the other 5% deserves a mention, then it should be given appropriate space, and also put into context (as above).

Stories about politics and social issues require obsessive balance, because these are mostly based on value-judgments and opinions. For these stories a journalist needs to get the facts right, and make sure that all credible sides have their say.

Science does not work like that. In science, some opinions are objectively better than others. Science stories are about the evidence and the process of science – about finding the best current answer. Science articles need to reflect that.

As soon as you put a pseudoscientist up against a genuine and respected scientist, you have elevated the pseudoscientist to a stature they likely do not deserve. You have framed the story in a very deceptive way that does not reflect the reality.

Bad science journalism generally falls into one of three categories in this regard. Some stories have false balance, where a pseudocontroversy is presented as if it is a real scientific controversy. This is  the false-balance fallacy.

Other stories have what we call token skepticism – most of the article is dedicated to giving a forum to the crank and glowing anecdotes, with scant mention of doubt and/or quick commentary by a real scientist. Dater’s article fell into this category.

The third is when even token skepticism is lacking – the story is presented without a hint of skepticism or actual investigation.

Conclusion

Good science journalism requires putting a science news story into a proper context, making sure the narrative that emerges is fair and appropriate to the actual story, and properly balancing different points of view to the scientific consensus and to scientific legitimacy.

This is not easy. It requires, in my opinion, at least a baseline of scientific literacy. It also requires significant background research into the topic, and into any experts upon which the journalist relies.

What we have from Adrian Dater is an excellent example of what happens when a non-science journalist thinks they can dabble in science reporting, without understanding any of the special requirements of competent science reporting. Even more telling than the article itself is Dater’s defense of his journalism. As is often the case with defensive overreaction, he just dug himself in deeper and deeper.

The exchange also highlights for me the new role that science blogs are playing in the reporting of science news. Journalists who write bad science news stories now have to contend with the second wave of science blog analysis. Now actual scientists, or at least dedicated science journalists, can add the missing context, deconstruct a misleading narrative, and rebalance a science news story.

Journalists, like Dater, who encounter this science-blog pushback when they write a naive and misleading piece would be better off if they embrace the criticism and try to learn from it, rather than get into an online fight with someone who actually knows what they are talking about.

Share

13 responses so far

13 Responses to “Science Journalism”

  1. Michael Finfer, MDon 10 Jun 2013 at 8:41 am

    An article in the Columbia Journalism Review recently addressed the issue of false balance and false controversies, in this case, vaccines: http://www.cjr.org/feature/sticking_with_the_truth.php

  2. locutusbrgon 10 Jun 2013 at 9:05 am

    Your Just a Doctor not a journalist. What do you know about writing news stories, generating interest or making a deadline. You are just some nit picking skeptic who cannot see the possibilities of some such something or other. Besides I whats’s the harm. You are just mean spirited kill joy.

    Didn’t read his response but I am guessing my rant is not far off.

  3. oldmanjenkinson 10 Jun 2013 at 9:41 am

    Adrian Dater appears to not get that he is susceptible to faulty logic, or it could be the cognitive dissonance causing a reaction formation to negate the dissonance. Acceptance that we can be deceived or that we are all susceptible to magical thinking is difficult for some. When one becomes credulous to the possibility of our shortcomings Mr. Dater displays the common responses; denial, minimizing, reaction formation, rationalizing, intellectualizing, externalizing. No one is immune to this type of thinking. It is analogous to my asthma and my periodic asthma attacks. I know it is there, I know my triggers, I recognize the signs/symptoms that it is happening and know what intervention(s) to utilize to bring the attack under control.

    As a skeptic in training the best I can hope for is recognition of my susceptibility to this type of magical thinking. The ways I combat it and would recommend Mr. Dater to as well is “arm” himself with knowledge and recognize our brains are programmed to think in such a way. Here is just a smattering of the books I have read which have helped me to combat my sometimes illogical, deception engine of a brain (in no particular order); Thinking Fast and Slow (Daniel Kahneman), Innumeracy (John Allen Paulos), The Invisible Gorilla: How our Intuitions Deceive Us (Christopher Chabris & Daniel Simons), Supersense (Bruce M. Hood), The Power of Persuasion: How we are Bought and Sold (Robert Levine) and my ultimate recommendation Unnatural Acts: Critical Thinking, Skepticism, and Science Exposed! (Robert Carroll).

    I believe we best serve our patients as well as the public when we realize and accept our shortcomings. Two books I am reading (one done, the second I have just started) speak directly to the shortcomings of science journalism; Science Left Behind (Alex B. Berezow and Hank Campbell) and Eco-Fads (Todd Myers) speak to whether you identify yourself as Left, Right or a Centrist science journalism leaves something to be desired.

  4. Keaneon 10 Jun 2013 at 10:07 am

    The saddest thing about this whole discussion is that Dater couldn’t see how wrong he was. He was so hurt that he couldn’t even read what you were actually saying. That, or he’s partially illiterate. I hope he’ll have someone patiently explain it to him in person.

  5. champenoiseon 10 Jun 2013 at 10:45 am

    Dater’s piece reads like an advertisement.

  6. ccbowerson 10 Jun 2013 at 10:54 am

    “Even more telling that the article itself is Dater’s defense of his journalism. As is often the case with defensive overreaction, he just dug himself in deeper and deeper.”

    I wonder how much of the criticism wasn’t understood due to the defensiveness versus a lack of appreciation of how scientific journalism is different than sports journalism often is.

    I wonder if showing uncritical stories written by some sports journalist years ago about the Powerbalance wrist bands would help illuminate the problem here. How about that fake bomb detector, and any uncritical stories that were done years ago about that?

    Both of these stories have had time to develop, and demonstrate how unproven claims are best dealt with a critical and skeptical aproach. Its not about cyncism, its about have a rigorous process for evaluating claims regardless of the narrative we would like to read or write. If after this rigorous process the product has been shown to work, then we can rejoice and write our narrative. Doing so prematurely does everyone a disservice, except for the promoters of the product

  7. Hosson 10 Jun 2013 at 2:07 pm

    http://www.denverpost.com/avalanche/ci_23067575/joey-hishon-improving-after-latest-concussion-avalanche

    This article appears to be the first written by Adrian Dater about GyroStim. The initial article is little more than a reporting of a claim from functional neurologist(chiropractor). I suspect Adrian was probably contacted by numerous professional athletes interested in GyroStim after its initial publication, which is probably one of the main factors that contributed to the decision to his follow up article on the machine.

    “The news is encouraging with former first-round Avalanche draft pick Joey Hishon, who suffered a second concussion last week.
    Hishon skated Thursday with the Lake Erie Monsters of the American Hockey League, and one of the functional neurologists tending to him said the outlook is positive.
    “He’s feeling much better than last week,” said Daniel Gallucci, an osteopath with the Carrick Institute in Marietta, Ga. “I think the stress and anxiety produced by the latest one was originally a concern, but his symptoms have improved.”
    Gallucci said Hishon will receive treatment at the Carrick Institute next week, likely receiving treatment in the same GyroStim spinning chair that helped him recover from a first concussion that sidelined him nearly two years.”

    I was curious how Adrian’s investigation of GyroStim lead him to the Carrick Institute. But after discovering the correlation with the first article, it makes sense.

    It appears Adrian was initially convinced by the anecdotes of professional athletes for GyroStim, but during his investigation found no evidence for the claims. This and a lack of scientific understanding(I’m assuming, but I think the content of the article and comments thereafter speaks well to his scientific illiteracy) helps explains the content of most of his article – heavy on the speculation and anecdotes, and little on science and skepticism.

    Thank you Steve for another great post.

  8. steve12on 10 Jun 2013 at 3:39 pm

    “I’ve also gotten a few “get off my lawn” emails and overall criticism of the story from people, most seemingly in the more “established” medical community.”

    I love the sneering dismissal of expertise that has, for reasons I don’t understand, become typical in hyper- masculine worlds like sports writing. Very Idiocracy. I find this conflation of stupidity and masculinity bizarre.

  9. SimonWon 10 Jun 2013 at 8:10 pm

    Is the popularity of chiropractic in the sport’s world perhaps also a root cause for Dater’s confusion?

    Probably hockey is a good choice if you want to bilk people with dodgy therapies, lots of fit young men who’ll mostly get better quickly pretty much whatever you do to them, and lots of injuries.

    I guess when we see “chiropractor” we know to be suspicious, but perhaps Dater simply is ignorant of such, and seemed to think you’d be in danger of slander. He is either very naive, or possibly worse.

    Perhaps when he reads pages like this

    http://www.nhs.uk/conditions/chiropractic/Pages/Introduction.aspx

    from esteemed health organizations the phrases:

    “Some uses of chiropractic treatments are based on ideas and an evidence base that are not recognised by the majority of independent scientists.”

    “There is some, mostly poor quality, evidence that that spinal manipulation is an effective treatment for some other musculoskeletal conditions involving the bones, joints and soft tissue. The evidence on spinal manipulation is not strong enough in these cases to form the basis of a recommendation to use the treatment.

    There is no evidence that treatments offered by chiropractors are effective for other conditions.

    There is also no scientific evidence to support the idea that most illness is caused by misalignment of the spine.”

    Don’t leap out enough. I suspect the NHS needs to practice writing for a popular audience, and simply put “not worth trying”.

  10. norrisLon 11 Jun 2013 at 6:55 am

    Some years ago (about 15 years ago) there was a medical news story on the front page of the Courier Mail (Brisbane, Australia) which stated that paracetomol (known in the US as acetominophen) caused asthma. Do you get the idea already that this article was not written by a genuine science journalist? I had an interest in this drug at that time as my son was going through teething and my wife was giving him paracetomol. So I phoned my brother, who is a GP, for an opinion on this article. It turned out that my brother had The Lancet article on his desk as I phoned.So the actual, science based story was that people who already had asthma and who took a lot of paracetomol may have a worsening of their asthma. This was a very different story from the newspaper front page story.

  11. Bill Openthalton 11 Jun 2013 at 10:32 am

    I think it was Alfred Lord Northcliffe who said “it is the business of a journalist to explain to others what he himself doesn’t understand.”

    I have blind faith in what I read in my newspaper, unless it is about something in my area of expertise, in which case it is bollocks.

  12. PharmD28on 13 Jun 2013 at 11:00 am

    strange how the guy could so fail to see how this is very poor journalism…

    “evidence” should matter to journalists…especially one reporting on science related topics….

    They need to understand the critical difference between anecdote and research.

    My experience in talking with my patients both in primary care, and in retail pharmacy, this basic distinction is SEVERELY blurred…and so when trying to make this distinction, many times the “nasty skeptic” or perception of “cynicism” rather than “proper skepticism” is concluded.

    Perhaps based on these observations we need some better science teaching in high school….all high school students could benefit from understanding this distinction I would think.

  13. eli_damonon 01 Jul 2013 at 6:49 pm

    A good article on an important subject. I think the problems you pointed out apply more broadly than you think. Bias, sloppy narratives, false balance, and lack of critical inquiry or investigation is a problem throughout journalism, not just in stories that are considered to be proper science stories. In fact, I would say that science is an element in all news stories, even ones that don’t directly relate to an established scientific field. In particular, intellectual rigor is terribly lacking in law and politics, and this rigor is desperately needed even though law and politics is not considered to be a scientific field.

    By the way, it would be very helpful to have a general guide for evaluating a scientific study. How to spot common flaws and such. Do you know of any such guide?

Trackback URI | Comments RSS

Leave a Reply

You must be logged in to post a comment.