Nov 16 2015

David Katz Does Not Understand Science-Based Medicine

David Katz is a prominent proponent of integrative medicine. We in the science-based medicine (SBM) community have on several occasions over the last seven years critically addressed some of his claims, for the purpose of public intellectual discourse on topics of vital interest to the public, namely health care. When Katz has responded, he has typically done so by attacking a strawman rather than the actual SBM position.

He has now done so again, in a transparent fashion. I suspect he is responding to our criticism of him from a couple of week ago. He has now written an article in the HuffPo in which he makes specific claims about SBM that are demonstrably false. When criticizing some one or group who advocates a position with which you disagree, it is critical to be fair, even charitable, to their position. At the very least you should endeavor to properly understand the position you are criticizing. Failing to do so falls somewhere on the spectrum from intellectually dishonest to lazy. Usually the pre-existing narrative holds sway and runs roughshod over the evidence, even when in plain sight.

Katz in his recent article is responding to an article in JAMA that reviews 10 standard practices the authors believe are not supported by evidence and should be questioned. This is exactly the kind of process that we support at SBM – examining all practices from a science-based point of view. The first half of Katz’s article is also quite reasonable, and in fact is something that could easily be found on the pages of SBM.

He did not stop there, however, and used the opportunity to go on a tear against SBM and in defense of his beloved Integrative Medicine. Here is where he goes south:

At the same time, and equally important, a certain sanctimony about evidence-based medicine results in contemptuous disregard for the “unconventional.” This broad designation may, at times, refer to so-called “alternative” medicine, where detractors will suggest one is headed toward voodoo. But it also refers to lifestyle interventions that are very far from the worrisome realm of “woo.”

Katz repeats the common CAM (complementary and alternative medicine) fiction that lifestyle factors are unconventional. This is nonsense, easily debunked by simply looking at historical evidence. Scientific conventional medicine identified the relationship between specific lifestyle factors and disease risk. Lifestyle recommendations have been slowly phased into convention medicine as the evidence has come to light. Sure, this has happened slower than we would have liked – in general the medical profession is slow to adapt to the evidence. They come around eventually, but we do need to explore ways to make this process happen more quickly. In any case, CAM does not own lifestyle factors. They have simply appropriated them to have some legitimate footing to their entire endeavor.

The rest of his statement needs to be put into the context of what he also says in the article:

If evidence matters, it matters equitably, and universally.

This has been a persistent theme of David Katz – accusing critics of CAM of having a double standard. The opposite is the truth. SBM specifically calls for one universal science-based standard of care. That is our very publicly and frequently stated position.

It is the proponents of CAM who are explicitly calling for a double standard. Health care freedom laws are about creating a double standard for CAM therapies. The National Center for Complementary and Integrative Health is about creating a double standard for CAM research funding. Dr. Katz himself has called for “a more fluid concept of evidence” when considering CAM.

The fact is, when held to the light of science, CAM therapies do not hold up well. That is precisely why they are alternative. Proponents don’t acknowledge this, so they have to create the fiction that CAM therapies are being treated unfairly. Katz repeats that accusation here:

In other words, the prevailing pattern is that “we” (i.e., conventional medicine) are innocent until proven guilty, but everyone else is guilty until proven innocent. No special olfactory acuity is required to discern how bad that smells.

What Katz is referring to here is not a double standard but simply considering scientific plausibility or prior probability. Katz has directly criticized the use of scientific plausibility in evaluating medical treatments. He is wrong, but at least here he is discussing the actual issue at hand.

It is true that conventional medicine will often use a treatment because it seems plausible before there is adequate evidence to show that it actually works. That, more than anything, is the central criticism put forth by the evidence-based medicine movement. EBM specifically eliminates considerations of plausibility in order to avoid this error.

SBM recognizes the problem but does not take that approach. Rather, we argue that treatments should be based on both scientific plausibility and rigorous clinical evidence. At the very least the clinical evidence needs to be put into the context of scientific plausibility.

Another way to look at the difference between SBM and EBM is that EBM tends to follow more of a frequentist statistical approach (with its over-reliance on p-values) while we prefer more of a Bayesian approach. The latter begins with prior probability and then evaluates the effect that any new clinical evidence has on that probability. In this way SBM advocates looking at all the scientific evidence to come to one overall conclusion about the likelihood that a treatment has benefits in excess of risks.

We advocate applying this standard to all of medicine.

I honestly don’t mind that Katz disagrees with us and advocates a different approach. Let’s have it out in open discussion. That is how better ideas prevail. I do mind when he mischaracterizes what SBM does and stands for. This is what he does here:

To the best of my knowledge, a rather boisterous group in cyberspace calling itself “science based medicine” is silent on all of this. They preferentially malign all alternatives to conventional medicine, implying that problems of evidence and its application lie entirely without, and not within. This, in turn, makes it clear that such protest is itself unconcerned with the underlying evidence, and born instead of ideological zealotry. If evidence matters, it matters equitably, and universally.

The link on SBM is not to SBM but to the recent article he wrote essentially calling us fools and fanatics (to which David Gorski and I responded two weeks ago).

Prefacing his claim with “to the best of my knowledge” does not save Katz from criticism for making a blatantly untrue statement. He is saying that SBM is silent when it comes to criticism of mainstream medicine. I am not saying that Katz should be highly familiar with the thousands of articles we have published on SBM. But even a casual perusal shows this claim to be false.

He could have also plugged something like “cancer screening” into the search window on SBM. He would have been greeted with 190 articles, most of which are discussing mainstream cancer screening practice. This was one of the actual topics of the JAMA article, and ironically David Gorski has addressed the very issue on SBM.

Just for fun I looked at the most recent 60 articles on SBM. Forty of them dealt with CAM, pseudoscience, science denial, a fringe treatment, or the regulation of fringe treatments or professions. That is undoubtedly our expertise and focus at SBM. However, 10 articles dealt with criticism of mainstream practice, 3 dealt with the nature of medical evidence itself, and 6 were about other topics. That is 10 articles in the last couple of months that Katz says “to the best of my knowledge” don’t exist. Often “to the best of my knowledge” means “I never bothered to look.”

In addition to being demonstrably factually wrong, Katz draws the wrong conclusion from his confirmation bias. He thinks it is “clear” that our focus on alternative treatments is due to “ideological zealotry.” This is a common tactic of CAM proponents – they try to depict the defenders of a reasonable standard of evidence as the zealots, while those trying to sell treatments based on magic and pseudoscience are just being “open-minded.”

We are actually quite open about our editorial policy. We focus on pseudoscience, unconventional treatments, and fringe claims for very good reasons.

1 – In establishing what the standard of science and evidence in medicine should be, it is useful to shine the light on the most egregious violators.

2 – The mainstream media does a generally poor job of reporting on fringe topics, falling for false balance, citing outliers as experts, and hyping sensational claims. We are filling a gap and correcting a great deal of bad science reporting.

3 – Understanding pseudscience is a specialty unto itself that requires specified knowledge. This knowledge is generally lacking in mainstream science and academia. That is our specialty, so of course we focus on it.

4 – Mainstream medicine already has an infrastructure of experts examining and commenting on practice and evidence. Our efforts there would largely be redundant. When we feel they aren’t, we comment.

The core of SBM, however, is an examination of the nature of scientific and clinical evidence, and the relationship between that evidence and the practice of medicine. We feel we have a very solid position, and Katz has failed to criticize it in any meaningful way. Instead he and others attack predictable and rather tired straw men.

In fact Katz’s entire article is just another iteration of the tu quoque logical fallacy common in CAM circles – trying to avoid criticism of CAM practice by saying that mainstream medicine has problems of its own.

In fact this is where Katz gets it most wrong – in his musings about how to fix the shortcomings of mainstream medical practice. He writes:

The cleanup will certainly not come courtesy of those calling themselves “science-based,” who live within its glass walls, tossing stones outward. They produce nothing more useful than shards of glass.

It will come courtesy of those who concede, with suitable humility, that no single domain of influence has a monopoly on dirty boots. It will come courtesy of those who like a level playing field, and respect the potential for baby and bathwater in any given tub.

This is a massive exercise in rewriting history and missing the point.  SBM is, in fact, the solution to the problems he discusses. SBM is about being humble before the evidence. What I call “neuropsychological humility” is a major theme of scientific skepticism, of which SBM is a part.

I and my colleagues have written many articles on SBM about how we need to increase the standard of evidence across the board. There are too many published false-positive studies, there is publication bias and citation bias, exploitation of researcher degrees of freedom, a problem with open-access journals, a reluctance to publish exact replications, a need for greater transparency, perhaps a rethinking of peer-review, and an overall problem of prematurely adopting new treatments with later reversals. These are all criticisms of mainstream medicine. To claim we do not address this issue is astounding.

Even more astounding is the fact that within CAM all of these problems are exacerbated, by orders of magnitude. The problems with conventional medicine that Katz criticizes are far worse in the world of CAM. It is difficult to take him seriously about “cleaning house” in mainstream medicine while he is simultaneously trying to give a free pass to his “integrative medicine.” He decries a double standard, while trying to create one. He calls for humility, while CAM at its very core is based on the hubris that personal experience and wisdom trump scientific evidence.

Conclusion

David Katz has his narrative – he believes in ancient wisdom and natural therapies, and would like for them to be integrated into mainstream medicine. The problem with this position is that there is no particular reason to suspect that ancient practices were based in reality, and there is no reason to think that the very vaguely defined “natural” is an advantageous property to have.

Unsurprisingly, when actually subjected to rigorous clinical study, treatments that are not based on scientific plausibility but instead on romantic notions of ancient wisdom and benign nature, turn out to be largely worthless. They don’t work.

Faced with this stark reality, proponents of integrative medicine have been doing a furious dance, trying to distract from the facts. They have attacked the messenger, and applied an impressive array of distractions and logical fallacies. Katz has become quite adept at this dance.

His criticisms all miss the mark, and in some cases are factually incorrect.

We could have a meaningful discussion of the optimal relationship between basic science, clinical science, and best medical practice. First, however, he would need to dispense with his obvious straw men positions, and acknowledge his factual errors.

Comments: 7

Nov 13 2015

Homeopathy on the Ropes

The British National Health Service (NHS) is considering blacklisting homeopathy prescriptions from general practitioners. While this would have an overall small effect on the homeopathy market, it is politically potentially very significant.

The NHS currently spends about £4m on homeopathy each year, of which only £110,000 is from GP prescriptions. The rest is from homeopathic hospitals (yes, hospitals). The real market, however, is in over the counter homeopathic products.

In the UK, Europe and the US homeopathy has ballooned into a multi-billion dollar industry. It is now potentially, it seems, the victim of its own success. When it was smaller it essentially flew under the radar – regulators and politicians didn’t think it was worth spending political capital to reign in a fringe treatment that people either wanted or did not know or care about.

In the US the FDA specifically decided to let the homeopathic industry regulate itself, because it was simply too small for them to spend their resources on. That has now changed.

Continue Reading »

Comments: 22

Nov 12 2015

Nutrient Density of Crops Over Time

A new video is making the rounds on social media making the incredible claim that you would have to eat five grocery store apples today to equal the nutrient density (not calories, but other nutrients) of a grocery store apple from 1936. The video makes a second claim, that the decrease in nutrient density is due to poor soil. The video is little more that pro-organic propaganda, and neither claim is sourced – because they are not true.

However, the story of nutrient density of fruits and vegetables is a complex and interesting one. It is interesting to compare food plants from thousands of years ago (prior to any cultivation), 100 years ago, and today. First we need to ask – what are the differences? Then we can try to explain them.

Let’s compare first fruits and vegetable from today to 50 and 100 years ago.  A systematic review by Donald David found:

Recent studies of historical nutrient content data for fruits and vegetables spanning 50 to 70 years show apparent median declines of 5% to 40% or more in minerals, vitamins, and protein in groups of foods, especially in vegetables.

So there does appear to be a real decline, although not the 80% across the board suggested by the pro-organic video. The declines are significant for vegetables, and less so in fruits (so apples was a poor example to use). Averaging out all the declines, you get about a 20% decline in nutrient density for produce.

Continue Reading »

Comments: 144

Nov 10 2015

Dunning-Kruger in Groups

This research was published in February of this year, but somehow I missed it when it came out. Fortunately, the internet never forgets, and the study is making the rounds again on social media, this time getting caught in my net.

The study is an examination of how we make decisions in groups, with the specific question of how we weight the opinions of different members of the group. The researchers studied pairs of subjects (which they call dyads) who were given a specific task, such as identifying a target in a photo they are able to briefly glimpse. Each member of the dyad registers their choice. If they disagree, then one member is chosen at random to be the arbiter. The question is – will the arbiter favor their own opinion, or that of their partner?

For each pair 256 trials were run, and after each one they were given feedback as to who was correct. The idea is that each member of the pair would learn who was performing better. In one experiment they were given a running tally, to make sure they knew who was performing better. In another the task was made more difficult for one partner, increasing the difference in their performance, and in a final experiment the pair was given a financial incentive to perform better.

Continue Reading »

Comments: 16

Nov 09 2015

History Validates Initial Skepticism

I understand that fervent skepticism can sometimes be annoying, especially when it is aimed at something you believe or at least think is either likely or should not be ruled out. It’s as if the skeptics are trying to disprove your belief.

Well, they are. That’s the point. Welcome to science.

Initial skepticism is a productive response to any new claim, and history bears that out. Often observations or even experimentation leads to the possibility of new phenomenon. Almost by definition this is based on a currently unexplained anomaly.

An anomaly is a phenomenon that does not fit with our current understanding of the universe. If we are encountering an entirely new phenomenon it is likely that what we are observing will be anomalous, because we can’t explain what we don’t know. In fact scientists love anomalies – they point toward new discoveries. Anomalies are where the action is.

Continue Reading »

Comments: 40

Nov 06 2015

Are People with Autism Psychic?

As a general rule of thumb, if a headline contains a question the answer is usually “no.” This headline is no exception.

Thanks to the low journalistic standards at the Daily Mail, an ESP proponent by the name of Diane Powell is getting another round of media attention for her claim that children with autism can have telepathic powers, which she feels is an evolutionary compensation for their otherwise limited ability to communicate.

The story revolved around a 5-year-old boy named Ramses Sanquino, whose mother believes is a savant as well as telepathic. I tend to be skeptical of specific savant claims, even though they are well documented in some cases. I don’t doubt the phenomenon, but I have also seen cases in which overzealous parents manufacture the perception of extraordinary abilities in their children.

I was even involved in a case in which a mother believed her young child, who was non-verbal, could read in two languages on a 10th grade level. She came by this determination through facilitated communication, which is not a legitimate technique. This was a clear case of projecting her hopes onto her mute and passive child.

Continue Reading »

Comments: 195

Nov 05 2015

The Ben Carson Contradiction

By all accounts, Ben Carson is a brilliant pediatric neurosurgeon. He was the Director of Pediatric Neurosurgery at Johns Hopkins Hospital until he retired, and received the Presidential Medal of Freedom for his surgical achievements.

Carson’s views have come under close scrutiny since he has become a presidential candidate and is closing in on the frontrunner position. Carson is a Seventh Day Adventist. He is a creationist who has stated that he believes Darwin came up with the idea of evolution because of Satan. He thinks the Big Bang is a “fairy tale.”

He famously suggested that those who believe in evolution have no basis for their morality, saying:

“Ultimately, if you accept the evolutionary theory, you dismiss ethics, you don’t have to abide by a set of moral codes, you determine your own conscience based on your own desires.”

This claim is transparently wrong, and discounts a vast and rich philosophical history of morality and ethics.

Continue Reading »

Comments: 77

Nov 03 2015

David Katz Tilts at Straw Men

How one responds to legitimate criticism is a very good indication of their intellectual fortitude and integrity. I pay specific attention to whether or not they address the actual criticism, rather than attack a convenient straw man, and whether or not they acknowledge fair points on the other side. Intellectual discourse, which often contains pointed criticism, is critically important. It is how we work out big ideas and move forward.

In a recent blog post on the HuffPo, David Katz launches into a fallacy-ridden attack on Science-Based Medicine, managing to entirely mischaracterize our position, despite the fact that our position has been exhaustively discussed on our blog and elsewhere. His post, Science and Medicine, Fools and Fanatics: The ‘Fluidity’ of Woo, contains the usual alternative medicine tropes draped in protests of his academic credentials. David Gorski has already responded over at SBM, and I would like to add to his analysis here.

David Katz first defends his infamous statement advocating that medicine should use a “more fluid concept of evidence.” He writes:

Colleagues and I proposed, based on years of wrestling with complex patients, many of whom, urgent medical needs still insufficiently addressed, had tried and exhausted all of the well-supported, conventional treatments, that evidence traversed 5 key considerations. Those include: what is known about a treatment’s safety; what is known about a treatment’s efficacy; how well those first two are known (i.e., the clarity of evidence); the patient’s preferences; and, importantly, the availability of other, untried treatments for the condition in question.

He argues that when science-based alternatives are exhausted, a caring clinician should consider treatments with a more “fluid” standard of evidence. Since he apparently did not understand our original criticism, I will spell it out carefully here.

Continue Reading »

Comments: 68

Nov 02 2015

Nerve Stimulation for Relaxation

A new device called Thync has been advertising quite extensively and this has prompted a number of questions to me about its legitimacy. The device is worn over the right temple and above the right eye and provides timed pulses of mild electrical stimulation. Their website describes the effects:

A soothing neck massage. A splash of cold water. A kiss from someone you love. These are common examples of how nerves signal the brain to change the way you feel. Thync works in the same way using signaling programs we call Vibes.

The device appears to have two settings, one with calming vibes and the other with energizing vibes. Let’s take a look at the state of the science

Plausibility

The idea of using electrical stimulation, of either nerves, the skin, or of muscles, in order to affect nervous system function is reasonable. The nervous system is both a chemical and electrical organ system and its function can be modified with electrical stimulation.

Continue Reading »

Comments: 4

Oct 30 2015

TCM on NPR

Journalists frustrate me. Whenever they cover a topic about which I have a fair degree of knowledge, or even expertise, they seem to do a generally poor job. There are excellent journalists out there, but the average mediocre journalist tends to fall for the fallacy of false balance, indulge in hype and sensationalism, overly rely on individual experts who may have quirky opinions, and often fail to put topics into a proper context.

These failings are exacerbated whenever the topic at issue requires critical thinking and a high degree of skepticism.

Even generally high quality news outlets, like NPR, tend to fail when they deal with topics which require both expertise and skepticism, such as alternative medicine. A recent episode of Marketplace with Colin McEnroe is an excellent example of how a generally reasonable journalist can completely fail when dealing with such topics.

Continue Reading »

Comments: 32

« Prev - Next »