Jan 09 2008

Pre-clinical vs Clinical Evidence

My blog entry today for Science-Based Medicine reviews a new study claiming to provide evidence for the healing power of magnets. I want to pick up on a theme I just touched upon in that article and extend it a bit – the relationship between pre-clinical and clinical evidence and their proper use in making and marketing health claims.

One of the biggest problems with the current supplement industry is that there is a significant disconnect between the scientific evidence and the claims made for specific products. Under the Dietary Supplement Health and Education Act of 1994 (DSHEA), in the US herbs, supplements, and other health products can make health claims in their marketing and literature without FDA approval or oversight, as long as those claims do not involve the treatment of a specific disease. A new category of health claims was invented for DSHEA – so-called structure/function claims: sellers can claim that a product supports or improves a biological function or structure. Therefore companies may claim that St. John’s Wort improves mood (a function), but they cannot say it treats depression (a disease).

There is always a political dimension to the question of regulations. Libertarians (even those who are solidly skeptical and scientific) may argue that the free market should sort it out and consumers should be free to choose for themselves (let the buyer beware). Nanny-state liberals might argue that regulations should be airtight at every level (from manufacturer, to provider, to consumer), and every abuse strictly prosecuted. And of course there is every position in between these extremes.

It is not my intent to take any particular political position (although I will disclose that my position is moderate – somewhere between the extremes I described) – but rather to argue that in any case regulation should be based upon solid science. It should also be the goal to insure (by whatever means are taken) that health products are safe and that the claims made for them are not fraudulent and are based upon appropriate science, or at least the public has sufficient information to make an informed decision.

As it stands now, under DSHEA, the scientific evidence for the use of supplements, herbal medicines, and health devices such as magnets is largely abused and misrepresented to the public. The most insidious aspect of this is the misuse of pre-clinical evidence to make clinical claims. This is the most damaging because it creates the impression that there is solid science behind the health claims that are being made for a product when there isn’t, and yet the details of the claims may be accurate.

Clinical evidence comes from experimental trials in which human subjects are given a specific treatment and then the outcomes are measured, including both benefits and side effects. The gold standard for such trials is the double-blind placebo-controlled randomized clinical trial, but lesser forms of clinical evidence are also useful and appropriate to use in making clinical decisions and claims (although with correspondingly lower degrees of confidence).

Pre-clinical, or basic science, evidence looks at the effects of treatments in cell cultures in a test tube or petri dish, or in animals. This type of evidence is called “pre-clinical” for a reason – it is a prelude to clinical trials, and is used to determine what substances are safe to test in humans (and in what doses), what side effects need to be monitored, and what the possible benefits could be. They help determine plausibility and prior probability, however, pre-clinical evidence should never be used as a basis for clinical claims.

I say this with confidence based upon a century of medical science. There are countless examples of therapies that looked very promising based upon the pre-clinical data but utterly failed to produce the expected clinical effect, or which had unpredicted side effects or toxicity. Only a very small percentage of treatments that look promising based upon pre-clinical data pan out when studied properly in humans.

The reasons for this are obvious – the human machine is incredibly complex and it is extremely difficult to predict the entire cascade of effects on the system resulting from any intervention. We therefore need to directly observe the net effect on the whole organism in order to make any clinical conclusions. Medical scientists have therefore learned humility in the face of this complexity and know that we must proceed from pre-clinical evidence with caution.

The marketers and promoters of unregulated health products, however, either have not learned this lesson, or (more likely) simply do not care. They can sell their products with pseudo-health claims so they do. Existing pre-clinical evidence for them is a marketing resource – a basis for giving the patina of scientific validity to their products.

For example, many products are marketed with the claim that they “boost the immune system.” Often, offered as evidence, is basic science studies showing that in cell culture the product increase some marker of immune function. But increasing immune system activity does not equal “boosting” immune function – meaning making a person more resistant to infections. Rather, it can simply mean inflammation, which can have very negative effects on a person. In order to support a claim for increasing resistance to infections, the product would have to be studied in people with a relevant outcome measured, like the overall incidence and severity of infections.

Claims made for magnets are another example. Even if we accept at face value the claims that a static magnetic field increases blood flow (which I do not), this does not necessarily translate into a specific clinical benefit. “Increasing” is not always “improving”. The body has evolved a complex system of feedbacks and autoregulation to keep most physiological parameters within an optimal range. Disrupting this balance based upon a simplistic notion that increasing one parameter – like blood flow – equals improved physiological function is naive in the extreme.

Again, medical scientists have learned this lesson numerous times. Even based upon careful thought and the best intentions the history of medicine is full of examples of treatments that should have worked but actually made the situation worse. For example, in the setting of intracranial hemmorhage, blood outside of blood vessels causes vasocontriction, reducing blood flow and causing brain damage. So this was treated with vasodilators to increase blood flow, or at least reduce the vasoconstriction. However, this had the unintended consequence of reducing overall perfusion pressure and thereby further reducing blood flow and worsening brain damage. Only clinical trials looking at the net effects of these treatments told the whole story.

The bottom line is that it is not legitimate to extrapolate from pre-clinical data to make clinical claims, and yet this practice is rife within the supplement industry. The citing of pre-clinical data is meant to confuse and impress the consumer, not to inform them. It also seems to be an effective marketing strategy. Under the current regulations, the only solution is a more skeptical and better informed public.

Consumers should always ask of any health claims: have the claims been tested in humans under properly controlled conditions? If not, be very suspicious of any claims, no matter how impressive they may seem.

3 responses so far

3 thoughts on “Pre-clinical vs Clinical Evidence”

  1. Freddy the Pig says:

    The marketers of these “Holistic” treatments are being excessively reductionistic while the pracitioners of “Western, Reductionist, Scientific Microfacist” (have I left anything out?) Evidence Based Medicine are looking at the holistic picture to see if something actually works.

    I think this Irony requires chelation.

  2. Well stated! You’re a top notch educator. I recently heard Mark Crislip’s podcast on “immune boosting” and its implications for inflamatory responses within the immune system. I highly recommend his podcast, Quackcast, as a “supplement” [& I guess I can use that term as long as I don’t mention any specific diseases] to this excellent piece.

Leave a Reply