Anti-vaccinationists who claim, against the prevailing scientific evidence, that there is a link between autism and vaccines, have been busy recently spamming science bloggers (at least those who have posted on this issue before) with new evidence they believe supports a connection. The evidence is a small study involving giving vaccines to macaques and measuring various neurological outcomes. Here’s mine – Kim Spencer left this comment on my recent post about the chelation related death case:

when are you going to get down and dirty on the new monkey study? waiting on your opinion on how this could be wrong… http://www.ageofautism.com/2008/05/pediatric-vacci.html

can’t wait for your explanations!!!

Notice the implication that I have been somehow avoiding taking on this new study. I am sorry to keep you waiting so long, Kim, given that this study was presented as a series of three abstracts at the latest International Meeting for Autism Research. You will note that they are dated May 16th 2008 – this is my very next blog post.

This also bears directly on my first criticism of this evidence – abstracts presented at such meetings do not pass the same rigorous peer-review as full papers published in respected journals. Most abstracts never see the peer-reviewed light of day.But let’s take this study at face value. Unfortunately, abstracts such as this do not include much information. Absent is any detailed description of methodology – all we get is a rough sketch. Here is what we do get.

Methods: Macaques were administered the recommended infant vaccines, adjusted for age and thimerosal dose (exposed; N=13), or saline (unexposed; N=3). Primate development, cognition and social behavior were assessed for both vaccinated and unvaccinated infants using standardized tests developed at the Washington National Primate Research Center. Amygdala growth and binding were measured serially by MRI and by the binding of the non-selective opioid antagonist [11C]diprenorphine, measured by PET, respectively, before (T1) and after (T2) the administration of the measles-mumps-rubella vaccine (MMR).

There are numerous shortcomings of this study, primarily the small N – 13 exposed and 3 controls. Why the disparity in numbers? Why the small numbers? Further, the outcomes that are being measured require careful methodology and some interpretation. They are not binary outcomes like survival vs death. These kinds of outcomes require greater subject numbers to have any statistical meaning. This alone – small numbers with “squishy” outcomes – means that this data is highly unreliable. At best this is a “pilot” study – meaning that it could be used to decide if it is worth doing a real study, but should NOT be used as the basis for any scientific conclusions.

There are also many pertinent questions not addressed in the abstracts: 1 – were the monkeys randomized to treatment vs control; 2 – were all of the outcome measures performed by a blinded examiner; 3 – how many different outcomes were measured? Just the ones listed, or were more looked at but not reported (this would provide the opportunity to cherry pick among many variables); 4 – what was the time frame for the series of vaccinations?

This last point is important because they say that the vaccines were “adjusted for age and thimerosal dose.” What does this mean, exactly? Was the injection schedule compressed to represent the shorter age to maturity and life expectancy of macaques? This would not necessarily make sense. Macaques probably do not clear ethylmercury (it is this mercury in thimerosal, which is a preservative that used to be in childhood vaccines, that some claim is the cause of autism) any quicker than human infants, despite the fact that they mature quicker. It is also difficult to extrapolate large differences in body weight. In other words – it is not a simple matter to give macaques equivalent exposure to vaccines and thimerosal to simulate the childhood vaccine schedule. We would need to know exactly what choices the researchers made and why.

Without all of this information these abstracts are impossible to evaluate. These findings would only be of interest if they ultimately are published in a peer-reviewed journal with a full methods section detailing how the studies were done. Even then, the small numbers render the results highly suspect -so the study would also need to be expanded to a more appropriate number of subjects.

There are other specific criticisms of these abstracts. Orac at Respectful Insolence has done his usual thorough job of picking it apart. Specifically he criticizes their methods of analyzing a genome microarray, which Orac does as part of his own research.

Orac also points out that perhaps the most suspect feature of this study (to clarify – this appears to be one study that was divided into three abstracts) is the people who carried it out. Listed as an author is Andrew Wakefield – he is the British researcher who started the vaccine-autism myth with his paper linking the MMR vaccine to autism. His research has been subsequently utterly rejected, and he even faces disciplinary actions for questionable ethical behavior. He did not disclose that he had a conflict of interest, for example, as he stood to gain financially as an expert consultant for lawyers who were suing for vaccine injury. Wakefield, therefore, would be highly motivated to produce new evidence to vindicate himself.

Another author is Laura Hewitson. It turns out she has a child with autism and is currently a litigant (#437) in the Autism Omnibus hearings – a special court established to determine the merit of thousands of claims for compensation by parents under the claim that their children’s autism was caused by vaccines. Hewitson did not disclose this obvious conflict of interest. Hewitson is also married to Dan Hollenbeck, who regularly contributes to the anti-vaccination website Age of Autism.

So what we have here is a small study presented in a loose format as a series of abstracts, without adequate documentation of methods, conducted largely by people who have an enormous motivation for a particular outcome.

In the battle of science, this is a butterknife.