Jun 27 2014

Seralini GMO Study Republished

I don’t recall this ever happening before. It probably has, but if so, it’s rare enough that I have never heard about it. The strange odyssey of the paper, Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize, by Seralini et. al., just got stranger.

The paper was published in 2012 in Food and Chemical Toxicity. It was greeted with intense criticism from the scientific community for its many shortcomings, culminating in the journal retracting the study. Now the study has been republished by a new open access journal, Environmental Sciences Europe.

The GMOSeralini website is celebrating the republication, writing:

“Now the study has passed a third peer review arranged by the journal that is republishing the study, Environmental Sciences Europe.”

They also claim the the article was retracted because of pressure from pro-GMO lobbying.

Nature, however, reports that the journal’s editor-in-chief, Henner Hollert, communicated to them:

“We were Springer Publishing’s first open access journal on the environment, and are a platform for discussion on science and regulation at a European and regional level.” ESEU conducted no scientific peer review, he adds, “because this had already been conducted by Food and Chemical Toxicology, and had concluded there had been no fraud nor misrepresentation.” The role of the three reviewers hired by ESEU was to check that there had been no change in the scientific content of the paper, Hollert adds.”

So the paper was not re-peer-reviewed, despite Seralini’s claim. It was just republished, with the addition of more raw data and commentary by Seralini.

I blogged about the paper when it was first published. The paper claims that exposure to GM corn resistant to Roundup, or to Roundup itself, increased tumors in a rat model. Seralini and his coauthors got off to a terrible start by sending a press release to reporters but not allowing them to seek outside comment from independent experts. The paper itself was quickly shredded by expert review. Here is a summary of the criticism:

- The population of rats used have a high propensity for tumors. This causes a great deal of background noise, and would likely favor a false positive result.

- There were only 20 rats in the control group, and 80 in the exposure groups, an atypical asymmetry.

- The data reports that “some” of the test groups had a higher tumor incidence, while others did not – sounds suspiciously like cherry picking the data.

- The statistical analysis done by the team was atypical, characterized by nutrition researcher Tom Sanders as ”a statistical fishing trip,” while a more standard analysis was excluded.

- Exposure to GM corn or the herbicide Roundup had the same negative effects. It is inherently implausible (admittedly not impossible) for such distinct mechanisms to have the same effect.

- There was no dose response at all – which is a critical component of demonstrating a toxic effect.

- The researchers did not control for total amount of food consumed, or fungal contaminants, both of which increase tumors in this population of rat.

A careful review of the pathology presented in the study was published as a comment, and concluded:

The ESTP comes to the conclusion that the pathology data presented in this paper are questionable and not correctly interpreted and displayed because they don’t concur with the established protocols for interpreting rodent carcinogenicity studies and their relevance for human risk assessment. The pathology description and conclusion of this study are unprofessional. There are misinterpretations of tumors and related biological processes, misuse of diagnostic terminology; pictures are not informative and presented changes do not correspond to the narrative. We would like to finish our commentary with a question: what is the scientific rationale that led the journal reviewers and the editorial board of Food and Chemical Toxicology to accept this article for publication?

It was stinging commentary such as this that likely led to the paper’s retraction. My own summary would be – this was a small study with sloppy methods using a noisy system and reeking of p-hacking.

Of course, this one small study is getting so much attention because of the political nature of the question – the safety of GMO. Some in the anti-GMO crowd still use the study to support their position, and they are likely happy with the republication.

Retraction Watch discussed this issue, giving a good quote from a journalist, Oransky:

“The ratio of politics to science when it comes to discussions of GMOs [genetically modified organisms] is so high that I think it often ceases to be useful.

“This is a good example of what happens when people with hardened beliefs manipulate a system for the result they want. Science should be about following the evidence, appropriately changing your mind if the evidence warrants it. But in this case people seem to reject the evidence that doesn’t suit their needs.”

Conclusion

The Seralini GMO rat study is now infamous for its poor quality and overstated conclusions. The republication of the paper extends the saga, but does nothing to correct the many failings of the study.

Defenders of the republication cite the virtue of access to data, and open communication, while decrying the “censorship” of the retraction. This completely misses the point, however. The peer-reviewed literature is not just about open communication of data, but puts into place a heavy filter for quality. A study needs to reach a certain minimum level of quality before it is worthy of consideration as part of the peer-reviewed literature.

If anything, the peer-review process (especially if you consider it across all such journals) should be tightened, not loosened.

There is a further problem with publishing preliminary or exploratory research. Such studies are meant only as an indicator of future confirmatory research, not as a basis of conclusions or recommendations. However, preliminary research is often treated by the press, and therefore the public (and often encouraged by authors overstating their data) as if it were confirmatory.

This problem is exacerbated when the topic is controversial, like GMO. I would argue that the threshold for publication should be higher for controversial topics, otherwise unreliable data is likely to confuse the public discourse.

Still, scientists need preliminary data to guide later research. The compromise I have suggested is that preliminary research be published with an editorial warning label – this is preliminary research meant only for professionals to guide later research and should not be used as a basis for recommendations, policy, or scientific conclusions. Publishing this study is not a statement by the editors that the results are likely to be true.

Share

19 responses so far

19 Responses to “Seralini GMO Study Republished”

  1. tmac57on 27 Jun 2014 at 9:40 am

    An editorial warning for preliminary studies sounds like an excellent and common sense idea Steve.This needs to happen!

    It has become such a “common knowledge ” so called “fact” in people’s minds that GMO’s are evil and unclean,that I have a hard time seeing how this mess will ever be resolved. Sure,we all want our food to be safe,but I have yet to see any convincing evidence that GMO’s are the danger that the average person has come to believe that they are.

  2. BillyJoe7on 27 Jun 2014 at 9:55 am

    Scientists react to republished Seralini GMO maize rat study:

    http://www.geneticliteracyproject.org/2014/06/24/scientists-react-to-republished-seralini-maize-rat-study/

    The claimed effects show no dose-response

    The breed of rats used is subject to spontaneous tumor development. To identify a statistically reliable increase in tumors in a group of rats requires a large number of individuals.

    Séralini did not follow conventional methods for assessing animal toxicity and made most of the measurements at the end of life.

    When a very large number of measurements are made, statistically significant differences will occur [by] chance.

    the design of the study and the described methods for data collection were fatally flawed in a number of ways.

    The sample sizes are too small

    there are too many treatments and not enough controls

    As [this breed of rat] is prone to high numbers of tumours, there is going to be a lot of noise and not enough statistical power.

    There is no dose response….ie they were just measuring noise.

    Looking at rat studies with a real carcinogen (Aristolochia herbs), the onset of tumors in a dose-dependent matter is rapid, and reaches 100% by 16 weeks

    the interpretation of findings included such basic errors that they would “be considered as a disqualifying mistake at an examination for pathologists”

    the paper has insufficient scientific merit even to be considered controversial or provocative and will likely to be essentially irrelevant to the mainstream scientific community.

    The wrong controls were used…there should have been an 11 per cent control for the 11 per cent GMO corn, a 22 per cent control for the 22 per cent GMO corn and 33 per cent standard corn for the 33 per cent GMO corn. As energy content, carbohydrate load and other components of the corn may affect tumour formation, this is a fundamental flaw which invalidates any conclusions.

    There is no dose response. For a substance to be an attributable cause of cancer, being exposed to more of the substance should result in more cases of cancer this just does not happen in this study.

    There is no consistent response to any of the measured outcomes that would even hint at a real adverse effect.
    - The GMO corn had no effect on the number of tumours
    – Roundup even decreased the number of tumours in male rats
    - The combination of roundup and GMO corn decreased the number of tumours in male rats
    - High levels of GMO corn and high levels of roundup both reduced spontaneous mortality and pushed back the onset of death in male rats.

    all we are seeing in these results is due to random variation in a poorly controlled experiment.

    the basic rule of toxicology is the dose makes the poison. Everything can be toxic if the dose is high enough. Therefore all proper toxicology studies show dose response curves (the higher the dose, the greater the effect). None of the data in the Séralini paper show dose response curves.

    the use of inappropriate strain of rats. Sprague-Dawley is a strain of rat that spontaneously generates tumors.

    The number of rats used was too small to detect a meaningful difference

    The strain of rats used (Sprague-Dawley) was inappropriate for this type of two-year long study, as these rats have a natural predisposition to form tumors, regardless of the treatment.

    There’s no dose response. In toxicity or carcinogenicity studies, increasing the dose of an actual toxin or carcinogen leads to greater effect.

  3. Amor De Cosmoson 27 Jun 2014 at 11:20 am

    Y’know… you’d think that if Seralini was so cocksure of himself, he would just repeat the experiments and address all the shortcomings outlined by his peers. If Seralini could properly repeat his experiments and prove that GMOs cause tumours, then that really would open up proper scientific discussion on the dangers of GMOs. There is so much money being spent on anti-GMO propaganda that surely it would be easy to raise the money to prove the Monsanto shills are just pushing poisons for profit. (/sarcasm)

    Instead, Seralini just acts like a petulant child trying to scream louder than anyone, “I don’t care what you say, I’m still right!”

  4. Mlemaon 27 Jun 2014 at 12:26 pm

    “The ratio of politics to science when it comes to discussions of GMOs [genetically modified organisms] is so high that I think it often ceases to be useful.

    “This is a good example of what happens when people with hardened beliefs manipulate a system for the result they want. Science should be about following the evidence, appropriately changing your mind if the evidence warrants it. But in this case people seem to reject the evidence that doesn’t suit their needs.”

    Amen to that.

    Being skeptical of a number of applications of genetic modification, especially as practiced by Monsanto – I would like to see equivalent peer review of the original safety studies by that company for these products. I’d also like to see it applied to the research reviews upon which the EFSA and, consequently, the AAAS have based their safety statements.

    Thanks for keeping us on track.

  5. Kawarthajonon 27 Jun 2014 at 2:17 pm

    “preliminary research is often treated by the press, and therefore the public (and often encouraged by authors overstating their data) as if it were confirmatory.”

    Not just the press and public. I have recently been to a conference during which researchers were promoting their small and poorly designed studies (i.e. only college psychology students as subjects) as the reason to drastically change public policy and the law! Most of the audience were clinicians and did not seem to understand the incredible failings of this study and the danger of making clinical decisions based on its flawed conclusions. Makes me very angry.

    “They also claim the the article was retracted because of pressure from pro-GMO lobbying.”

    Steve, I guess you’re in the pocket of BIG GMO! ;) What a great way to dismiss constructive criticism.

  6. GeoManon 28 Jun 2014 at 8:26 am

    Love these types of posts as it gives us an insight to how real science is done! Have been enjoying your blogs so keep it coming!

  7. hardnoseon 29 Jun 2014 at 7:09 pm

    There were 80 rats and only 20 in the control group because there were several different treatments. They needed at least 20 rats in each treatment group, but they only needed 20 in the control group. So I don’t see anything strange about that.

    Since there was a control group, it doesn’t matter that these rats are likely to get tumors.

    And they explained the lack of a dose response — they said it was a threshold effect.

    I am not saying I think the study is beyond criticism. Just that some of the criticisms have seemed unfair.

    And I don’t see anything implausible about the idea that eating herbicides might damage your health. I would be kind of surprised if it didn’t.

  8. Steven Novellaon 29 Jun 2014 at 8:05 pm

    Saying it’s a threshold effect is special pleading, nothing more. There is no reason to think there is a threshold effect,when toxicity is overwhelming dose related.

    The fact that this breed has a high tumor rate creates a lot of background noise, which means you need more numbers (more power) to pick out an effect from this noise, and this was a small study. The only 20 controls is just part of the overall fact that this was a small study.

  9. Davdoodleson 29 Jun 2014 at 8:57 pm

    So:

    1. Setralini did the experiment wrong.
    2. When the obvious shortomings and errors in his approach were drawn to his attention, by multiple independant sources, he ignored every one of them.
    3. it would be really simple to conduct this experiment in a way that results in probative evidence, and her now knows it, yet he chooses not to re-do the experiment in that manner.

    There is no reason to believe he is conducting his “research” in good faith.
    .

  10. Bill Openthalton 30 Jun 2014 at 4:02 am

    hardnose –

    And I don’t see anything implausible about the idea that eating herbicides might damage your health. I would be kind of surprised if it didn’t.

    But that wasn’t what the study purported to prove. It was hailed (in the European media at least) as proving that GM grain (and by extension all GMOs) caused cancer. Even before this “study”, large swathes of the European public opinion accepted as self-evident that GMOs are dangerous, and should be avoided at all cost. To Séralini and his defenders, scientific fraud is obviously a small price to pay to further their ideology.

  11. hardnoseon 30 Jun 2014 at 9:16 am

    “he fact that this breed has a high tumor rate creates a lot of background noise, which means you need more numbers (more power) to pick out an effect from this noise, and this was a small study. The only 20 controls is just part of the overall fact that this was a small study.”

    Steve N — High variance and a low N makes it less likely that you will find an effect, if there is one. The fact that they got results means the variance was probably not too high and the N was probably not too low.

    And 20 subjects is pretty conventional, not usually considered small, in experimental research.

    When you get positive results, you don’t worry about the power being too low. I have seen you make that mistake repeatedly.

    Yes of course the research has to be replicated and refined. But low power is not the problem.

  12. Mlemaon 30 Jun 2014 at 10:05 pm

    The same study was done by Monsanto 8 years prior, and published in the same journal. The only difference was that was for 9 months and Séralini’s was for 2 years. No paper has been retracted for being “inconclusive” before.

  13. Mlemaon 30 Jun 2014 at 10:15 pm

    sorry – 90 days, not 9 months.

  14. Newcoasteron 01 Jul 2014 at 12:21 pm

    I’m terrible at statistics. I remember taking it as a required course in Pre-med, but the only slot I could get was 6-9pm on a Thursday night, and it was taught by a grad student whose command of English was less than ideal. I struggled and managed to squeak by, but I’ve never really had a firm grasp on Chi-square, P-values, Bayesian etc.

    So, when I saw “p-hacking” I realized there was yet another term I really didn’t have a good handle on, so decided to use trusty Google for an explanation. The 3rd item that popped up was Neurologica and a piece Steve did in February. Thanks Steve !

  15. Mlemaon 03 Jul 2014 at 3:26 pm

    Seralini claims there’s a double standard, and gives good evidence thereof.
    http://www.sciencedirect.com/science/article/pii/S0278691514002002
    Here’s a link to the Monsanto abstract of the same study (Also – Monsanto tends to NOT use isogenic lines as controls, and may in fact use feed contaminated with gmos).
    http://www.sciencedirect.com/science/article/pii/S0278691504000547
    I provided a link to the raw data in a pdf in earlier conversations. It’s out there if you’re interested – made public as the result of a law suit in Europe. Liver and kidney lesions.

  16. hardnoseon 04 Jul 2014 at 2:23 pm

    Yes Mlema, it certainly does seem to be a double standard.

    And criticizing a study with positive results for having a low N and high variance is just plain ignorant.

  17. rezistnzisfutlon 25 Jul 2014 at 11:23 pm

    There is no double standard when it comes to the Monsanto studies. For one, there is nothing contradictory with established science in their studies – the Seralini study is. When several thousand varying lines of evidence tend to converge on similar conclusions, that makes the results fairly reliable. Furthermore, when relying on one study to inform opinions (single study fallacy), especially when that study is runs contrary to the myriad other studies and established science, that will only serve to fuel confirmation biases rather than serve to inform. What’s more, when the study is weak and fraught with problems, the likeliest conclusion is that the study is unreliable.

    Monsanto studies have only one purpose – to establish safety and efficacy of the seed in question. When results turn up negative on basic tests, there is no hypothesis to test from there. In fact, there is no hypothesis anywhere that indicates that GMOs are at all unsafe, either scientifically or empirically. The thousands of independent published studies that corroborate this is more than enough. If the independent studies were more mixed, inconclusive, or had at least some indication of harm, that may be a different story. The fact of the matter is, nothing of the sort has come to light.

    Then we get the conspiracy theories that Monsanto is somehow suppressing evidence or colluding with scientists from around the world in a massive effort to skew results.

  18. Mlemaon 25 Jul 2014 at 11:41 pm

    The Seralini study had some of the same results as the Monsanto study. The difference was: Seralini broadcast despicable photos of tumorous rats all over the internet and Monsanto hid its raw data behind “trade secret” legal wall.

    You can say there are thousands of independent published studies that corroborate that GMOs are safe, but that doesn’t make it true. You don’t see what you don’t look for, and you don’t look for what you don’t want to see.

  19. rezistnzisfutlon 25 Jul 2014 at 11:55 pm

    The Seralini study had some of the same results as the Monsanto study.

    Really? So the Monsanto study demonstrates that their GM corn, with glyphosate residue, causes tumors in rats? in fact, I’d be surprised if any Monsanto scientist came up with any study as egregious as Seralini’s. The blowback on Monsanto would be tremendous and would only serve to fuel the flames of anti-GMO hysteria even further. Monsanto has to play it utterly straight and clean because anti-GMO fear-mongers have created a narrative that has formed much of public consciousness, mostly based on misinformation, if not lies.

    You can say there are thousands of independent published studies that corroborate that GMOs are safe, but that doesn’t make it true. You don’t see what you don’t look for, and you don’t look for what you don’t want to see.

    That’s pretty much meaningless. You assume that none of us have critically evaluated any of these studies. Sorry to burst your bubble, but we have. But it’s not just the strength of the studies alone, it’s the repeated efforts as well as the converging lines of evidence. What you have are single studies that are highly flawed to begin with to support your narrative, studies that have been roundly refuted by the vast majority of scientists. That’s simply not how science operates. Out of all of those studies, none have produced positive results in regards to safety. Furthermore, we have decades of use that have shown no link whatsoever to GMOs causing any sort of health issues, nor does the science itself support such a notion. All that comes together to point to the conclusion that GMOs are safe.

    Even IF there was ever a study that demonstrated a GMO was unsafe (and there have been), that doesn’t impugn all GMOs, because every GMO expresses differently. The few studies that did show harm were preliminary to public release of the product (Starlink corn, for instance). This demonstrates that the system works.

    No, the true double standard is that Organic products have never had any sort of safety testing done on them, and it’s just as new of a farming technique as GMOs are, and we know virtually nothing about the effects of organic methods other than the several outbreaks of salmonella, etc, directly due to organic farming methods (primarily the use of uncured manure and compost).

Trackback URI | Comments RSS

Leave a Reply

You must be logged in to post a comment.