Very little of the public gets their information directly from scientists or the publications they write. Instead, most of us rely on accounts in the media, which means reporters play a key role in highlighting and filtering science for the public. And—through embargoed material, press releases, and personal appeals—journals and institutions vie for press attention as a route to capturing the public's imagination.
This system doesn't always work smoothly. Just this year, we've seen a university promote a crazed theory of everything and researchers and journals combine to rewrite the history of science in order to promote their new results. But these unfortunate events are relatively minor compared to a completely cynical manipulation of the press that happened last week.
In this case, the offenders appear to be the scientists themselves. After getting a study published that raised questions about the safety of genetically modified food (GMOs), the researchers provided advanced copies to the press only if they signed an agreement that meant they could not consult outside experts. A live press conference and the first wave of press appeared before outside experts could weigh in—and many of them found the study to be seriously flawed.
Science journalism and the embargo system
Each week, reporters around the world get a jump on the scientific community. Nearly a week before the new editions of major journals are released, the press gets a chance to download many of the papers that will appear within them. That access is predicated on a simple agreement: nobody runs any news stories about the contents until after a date and time set by the journal. This embargo system is why key scientific findings tend to appear everywhere at the same time, with hundreds of similar stories published within minutes of each other.
When it works well, the embargo system provides a valuable sanity check on media hype. While preparing their stories, reporters are allowed to share the papers with relevant experts and scientists with contrary opinions, who can warn the public about the possibility of over-interpretations or shaky data (provided they, too, agree not to publicize the results early).
This system has its flaws—embargoes can be capricious or get broken, and the sources can be selective about who they allow to access the papers. But it can provide what's essentially an additional level of peer review before the results are set out before the public.
Manipulating the system
The important checks provided by that system have now been systematically undermined by a group of French researchers, primarily at the University of Caen. The researchers managed to get a paper published in the journal Food and Chemical Toxicology. Their paper examined the long-term viability of rats fed a diet supplemented with either the herbicide Roundup or a crop engineered to tolerate high levels of Roundup. In their study, the researchers claim to have found that both the pesticide and the GMO crop reduced the lifespan of the rats and caused a high incidence of tumors.
People with relevant expertise, when given a chance to look at the study, found significant and systematic flaws with both the experimental approach and the data. Presumably, this is precisely why these scientists went out of their way to avoid giving them the chance to look.
At the blog Embargo Watch, Ivan Oransky has tracked how the researchers controlled access to their paper. Any journalist that wanted to receive an advance copy was required to sign a non-disclosure agreement before receiving one. That agreement prohibited the outlet from sharing the results with any outside experts before the embargo lifted. In other words, if a press outlet wanted to be one of the first to cover the story, it would have to run the story without having any experts sanity check the paper.
The manipulation didn't end there. The embargo lifted during a live press conference from the researchers, hosted in London in cooperation with the Sustainable Food Trust. The SFT conveniently had a press release prepared; a release claiming that the research was "supported by independent research organization, CRIIGEN." However, this neglected to note that the paper's lead author, Eric Seralini, is on the CRIIGEN board.
The researchers got exactly what they'd intended. The first coverage was largely uncritical. Slashdot linked to the press release when mentioning the story, while the first coverage from Reuters didn't address any scientific questions whatsoever (although it did raise some cautions about the researchers themselves).
Problems with the research
We looked into the study immediately after it began appearing in a variety of outlets. While we didn't have the sort of expertise in toxicology needed to critique some of the details, a few things stood out. First, there appeared to be no dose sensitivity for either Roundup or the level of GMO food provided—they saw the same effects at any of the doses they tested. In addition, the GMO food produced the exact same effects as the Roundup, something the authors didn't provide a reasonable explanation for.
But these problems were only the beginning. As more critical reports began to appear and scientist/bloggers looked at the results, huge issues were made clear. The authors used a strain of rats that is prone to tumors late in life. Every single experimental condition was compared to a single control group of only 10 rats, and some of the experimental groups were actually healthier than the controls. The authors didn't use a standard statistical analysis to determine whether any of the experimental groups had significantly different health problems. And so on.
The experts who weighed in were dismissive. One called the work "a statistical fishing trip" while another said the lack of proper controls meant "these results are of no value." One report quoted a scientist at UC Davis as saying, "There is very little scientific credibility to this paper. The flaws in the test are just incredible to me."
However, by this point, the promotion of the paper already had its desired effect. Both the European Union and French governments were asking their food safety organizations to look into the results, which may have implications for France's attempt to ban GM crops.
Ethical failures all around
If the previous examples we've covered have been about scientists who have gone too far in promoting their work, at least in those cases the goal appears to have been publicity. In this case, the researchers clearly seem to be focused on achieving political ends, namely a ban on the use of genetically modified crops. All indications are that they have performed sloppy science, presented it as indicating something it hasn't, and then knowingly manipulated the press coverage of their work. All in order to ensure the paper had an oversized impact in the public sphere.
In that, apparently, they were unknowingly aided by the peer reviewers who allowed the article to be published. Food and Chemical Toxicology is a small, specialized journal, but there's little reason that its reviewers shouldn't have expected a basic statistical analysis of these results. (We've reached out to the Editor-in-Chief of the journal, but haven't received a response at publication time).
But the use of a nondisclosure agreement to limit press coverage should have set off alarm bells within the press. As Ivan Oransky put it in his Embargo Watch report, this threatens to turn reporters into little more than stenographers, copying down only what researchers want them to say, and performing no independent evaluations. Journalist Carl Zimmer, on his blog, calls out the AFP and Reuters (Oransky works for a different group within Reuters) for having agreed to the conditions in the first place. He writes, "This is a rancid, corrupt way to report about science."
In this case, however, the press wasn't reporting about science at all. It was simply being used as a tool for political ends.
reader comments
214