The other day, the most disturbing thing happened to me. I found out that I was being lied to.

And not just about something trivial, but about an issue of the utmost importance to my family’s and community’s health, not to mention my own. I had been lied to about the un-safety-ness of vaccines.

You see, I had been under the ignorant misconception that the issue of vaccine safety was a complicated one. I had thought, based on quite a bit of research, that there were risks associated both with vaccinating and not vaccinating your child. I thought that high quality research was accumulating on both sides of the debate. And I thought that the relative risks of vaccinating your child might depend on different factors, such as where in the world you live and your genetics.

But then I read a blog post by Jennifer Raff called “Dear Parents, You are being lied to” And boy did I have egg on my face.

Because apparently, according to the post by this post-doctorate research fellow out of the University of Texas, the only reason that my head was full of such nonsense was because I had been lied to, lied to by people who have an interest in making me afraid of “science-based medicine.”

Phew! You can imagine my relief that a colleague of mine had brought this post to my attention. After all, I may have a family of my own one day, and I really didn’t want to be burdened by the irrelevant nuances of an issue that I had been deceived into believing was complicated, when in reality it was very simple, very black and white. I just needed to be unburdened of all the “unscience-y” lies and do the responsible thing.

On first pass, I read through the post without clicking on the links. It’s like she’s inside my head, I thought to myself, eagerly soaking up what was promising to be a thorough piece of research. The post listed a series of “lies” that I and millions of others had been told followed by links to references which promised to point to the actual scientific literature to handily refute them, once and for all. Hyperlinks danced across the page and the piece culminated with some recommendations on how to read “scientific papers,” such as the ones linked to in the piece, “intimidating” though they may be. To gather some courage, I took a quick look at her “guide for non-scientists on how to read and understand scientific literature.” Wow, this was going to be great. Armed with her guide and a hot cup of green tea, I rolled up my sleeves eager to jump into the scientific studies.

“In light of recent outbreaks of measles and other vaccine preventable illnesses, and the refusal of anti-vaccination advocates to acknowledge the problem, I thought it was past time for this post” she begins. I clicked on the first link and was taken to a self-published article on about.com. “Huh, that’s odd,” I thought, “this isn’t a scientific study.”

Not to be discouraged, I continued on. “Dear Parents,” she continues, “you are being lied to.” Wow, heavy stuff. I click on the second link and am greeted by an article published in Forbes Magazine.

“The people who claim to be acting in the best interests of your children are putting their health and even lives at risk.” Goodness, well that’s a bold statement. With baited breath I clicked on the third link, and rather than being directed to a peer-reviewed controlled study, I found myself looking at a self-published article on weather.com. No, seriously, weather.com. Now, I was completely and utterly confused.

As I made my way through the rest of the links in the article, I experienced a range of emotions, most of which were highly unpleasant. It was slowly beginning to dawn on me that this was not an objective and well-researched article about an important health issue written by a well-trained and unbiased “scientist” trying to fairly represent the scientific literature. No, this was actually a textbook example of science abuse. Flagrant wanton bias. This lady had turned science into a red-headed stepchild.

Opportunity really does present itself in the funniest of places. After composing myself from the double indignity of being lied to about being lied to, I realised that what the author had actually presented us with is a valuable opportunity to study the black art of deception by “science.” What I had in front of me was a textbook example of how someone claiming the role of scientist might misuse science to support an opinion all while pretending to be scientific. A bit of misdirect here, a bit of razzle-matazzle there. It’s most ironic, given the title of her piece.

But the techniques employed are very common indeed, and learning them will be the best way to spot them in the future. So with your permission, I shall take you through some of the most flagrant misuses, so that the next time someone tells you that you are being lied to you’ll know if that’s true because you’re being lied to or because the person saying that you’re being lied to is lying.

Before we begin, I want to say that this is not primarily a post about vaccines. This is a post about the abuse of science, which I will loosely define as presenting a subjective opinion and intentionally misrepresenting the scientific literature-base to present this opinion as fact, all while pretending to teach people about the scientific literature. You may learn a bit about vaccine safety along the way, as I did, but this is just a positive side effect of learning a very important lesson. So when you’re ready, let’s dive into the first dirty trick she turned.

Trick Number One: Ignore the Hierarchy of Evidence and pretend that all “science” is created equal

In scientific research, there are different study designs and methods used, depending on the question being asked. A case study is an example of a method where we might follow one patient to see if a certain medication or treatment helps them get better. The problem with this study design is that we don’t know if they would have gotten better anyways without the treatment. It’s not controlled and it has too few study subjects. In other words, the study design itself has some built in flaws that predispose it to bias, which is when the outcome suggests an effect that isn’t really happening.

All research methods can be categorised by the strength of evidence that they provide when done well. A randomized controlled trial is considered to give stronger evidence than a case controlled trial, for example. A systematic review is considered to be at the top of the evidence hierarchy because it takes all the available literature that seeks to ask a certain question and then systematically evaluates it as a whole.

What is not usually represented in the hierarchy of evidence, however, is newspaper articles, blogs, or unreferenced, non-peer reviewed agency websites. Because they’re not technically part of the scientific, medical literature. But if you’re a visual learner you can imagine them underneath the bottom level, if you’d like, right next to your uncle’s “expert opinion” (read “drunken rants”) about the length of your friend’s miniskirt.

I looked at the first 19 statements that she made in the post (right before she digressed into a fantasy about how people would react to having so many “lies” exposed all at once) and the 30 references she used to support them. Out of 30 references, eight of the references were to peer-reviewed scientific research studies (there was one link that was broken, so let’s be generous and assume that was an actual scientific study as well). So out of all the “evidence” she used to support her testable claims about what the research literature shows about vaccines, only 30% of this was actual research literature. The rest were unreferenced websites (of the type she warns us at the end of her post to stay away from) and blogs, including a guest post on her own. How bizarre.

As she points out, “not all evidence is equivalent, and it takes training and a willingness to admit to ignorance in order to be able to identify good evidence from bad evidence.” Indeed.

Trick Number 2: Summarize the available research (but ignore any studies that contradict your argument)

In the middle of the post, the author makes the following statements: “The question of whether vaccines cause autism has been investigated in study after study, and they all show overwhelming evidence that they don’t.”

In an uncharacteristic fit of using actual peer-reviewed research to support a testable claim, Dr Raff references five actual studies (with a couple of newspaper articles sprinkled in for good measure). And she uses these to support her statement about what all studies show. She does not, however, tell us in which databases she searched, which search terms she used, or how she evaluated which studies were worth considering. In other words, she does not reveal the methods that she used to present as truth that there were no studies in existence that had come to a different conclusion from her own.

Being a naturally curious person, I hopped over to Google Scholar and typed in the words “vaccines” and “autism” and wouldn’t you know, but I came across a study published in 2011 entitled: “Do aluminum vaccine adjuvants contribute to the rising prevalence in autism?” published in the Journal of Inorganic Biochemistry. The authors found a significant correlation between the two and using a bit of statistical juju found that the relationship could be causal. Huh.

So let’s put this into perspective: the vast majority of studies looking at the relationship between vaccines and autism find no connection, and I’m not trying to suggest otherwise. I’m also not saying that the study that I found is perfect. But what I am saying is that the study that I found exists and cherry picking studies that support your claims while pretending that studies that contradict your viewpoint do not exist even though they do, is really really unscientific (even, and especially, when done by a “scientist”).

Trick Number 3: Got a “Special Article” that supports your argument? Run with it

The two articles that this post referenced most often were published in a journal called Pediatrics – one is called “The Problem with Dr Bob’s Alternative Vaccine Schedule” and the other is “Addressing Parents’ Concerns: Do Vaccines Contain Harmful Preservatives, Adjuvants, Additives, or Residuals?”

Ok, so we’ve got a couple of articles out of a “peer-reviewed” journal. We’re looking good so far. When I come across a paper, one of my first questions is, “What was the study design?” and after a quick look around I see that there is no study design in either of these papers because these are not research reports at all, but rather they are “Special Reports.”

So one might be tempted to ask, “just what, exactly, makes these reports so gosh darn ‘special’ anyways?”

Their specialness comes from not being reports of research studies but rather being op-ed pieces by doctors or other scientists. They could come under the heading of “expert” opinion. So it’s at the bottom of the hierarchy of evidence, just above referencing your own blog to answer a question about what scientific research shows about an important public health issue. But they don’t constitute scientific research in themselves, unless your question wasn’t about vaccine safety but about the mis-use of “special reports” to back up quantitative claims.

Lesson Number 4: Reference a study that doesn’t support your argument but pretend that it does

Let’s take a look at what she says: “They say that vaccines haven’t been rigorously tested for safety. But vaccines are subjected to a higher level of scrutiny than any other medicine. For example, this study tested the safety and effectiveness of the pneumococcal vaccine in more than 37,868 children.”

Gosh, that is a lot of children! Let’s take a look at that study. First the design: a double-blind controlled trial. Very good. Next, what were the methods: “37,868 children were randomly assigned 1:1 to receive either the pneumococcal conjugate vaccine or meningococcus type C CRM197 conjugate.”1

Hang on a minute, I’m confused. The study group received a vaccine and the control group also received a vaccine, just a different one?

Tell me if I’m out of line here, but when concerned parents, or as the author calls them, “anti-vaccine activists,” question the relative safety of vaccines, aren’t they usually comparing being vaccinated to not being vaccinated? This study is comparing the safety of one vaccine over the other, not ‘vaccinating’ vs ‘not vaccinating’. So when the researchers discuss the differences, for example, between febrile seizures in the “vaccinated group” vs the “control group” you gotta remember that the “control group” was also vaccinated. It’s a bit subtle, I know.

We find another example of this in the author’s treatment of the connection between autism and the MMR vaccination . In support of the claim that no connection has ever been found, she uses what I consider to be the highest quality publication that I’ve yet come across on the subject vaccine safety. It’s an Immunzation Safety Review created by an impartial committee put together by the National Academies. The 153-page document systematically walks through each issue, the evidence presented, and then forms a conclusion with recommendations.2

Dr Raff uses this document to support the claim that there is no evidence for a link between the MMR vaccine and autism. So what does this document actually say?

“The committee concludes that the evidence favors rejection of a causal relationship at the population level between measles-mumps-rubella (MMR) vaccine and autistic spectrum disorders (ASD). However, this conclusion does not exclude the possibility that MMR vaccine could contribute to ASD in a small number of children.

The committee concludes that further research on the possible occurrence of ASD in a small number of children subsequent to MMR vaccination is warranted, and it has identified targeted research opportunities that could lead to firmer understanding of the relationship.

The committee recommends that the relationship between the MMR vaccine and autistic spectrum disorders receive continued attention.”

Wow, I don’t even know what to say. What the report says is so incredibly different from how Jennifer summarised it, which is frankly insulting to the intelligence of her readers. I’m really glad that she told me not to take her word for it and to look at the original publication. Otherwise, I would have been profoundly mislead.

(By the way, this report also found “weak support for increased risk of allergy and for autoimmunity and strong support for increased risk of infection” when having multiple vaccines. I wonder why Jennifer found the report to be strong enough evidence to use in support of her claims about MMR but not worth including in her treatment of these other topics?)

Trick Number 5: Cite a website with “science” in the name and hope that nobody notices that it’s unreferenced, biased and factually inaccurate

The “lie” that she is exposing: “They say that ‘natural’, ‘alternative’ remedies are better than science-based medicine. They aren’t.” This statement is supported with a link to an unreferenced article on the Autism Science Foundation website entitled “Beware of Non-Evidence-Based Treatments.”

We’re first delighted by a lesson in research literacy: “To be considered evidence-based, a treatment must be thoroughly investigated in multiple well-designed scientific studies and show measurable, sustained improvements in targeted areas.” It then goes on to give a list of “Biomedical Non-Evidence-Based Treatments.”

First up, chelation. “There is no evidence that supports chelation as a safe treatment alternative because autism is not caused by metal poisoning … No paper published in the peer-reviewed literature has reported abnormal levels of mercury in individuals with autism spectrum disorder.”

Let’s break this down. First, this “science” website says that there is no evidence that supports chelation – not that the literature is mixed, or that there is evidence but that the methods used were flawed, or that the evidence is ‘weak’, but that there is no evidence in support. Ok google scholar, you’re up.

I typed “austism” and “chelation” into the search thingy and was greeted with non other than a systematic review (the pinacle of the evidence hierarchy) that states “Multiple studies were identified that suggest some individuals with an ASD manifest clinical and behavioral improvements with chelation.”3 Riiiiight.

Next up, “No paper published in the peer-reviewed literature has reported abnormal levels of mercury in individuals with autism spectrum disorder.” No paper. None, nada, zilch.

Typing “mercury” and “autism” into the box, I am overwhelmed by a veritable research avalanche. I click on one listing at random entitled “Blood levels of mercury are related to diagnosis of autism: a reanalysis of an important data set” published in the Journal of Child Neurology.4 The authors state:

“We have reanalyzed the data set originally reported by Ip et al. In 2004 and have found that the original p value was in error and that a significant relation does exist between the blood levels of mercury and diagnosis of an autism spectrum disorder.”

It’s like the author of this article went to the same school of “science” as the blog author.

Again, I am the first to admit that what I am presenting is what one study says. However, either the Autism Science Foundation was not aware of its existence, and thus was not in a position to comment on the totality of published research on the subject, or they were aware of its existence but pretended not to be. Either way, again, we’re looking at a case of wanton science abuse. Oh, the humanity.

Trick Number 6: Pretend that two very closely related issues are not connected at all

Towards the end of the article, Dr Raff encourages us to read about the real cause or causes of autism, so we can stop ignorantly and unscientifically accusing vaccines of playing some part. She provides a couple links looking at the genetic causes.

Ah genetics, gotta love genetics. Ok, so its genetics, not vaccines, that are causing autism. I follow ya.

But wait a minute, let’s get a bit more specific – which genes are they? I mean, have they found the “autism gene”? No? Ok, so there are some genetic defects that show up more commonly in autistic people than those who aren’t. So, what exactly do those genes do?

Well one set of genes that have been implicated is involved in the production and regulation of something fancy-sounding called glu-ta-thi-one.5 Glutathione is a molecule involved in the detoxification of heavy metals. It turns out that due to their genetics, some autisic children aren’t very good at producing or recycling glutathione, and thus have impaired ability to detoxify heavy metals, so what might be “safe” for some kids really cause a lot of problems in these kids. Hang on, wait a minute . . .

Listen, don’t get your panties in a twist, I’m not saying that “vaccines cause autism.” Anyone who has actually looked in depth at the issue knows that it’s a multi-factorial condition without a single cause. What I’m saying, based on all of the scientific evidence that I’ve come across (not just the bits that I like), is that it seems possible that vaccines are safer for some kids than others. It seems reasonable to me that some kids can handle a little aluminium injected into their veins, others not so much. I think it’s reasonable, even “scientific”, to explore the possibility that there isn’t some magic uniform value of aluminium or any other toxin that’s safe for everyone, as Jennifer suggests.

And that’s an opinion that I’ve formed based on looking at a variety of high-quality scientific studies. And as time progresses, and our knowledge increases, and the literature-base becomes broader, I fully suspect that my opinions, feelings, and knowledge of the matter will change over time.

Wrap Up

I am not anti-vaccine. I am not pro-vaccine. I do not see vaccination as a black and white issue. I believe, based on copious amounts of scientific research, that the issue is complicated. I believe that there are risks associated both with vaccinating your child and not vaccinating your child and like many other times, parents have to make a difficult decision. A cost benefit analysis of vaccinating vs not vaccinating will not be the same for every parent and every child. Where in the world you live, how well your child’s immune system works, and genetics are just some of the factors that might play a part in making the best decision you can.

But this really isn’t a post about vaccine safety. It’s a post about the abuse of science. I have no objections to Dr Raff’s opinions; but I strongly object to the intentionally misleading way she supports them.

I don’t understand why a pithy post debunking all the lies of “science haters” was backed up, not by the peer-reviewed controlled trials that the author seemingly encourages us to read, but overwhelmingly by sources that were neither peer-reviewed nor for the most part actual scientific studies. They were backed up mainly by reports firmly outside of the research literature, that happened to share the author’s opinion.

And when she did finally demonstrate that she could locate some actual peer-reviewed scientific research, she misrepresented it. That is some piss poor science. Using science to back an opinion is known as “bias” in science-y circles, and it’s something that good scientific practice seeks to minimise. When it’s done intentionally, it’s better known as “lying” . . . and trying to sound clever while doing so.

What are your thoughts? About vaccines? About science? About science abuse?