A friend of mine recently posted a letter from Kevin Donka, a chiropractor, criticizing a CBS report on vaccine safety in 2004. Unfortunately, as far as I can tell the full text is only available on Facebook, and I’m not sure if Donka actually made it public or if I can see it because he and I have a mutual friend. I know nothing about this issue, and I imagine Donka is more informed about it than me — so I’m not going to go for factual issues like whether there’s a double-blind study indicating that vaccines prevent the flu. What I’m interested in is when he gets into numbers.
Your report was based on the theory of “herd immunity,” i.e., the idea that once a great enough percentage of the population has reached immunity that the incidence of disease becomes insignificant. However, this phenomenon was observed in populations that had achieved this through natural means, not vaccination. This theory does not apply to artificially immunized populations as evidenced by the fact that epidemics have often occurred in 98% and 100% vaccinated populations. This was reported in the Journal of the American Medical Association in 1998; 280:635-637.
Having said I’m not pursuing factual errors, I think it’s worth summarizing the results, because they are kind of laughably at variance with this description, which suggests that maybe the entire city of Topeka got measles a few years back, or something of the kind. The basic finding is that, in a sample of Finlanders with paroxysmal cough (n=584), 26% had pertussis or parapertussis (n=153). Finland is 98% vaccinated against pertussis, so even if this 153-person catastrophe isn’t an epidemic, it’s still kind of an indictment of the vaccine, right? In an almost-600-person sample, no more than 12 people should –
– oh, forget it, everyone knows that when you say “right?” it means it’s wrong. The trick is sampling. This pertussis sub-epidemic wasn’t found in 584 randomly selected Finlanders, it was in 584 Finlanders specifically selected to have one of the signal symptoms of pertussis. Note also (or take my word for it, since the article is paywalled) that although the study reports Finland’s overall pertussis vaccination coverage, it does not actually report the patients’ vaccination status — so not only is the sample biased toward people with pertussis, there’s nothing to ensure that anyone at all was vaccinated, much less the 153 people who actually had the disease.
This is arguably as much a problem with the authors of the study as with the hapless chiropractor who further warped them. The next two are a little bit more interesting to me, inasmuch as you can do a lot assuming that Donka’s reporting is rock-solid.
Even the Centers for Disease Control reported in its morbidity and mortality weekly report that 80% of all measles cases are in fully vaccinated people. (US Govt. 6/6/86; 35(22): 366-370.
The CDC’s own figures put the number of childhood deaths in the 2003-04 “flu season” at 135. Of these, 59 had received their flu shots. 43% of these children did as they were told and died from the flu anyway. This means that the odds that the flu vaccine will “save” a child are not much different statistically than simply flipping a coin. (http://www.cdc.gov/flu/avia/gen/info/pandemics.html)
80% of measles cases are in vaccinated people. 43% of flu deaths are in vaccinated people. This is serious. Right? (See what I did there?)
Problem 1: “not much different statistically than simply flipping a coin” is only bad in the context of the base rate of survival. Lots of cancer patients would kill to have a coin-flip’s chance of living. But the death rate from flu is less than 50%, so this is purely a quibble. But, while we’re here, I’ve got another: Although Kevin Donka seems sanguine about dismissing an improvement of 7% on a coin flip, I’m sure the parents of the 9 kids that improvement would have saved (0.07*135=9.45 — all assuming his setup is correct, which it isn’t; see below) would feel differently.
More gravely, though, Donka is basing all his arguments on his calculation of P(vaccinated | died)=0.43. In words: GIVEN THAT YOU DIED, the probability that you were vaccinated was 43%. This is an example of how formal phrasing can really clarify the point at hand. Donka’s number characterizes only dead children. It definitionally has nothing to say about what can “save” anybody, because people who are saved don’t die. What you want to know is P(died | vaccinated), which you can’t calculate from the numbers he’s using. But that isn’t the only thing you want to know, because you have to compare it to P(died | not vaccinated). Only if the two numbers are the same can you conclude that the vaccine has no effect.
But you can actually get some purchase on the problem from the death stats. If the vaccine has no effect, that means P(died | vaccinated) = P(died | not vaccinated) = P(died). A little algebra (this can be skipped, the end point is intuitive):
P(D|V) = P(D)
By Bayes’ rule,
(P(V|D)*P(D))/P(V) = P(D)
A couple substitutions:
P(V|D)*P(D) = P(V)*P(D)
P(V|D) = P(V)
… which is just a formal derivation of the natural intuition that, if the vaccine has no effect, the death stats should reflect the vaccination stats, i.e. the percentage of dead kids who were vaccinated should be the same as the percentage of kids overall who are vaccinated. If P(V|D) is LESS than P(V), then the vaccination has a protective effect — just redo the derivation starting with P(D|V) < P(D). Donka says P(V|D) is 0.43. Unfortunately, due to a lack of either data or creativity, I can’t find P(V). I think it’s higher than 0.43, but I don’t know for sure. The same logic applies to the measles stat — sure, 80% of measles cases are in vaccinated people, but how many people are vaccinated? Wikipedia says MMR vaccination dropped to less than 80% in the UK starting in 1998, when people started freaking out about this stuff; I assume the overall vaccination rate is still much higher, but that figure from the UK underscores that I don’t know for sure.
Since Donka’s letter isn’t exactly making the rounds, I’ve implied by posting this analysis that this sort of thing isn’t obvious to you, for which I apologize if I’m wrong. And I admit I kind of worry about how people will react when I post things like this: Is it just more evidence that statistics can say whatever you want? I mean, of course it isn’t — but of course I would think it isn’t.