Last weekend, I noted a disturbance in the antivax crankosphere. Apparently, once again, a horrible antivax “study” that was ultimately retracted had been resurrected, as we have seen more times than I can remember going back to the very beginnings of this blog in 2005. I’m referring to a study published in BMC Infectious Diseases in January with a single author, Michigan State University economics professor Mark Skidmore, who used an anonymized survey that actually had some interesting findings that made a fair amount of sense, such as the observation that political orientation and antivax beliefs influence people’s perception of whether health problems in their social circle were caused by COVID-19 or COVID-19 vaccines. If Prof. Skidmore had stuck with those modest findings, there would have been little to complain about, but he didn’t. Instead, he used the results of his survey to extrapolate an estimate as high as 278,000 people “killed” by COVID-19 vaccines. I applied some much deserved not-so-Respectful Insolence to the study in January, and by April it was retracted, a fate that it richly also richly deserved. Although there also remained a question of whether, in using survey results to make an estimate of a health outcome in the US population, Prof. Skidmore had gone beyond the protocol that MSU’s Institutional Review Board (IRB) had deemed exempt (because it was an anonymized survey). I moved on, only mentioning Skidmore’s execrable “science” one more time in the context of its being an example of retracted studies that remain far more influential than they deserve. Then came Steve Kirsch.
A week ago, deluded tech bro turned rabid antivax and COVID-19 pseudoscience and conspiracy mongerer Steve Kirsch posted an article on his Substack entitled MSU Professor Mark Skidmore was exonerated by MSU; his landmark paper showing over 250,000 killed by the COVID vaccine is now back in the peer-reviewed literature:
Mark Skidmore wrote a paper that showed that 217,000 Americans were killed in 2021 by the COVID vaccine.
The journal retracted the article and Mark’s university commenced a 7-month investigation into unethical behavior by Professor Skidmore.
Today, I’m pleased to announce that Professor Skidmore has been exonerated on all charges and his paper, with some helpful additions suggested by Dr. Susan Oliver (and her dog, Cindy), has now been published in another peer-reviewed journal.
This is the usual sort of blather that I’ve come to expect from Steve Kirsch, who wouldn’t know good science from bad if it bit him on the posterior. (As a reminder, Susan Oliver did an excellent video deconstruction of Mark Skidmore’s dreck as well that is very much worth watching.) This post isn’t really so much to rehash what was so awful about Prof. Skidmore’s paper (and there were many, many awful things about it), but rather to discuss Mr. Kirsch’s hilariously ridiculously incompetent survey. I will, however, give my readers a heads up to mention that I did write an update for my not-so-secret other blog describing how the new version of Prof. Skidmore’s paper has been only slightly modified and still has all the problems that I discussed, what those problems are, and what a humiliation it must be for him to have been forced to republish his retracted paper in a crank antivax journal, Science, Public Health Policy & the Law, published by antivax scientist James Lyons-Weiler‘s Institute for Pure and Applied Knowledge (IPAK) and having been represented by the Christian dominionist, antivaccine, and rabidly anti-LGBTQ+ legal group, the Liberty Counsel. It will be published on Monday at the latest.
In the meantime, let me just say that Prof. Skidmore’s “estimate” for how many deaths COVID-19 vaccines “might have caused” is not really an estimate of what he claims, but rather an estimate based on an extrapolation of how many people perceived that someone in their social circles had died as a result of the vaccines. That’s it. His estimate has no epidemiological validity whatsoever and is only interesting as a result showing how successful antivaxxers have been at feeding the perception that the vaccines are dangerous.
Another issue is that apparently the MSU IRB found that Prof. Skidmore had committed no violations of regulations governing human subjects research protections. I’ll say here as I wrote in my more lengthy explanation to be published Monday: It’s not really the job of the IRB to determine if the science is good or bad (although often the IRB will do that if it comes to the conclusion that bad scientific design can cause potential harm to human subjects in a study), but rather if human subjects protections have been followed. Since Prof. Skidmore’s study was an anonymized study, the potential for harm for the human subjects who responded to the survey was deemed small, hence the finding that the study was “exempt” from detailed IRB oversight. Sadly, the MSU IRB doesn’t consider whether Prof. Skidmore’s study would be harmful to society at large, which it most definitely was, becoming one of the top 5% studies shared on social media.
Of course, Steve Kirsch, being Steve Kirsch and thus utterly full of the arrogance of extreme ignorance, decided that he would try to replicate Prof. Skidmore’s results, which led to an article on his Substack the other day entitled “Vaccine” killed 3.5X more Americans than COVID virus. To this, Mr. Kirsch added this challenges:
The data is clear and consistent. I challenge any qualified scientist to challenge this data in an open public debate.
Consider your challenge partially accepted, Mr. Kirsch. I will gladly challenge your “data” here, publicly, on my blog, but given your history of “Debate me, bro!” nonsensical unwinnable “debate” challenges, I will limit my not-so-Respectful Insolence to my blog. If you want to publicly debate, you are more than welcome either to comment below or to post a response on your monetized Substack for clicks and cash.
So let’s take a look at Mr. Kirsch’s “results”—in which he claims that his survey shows that COVID-19 vaccine killed 3.5x more people than COVID-19—such as they are:
The irresponsible attacks by an LA Times journalist Michael Hiltzik on MSU Professor Mark Skidmore’s paper motivated me to run my own survey of my readers to see what the actual harm numbers really are.
Over 10,000 readers responded.
The survey clearly showed that the COVID vaccines have killed 3.5 times as many people as COVID. This is a disaster.
I’ve had expert statisticians and epidemiologists review the survey, the methodology, and the results. None could find any errors.Who are these “expert statisticians and epidemiologists”? Inquiring minds want to know!
I laughed out loud at this part. (I will also note that Michael Hitlzik’s article about the Skidmore fiasco is quite good and you should read it if you can get past the paywall.) Once again, Mr. Kirsch thinks that an Internet poll can give anything resembling an epidemiologically valid estimate of any health condition in a population. Hell, Mr. Kirsch still thinks that amateurish Internet polls can provide an accurate estimate of any public opinion. They don’t, given how easily they are gamed. More importantly, even if they aren’t gamed such polls do not attract a representative sampling of the population. The people who read and subscribe to Mr. Kirsch’s Substack tend overwhelmingly to be antivax and COVID-19 conspiracy theorists. If you don’t believe me, just peruse the comments after any of his posts.
What made me laugh the hardest at Mr. Kirsch as I read this incredible pile of stinking fetid dingos’ kidneys was this passage in which he tries to defend the validity of his methodology, such as it is (or if you can even call it methodology in the scientific or even survey sense):
When I called Professor Norman Fenton and informed him of the 3.5X figure he calmly replied “I’m not surprised.”
The results of this survey are entirely consistent with the surveys by others as well as individual anecdotes that would have been very unlikely for me to have located if the vaccine didn’t kill at least 3.5X more people than the virus.Norman Fenton? Hilarious!
I note that Norman Fenton isn’t exactly a great person to go to to validate your awful methodology, given that he himself has engaged in a bit of data dredging in the cause of antivax narratives and appears unable to recognize that deaths due to secondary bacterial pneumonias due to COVID-19 are COVID-19 deaths. So of course he’s “not surprised” that Steve Kirsch’s silly survey found that 3.5x more people died due to COVID-19 vaccine complications than due to COVID-19 itself:
Even more hilarious is how, before he lays out his “data,” Mr. Kirsch shifts the burden of evidence:
Therefore accusations of “the survey was biased” are simply “hand-waving” arguments with absolutely no evidentiary basis of support. Could there be bias? Of course. Is the bias significant is the question! Since these people are anti-vaxxers, they are simply less likely to vaccinate and so the number of vaccine injuries will be LOWER than an unbiased group who vaccinates. So yes, there may be bias, but if anything the bias suggests that the actual ratio is higher than 3.5. I’m happy to have that discussion. Bring it on.
The best way to challenge these results is to show data that is 100% independently verifiable (which government statistics are not). So they will have to show us their survey and their verifiable anecdotes supporting their hypothesis. No one has any interest in doing that for some reason. These people are all perfectly content with having the number be “unknown.” I have a big problem with that.
No, this isn’t how science works. It’s not the duel between anecdotes, which is what Mr. Kirsch seems to think that it is. Moreover, it is not up to those challenging a claim to prove that Mr. Kirsch’s survey sample is not representative of the general population, although that is very easy to do just by pointing out that it’s an Internet survey that anyone can answer whose links were posted on the substack of a raving antivax conspiracy theorist.(Seriously, Mr. Kirsch, pollsters have standards in methodology to try to assure that their samples are as representative of the population being studied as is feasible, and even those sometimes fall short.) It’s not up to those criticizing Mr. Kirsch’s study to show that his survey questions can’t answer the question that he is claiming to answer. It’s up to him to show that they are valid according to standard polling methodology. (They’re not, obviously.) It’s not up to critics to show that his statistical methodology is incorrect and inappropriate for the survey design and sample chosen. It’s up to him to show that they are. One way that he could do that would be to subject his work to peer review in a journal of epidemiology. He won’t, because peer reviewers other than Norman Fenton and other antivaxxers would likely laugh out loud at his manuscript.
I do like Mr. Kirsch’s attempt to claim that, if anything, he’s underestimating the number of deaths due to COVID-19 vaccines because his audience is all antivaxxers and therefore there should be fewer vaccine injuries. This is one of the most incredibly ridiculous arguments I’ve ever seen from an antivaxxer, and I’ve been at this blogging thing for nearly 19 years now. What he ignores is that, if his audience is made up mainly of antivaxxers, then one would expect that respondents would massively misattribute health problems in their family and extended social circle to vaccines. He also neglects to note that antivaxxers who think that one of their extended family members was injured or killed by COVID-19 vaccines would be far more likely to respond to his poll than an average person in the population who might see this survey.
Also, just look at the survey! First of all, Mr. Kirsch wants names, email addresses, and phone numbers for “verification.” Even Prof. Skidmore’s survey, as misused and incompetently designed as it was, anonymized the respondents’ personal data through a third party, the company from which he accessed a supposedly “representative” sample of the US population. Who on earth would be foolish enough to give Mr. Kirsch that personal information? Mr. Kirsch waves away such concerns in the comments of his announcement for his survey:
READ THIS FIRST: I have never sold a phone number or contact info for anyone ever in my life.
The phone number is for verification. I call people on their responses at random. Live validation is much better than email. Jay Bonnar’s story for example, was because he filled out a survey and it looked odd and I called him.
The name, phone, and emails are for my use only and are never disclosed externally to anyone (including “fact checkers”) without getting your express permission.
And no, for this survey, I do not need to collect demographics. The signals here are incredibly strong and random draws of 10% of the samples show the values are very consistent.“Demographics? I don’t need no stinking demographics!”
Seriously, who would be stupid enough to believe this and trust Mr, Kirsch with their phone number particularly if they are not antivax and wanted to answer the questions truthfully in a way that goes against his obvious preexisting belief that the vaccines are deadly? I thought about answering, but then saw that he wanted a phone number and said not just “No!” but “Oh, hell no!” True, I could have used the same solution to the issue of providing a phone number that the Blues Brothers famously used when providing a mailing address for registering the Bluesmobile. (It came in quite handy, you might remember, for avoiding Illinois Nazis.) Sure, I could provide my work address and email, which are on public websites, but why would I give a litigious bully and social media harasser like Mr. Kirsch an excuse accuse me of trying to sabotage one of his silly surveys and, more importantly, to possibly bother the people at my cancer center who answer the phones?
I’ll just recount Mr. Kirsch’s results, even though his methodology and sample render them utterly meaningless and uninterpretable:
Analysis of the first 9,620 found 804 deaths from COVID and 2,830 deaths from the COVID vaccine. Those results were generated from a minimum of 108,000 people covered by the survey (some extended families were over 25 people and the survey didn’t track this so the number of total family members covered by the survey is a lower bound). We also didn’t ask about the age of each family member as this would have made the survey unmanageable.
Seriously? Mr. Kirsch didn’t even track the number of extended family members? He didn’t ask about age? After all, age is a very major contributor to the probability of someone dying if they get COVID-19! So basically he utterly ignored one potential contributory factor to deaths because…reasons.
We were primarily interested in simply the ratio of COVID deaths to vaccine deaths in the extended family (excluding the immediate household). The reason for excluding the immediate household is to reduce the bias effect since most of the respondents didn’t vaccinate themselves or their household. This is reflected in the lower ratio for the household statistics (and even then, the vaccines killed more people than COVID which is astonishing).
No, it’s not at all “astonishing” given the population from which the sample was drawn, readers of Steve Kirsch’s Substack who not only were willing to take the time to answer his survey but were willing to trust him with their phone numbers.These are likely hardcore antivaxxers who like Mr. Kirsch so much that they are willing to trust him with their names, email addresses, and phone numbers, which most people would wisely decline to do.
You know what? Remember how hard I said I laughed at Mr. Kirsch’s bit about having had epidemiologists and statisticians evaluate his methodology? I laughed even harder at this part, where Mr. Kirsch tries to defend against the charge that his respondents misattributed the causes of their family members’ deaths to COVID-19 vaccines:
At first, you may think “3.5X… that’s way too high. Surely these anti-vaxxers are misclassifying normal deaths as “vaccine deaths.”
There are 10,000 different people making these assessments. We can randomly draw 20 names and check on the details of each death to assess whether this is the case.
But there is a much easier method to validate that the 3.5X number is sane: a single anecdote that is 100% verifiable.
I reported earlier on a high tech sales executive Jay Bonnar who told me 15 of his friends “died suddenly” after getting the vax. His life experience otherwise is devoid of deaths. The stories are all in the public domain and are verifiable. They were all his friends; they all died suddenly after the vaccine. Jay also had 1 friend who died in the hospital from COVID after receiving Remdesivir (which is probably what really killed his friend, but let’s just give the COVID virus a death).
Jay’s observations are all objective counts of deaths. He did not make any subjective assessment as to cause. In the 10 years prior to the vaccine rollout, he had lost only 1 friend. Post vaccine he lost 15 friends, several of whom died within 1 week of being vaccinated. There’s a big signal there.
So if Jay saw one COVID death, with a 3.5 multiple, Jay should have seen 3.5 vaccine deaths. But he saw 15. The probability of that happening is 4.26e-6 which means that only 1 person in 234,515 would have observed a story like Jay’s.
There is no facepalm epic enough for this. Even the Godzilla facepalm is not large enough, but it is the largest that I have; so I’ll use it:
As for misattribution bias, let’s just say that Mr. Kirsch has rather odd ideas—from a scientific standpoint—on how to adjudicate such bias:
But there might also be a bias in assessing a death to be from the vaccine when it wasn’t caused by the vaccine. Experts can adjudicate these deaths and we can apply a correction factor that might correct in either direction. Here’s the interesting thing about this bias: I don’t think anyone knows which direction this bias is! I don’t. Do you? Were my readers more astute than trained professionals in assessing vaccine deaths? Or less astute? We can adjust for this bias, but the problem will be: who do you trust to make the professional assessments of the death? Any medical expert I suggest who I think is astute can be accused of being biased. So the bias accusation can always be made.
The simplest approach is the Occam’s razor method and assume that the assessments are “close” and consider this as one experiment that generates a value.
Again, no, Mr. Kirsch. That’s not how science works at all! He is correct about one thing, though. Antivaxxers like him never trust medical professionals who haven’t drunk the antivax Kool-Aid, whom they reject as “biased” or “pharma shills.”
Then he’ll cite nonsense like Ed Dowd’s book Cause Unknown:
It lists 550 cases of people who died suddenly. That was the only criteria. He never researched vaccination status for any of these cases.
Only one person was found to be not vaccinated on the list. The AP fact checkers didn’t find any unvaccinated people on the list.
If the vaccine doesn’t kill anyone, around 25% of the people would have been unvaccinated.
If the vaccine didn’t cause these deaths, then how do you explain the correlation?
Notice the assumption made without evidence, namely that the vaccine caused all those sudden deaths. This is what lawyers would refer to as “assuming facts not in evidence,” one of the few legal principles that works pretty well in science too. Again, sudden cardiac deaths are nothing new. Sudden arrhythmic death syndrome (SADS) was first described over 50 years ago, and we know many of the conditions that predispose to it, such as gene mutations and undetected congenital heart abnormalities. Moreover, there is no evidence that the incidence of SADS has been increasing since the vaccines rolled out. Just because antivaxxers didn’t know about SADS before the pandemic does not mean that physicians and nurses didn’t. The existence of SADS has been the reason for sports physicals in school, where the physician or nurse practitioner will screen for conditions and history that suggest a predisposition to SADS. It’s why there has been a push to make public access automatic external defibrillators (AEDs) more ubiquitous in public places going back at least a decade. Even leaving aside SADS, approximately 356,000 out-of-hospital cardiac arrests (OHCAs) have been reported annually in the United States, with an estimated 70% to 90% of people experiencing OHCA dying before reaching the hospital. Again, just because antivaxxers did not (and, apparently still do not) know this simple fact does not mean that the medical profession didn’t and doesn’t.
Mr. Kirsch keeps going on and on about his “survey” results, including “serious injuries” from vaccines, seemingly without understanding that if your sample, questions, and methods are crap, no matter how you spin it, your results are utter crap, challenges to vaccine advocates to produce data notwithstanding. I also note that vaccine advocates do provide data and studies. Mr. Kirsch always finds a reason to reject them. So why bother to try to school him with actual science and data again? He’ll only misrepresent it again, and then reverse the burden of proof again, the way he does near the end of his long post:
It’s time for the vaccine advocates to put up or shut up. Show me your numbers, show me at least 6 independent ways you validated it, and show me the similar extreme anecdotes showing that your number must be right and mine is wrong (like show me the guy who knows 30 people who died from COVID and only 1 person who died from the vaccine), and let’s get to the bottom of this and find out the correct number. Or is it better not to look and not to discuss it? Will that save more lives?
Not showing up with any data or any willingness to resolve this issue is unacceptable.
Even less acceptable is basically pulling bad data out of your nether regions, which is what Mr. Kirsch’s “survey” results in him doing. Let me suggest another alternative to Mr. Kirsch. Perhaps he should submit his results to a decent epidemiology journal, such as the American Journal of Epidemiology, Epidemiology, Journal of Epidemiology and Community Health, or even the European Journal of Epidemiology, the one most likely to be sympathetic to antivax ideas given that it’s let John Ioannidis use it as a venue to attack and trash his enemies. Put up, or shut up, Mr. Kirsch. Prove that your results stand up to peer review.
No doubt Mr. Kirsch will demur, claiming that all these epidemiologists are hopelessly “biased.” Same as it ever was.