Autism Medicine Quackery

One last word on the Geiers: So good it shouldn’t be buried in the comments

I was going to give this a rest for a while, but this is too good not to post a brief note about.

Posted in the comments of my piece debunking the Geiers’ pseudoscience and their laughable “scientific” article claiming to show a decrease in the rate of new cases of autism since late 2002, when thimerosal was removed from vaccines completely other than some flu vaccines was this gem of a comment, by one MarkCC, which stated the essence of what was wrong with the Geiers’ so-called “statistical analysis” of the VAERS database:

Here’s the key, fundamental issue: when you’re doing statistical analysis, you don’t get to look at the data and choose a split point. What the Geiers did is to look at the data, and find the best point for splitting the dataset to create the result they wanted. There is no justification for choosing that point except that it’s the point that produces the result that they, a priori, decided they wanted to produce.

Time trend analysis is extremely tricky to do – but the most important thing in getting it right is doing it in a way that eliminates the ability of the analysis to be biased in the direction of a particular a priori conclusion. (In general, you do that not to screen out cheaters, but to ensure that whatever correlation you’re demonstrating is real, not just an accidental correlation created by the human ability to notice patterns. It’s very easy for a human being to see patterns, even where there aren’t any.)

Redo the Geiers analysis using any decent time-trend analysis technique – even a trivial one like doing multiple overlapping three-year regressions (i.e., plot the data from ’92 to ’95, ’93 to ’96, ’94 to ’97, etc) and you’ll find that that nice clean break point in the data doesn’t really exist – you’ll get a series of trend lines with different slopes, without any clear break in slope or correlation.

So – to sum up the problem in one brief sentence: in statistical time analysis, you do not get to pick break points in the time sequence by looking at the data and choosing the break point that is most favorable to your desired conclusion.

Exactly! Unfortunately, that’s exactly what the Geiers did.

A proper statistical analysis of such data, looking for time points at which a rate of change in a variable changes, is designed such that there is no bias in selecting a time point at which a significant change in slope is observed. As much as the Geiers might want to believe that there is a marked change in the slope of the curve beginning around late 2002 to early 2003, they can’t assume that there is such a breakpoint before doing the analysis.

Once again, what pseudoscientists like the Geiers never seem to understand is that all those precautions we scientists take with control groups and statistical analyses designed to minimize investigator bias exist because we realize how easy it is for a scientist, particularly a medical scientist who is invested in finding a cure for a particular disease or condition, to be seduced into believing something that is not supported by data. (If they did understand, they wouldn’t use such simplistic and easily debunked “scientific” methodology.) It’s a very human tendency, and the scientific method is designed to minimize that tendency. That’s why it takes so much training to overcome.

Some scientists never do overcome this tendency, and if they fall deeply enough into belief over evidence they become pseudoscientists.

Like the Geiers.

Thanks, MarkCC.

By Orac

Orac is the nom de blog of a humble surgeon/scientist who has an ego just big enough to delude himself that someone, somewhere might actually give a rodent's posterior about his copious verbal meanderings, but just barely small enough to admit to himself that few probably will. That surgeon is otherwise known as David Gorski.

That this particular surgeon has chosen his nom de blog based on a rather cranky and arrogant computer shaped like a clear box of blinking lights that he originally encountered when he became a fan of a 35 year old British SF television show whose special effects were renowned for their BBC/Doctor Who-style low budget look, but whose stories nonetheless resulted in some of the best, most innovative science fiction ever televised, should tell you nearly all that you need to know about Orac. (That, and the length of the preceding sentence.)

DISCLAIMER:: The various written meanderings here are the opinions of Orac and Orac alone, written on his own time. They should never be construed as representing the opinions of any other person or entity, especially Orac's cancer center, department of surgery, medical school, or university. Also note that Orac is nonpartisan; he is more than willing to criticize the statements of anyone, regardless of of political leanings, if that anyone advocates pseudoscience or quackery. Finally, medical commentary is not to be construed in any way as medical advice.

To contact Orac: [email protected]

Comments are closed.


Subscribe now to keep reading and get access to the full archive.

Continue reading