I’ve railed on more than one occasion about how much I detest science by press release. For one thing, it bypasses the peer review process and reports results directly to the public, which to me is a strike against any study. Indeed, releasing results by press release or using a press release to tout a study before it’s even published is, as far as I’m concerned, quite dubious, and when I see it I’m automatically way more suspicious of whether the study represents good (or even OK) science.
One form of science by press release that can be a bit more subtle than, say, the cold fusion guys holding a huge press conference to claim that they’ve discovered how to do something that scientists have been trying to achieve for decades, is the press release touting a study that is either ongoing or hasn’t been published yet. Such press releases can serve a valid purpose, for example to encourage recruitment to an ongoing clinical trial or to tout cutting edge research, but it’s very, very easy for such press releases to come across as a bait-and-switch or as a report of success that may not be there because the study isn’t done yet. I came across just such an example of this yesterday in a press release entitled Acupuncture may hold promise for women with hormone disorder. Here’s how it started:
Charlottesville, Va., Sept. 3, 2008 — Getting pregnant with her first child was difficult, but when Rebecca Killmeyer of Charlottesville, Va. experienced a miscarriage during her second pregnancy, she wasn’t sure if she would ever have another baby. When she decided to enter a study testing the impact of acupuncture on women with polycystic ovary syndrome (PCOS) at the University of Virginia Health System, she came out with a miracle.
“To our great surprise we were blessed with a third pregnancy during the PCOS study,” said Killmeyer. “I’m absolutely certain the acupuncture treatments helped me ovulate regularly, which allowed me to become pregnant.”
Lisa Pastore, assistant professor of obstetrics and gynecology at UVA Health System and principle researcher of the study, was hoping for results like this. Her goal has been to help women with PCOS have regular menstrual cycles. PCOS causes a hormonal imbalance, interfering with ovulation and ultimately, fertility. With several women in the study reporting pregnancies, Pastore believes that acupuncture could be an important alternative, non-drug therapy for women with this disorder.
“Over the last year we have seen women who never had a regular menstrual cycle start having regular periods. We can also boast several pregnancies since the study began,” said Pastore. “Now we would like to recruit more people to the study in order to complete the study. It is important for research to have enough participants to ensure that the results are scientifically credible and not due to chance.”
Scared and skeptical was how Killmeyer described her initial feelings towards the experimental treatment, but soon her worries gave way to relaxation.
“When I saw those tiny little needles coming at me I thought to myself, ‘I didn’t sign up for this!’ but I tried it and after a few minutes I was asleep on the table,” Killmeyer said. “The sessions were completely refreshing after awhile.”
After I had seen that headline in my EurekaAlert! newsfeed and then read the press release, I thought Aha! Possible blog material. Let’s go and find the actual study to see if there’s anything to it. So I went looking for the study. After all, I’ve analyzed a number of acupuncture studies before. So I searched PubMed for publications by Lisa M. Pastore.
To my surprise, there was no publication under her name about using acupuncture to treat polycystic ovary syndrome.
My next thought was that maybe it was presented as an abstract or published in a journal not indexed by PubMed. So I searched Google Scholar to see if I could find any evidence of that this study had been published. Nope. Nothing. Nada. Zip. So I did a little more general Googling. The first thing I found was Dr. Pastore’s faculty webpage. I also found other press releases. For example, this one appears to be the one announcing the study. That’s perfectly fine; universities often send out press releases when one of their investigators scores an NIH grant. I have no problem with that. In fact, if (no, when!) I score my next grant, I hope my university will send out a press release about it. Unfortunately, this particular press release was painful to read:
Charlottesville, Va., Nov. 17, 2006 – Can acupuncture normalize hormones in women with irregular periods? The ancient Chinese therapy has long been thought to cure everything from cancer to addictions. Researchers at the University of Virginia Health System have been studying the possibility that acupuncture could stabilize the hormones of women with polycystic ovarian syndrome (PCOS) in the first federally funded acupuncture study at the Medical Center.
PCOS affects 5 percent of reproductive age women. The syndrome is characterized by a hormonal imbalance, interfering with ovulation and ultimately, fertility. Some women with PCOS have small cysts on their ovaries. Others may not have cysts, but may have symptoms such as heavy or irregular vaginal bleeding, male-pattern hair growth and acne. Insulin resistance and pre-diabetes also can occur.
While there are many traditional drugs and therapies that manage this syndrome, acupuncture may succeed in regulating hormones and curing the symptoms of PCOS. In Asian medicine, practitioners believe that everything in the universe forms a unified whole made up of two opposing and complimentary forces called Yin and Yang. Illness is thought to be the result of those forces being out of balance. For the menstrual cycle, that delicate balance of Yin and Yang lies within the kidney meridian. By pricking certain pressure points affecting this region with tiny needles, researchers hope to learn whether acupuncture restores this balance.
Score another one for the Academic Woo Aggregator! The above press release is some serious woo. It has everything you could want in good woo: references to “Asian” or “Eastern” medicine (as opposed, I suppose, to that evil, reductionist, Western, allopathic medicine); a full buy-in to prescientific concepts of Yin and Yang, which, let’s face it, are philosophical and religious concepts, not scientific concepts; and a highly implausible treatment and mechanism. But, wanting to give the benefit of the doubt to Dr. Pastore, I initially assumed that this credulous press release was something put out by the P.R. department at the University of Virginia. Then I read this:
According to Dr. Lisa Pastore, assistant professor of obstetrics and gynecology at UVa Health System, hormones normally go up and down throughout women’s menstrual cycles. In women with PCOS, the hormone levels stay fairly flat.
“In western medicine we tend to think about treatment in terms of making things go up or down and stay there, such as medication to bring down blood pressure or medication to bring estrogen levels back up after menopause,” Pastore said. “But with PCOS patients we want to make those ups and downs happen. We want women to experience their natural cycles and get their rhythm back.”
Ugh. More with the “Western” medicine stuff. Allow me a brief digression here: There is no such thing as “Western” or “Eastern” medicine or “alternative” medicine. There is either medicine that has been shown by science to be effective; medicine that has been shown to be ineffective; and medicine whose efficacy has not yet been demonstrated one way or the other.
There, I feel better now. Back to the topic at hand.
As I said before, I’m not opposed to press releases to announce a study or the award of an NIH grant. The above press release, for all its credulousness and woo, is nothing more than the natural consequence of the NIH funding pseudoscience. When that happens, there really is little reason for universities or the press not to treat press releases about CAM studies the same way they treat any other grants their faculty may obtain that are funded by the NIH. Also, the above press release didn’t really describe results. It was more or less a straight announcement, my dislike of its credulity towards pseudoscience notwithstanding. Again, if the NIH is going to fund stuff like this and accept a rationale like that, then there’s no reason why universities shouldn’t include that rationale in press releases. That’s one reason why I’ve been so disturbed at the sorts of studies that the National Center for Complementary and Alternative Medicine (NCCAM) funds.
Curious, I headed on over to ClinicalTrials.gov to find the study and see how it was designed. As I’ve said before, I used to think there was something to acupuncture; that is, before I actually started reading the scientific literature on acupuncture in depth. One thing I did learn is that the best methodology for studying acupuncture, if study it you must, is to use a double-blinded design. To achieve this the use of sham acupuncture, which involves the use of “sham” needles that do not actually penetrate the skin, the best of which are so well-designed that even the practitioner can’t distinguish them from real. This study appears to use sham acupuncture, although the exact nature of the sham is not described. However, the study is described as “triple-blinded,” which usually means that the researcher, patient, and the person administering the therapy are blinded. (Actually, I don’t like the term “triple-blinded,” because it usually goes without saying in a double-blind study that the researcher and patient are blinded to the.) So, on the surface, this study looks as though it’s probably pretty well-designed, although without details you just can’t tell for sure. Of course, once again, the physiologic mechanism by which acupuncture might modulate hormone levels to correct the symptoms and signs of PCOS is, as far as we can tell, nonexistent, but that’s just par for the course for most CAM therapies.
So what bothered me about the original press release? Certainly there was the dwelling on a single anecdote that made it sound as though acupuncture definitely resulted in Rebecca Killmeyer’s pregnancy. Look at this part of this week’s press release in particular:
Killmeyer learned of her PCOS in 2005. Over the past five years she did not have regular, monthly periods. One month after she started acupuncture treatments she got a period and for the next three months, they continued.
“I had finished all my acupuncture treatments and was in the end stages of the study when I became pregnant,” Killmeyer said. “We had already scheduled our follow-up appt with our fertility doctors when we found out we were pregnant.”
If the study was truly blinded, how did Mrs. Killmeyer know she was getting “true” acupuncture rather than sham acupuncture? Wouldn’t it be ironic if she were in the control group? I’m guessing that, if she’s finished the study, her case was probably unblinded–to her at least. It’s not uncommon to let patients know which group they were in after they have completed the trial. However, having interviewing her for a press release risks unblinding her case to everyone. I just hope the study was designed well enough that the data management system keeps investigators from linking her patient number to her name, something that’s not easy because her clinical history as described in the press release could allow investigators examining the evidence to make an educated guess about her.
In the end, though, the real purpose of this press release appeared to be to recruit more patients to this study. Indeed, the press release says as much. In general, there is rarely anything wrong with trying to promote a study that’s having poor accrual by issuing a press release. However there are some things that should not be in such a press release. This particular release crossed the line when it presented what was, in effect, a glowing testimonial not unlike other testimonials for “alternative” medicine therapies. In doing so the press release strongly implied that acupuncture could do the same thing for patients who might enroll on the study, holding it out as a “carrot” to attract potential recruits. That’s definitely going too far.
In the meantime, I’ll wait for the results of this study to come out to see if it shows anything and whether its conclusions are justified by its methodology and results. I suspect, however, that I’ll wait a long time.
