Last week, The New York Times started a rather unusual series in its medical section entitled, The Evidence Gap, described thusly:
Articles in this series will explore medical treatments used despite scant proof they work and will consider steps toward medicine based on evidence.
When I first saw the series, I was prepared for a crapfest. My experience has generally been that when reporters start examining the evidence for and against a treatment they usually do a pretty lousy job. This is most obvious when it comes to “complementary and alternative medicine” (CAM), where we are routinely treated to utterly credulous articles touting the latest poorly designed study demonstrating that this or that fairy dust cures cancer and the common cold (I exaggerate, but not by much) or “exposes” on the latest misdeeds by big pharma, in which risks are either exaggerated or not put into perspective. Oddly enough (or not so oddly enough) seldom does the misinformation go in the other direction, with excessive credulity towards pharma-produced evidence and excessive skepticism for evidence relating to the efficacy and safety of CAM modalities, but that’s a topic for another day. I don’t so much mind the tighter skepticism regarding the claims of big pharma, given its history; I only wish that similar skepticism applied to the claims of big CAM. In any case, the point is that generally medical reporting tends to be very poor at doing a fair, objective analysis of the state of the actual evidence regarding a treatment modality, so much so that I sit up and take notice when I find an exception.
Depending on how the the rest of the articles in the series play out, this New York Times series may very well be one of those exceptions. As a maven of science- and evidence-based medicine, I approve–thus far, anyway. The first two entries are on cardiovascular testing and cancer chemotherapy. I’ll spend the most time on the latter, but I can’t resist a brief treatment of the former before I get to the latter.
The first installment in the series was Weighing the Costs of a CT Scan’s Look Inside the Heart, which provided a mostly balanced look at the latest bandwagon fad among cardiologists, CT angiography. Unfortunately, the article gave Dr. Harvey Hecht of the Lenox Hill Heart and Vascular Institute of New York license to say some spectacularly stupid things, like:
“It’s incumbent on the community to dispense with the need for evidence-based medicine,” he said. “Thousands of people are dying unnecessarily.”
Bullshit. More like there are hundreds of ideologue doctors with a financial incentive that distorts their objectivity convinced that their shiny, expensive new machines are the latest and greatest–doctors like Dr. Hecht, who has, as far as I can tell, not produced one whit of evidence that using his device to guide therapy saves even a single life.
Fortunately, the NYT actually did a pretty good job of pointing out the price and lack of evidence supporting the widespread use of these scans for screening. When used indiscriminately, they also could risk leading to more unnecessary angioplasties and coronary artery stenting in much the same way that ever more sensitive tests for cancer can lead to overtreatment. Unfortunately, Dr. Hecht appears no better than woo-bosters Dr. David Katz, who wants a “more fluid concept of evidence” supporting CAM or Dr. Andrew Weil, who wants to emphasize “uncontrolled clinical observations” (i.e., anecdotes) in evaluating evidence. The only thing different is what kind of modality he’s trying to change the rules to support.
Cardiologist blogger Dr. Wes, as well as medical bloggers Matthew Holt, Dr. Ray Poses, and Dr. Guy Clifton have already ripped Dr. Hecht a much-deserved new one for his idiocy. I also happen to know that Dr. Harriet Hall over at Science-based Medicine is planning an installment for tomorrow. The issue is well-covered, even by medical students, and I’ve also touched on many of the same issues before the last time this sort of technology was overhyped. From my perspective, the only shortcoming of the NYT article is that it didn’t adequately discuss the question of what the best science-based recommendations are for when a CT-angiogram is appropriate.
More interesting and relevant to me is the second article in the series, Costly Cancer Drug Offers Hope, but Also a Dilemma. It’s right up my alley in that (1) it’s about cancer and (2) it’s about my area of research interest, angiogenesis inhibitors. Specifically, it’s about one angiogenesis inhibitor in particular, namely Avastin.
Avastin is the trade name for bevicuzimab, which is a humanized monoclonal antibody against a protein called vascular endothelial growth factor (VEGF). A “humanized” antibody is an antibody raised in mouse cells the nonspecific or invariable part of which (in other words, the part that doesn’t recognize and bind to the target antigen) have been replaced with its human counterpart in order to reduce the likelihood of an immune reaction against a foreign protein. VEGF is a protein that is secreted by both normal cells and cancer cells in order to stimulate the ingrowth of new blood vessels, a process known as angiogenesis. In the case of normal cells, VEGF is only secreted in response to injury or a physiologic need for more nutrients and oxygen. Tumors, on the other hand, hijack this normal physiological function to their own nefarious ends, often by cranking out huge quantities of VEGF or other angiogenesis-stimulating proteins to feed their ravenous glucose and oxygen requirements. Targeting tumor angiogenesis as a means of “starving” tumors has been an area of my research interest for over ten years now, and it is the area of research pioneered by one of my scientific and surgical heroes, the late Dr. Judah Folkman, who demonstrated the feasibility and effectiveness of antiangiogenesis as a therapeutic strategy against cancer.
Antiangiogenic therapy is attractive because, in theory at least, it could target almost any solid tumor. In addition, in the mouse and, as we now know, in humans it is a therapy that generally produces low toxicity, at least compared to chemotherapy. Hypertension, edema, poor wound healing, and, paradoxically, bleeding appear to be the most common side effects. Another less common but very serious side effect is gastrointestinal perforation, thought to be due to compromise of the blood supply to an area in the GI tract. Still, all in all, the drug has a favorable side effect profile compared to many cytotoxic chemotherapy regimens, and it’s been shown to be effective in prolonging life for certain metastatic tumors, particularly when combined with chemotherapy, a phenomenon I’ve described before.
The problem, as the NYT article points out, however, is that Avastin is very, very expensive. It can cost as much as $100,000 a year, depending on the specific tumor, which influences the dose regimen necessary. It’s here, as the article describes, where a question that we as Americans find far more uncomfortable than perhaps many other nations with nationalized health care systems do, rears its ugly head: How much are an additional few months of life worth?
Antiangiogenic therapy was all the rage 10 years ago in the research realm. Dr. Folkman’s dramatic results in mice using angiostatin and endostatin fueled hype in the mainstream media that a “cure” for cancer was at hand or that, at least, Dr. Folkman’s dream of turning cancer into a chronic, manageable disease like diabetes was finally being realized. Avastin was the first of the new generation of antiangiogenic drugs to make it into widespread clinical testing, thanks to the efforts and R&D money of Genentech. Unfortunately, its results when used alone against various cancers were disappointing to nonexistent. Fortunately, against metastatic colorectal cancer, combining Avastin with standard chemotherapy prolonged median survival from 15.6 months to 20.3 months. While chemophobes (as I like to call those who can’t ever admit that chemotherapy does any good) would scoff, I’d point out that that represents a 30% increase in survival, generally with good quality of life.
However, as the article points out, newer chemotherapy regimens seem to be more effective on their own and less synergistic when combined with Avastin:
In February 2004, 15 years after Dr. Ferrara’s initial discovery, the Food and Drug Administration approved Avastin for patients with advanced colon cancer. A blockbuster was born.
But now there is a question mark over that evidence. That first exciting result compared Avastin with a type of chemotherapy that has since been widely replaced by a more effective regimen.
In a later, larger study comparing Avastin with current chemotherapy, Avastin slowed the growth of tumors but did not extend life by an amount considered statistically significant.
Although GI cancers are not my specialty, I do know enough to say that the therapy of metastatic colorectal cancer and the evidence supporting it is in a state of flux, and this may indeed be the “true” case rather than a fluke in a study. Still, the evidence for the efficacy of Avastin is probably the strongest for colorectal cancer, both in terms of magnitude of the benefit in terms of the increase in life expectancy and quality of evidence. On less compelling evidence, Avastin was also approved for lung cancer, as well.
Perhaps the approval for Avastin that caused the most controversy was one that occurred earlier this year. I really should have written about it at the time, but for some reason I never did. Here’s what how the article describes the approval:
The third approval for Avastin, for advanced breast cancer, came in February of this year. The clinical trial found it significantly slowed the progression of cancer but did not significantly extend life. The F.D.A. went against its own panel of outside experts, who had voted 5 to 4 against approval.
The agency’s action has not sat well. Senator Charles E. Grassley, Republican of Iowa, asked the Government Accountability Office to look into the F.D.A.’s approval of Avastin and some other drugs that “appear to have little to no effect in protecting lives and increasing health.”
Dr. Lee Newcomer, an oncologist and executive at the insurer United HealthCare, said patients were not well served, and neither were insurers, nor the public, which ultimately foots the bill. If a drug just stops tumor progression, without the woman’s living longer or feeling better, without her noticing anything different, Dr. Newcomer said, “you’re treating an X-ray.”
I must admit that I was even skeptical of this approval. Basically, what is being discussed here is whether a drug affects overall survival (OS), which is mortality from all causes in a cancer patient, versus whether it affects progression-free survival (PFS), which is the period of time before a given tumor progresses. Surprisingly, at least to non-oncologists and lay people, OS and PFS are often unrelated. If, for example, a drug slows tumor growth sufficiently to demonstrate a significant affect on PFS, it doesn’t necessarily mean that OS will be better too. Sometimes it will, sometimes it won’t. Moreover, it’s long been a debate over whether PFS is a valid endpoint for approving a drug. Traditionally, the thinking has been that if a drug does not improve OS, then it probably shouldn’t be approved as a first line agent given up front to new cancer patients who have not been treated yet, although it can be approved as a second-line or third-line agent, to be tried after first line agents fail.
However, thinking has been evolving over the last few years towards accepting a somewhat looser standard of valuing PFS. Dr. Kathy Albain of Loyola University Medical Center and the Cardinal Bernardin Cancer Center, a passionate defender of using using PFS as an endpoint, makes the argument this way:
“Assuming that there is a real treatment benefit for whatever agent you are testing, I believe that PFS should be accepted as a proper surrogate,” Albain maintained. She pointed out that in metastatic breast cancer in particular, it may never be possible to prove surrogacy for overall survival, given the thousands of patients that would be needed.
Says Dr. David Miles, lead investigator on the AVADO study on Avastin and breast cancer:
“As a practicing physician,” AVADO lead investigator David Miles said he views PFS as a reflection of the period of time where the disease is controlled and the patient remains relatively well. There is an expectation that the patient is “doing better than if their disease is not controlled,” he told the press briefing.
“We have studies where PFS does reflect overall survival and studies where it doesn’t,” Miles observed. “I think when you have a patient whose disease is well-controlled, there is benefit to that, there is utility to that. Particularly when studies are not powered for overall survival.” None of the Avastin breast cancer studies have been designed for a full survival analysis.
Indeed, Dr. Albain polled oncologists about whether they accept PFS as a valid endpoint, and this is what she found:
Albain took it upon herself to conduct a survey, sending out a questionnaire to a sample of 47 experts. She was overwhelmed by the volume and vehemence of the replies. The clear consensus was that PFS should be considered a meaningful approval endpoint. Getting to “the crux of the matter,” Albain’s group suggested PFS should be used in Phase III trials, but maybe it should be prospectively determined how large a benefit is desired. For full approval perhaps two trials should be required, or perhaps there should be a mandatory cross-over design.
In a world where cost is no option, I’d agree. However, resources are not endless, and one has to ask how much a few months of PFS without a concomitant increase in OS are worth. I don’t know the answer to that one, as this is a very difficult debate that we in the U.S. have thus far been able to avoid. Our colleagues in nations with nationalized health care systems cannot avoid it, though. For example, in Canada, it’s been estimated that the addition of Avastin to breast cancer and lung cancer treatment would add $299 million a year to Canada’s health care costs. In a country like Canada, the only choices are to cut money out of other programs to pay for this or to raise taxes. In the U.S., our insurance premiums just go up.
These issues are difficult, and the NYT article gives a fairly balanced overview up until this point. However, when it comes to discussing off-label use of Avastin, the bioethical and financial issues become harder still:
Doctors are free to prescribe Avastin, or any other drug on the market, for unapproved uses, at their discretion. As much as 75 percent of cancer drug use is of this “off label” variety, according to an estimate by the National Comprehensive Cancer Network, a group of big cancer centers. And some doctors say that with patients dying, they simply cannot wait for airtight evidence.
“Of course we want everything to be evidence-based,” said Dr. Yashar Hirshaut, an oncologist in Manhattan. “I also like the American flag and apple pie.”
But, he explained, “You say, ‘This person is dying right here and I need something that will help, and there’s a logical construct that I can see how it will help.’ “
I must point out that in this case “off-label” use does not usually mean non-evidence-based. What it usually means is that the evidence supporting the treatment choice is weaker or that the clinical trials haven’t been done yet to show if a therapy works or not. Even the most die-hard advocate of EBM understands that if you have a patient in front of you dying right now, you’re willing to try something for which the evidence may not be particularly strong. That’s why I’m not making sarcastic comments about Dr. Hirshaut the way I did about Dr. Hecht; the two situations are apples and oranges.
The problems, however, are two-fold: First, there’s a financial incentive on the part of the drug company to encourage such off-label use, or at least not to object to it. Second, off label use makes doing the correct clinical trial difficult, and the NYT article describes one way how:
But the anecdotes and evidence from small trials that may seem to justify off-label use sometimes turn out to be misleading. That happened with pancreatic cancer. After patients and doctors decided Avastin had to be helping, cancer researchers themselves conducted a large study. So did Roche. Avastin, both studies concluded, did not prolong life for people with cancer of the pancreas.
The moral of the story above? Anecdotes can deceive “conventional” physicians as much as they can CAM practitioners. Never forget that. This is in reality not much different than the issues I discussed about whether or not patients have a right to experimental therapy. The only difference is that it’s much easier to prescribe drugs off-label than it is to prescribe experimental therapy.
One thing that bothers me about this trend is indeed the potential harm it can do to science- and evidence-based medicine. One can make an argument that perhaps PFS is a valid endpoint for approving a cancer drug (if we’re willing to pay for it), but it’s hard for me to accept arguments like this:
As word spread, Dr. Friedman at Duke and Genentech organized studies of a type generally considered less than definitive. There was no control group that took another drug or got a placebo. Everyone got Avastin. Otherwise, no one would enroll in the study, doctors argued.
Then the investigators compared the results with what they thought would have happened without Avastin. The patients lived a median of about nine months, about three months longer than the researchers estimate would have been expected.
But such comparisons have led scientists seriously astray in the past because the people being treated with a new drug often are very different from previous patients who did not take it and because overall medical care steadily improves. Nonetheless, Genentech has said it planned to apply this year to the F.D.A. for approval for Avastin to treat glioblastoma, the deadliest form of brain cancer.
Single-arm studies like this using historical controls can sometimes be used as valid evidence, but the difference between the treated patients and historical controls have to be much, much more impressive, to the point where nothing other than a major effect from the drug is a reasonable explanation for the improvement in survival. After all, selection bias alone could potentially account for a 3 month survival benefit compared to historical controls. Trying to argue for approval of a drug on the basis of such thin gruel isn’t in essence any different than Andrew Weil or David Katz, either.
With complex medical technology and expensive drugs whose benefits seem modest in relationship to their price, our health care system appears to be heading for a crisis. We in the U.S. absolutely hate to address real world questions such as whether how much these drugs cost is worth their benefit. A cancer patient will often answer a resounding yes. We’d all like to say yes to that question as well. Unfortunately, such idealism comes crashing against fiscal reality, and it’s not clear where we will draw the line and to which patients we will end up having to deny potentially life-prolonging treatments because they are just too expensive.
