[NOTE FROM ORAC (June 4, 2020): This study was retracted today, due to issues of discrepancies in the data and a lack of transparency from Surgisphere, the company running the database used. Orac has written a long, detailed post about what happened. He’s not proud of the post below now, but won’t delete or edit it, other than to add this update, because, well, he’s more transparent than Surgisphere.]
So hydroxychloroquine is in the news again. It’s as though it can’t be escaped, probably because it can’t. When President Trump announced earlier this week that he had been taking hydroxychloroquine to protect himself against contracting COVID-19, I briefly debated over whether to write about it but ended up deciding not to, not so much from a sense of the uselessness of doing so but rather due to a profound ennui over the topic, an ennui that, I confess, has hampered my usual ridiculously prolific blog output of late. After all, no matter how much the evidence is trending (and has been for nearly two months now) in the direction that hydroxychloroquine is very likely ineffective against COVID-19 (or, at best, so mildly effective that only large double-blind, randomized, placebo-controlled clinical trials will be necessary to detect any effect) and causes harm through cardiac arrhythmias, the hydroxychloroquine con men, grifters, and cultists are unrelenting. But this morning I’ve been feeling signs of snapping out of it. Why? I don’t know, but one reason might be a study in The Lancet that’s hot off the presses, which I learned about thanks to Eric Topol, that demonstrates increased mortality and ventricular arrhythmias from hydroxychloroquine:
Here’s the study. Again, it’s not a randomized study (none has been published yet), but it is the largest retrospective study to date examining the use of hydroxychloroquine to treat COVID-19. The study is by Mandeep Mehra, a Harvard Medical School professor and physician at Brigham and Women’s Hospital, and colleagues at other institutions. It was a registry analysis that looked at over 96,000 patients with COVID-19 treated with hydroxychloroquine or chloroquine with or without a macrolide antibiotic (e.g., azithromycin) using de-identified data obtained by automated data extraction from from inpatient and outpatient electronic health records, supply chain databases, and financial records. The specific database used is the Surgical Outcomes Collaborative (Surgisphere Corporation, Chicago, IL, USA), which consists of de-identified data obtained by automated data extraction from inpatient and outpatient electronic health records, supply chain databases, and financial records, all taken from 671 hospitals located on six continents.
There are a couple of important things to note about the Surgical Outcomes Collaborative. First, it’s primarily designed to look at cardiovascular disease and outcomes, making it better suited than most databases to examine outcomes related to cardiac events. Second, as described in the study:
The registry uses a cloud-based health-care data analytics platform that includes specific modules for data acquisition, data warehousing, data analytics, and data reporting. A manual data entry process is used for quality assurance and validation to ensure that key missing values are kept to a minimum. The Surgical Outcomes Collaborative (hereafter referred to as the Collaborative) ensures compliance with the US Food and Drug Administration (FDA) guidance on real-world evidence. Real-world data are collected through automated data transfers that capture 100% of the data from each health-care entity at regular, predetermined intervals, thus reducing the impact of selection bias and missing values, and ensuring that the data are current, reliable, and relevant. Verifiable source documentation for the elements include electronic inpatient and outpatient medical records and, in accordance with the FDA guidance on relevance of real-world data, data acquisition is performed through use of a standardised Health Level Seven-compliant data dictionary, with data collected on a prospective ongoing basis. The validation procedure for the registry refers to the standard operating procedures in place for each of the four ISO 9001:2015 and ISO 27001:2013 certified features of the registry: data acquisition, data warehousing, data analytics, and data reporting.
The standardised Health Level Seven-compliant data dictionary used by the Collaborative serves as the focal point for all data acquisition and warehousing. Once this data dictionary is harmonised with electronic health record data, data acquisition is completed using automated interfaces to expedite data transfer and improve data integrity. Collection of a 100% sample from each health-care entity is validated against financial records and external databases to minimise selection bias. To reduce the risk of inadvertent protected health information disclosures, all such information is stripped before storage in the cloud-based data warehouse. The Collaborative is intended to minimise the effects of information bias and selection bias by capturing all-comer data and consecutive patient enrolment by capturing 100% of the data within electronic systems, ensuring that the results remain generalisable to the larger population. The Collaborative is compliant with the US Agency for Healthcare Research and Quality guidelines for registries. With the onset of the COVID-19 crisis, this registry was used to collect data from hospitals in the USA (that are selected to match the epidemiological characteristics of the US population) and internationally, to achieve representation from diverse populations across six continents. Data have been collected from a variety of urban and rural hospitals, academic or community hospitals, and for-profit and non-profit hospitals.
The primary outcome examined was in-hospital mortality, and the study looked for an association between use of a treatment regimen containing chloroquine or hydroxychloroquine, with or without a second generation macrolide like azithromycin, when initiated early after COVID-19 diagnosis, and this endpoint. Secondary outcomes were the occurrence of clinically significant ventricular arrhythmias. or sustained ventricular tachycardia or fibrillation (both life-threatening arrhythmias often requiring cardioversion). Other outcomes examined were rates of progression to mechanical ventilation and total and intensive care unit lengths of stay for patients in each group. A large number of patient demographics and characteristics were also recorded, including age, body-mass index (BMI), sex, race or ethnicity, and continent of origin were obtained. Underlying comorbidities were also recorded, including cardiovascular disease , smoking history, hypertension, diabetes, hyperlipidemia, or chronic obstructive pulmonary disease (COPD), and presence of an immunosuppressed condition, as well as use of medications at baseline, including cardiac medications or use of antiviral therapy other than the drug regimens evaluated.
The initiation of hydroxychloroquine or chloroquine during hospital admission was recorded, including the time of initiation. The use of second-generation macrolides, specifically azithromycin and clarithromycin, was similarly recorded. A quick sepsis-related organ failure assessment (qSOFA) was calculated for the start of therapy (including a scored calculation of the mental status, respiratory rate, and systolic blood pressure) and oxygen saturation (SPO2) on room air was recorded, as measures of disease severity.
Confounding factors were controlled for, including demographic characteristics, comorbidities, disease severity at presentation, and other medication use (cardiac medications and other antiviral therapies).
So what were the results? Not good. Actually, that’s an understatement. Of the 96,032 patients with COVID-19 (mean age 53.8 years, with 46.3% women), 14,888 patients were in the treatment groups, with 1,868 receiving chloroquine, 3,783 receiving chloroquine + macrolide, 3,016 receiving hydroxychloroquine, and 6,221 receiving hydroxychloroquine + macrolide. That left 81,144 patients in the control group. Overall 10,698 patients (11.1%) died in the hospital. Then there was this:
After controlling for multiple confounding factors (age, sex, race or ethnicity, body-mass index, underlying cardiovascular disease and its risk factors, diabetes, underlying lung disease, smoking, immunosuppressed condition, and baseline disease severity), when compared with mortality in the control group (9·3%), hydroxychloroquine (18·0%; hazard ratio 1·335, 95% CI 1·223–1·457), hydroxychloroquine with a macrolide (23·8%; 1·447, 1·368–1·531), chloroquine (16·4%; 1·365, 1·218–1·531), and chloroquine with a macrolide (22·2%; 1·368, 1·273–1·469) were each independently associated with an increased risk of in-hospital mortality. Compared with the control group (0·3%), hydroxychloroquine (6·1%; 2·369, 1·935–2·900), hydroxychloroquine with a macrolide (8·1%; 5·106, 4·106–5·983), chloroquine (4·3%; 3·561, 2·760–4·596), and chloroquine with a macrolide (6·5%; 4·011, 3·344–4·812) were independently associated with an increased risk of de-novo ventricular arrhythmia during hospitalisation.
Or, to boil it down, after controlling for confounders, the investigators found that all four groups treated with anti-malaria drug ± macrolide were more likely to die in the hospital than the control groups receiving neither chloroquine nor hydroxychloroquine. Those receiving hydroxychloroquine were 34% more likely to die, hydroxychloroquine + macrolide 45% more likely, and chloroquine 37% more likely, chloroquine + macrolide 37% more likely. Depending upon the specific group, the treated patients were between 2.37 and 5.1 times more likely to suffer a new ventricular arrhythmia, with the highest risk being among patients receiving Didier Raoult’s combination of hydroxychloroquine plus azithromycin or other macrolide. The authors also did a tipping point analysis to estimate the effects of an unmeasured confounder on the findings of significance with hydroxychloroquine or chloroquine and found:
For chloroquine, hydroxychloroquine, and chloroquine with a macrolide, a hypothetical unobserved binary confounder with a prevalence of 50% in the exposed population would need to have an HR of 1·5 to tip this analysis to non-significance at the 5% level. For a comparison with the observed confounders in this study, if congestive heart failure (which has an HR of 1·756) were left out of the model, it would need to have a prevalence of approximately 30% in the population to lead to confounding in the analysis. Similarly, for hydroxychloroquine with a macrolide, a hypothetical unobserved binary confounder with a prevalence of 37% in the exposed population would need to have an HR of 2·0 to tip this analysis to non-significance at the 5% level. Again, congestive heart failure (which has an HR of 1·756) would need to have a prevalence of approximately 50% in the population to lead to confounding in the analysis, had it not been adjusted for in the Cox proportional hazards model.
In other words, the finding is robust, at least as robust as a retrospective analysis can be, leading the authors to conclude that they saw no benefit, but did see a signal suggestive of a higher risk of cardiac arrhythmias and death in the groups treated with chloroquine and hydroxychloroquine:
In summary, this multinational, observational, real-world study of patients with COVID-19 requiring hospitalisation found that the use of a regimen containing hydroxychloroquine or chloroquine (with or without a macrolide) was associated with no evidence of benefit, but instead was associated with an increase in the risk of ventricular arrhythmias and a greater hazard for in-hospital death with COVID-19. These findings suggest that these drug regimens should not be used outside of clinical trials and urgent confirmation from randomised clinical trials is needed.
Or, as Eric Topol put it:
It is actually interesting to note that, while the incidence of life-threatening cardiac arrhythmias was 2-5 time higher in the treatment groups, the increased mortality was only 34-45%. One possible explanation is that we’re very good at treating such arrhythmias in hospitalized patients on cardiac monitor, so that most of the arrhythmias are reversed before the patient dies. We can’t tell, though, from this study, because, as they authors conceded, they did not study the relationship between ventricular arrhythmias and deaths (i.e., death due to arrhythmia versus death due to other causes). Also, it’s important to note that this study only examined hospitalized patients; so its results can’t be extrapolated to prophylactic use. Of course, to determine the efficacy of prophylactic use of drugs like this would require thousands of patients in a randomized, double-blind, placebo-controlled clinical trial, at least if the primary outcome is death from COVID-19.
Cardiologist Steven Nissen of the Cleveland Clinic said the new data, combined with data from smaller previous studies, suggests that the drug “is maybe harmful and that no one should be taking it outside of a clinical trial.”
Jesse Goodman, a former FDA chief scientist who is now a Georgetown University professor, called the report “very concerning.” He noted, however, that it is an observational study, rather than a randomized controlled trial, so it shows correlation between the drugs and certain outcomes, rather than a clear cause and effect.
Peter Lurie, a former top FDA official who now heads the Center for Science in the Public Interest, called the report “another nail in the coffin for hydroxychloroquine — this time from the largest study ever.”
He said it was time to revoke the emergency use authorization issued by the FDA, which approved the drug for seriously ill patients who were hospitalized or for whom a clinical trial was not available.
I agree 100%. The EUA should never have been issued, but, given that President Trump apparently still believes in hydroxychloroquine, I highly doubt that the FDA will revoke the EUA any time soon. Dr. Topol brings up a more interesting question, though:
He’s right, too. There’s a principle known as clinical equipoise, meaning a genuine uncertainty over whether a treatment is effective and safe or not. For a clinical trial to be ethical, there must be clinical equipoise, because, if there isn’t clinical equipoise, then it is unethical to randomize patients because there will be patients whom investigators knowingly randomize to a group that will receive inferior care. It’s the reason why a randomized, double-blind, placebo controlled trial of the vaccination schedule is unethical; the control group would be randomized to be left unprotected against potentially deadly infectious diseases. In this case, we now know that in hospitalized patients treatment with hydroxychloroquine very likely results in more dangerous arrhythmias and more death, not just from this study but from others.
Of course, as is always the case with treatments like these, be they vitamin C or whatever, the treatment can never fail; doctors and patients can only fail the treatment. What then happens is, inevitably, the claim that the drug or intervention has to be started “early”:
I’m definitely getting a vitamin C vibe now from the hydroxychloroquine cultists, or, as Mark Hoofnagle points out:
Hydroxychloroquine acolytes are definitely well down this progression, at least to #5 and #6. Because I’m dedicated to evidence and science when it comes to medical decision making, I always concede that it is still possible that hydroxychloroquine might still be found to have some anti-COVID-19 activity, although it’s becoming increasingly clear that, if there is any activity it will be modest and require large clinical trials to detect and might be outweighed by potential complications from the drugs, most prominently cardiac arrhythmias. However, my gut feeling, for what it’s worth, is that the drug does not work and has likely killed patients who might otherwise have survived.