The safety and efficacy of hydroxychloroquine for the treatment of COVID-19 has quickly become an important medical question in managing this pandemic, although not by far the most important. There are many drugs under consideration, and some with promising early results. But hydroxychloroquine has garnered the majority of attention for purely political reasons. I most recently wrote about the scientific evidence for hydroxychloroquine on May 18th, referring to four studies all showing no benefit. Since then there have been more studies, including this one in NEJM showing no benefit from hydroxychloroquine in terms of preventing the contraction of COVID-19. Systematic reviews, which are being done in an ongoing manner, also conclude no benefit from this drug.

But at the end of my May 18th blog post, on May 22nd, I added a brief addendum because another study had just come out I thought was worth noting – a multinational study which compiled evidence from 120 different hospitals involving over 90,000 patients. This study found no benefit but significantly an increased risk of heart complications and death from hydroxychloroquine. If you follow this link now you will see a giant “retracted” posted over the study. The Lancet reports:

But in an open letter last week, a group of scientists raised “both methodological and data integrity concerns” about it.

These included a lack of information about the countries and hospitals that contributed to the data provided by Chicago-based healthcare data analytics firm Surgisphere.

One other hydroxychloroquine study used data from Surgisphere, this one published in the NEJM, and has also since been retracted. So what happened and what does all this mean?

First, at this point, there is no evidence that there was any political motivation to this fiasco. The company, Surgisphere, has been in existence since 2008. Last year they started a new service in which they claim to gather medical information from electronic medical records (EMR) systems from 120 participating hospitals. This allows them to gather de-identified information in almost real time. They also claim to have AI algorithms to help them analyze all that data. This claim itself is not implausible. In fact, this was one of the great promises of EMR, one that has not been fully realized. They have a number of studies published on their website on various topics.

However, as soon as the large hydroxychloroquine study was published in the Lancet, some experts notices discrepancies. The number of deaths in Australia, for example, did not match the number published elsewhere. The more experts looked at the data the more curious it became. The overall death rate from COVID-19 was too high, for example. Then they started looking into the company itself, and asking for the raw data. Surgisphere, it turns out, only had 6 employees (and three dropped off the list, so there is officially only 3). Two of the remaining three do not appear to have any science or programming credentials. It seems implausible that such a small company could be managing such a massive database. Plus, it would have taken years to arrange for all those contracts with 120 hospitals. The whole thing appears to be a scam.

The Guardian did an independent investigation of the company and found that they are dubious. The head, Desai, for example, also ran an Indiegogo campaign for  “next generation human augmentation device that can help you achieve what you never thought was possible”. The campaign failed.

We are still early in the investigation of Surgisphere, but from what is known at this time my guess is that this is a dubious startup, making semi-plausible claims but not having the reality to back it up. Perhaps they were hoping to make a splash as a way of fishing for investors (a common ploy). The whole thing sticks of the Theranos episode.

What does all this mean for the question of hydroxychloroquine and COVID-19? In terms of efficacy, nothing. There are many other studies and the overwhelming consensus so far is that hydroxychloroquine lacks any efficacy in treating or preventing COVID-19. The retracted studies were in line with this conclusion and neither they or their retraction change it. But the retraction of the Lancet study does change the prevailing conclusion regarding the safety of hydroxychloroquine. Most studies find no benefit, but also no harm attributable to hydroxychloroquine. There are legitimate concerns about increased heart complications, but this has not translated into a statistically significant difference in death or other major outcome. The Lancet study was really the first to do so, which was attributed to the fact that it was the most powerful study to date. With that study retracted, we are back to – no benefit, but no significant evidence of harm either.

All this means that further studies of hydroxychloroquine are at least potentially safe. The WHO had cancelled some hydroxychloroquine studies based on evidence of harm, but those studies are now back on. We can debate whether or not they are necessary, but at least they are safe enough to conduct.

Unfortunately, because hydroxychloroquine is already a political issue, the Surgisphere scandal is being interpreted politically, as if it is a political hit-job against hydroxychloroquine. There is no evidence for that. A full investigation is under way, so time will tell exactly what happened, but the simplest explanation at this point is that the company is just a dubious startup. But also we should not how quickly these studies unraveled. Studies of COVID-19 are being fast-tracked because we are in the middle of a pandemic. This is not an excuse, but does contribute to perhaps more problematic studies slipping through the cracks. Part of the investigation will involve the failure of the editorial and peer-review processes at the Lancet and NEJM.

But we should note that “peer-review” does not end with publication. That is when expert crowdsourcing begins. In this case experts began examining the data for flaws, and found them. They then asked the right questions, and within a couple of weeks exposed the massive problems with Surgisphere and these studies, leading to their retraction. So while publication may have been fast-tracked, so was exposure and retraction. The entire episode is unfortunate, and should never have happened, but in fairness it not only exposed the flaws in peer-review, but the strength of the scientific community and the self-corrective process of science.

And in the end – hydroxychloroquine still does not work for COVID-19.