In this latest contribution to Croakey, the Naked Doctor columnist, Dr Justin Coleman, looks at a confronting paper recently published in the JAMA.
Dr Coleman writes:
An astonishing article published recently in JAMA (1) flew largely under the radar, but is an extreme example of why doctors should exhibit caution when relying on individual trials for evidence about a medication, particularly where the trial sponsor stands to gain from the results.
We owe a debt to Charles Seife, Professor of Journalism at New York University, who painstakingly tracked down research trials which severely breached acceptable standards. What he discovered is shocking.
Background
The US Food and Drug Administration (FDA) conducts regular inspections of clinical trial sites, two per cent of which result in the worst category of warning, called Official Action Indicated (OAI). To earn an OAI, the trials must do significantly unconscionable things such as falsify results, deliberately ‘unblind’ a blinded trial, or destroy records of adverse outcomes. The word ‘fraud’, while not legally conferred by the OAI rating, is at least sniffing around the neighbourhood.
Two per cent probably isn’t too awful. Simply throw out these bad apples—never to enter the medical literature—and get on with the other 98 per cent. But clinical trials are expensive things, and favourable outcomes are highly profitable, so it turns out most of the two per cent continue on to publication, regardless.
In which case one would expect, as a bare minimum, a clear caution that the trial had received an OAI rating.
Charles Seife, data hunter.
Remarkably, there is no easy way of finding out how often this public admission actually eventuates. In fact there’s not even a ‘not-so-easy’ way. Luckily for us, Prof Seife discovered a ‘ridiculously hard’ way, and tenaciously followed the trail like a sniffer dog.
The FDA does not publish a list of OAI ratings (a concern in itself), so the author used a Freedom of Information request and exhaustive Googling to track down infringement notices one-by-one. Even then, heavy redactions were so widespread that most of the trials were completely unidentifiable. The author eventually identified about a sixth of the OAI trials, and looked at the 78 publications in medical journals about those trials.
What was the result?
So, the big question – of those 78 publications, how many mentioned the fact that their trial site had received notice of OAI, the most severe FDA breach?
None.
Although three alluded to it.
The three admissions were couched in vague terms – “site monitoring raised questions in regard to certain data” – but at least the reader had some means of guessing that something smelled fishy.
The other 75 smelled like roses.
Is it important?
Fraud, misconduct and incompetence matter very much to patients. These 78 publications (and the hundreds of unidentified ones) are, right at this moment, being peddled to doctors around the world, cited as reasons why they should prescribe a particular medication.
These doctors are good doctors. They want to see the evidence, not just the marketing. The evidence is all there in front of them…it’s just that the doctor has no way of even suspecting that the FDA found severe faults in the trial.
Example 1.
Rivaroxaban (Xarelto) is a novel anticoagulant (warfarin alternative) and one of the drugs most heavily marketed to me, as a GP, in recent years. You can read all about its seminal RECORD trial on the manufacturer’s website.
“XARELTO® was studied extensively in well-controlled trials” says the first paragraph, “designed with scientific rigor in mind.”
What the manufacturer doesn’t mention – nor does any author in any of its journal publications –is the fact that eight of the 16 FDA inspections warranted an OAI warning. Not just one trial site with one rogue researcher – eight separate RECORD inspections copped the severest warning level!
The various inspectors found falsification, deliberate unblinding, and systemic discarding of medical records. It’s a bit like finding eight positive urine samples in the one cycling team. Yet until Prof Seife trawled through the mountains of paper trails and published last week, no doctor would have had the slightest inkling of a problem.
Even now, the vast majority of orthopaedic surgeons and GPs prescribing rivaroxaban to prevent DVTs will remain entirely unaware that the RECORD trial has such a dubious history.
Example 2
A trial site for another novel anticoagulant, apixaban (Eliquis) was found to have actively altered patient records. The FDA recommended that data from this site be excluded from analysis (along with 23 other suspect sites). The problem for the company was that exclusion of data from that one site tipped the mortality benefit of the drug from statistically significant to non-significant.
So, in order to protect the public by publishing the truth, what happened?
Again…nothing.
Not only was no admission published regarding the OAI rating, the false data were actually included, which made the published mortality outcomes appear better than they were. The fraudulent data demonstrating statistical significance has been published, reanalysed and published again 18 months later. It was used as recently as August 2014 to gain FDA approval for the drug label.
Statistical significance in the multi-billion-dollar anticoagulant market is the difference between laying rotten eggs and golden eggs.
Doctors, even those cautious sceptics who care deeply about evidence, are therefore making prescription decisions based at least partly on cheating.
The researchers were caught out by the umpire, banned from the field, snuck back on while no-one was looking and won man-of-the-match while everyone except one investigative journalist stood by.
It’s only two per cent of trial sites, but what a fundamentally rancid two per cent! Someone needs to do something about the smell.
Seife, C. Research misconduct identified by the US Food and Drug Administration: : Out of sight, out of mind, out of the peer-reviewed literature. JAMA Intern Med. Published online February 09, 2015. doi:10.1001/jamainternmed.2014.7774. Avail at http://archinte.jamanetwork.com/article.aspx?articleid=2109855
Justin Coleman
