The FDA has a big problem. It likes to approve new drugs and devices. Approvals look good politically and allow the FDA to boast each year about their large number of approvals. But when the FDA approves a drug or device that doesn't work or is dangerous, then it can get into hot water, fast.
Most recently, the FDA has caught fire for its approval of the muscular dystrophy drug from Sarepta. FDA Commissioner Robert Califf, MD, received searing criticism from high-ranking FDA staff members, including Ellis Unger, MD, the acting director of the large FDA office that reviews new neurology drugs.
"I think it will be important for the regulatory record to reflect that there was no scientific basis underlying the conclusion of 'reasonably likely' in this case. This was simply a judgment call by Dr. Woodcock," wrote Unger, as reported by . Unger continued: "Perhaps granting accelerated approval to drugs that show a mere scintilla of an effect on a surrogate endpoint represents a stroke of brilliance – one that will stimulate investment in the development of drugs for these disorders. But in my opinion, this approach should receive broader public (and FDA) input before being implemented."
One of the surprising aspects of this story is Califf's role. The constant and dominant themes of his long career prior to joining the FDA was his support for randomized controlled trials (RCTs). Prior to his approval as commissioner, . It now looks like I was wrong.
Cardiac Devices On Shaky Ground
The approval of a whole new generation of cardiac devices illustrates the FDA's new laxity. What would happen if one of these new devices blew up in a big public way?
, (Yale University) points to a whole list of new cardiac devices that have been approved despite either equivocal or negative findings in their pivotal trials:
- Absorb bioresorbable scaffold stent (BVS): It demonstrated noninferiority, but all the endpoints went in the wrong direction.
- CardioMems: There were significant interactions with patients by study nurses employed by the company that might have biased findings. Although the FDA requested a new trial, the company just continued the initial study after blinding had been broken.
- Watchman: It did not meet both co-primary endpoints in PREVAIL.
- Mitraclip: EVEREST II was initiated as the pivotal study, but then the approval decision was made based on the REALISM high-risk registry data.
- Impella: PROTECT II prematurely terminated for futility and safety concerns. As Ross and co-authors wrote in their excellent , the device was primarily approved on no greater mortality risk compared to intra-aortic balloon pumps, although these are no longer recommended in some guidelines and may have no benefit.
- Amplatzer: The prespecified intent-to-treat analysis of RESPECT found no statistically-significant ischemic stroke reduction and, therefore, the primary endpoint was not met. But the FDA deemed the 50% relative risk reduction "clinically meaningful."
It wasn't so long ago that Califf wanted to hold interventional cardiologists to higher standards. "I used to really enjoy making fun of interventional cardiologists as knuckle dragging individuals who didn't know much about anything except about how to work in the cath lab," he said at a large interventional cardiology meeting in Washington, D.C., last week. But now, as reported by Crystal Phend in 鶹ý, he has a different view: "It's amazing what's happened. You're now the paradigm of evidence generation," he told the interventionalists.
But it's important to point out that Califf isn't really suggesting that interventionalists have raised their standards. Instead, Califf has simply moved the goal posts and made it easier for the people who test devices to generate "evidence." Prior to joining the FDA, Califf's "paradigm of evidence generation" was the RCT. But RCTs are playing a diminished role in the development and approval of cardiac devices.
It's only responsible to point out that Califf can't be held responsible for every single FDA decision, and many of the cardiac devices mentioned below were approved before he became commissioner. But the Sarepta case suggested that Califf is likely to value bureaucratic and political expediency over principles. In his explanation for his Sarepta decision, Califf said that he was not an expert in muscular dystrophy. But as a leading cardiologist, he can't use that excuse when it comes to cardiac devices. So he bears at least some of the responsibility for the recent approvals under his watch of the Absorb bioresorbable stent and the Amplatzer patent foramen ovale (PFO) closure device.
Drugs and Devices
It is important to first understand that there are significant regulatory differences in approving drugs and devices. Devices, in general, do not require the same level of evidence as drugs.
, (Cedars-Sinai) explained in some detail why the FDA has modified its approach to device approvals:
"In response to the criticism, driven primarily by the industry and investigators, that the U.S. regulatory system is too slow, risk averse and expensive, recent reforms have been introduced in the United States that are principally focused on streamlining the regulatory processes, enhancing postmarket regulation through more robust surveillance systems, and improving the traceability and monitoring of devices. The driving factor has been seeking parity with the European system, which is somewhat faster providing earlier access to some high-risk technologies such as TAVR [transcatheter aortic valve replacement], MitraClip, and Watchman device. Although all of these are a step in the right direction, additional actions need to be considered. Chief among these is requiring high-quality evidence of benefit from medium- and high-risk devices. In an effort to provide earlier access to devices, the FDA appears to have shifted the statutory burden of proof of reasonable assurance of safety and effectiveness (RASE) from exclusively pre-approval stage to a lifecycle approach with greater emphasis on post-approval studies, especially the 'reasonable assurance of effectiveness' component. In addition to RASE being a lower evidentiary standard than the 'substantial evidence of safety and effectiveness' required for approval of drugs and biologics, the problem with the post-approval approach (as acknowledged by the FDA) is that 'many of the FDA-mandated postmarket studies have been delayed, scaled back or never finished', as . So, it is not clear how effective these postmarket studies are going to be in providing sufficient safeguards for technologies that affect morbidity, mortality, and health-related quality of life."
"For the life cycle approach to provide an effective 'checks and balances', there should be a complete and transparent accounting of the post-approval studies. If these studies are not completed in a timely manner or they do not confirm that the benefits outweigh risks in the intended population, then the approval should be revoked. Making a good-faith effort in performing postmarket studies should no longer be a sufficient safeguard against penalty by the FDA."
The FDA has approved the new cardiac devices, noted Dhruva, based on the weaker regulatory requirements for devices. The FDA's case is that "despite the limitations of the RCTs," the FDA "found sufficiently compelling data for approval of all these devices and is bound by statute to require the 'least burdensome' data."
Dhruva agreed with Kaul: "The key becomes ensuring that post-approval commitments are fulfilled in a timely manner. And, just as importantly, if those post-approval studies show no benefit -- or worse, that harms outweigh benefits -- the FDA should revoke approval." The problem is that "postmarketing studies are often not completed in a timely manner. Further, it is quite hard to revoke approval once a device has been adopted into the repertoire of clinical practice."
For now, the potential problems with the cardiac devices approved on weak grounds remain theoretical. But if any of those devices turn out to be harmful, the backlash could be powerful and immediate. Currently, the most worrisome device is the Absorb stent, especially since . One interventional cardiologist, who tweets under the name "Jeddacath," said he wouldn't "sugarcoat" the data and call it "worrisome."
"Cardiologists should really think hard on why they would use it in a given patient," he said. "I expect 1-800-SUE-ME soon asking patients 'Did you get a BVS and have a heart attack? Then call us now.'"