by Ben Goldacre
Surrogate outcomes
Placebo controls are not the only problem with the trials that are used to obtain marketing approval. Often, drugs are approved despite showing no benefit at all on real-world outcomes, such as heart attacks or death: instead, they are approved for showing a benefit on ‘surrogate outcomes’, such as a blood test, that is only weakly or theoretically associated with the real suffering and death that we are trying to avoid.
This is best understood with an example. Statins are drugs that lower cholesterol, but you don’t take them because you want to change your cholesterol figures on a blood test printout: you take them because you want to lower your risk of having a heart attack, or dying. Heart attack and death are the real outcomes of interest here, and cholesterol is just a surrogate for those, a process outcome, something that we hope is associated with the real outcome, but it might not be, either not at all, or perhaps not very well.
Often there is a fair reason for using a surrogate outcome, not as your only indicator, but at least for some of the data. People take a long time to die (it’s one of the great problems of research, if you can forgive the thought), so if you want an answer quickly, you can’t wait around for them to have a heart attack and die. In these circumstances, a surrogate outcome like a blood test is a reasonable thing to measure, as an interim arrangement. But you still have to do long-term follow-up studies at some stage, to find out if your hunch about the surrogate outcome was right after all. Unfortunately, the incentives for companies – which are by far the largest funders of trials – are all focused on short-term gains, either to get their drug on the market as soon as possible, or to get results before the drug comes off patent, while it still belongs to them.
This is a major problem for patients, because benefits on surrogate endpoints often don’t translate into real-life benefits. In fact, the history of medicine is full of examples where quite the opposite was true.
Probably the most dramatic and famous comes from the Cardiac Arrhythmia Suppression Trial (CAST), which tested three anti-arrhythmic drugs to see if they prevented sudden death in patients who were at higher risk because they had a certain kind of abnormal heart rhythm.20 The drugs prevented these abnormal rhythms, so everyone thought they must be great: they were approved onto the market to prevent sudden death in patients with abnormal rhythms, and doctors felt pretty good about prescribing them. When a proper trial measuring death was conducted, everyone felt a bit embarrassed: the drugs increased the risk of death to such a huge extent that the trial had to be stopped early. We had been cheerfully handing out tablets that killed people (it’s been estimated that well over a hundred thousand people died as a result).
Even when they don’t actively increase your risk of death, sometimes drugs which work well to change surrogate outcomes simply don’t make any difference to the real outcomes that we’re most interested in. Doxazosin is an expensive branded blood-pressure drug, and it works extremely well for lowering the blood-pressure reading in a doctor’s office – about as well as chlorthalidone, a simple old-fashioned blood-pressure drug that has been off patent for many years. Eventually, a trial was done comparing the two on real-world outcomes like heart failure (using government funding, since it was in nobody’s financial interest); and it had to stop early, because patients on doxazosin were doing so much worse.21 The manufacturer of doxazosin, Pfizer, mounted a magnificent marketing campaign, and there was barely any change in the use of the drug.22 I will discuss this kind of campaigning later.
There are endless examples of drugs for which the only evidence available uses surrogate outcomes. If you have diabetes, the thing you’re worried about is death, and horrible problems in your feet, your kidneys, your eyes and so on. You worry about your blood-sugar level and your weight because they are useful guides to whether your diabetes is under control, but they’re nothing compared to the basic important question: will this drug actually reduce my risk of dying? Right now, there are all sorts of new diabetes drugs on the market. The ‘glucagon-like peptide-1 receptor drugs’, for example, are pretty exciting to a lot of doctors. If you look at the latest systematic review of their benefits, published in December 2011 (this one just happens to be open in front of me – it could have been any number of drugs) you will see that they lower blood sugar, lower blood pressure, lower cholesterol, and all this great stuff;23 but nobody’s ever checked to see if they actually stop you dying, which is all that the people taking them really care about.
The same goes for side effects. Depo-provera is a reasonably good contraceptive, but there’s some concern about whether it makes you more vulnerable to fractures. The research into this looks at bone mineral density, rather than actual fractures.24
When you come to get your drug approved to go on the market, regulators will often permit you to show proof of effectiveness only on surrogate outcomes. For ‘accelerated approval’, for drugs that are the first in a new class, or are treating a condition that has no current treatment, they may let you get away with a surrogate outcome that has barely been validated, which means there is very little research into how well it really is associated with the real-world outcomes of the disease. For context here, it’s worth remembering that the examples above, where we were misled, came from surrogate outcomes that are regarded as ‘well validated’. This would be fine if getting onto the market was just the beginning of the story, a starting gun for cautious prescription, in the context of larger monitoring of real-world outcomes. Unfortunately, as we will now see, things aren’t like that.
Accelerated approval
Gathering and assessing trial evidence takes a long time, but regulators have to balance several opposing forces. Doctors with an eye on public health are often keen to make sure that the evidence for a new product is as good as possible, partly because many new drugs are only trivially useful in comparison with what already exists; but also because the pre-approval period is the time when demands on a drug company for compelling research are most likely to be met.
Drug companies, meanwhile, would like to get their drug on the market as swiftly and cheaply as possible. This isn’t just impatience for revenue; it’s also a fear of losing revenue outright, because the clock is already ticking for patent expiry even before the approval process starts. That strong commercial incentive is communicated in muscular fashion to governments, which push regulators to approve rapidly, and often measure speed of approval as a key outcome for the regulator.
This can have worrying effects, which might lead you to believe that quality of evidence is not the only factor affecting a drug’s approval. For many decades, for example, the FDA’s performance was measured by how many drugs it managed to approve in each calendar year.25 This led to a phenomenon known as the ‘December Effect’, whereby a very large proportion of the year’s approvals were rushed through in a panic during the last few weeks around Christmas. By graphing the proportion of approvals that were made in December over the course of thirty years (below, from Carpenter 2010), we can see the size of this effect, and also trace the arrival of a more aggressive pro-industry stance during Ronald Reagan’s presidency (1981–89). If approvals were evenly distributed throughout the year, we’d expect to only see 8 per cent in each month: during the late eighties, the proportion passing in December rose to more than half, and it’s hard to believe that this was simply when the assessments were complete.
These kinds of pressures can also be seen in the approval times for medicines, which have fallen hugely around the world: in the US they have reduced by half since 1993, on the back of previous cuts before that; and in the UK they dropped even more dramatically, from 154 working days in 1989 to forty-four days just a decade later.
It would be a mistake to imagine that drug companies are the only people applying pressure for fast approvals. Patients can also feel they are being deprived of access to drugs, especially if they are desperate. In fact, in the 1980s and 1990s the key public drive for faster approvals came from an alliance
forged between drug companies and AIDS activists such as ACT UP.
At the time, HIV and AIDS had suddenly appeared out of nowhere, and young, previously healthy gay men were falling ill and dying in terrifying numbers, with no treatment available. We don’t care, they explained, if the drugs that are currently being researched for effectiveness might kill us: we want them, because we’re dying anyway. Losing a couple of months of life because a currently unapproved drug turned out to be dangerous was nothing, compared to a shot at a normal lifespan. In an extreme form, the HIV-positive community was exemplifying the very best motivations that drive people to participate in clinical trials: they were prepared to take a risk, in the hope of finding better treatments for themselves or others like them in the future. To achieve this goal they blocked traffic on Wall Street, marched on the FDA headquarters in Rockville, Maryland, and campaigned tirelessly for faster approvals.
As a result of this campaign, a series of new regulations were implemented to allow accelerated approval of certain new drugs. This legislation was intended for use on lifesaving drugs, in situations where there was no currently available medical treatment. Unfortunately, now that they have been in place for more than a decade, we can see that this is not how they have been used.
Midodrine
Once a drug has been approved it is very rare for a regulator to remove it from the market, especially if the only issue is lack of efficacy, rather than patients actively dying because of side effects. Where they do finally make such a move, it is usually after phenomenal delay.
Midodrine is a drug used to treat ‘orthostatic hypotension’, a drop in blood pressure causing dizziness when you stand up.26 While this is doubtless unpleasant for those who experience it, and there may be an increased risk of, say, falls when feeling dizzy, this condition is generally not what most people would regard as serious or life-threatening. Furthermore, the extent to which it is regarded as a singular medical problem varies between countries and cultures. But because there were no previous drugs available to treat it, midodrine was able to be approved through the accelerated programme, in 1996, with weak evidence, but the promise of better studies to follow.
Specifically, midodrine was approved on the basis of three very small, brief trials (two of them only two days long) in which many of the people receiving the drug dropped out of the study completely. These trials showed a small benefit on a surrogate outcome – changes in blood-pressure recordings when the participants stood up – but no benefit on real-world outcomes like dizziness, quality of life, falls, and so on. Because of this, after midodrine was approved through the urgent approval scheme, the manufacturer, Shire, had to promise it would do more research once the drug was on the market.
Year after year, no satisfactory trials appeared. In August 2010, fourteen years later, the FDA announced that unless Shire finally produced some compelling data showing that midodrine improved actual symptoms and day-to-day function, rather than some numbers on a blood-pressure machine after one day, it would take the drug off the market for good.27 This seemed like an assertive move which should finally provoke compliance, but the result was quite the opposite. Effectively, the company said: ‘Fine.’ The drug was off patent: anyone could make it, and indeed Shire now only made 1 per cent of the midodrine sold, with Sandoz, Apotex, Mylan and other companies making the rest. In such a crowded market there was very little money to be made from selling this medicine, and certainly no incentive to invest in research that would only help other companies sell a hundred times more of the same product. Fourteen years on from midodrine’s original approval, the FDA found that there is such a thing as being simply too late.
But that was not the end of the story. Suddenly a vast army of midodrine users and special-interest patient groups appeared, with politicians at their helm: 100,000 patients had filed prescriptions for the drug in 2009 alone. To them, this pill was a life-saver, and the only drug available to treat their condition. If all companies were going to be banned from making it, with the drug taken off the market, this would be a disaster. The fact that no trial had ever demonstrated any concrete benefit was irrelevant: quack remedies such as homeopathy continue to maintain a viciously loyal fan base, despite the fact that homeopathic pills by definition contain no ingredients at all, and despite research overall showing them to perform no better than placebo.* These midodrine patients didn’t care about what trials found: they ‘knew’ that their drug worked, with the certainty of true believers. And now the government was planning to take it away because of some complicated administrative transgression. Surrogate what? This must have sounded like irrelevant word-play from where they were trying to stand.
The FDA was forced to backtrack, and leave the drug on the market. Slow negotiations have continued over the post-marketing trials, but the FDA now has very little leverage with any company over this drug. Almost two decades after midodrine was first approved as an urgent, exceptional case, the drug companies are still making promises to do proper trials. As of 2012, these trials are nowhere to be seen.
This is a serious problem, and it goes way beyond this single, rather trivial medicine. The General Accounting Office is the investigative audit branch of the US Congress. In 2009, it examined the FDA’s failure to chase these kinds of post-approval studies, and its findings were damning: between 1992 and 2008, ninety drugs had been given accelerated approval on the basis of surrogate endpoints alone, with the drug companies making a commitment to conduct 144 trials in total. As of 2009, one in every three of those trials was still outstanding.28 No drug had ever been taken off the market because its manufacturer had failed to hand over outstanding trial data.
The British academic John Abraham is a social scientist who has done more than anyone to shine a light into the traditions and processes of regulators around the world. He has concluded that accelerated approval is simply part of a consistent trend towards deregulation, for the benefit of the industry. It’s useful to walk through just one of the case studies he has worked on, with colleague Courtney Davis, to see how regulators around the world dealt with the best possible candidates for these urgent assessments.29
Gefitinib (brand name Iressa) is a cancer drug made by AstraZeneca, for desperate patients who have reached the end of the line. It’s approved for non-small-cell lung cancer, which is a serious life-threatening diagnosis, and it’s approved for use as ‘third-line’ treatment, after all else has failed. Its accelerated approval was driven partly by patient campaigning, just like the AIDS campaigners who drove the introduction of accelerated approval legislation in the first place. It’s also a good case study, because the manufacturer did actually conduct its follow-up studies, which is fairly unusual (only 25 per cent of the cancer drugs being studied by Abraham have done so).
For standard approval of a lung-cancer treatment you need to show meaningful improvement in either survival or symptoms. But ‘tumour response’ – a reduction in tumour size seen on a body scan – is a fairly typical surrogate outcome for cancer drugs, which can be used to get accelerated approval; after doing so, you will then need to do more trials to find out if this translates into benefits that actually matter to patients.
Initially, AstraZeneca provided evidence from a small trial showing a 10 per cent drop in tumour size on Iressa. This was regarded by the FDA as unimpressive, especially since the patients in the trial were unusual, with slower-growing tumours than you’d usually see. But the company pressed on, and began much larger trials measuring the impact on survival. It had expected to finish these studies after the drug was rapidly approved, but in fact they were completed before then. The trials on real-world outcomes found no survival benefit. What’s more, contradicting the smaller, earlier study, they found no improvement in tumour size. One FDA scientist summarised the findings fairly bluntly: ‘It’s 2,000 patients saying Iressa doesn’t work versus 139 saying it works marginally.’
At the same time, the company was also giving the drug to 12,000 desperate, dying pat
ients with no other option, through something known as an ‘expanded access programme’. This is common when patients have shown no response to any other medication, and are regarded as too unfit for clinical trials (although I would argue that trials should ideally include everyone eligible for treatment, since we only do them to try to answer the question of whether a drug works in real-world patients). These programmes can cost companies money, but they also generate a huge amount of goodwill from desperate people, their families and their organised patient groups.
Regulators now, like so many public bodies, set high store by ‘public engagement’, and this is an admirable goal, if done well. But what we see here is not an example of good public engagement. Large, well-conducted, fair tests of Iressa had shown that it was no better than a dummy sugar pill containing no medicine. Yet many dying patients from the expanded access programmes, who had been given the drug for free, travelled with advocacy groups to give compelling evidence to the FDA. From their perspective this was ‘a wonderful drug’, they explained, ‘light-years better than previous treatment’. It ‘began to eliminate cancer symptoms in seven days’. Tumours were ‘90 per cent gone in three months’, said one. Whether that was exaggeration or fluke, the reality is that fair tests showed no benefit. But the desperate patients disagreed, and asserted their case plainly and simply: Iressa ‘will save lives’. This personal testimony was in all likelihood a combination of the placebo effect and the natural fluctuation in symptoms that all patients experience. That didn’t seem to matter.