by Ben Goldacre
And crucially, it is for these drugs – the ones we actually use – that we need the evidence: from trials completed in 2005 or 1995. These are the drugs that we are prescribing completely blind, misled by a biased sample of trials, selectively published, with the unflattering data buried in secure underground data archives somewhere in the hills (I am told) of Cheshire.
But there is a second, more disturbing reason why these regulations should be taken with a pinch of salt: they have been widely ignored. A study published in January 2012 looked at the first slice of trials subject to mandatory reporting, and found that only one in five had met its obligation to post results.62
Perhaps this is not surprising: the fine for non-compliance is $10,000 a day, which sounds spectacular, until you realise that it’s only $3.5 million a year, which is chickenfeed for a drug bringing in $4 billion a year. And what’s more, no such fine has ever been levied, throughout the entire history of the legislation.
So that, in total, is why I regard the ICMJE, the FDA and the EU’s claims to have addressed this problem as ‘fake fixes’. In fact, they have done worse than fail: they have given false reassurance that the problem has been fixed, false reassurance that it has gone away, and they have led us to take our eyes off the ball. For half a decade now, people in medicine and academia have talked about publication bias as if it was yesterday’s problem, discovered in the 1990s and early 2000s, and swiftly fixed.
But the problem of missing data has not gone away, and soon we will see exactly how shameless some companies and regulators can be, in the very present day.
What can be done?
The ICMJE should keep its promises, the EU should be less risible, and the FDA Amendment Act 2007 should be muscularly enforced. But there are many more disappointments to come, so I will leave my action plan on missing data for later in this chapter.
Blood from a stone: trying to get data from regulators
So far we’ve established that doctors and patients have been failed by a range of different people and organisations, all of whom we might have expected to step up and fix the problem of missing data, since it harms patients in large numbers around the world. We have seen that governments take no action against those who fail to publish their results, despite the public pretence to the contrary; and that they take no action against those who fail to register their trials. We have seen that medical journal editors continue to publish unregistered trials, despite the public pretence that they have taken a stand. We have seen that ethics committees fail to insist on universal publication, despite their stated aim of protecting patients. And we have seen that professional bodies fail to take action against what is obviously research misconduct, despite evidence showing that the problem of missing data is of epidemic proportions.63
While the published academic record is hopelessly distorted, you might hope that there is one final route which patients and doctors could use to get access to the results of clinical trials: the regulators, which receive large amounts of data from drug companies during the approval process, must surely have obligations to protect patients’ safety? But this, sadly, is just one more example of how we are failed by the very bodies that are supposed to be protecting us.
In this section, we will see three key failures. Firstly, the regulators may not have the information in the first place. Secondly, the way in which they ‘share’ summary trial information with doctors and patients is broken and shabby. And finally, if you try to get all of the information that a drug company has provided – the long-form documents, where the bodies are often buried – then regulators present bizarre barriers, blocking and obfuscating for several years at a time, even on drugs that turn out to be ineffective and harmful. Nothing of what I am about to tell you is in any sense reassuring.
One: Information is withheld from regulators
Paroxetine is a commonly used antidepressant, from the class of drugs known as ‘selective serotonin reuptake inhibitors’, or SSRIs. You will hear more about this class of drugs later in this book, but here we will use paroxetine to show how companies have exploited our longstanding permissiveness about missing trials, and found loopholes in our inadequate regulations on trial disclosure. We will see that GSK withheld data about whether paroxetine works as an antidepressant, and even withheld data about its harmful side effects, but most importantly, we will see that what it did was all entirely legal.
To understand why, we first need to go through a quirk of the licensing process. Drugs do not simply come onto the market for use in all medical conditions: for any specific use of any drug, in any specific disease, you need a separate marketing authorisation. So, a drug might be licensed to treat ovarian cancer, for example, but not breast cancer. That doesn’t mean the drug doesn’t work in breast cancer. There might well be some evidence that it’s great for treating that disease too, but maybe the company hasn’t gone to the trouble and expense of getting a formal marketing authorisation for that specific use. Doctors can still go ahead and prescribe it for breast cancer, if they want, because the drug is available for prescription, and there are boxes of it sitting in pharmacies waiting to go out (even though, strictly speaking, it’s only got marketing approval for use in ovarian cancer). In this situation the doctor will be prescribing the drug legally, but ‘off-label’.
This is fairly common, as getting a marketing authorisation for a specific use can be time-consuming and expensive. If doctors know that there’s a drug which has been shown in good-quality trials to help treat a disease, it would be perverse and unhelpful of them not to prescribe it, just because the company hasn’t applied for a formal licence to market it for that specific use. I’ll discuss the ins and outs of all this in more detail later. But for now, what you need to know is that the use of a drug in children is treated as a separate marketing authorisation from its use in adults.
This makes sense in many cases, because children can respond to drugs in very different ways to adults, so the risks and benefits might be very different, and research needs to be done in children separately. But this licensing quirk also brings some disadvantages. Getting a licence for a specific use is an arduous business, requiring lots of paperwork, and some specific studies. Often this will be so expensive that companies will not bother to get a licence specifically to market a drug for use in children, because that market is usually much smaller.
But once a drug is available in a country for one specific thing, as we have seen, it can then be prescribed for absolutely anything. So it is not unusual for a drug to be licensed for use in adults, but then prescribed for children on the back of a hunch; or a judgement that it should at least do no harm; or studies that suggest benefit in children, but that would probably be insufficient to get through the specific formal process of getting marketing authorisation for use in kids; or even good studies, but in a disease where the market is so small that the company can’t be bothered to get a marketing approval for use in children.
Regulators have recognised that there is a serious problem with drugs being used in children ‘off-label’, without adequate research, so recently they have started to offer incentives for companies to conduct the research, and formally seek these licences. The incentives are patent extensions, and these can be lucrative. All drugs slip into the public domain about a decade after coming onto the market, and become like paracetamol, which anyone can make very cheaply. If a company is given a six-month extension on a drug, for all uses, then it can make a lot more money from that medicine. This seems a good example of regulators being pragmatic, and thinking creatively about what carrots they can offer. Licensed use in children will probably not itself make a company much extra money, since doctors are prescribing the drug for children already, even without a licence or good evidence, simply because there are no other options. Meanwhile, six months of extra patent life for a blockbuster drug will be very lucrative, if its adult market is large enough.
There’s a lot of debate about whether the drug companies have pl
ayed fair with these offers. For example, since the FDA started offering this deal, about a hundred drugs have been given paediatric licences, but many of them were for diseases that aren’t very common in children, like stomach ulcers, or arthritis. There have been far fewer applications for less lucrative products that could be used in children, such as more modern medicines called ‘large-molecule biologics’. But there it is.
When GSK applied for a marketing authorisation in children for paroxetine, an extraordinary situation came to light, triggering the longest investigation in the history of UK drugs regulation. This investigation was published in 2008, and examined whether GSK should face criminal charges.64 It turned out that what the company had done – withholding important data about safety and effectiveness that doctors and patients clearly needed to see – was plainly unethical, and put children around the world at risk; but our laws are so weak that GSK could not be charged with any crime.
Between 1994 and 2002 GSK conducted nine trials of paroxetine in children.65 The first two failed to show any benefit, but the company made no attempt to inform anyone of this by changing the ‘drug label’ that is sent to all doctors and patients. In fact, after these trials were completed, an internal company management document stated: ‘it would be commercially unacceptable to include a statement that efficacy had not been demonstrated, as this would undermine the profile of paroxetine’. In the year after this secret internal memo, 32,000 prescriptions were issued to children for paroxetine in the UK alone: so, while the company knew the drug didn’t work in children, it was in no hurry to tell doctors that, despite knowing that large numbers of children were taking it. More trials were conducted over the coming years – nine in total – and none showed that the drug was effective at treating depression in children.
It gets much worse than that. These children weren’t simply receiving a drug that the company knew to be ineffective for them: they were also being exposed to side effects. This should be self-evident, since any effective treatment will have some side effects, and doctors factor this in, alongside the benefits (which in this case were non-existent). But nobody knew how bad these side effects were, because the company didn’t tell doctors, or patients, or even the regulator about the worrying safety data from its trials. This was because of a loophole: you only have to tell the regulator about side effects reported in studies looking at the specific uses for which the drug has a marketing authorisation. Because the use of paroxetine in children was ‘off-label’, GSK had no legal obligation to tell anyone about what it had found.
People had worried for a long time that paroxetine might increase the risk of suicide, though that is quite a difficult side effect to detect in an antidepressant, because people with depression are at a much higher risk of suicide than the general population anyway, as a result of their depression. There are also some grounds to believe that as patients first come out of their depression, and leave behind the sluggish lack of motivation that often accompanies profound misery, there may be a period during which they are more capable of killing themselves, just because of the depression slowly lifting.
Furthermore, suicide is a mercifully rare event, which means you need a lot of people on a drug to detect an increased risk. Also, suicide is not always recorded accurately on death certificates, because coroners and doctors are reluctant to give a verdict that many would regard as shameful, so the signal you are trying to detect in the data – suicide – is going to be corrupted. Suicidal thoughts or behaviours that don’t result in death are more common than suicide itself, so they should be easier to detect, but they too are hard to pick up in routinely collected data, because they’re often not presented to doctors, and where they are, they can be coded in health records in all sorts of different ways, if they appear at all. Because of all these difficulties, you would want to have every scrap of data you could possibly cobble together on the question of whether these drugs cause suicidal thoughts or behaviour in children; and you would want a lot of experienced people, with a wide range of skills, all looking at the data and discussing it.
In February 2003, GSK spontaneously sent the MHRA a package of information on the risk of suicide on paroxetine, containing some analyses done in 2002 from adverse-event data in trials the company had held, going back a decade. This analysis showed that there was no increased risk of suicide. But it was misleading: although it was unclear at the time, data from trials in children had been mixed in with data from trials in adults, which had vastly greater numbers of participants. As a result, any sign of increased suicide risk among children on paroxetine had been completely diluted away.
Later in 2003, GSK had a meeting with the MHRA to discuss another issue involving paroxetine. At the end of this meeting, the GSK representatives gave out a briefing document, explaining that the company was planning to apply later that year for a specific marketing authorisation to use paroxetine in children. They mentioned, while handing out the document, that the MHRA might wish to bear in mind a safety concern the company had noted: an increased risk of suicide among children with depression who received paroxetine, compared with those on dummy placebo pills.
This was vitally important side-effect data, being presented, after an astonishing delay, casually, through an entirely inappropriate and unofficial channel. GSK knew that the drug was being prescribed in children, and it knew that there were safety concerns in children, but it had chosen not to reveal that information. When it did share the data, it didn’t flag it up as a clear danger in the current use of the drug, requiring urgent attention from the relevant department in the regulator; instead it presented it as part of an informal briefing about a future application. Although the data was given to completely the wrong team, the MHRA staff present at this meeting had the wit to spot that this was an important new problem. A flurry of activity followed: analyses were done, and within one month a letter was sent to all doctors advising them not to prescribe paroxetine to patients under the age of eighteen.
How is it possible that our systems for getting data from companies are so poor that they can simply withhold vitally important information showing that a drug is not only ineffective, but actively dangerous? There are two sets of problems here: firstly, access for regulators; and secondly, access for doctors.
There is no doubt that the regulations contain ridiculous loopholes, and it’s dismal to see how GSK cheerfully exploited them. As I’ve mentioned, the company had no legal duty to give over the information, because prescription of the drug in children was outside of paroxetine’s formally licensed uses – even though GSK knew this was widespread. In fact, of the nine studies the company conducted, only one had its results reported to the MHRA, because that was the only one conducted in the UK.
After this episode, the MHRA and the EU changed some of their regulations, though not adequately. They created an obligation for companies to hand over safety data for uses of a drug outside its marketing authorisation; but ridiculously, for example, trials conducted outside the EU were still exempt.
There is a key problem here, and it is one that recurs throughout this section of the book: you need all of the data in order to see what’s happening with a drug’s benefits, and risks. Some of the trials that GSK conducted were published in part, but that is obviously not enough: we already know that if we see only a biased sample of the data, we are misled. But we also need all the data for the more simple reason that we need lots of data: safety signals are often weak, subtle and difficult to detect. Suicidal thoughts and plans are rare in children – even those with depression, even those on paroxetine – so all the data from a large number of participants needed to be combined before the signal was detectable in the noise. In the case of paroxetine, the dangers only became apparent when the adverse events from all of the trials were pooled and analysed together.
That leads us to the second obvious flaw in the current system: the results of these trials – the safety data and the effectiveness data – are given in secret to the regu
lator, which then sits and quietly makes a decision. This is a huge problem, because you need many eyes on these difficult problems. I don’t think that the people who work in the MHRA are bad, or incompetent: I know a lot of them, and they are smart, good people. But we shouldn’t trust them to analyse this data alone, in the same way that we shouldn’t trust any single organisation to analyse data alone, with nobody looking over its shoulder, checking the working, providing competition, offering helpful criticism, speeding it up, and so on.
This is even worse than academics failing to share their primary research data, because at least in an academic paper you get a lot of detail about what was done, and how. The output of a regulator is often simply a crude, brief summary: almost a ‘yes’ or ‘no’ about side effects. This is the opposite of science, which is only reliable because everyone shows their working, explains how they know that something is effective or safe, shares their methods and their results, and allows others to decide if they agree with the way they processed and analysed the data.
Yet for the safety and efficacy of drugs, one of the most important of all analyses done by science, we turn our back on this process completely: we allow it to happen behind closed doors, because drug companies have decided that they want to share their trial results discretely with the regulators. So the most important job in evidence-based medicine, and a perfect example of a problem that benefits from many eyes and minds, is carried out alone and in secret.
This perverse and unhealthy secrecy extends way beyond regulators. NICE, the UK’s National Institute for Health and Clinical Excellence, is charged with making recommendations about which treatments are most cost-effective, and which work best. When it does this, it’s in the same boat as you or me: it has absolutely no statutory right to see data on the safety or effectiveness of a drug, if a company doesn’t want to release it, even though the regulators have all of that data. As a result, NICE can be given distorted, edited, biased samples of the data, not just on whether a drug works, but also on how likely it is to have unpleasant side effects.