by Ben Goldacre
There is only one conclusion that we can draw from this: consent forms routinely lie to patients participating in trials. This is an extraordinary state of affairs, made all the more extraordinary by the huge amounts of red tape that everyone involved in a trial must negotiate, closely monitoring endless arcane pieces of paperwork, and ensuring that patients are fully informed on the minutiae of their treatment. Despite all this regulatory theatre, which hinders good research on routine practice (as we shall see – p.230), we allow these forms to tell patients outright lies about the control of data, and we fail to police one of the most important ethical problems in the whole of medicine. The deceit of these consent forms is, to me, a good illustration of how broken and outdated the regulatory frameworks of medicine have become. But it also, finally, poses a serious problem for the future of research.
We desperately need patients to continue to believe that they are contributing to society, because trial recruitment is in crisis, and it is increasingly hard to persuade patients to participate at all. In one recent study, a third of all trials failed to reach their original recruitment target, and more than half had to be awarded an extension.52 If word gets out that trials are often more promotional than genuinely scientific, recruitment will become even more difficult. The answer is not to hide this problem, but rather to fix it.
So universities and ethics committees may have failed us, but there is one group of people we might expect to step up, to try to show some leadership on missing trial data. These are the medical and academic professional bodies, the Royal Colleges of General Practice, Surgery and Physicians, the General Medical Council, the British Medical Association, the pharmacists’ organisations, the bodies representing each sub-specialty of academia, the respiratory physiologists, the pharmacologists, the Academy of Medical Sciences, and so on.
These organisations have the opportunity to set the tone of academic and clinical medicine, in their codes of conduct, their aspirations, and in some cases their rules, since some have the ability to impose sanctions, and all have the ability to exclude those who fail to meet basic ethical standards. We have established, I hope, beyond any doubt, that non-publication of trials in humans is research misconduct, that it misleads doctors and harms patients around the world. Have these organisations used their powers, stood up and announced, prominently and fiercely, that this must stop, and that they will take action?
One has: the Faculty of Pharmaceutical Medicine, a small organisation with 1,400 members. And none of the others have bothered.
Not one.
What can be done?
There are several simple solutions to these problems, which fall into three categories. There is no argument against any of the following suggestions that I am aware of. The issue of missing data has been neglected through institutional inertia, and reluctance by senior academics to challenge industry. Their failure to act harms patients every day.
Gagging clauses
If there is nothing to be ashamed of in gagging clauses – if companies, and legislators, and academics, and university contracts departments, all believe that they are acceptable – then everything should be out in the open, prominently flagged up, so that everyone outside those systems can decide if they agree.
Until they can be eradicated, where gagging clauses exist, patients should be told, at the time they are invited to participate in a trial, that the company sponsoring it is allowed to hide the results if it doesn’t like them. The consent form should also explain clearly that withholding negative results will lead to doctors and patients being misled about the effectiveness of the treatment being trialled, and so exposed to unnecessary harm. Trial participants can then decide for themselves if they think these contracts are acceptable.
Wherever the sponsoring company has the contractual right to gag publication, even if it doesn’t exercise that right, the fact that a gagging clause existed should be stated clearly: in the academic paper; in the trial protocol; and in the publicly available trial registry entry that goes up before the trial starts. Readers of the trial findings can then decide for themselves if they trust that sponsor and research group to publish all negative findings, and interpret any reported positive findings in that light.
All university contracts should follow the same boilerplate format, and forbid gagging clauses. Failing that, all universities should be forced to prominently declare which contracts with gagging clauses they have permitted, and to publish those clauses online for all to see, so that all can be alerted that the institution is producing systematically biased research, and discount any findings from them accordingly.
In legislation, gagging clauses should be made illegal, with no possibility of quibbles. If there is a dispute about analysis or interpretation between the people running the trial and the people paying for it, it should take place in the published academic literature, or some other public forum. Not in secret.
Professional bodies
All professional bodies should take a clear stand on unpublished trial data, declare it clearly as research misconduct, and state that it will be handled like any other form of serious misconduct that misleads doctors and harms patients. That they have not done so is a stain on the reputation of these organisations, and on their senior members.
All research staff involved in any trial on humans should be regarded as jointly and severally liable for ensuring that it is published in full within one year of completion.
All those responsible for withholding trial data should have their names prominently posted in a single database, so that others can be made aware of the risk of working with them, or allowing them access to patients for research, in future.
Ethics committees
No person should be allowed to conduct trials in humans if a research project they are responsible for is currently withholding trial data from publication more than one year after completion. Where any researcher on a project has a previous track record of delayed publication of trial data, the ethics committee should be notified, and this should be taken into account, as with any researcher guilty of research misconduct.
No trial should be approved without a firm guarantee of publication within one year of completion.
How regulators and journals have failed us
So far we’ve established that ethics committees, universities and the professional bodies of medical researchers have all failed to protect patients from publication bias, even though the selective non-publication of unflattering data is a serious issue for medicine. We know that it distorts and undermines every decision that researchers, doctors and patients make, and that it exposes patients to avoidable suffering and death. This is not seriously disputed, so you might reasonably imagine that governments, regulators and medical journals must all have tried to address it.
They have tried, and failed. Worse than simply failing, they have repeatedly provided what we might regard as ‘fake fixes’: we have seen small changes in regulations and practices, announced with great fanfare, but then either ignored or bypassed. This has given false reassurance to doctors, academics and the public that the problem has been fixed. What follows is the story of these fake fixes.
Registers
The earliest and simplest solution proposed was to open registers of trials: if people are compelled to publish their protocol, in full, before they start work, then we at least have the opportunity to go back and check to see if they’ve published the trials that they’ve conducted. This is very useful, for a number of reasons. A trial protocol describes in great technical detail everything that researchers will do in a trial: how many patients they’ll recruit, where they’ll come from, how they’ll be divided up, what treatment each group will get, and what outcome will be measured to establish if the treatment was successful. Because of this, it can be used to check whether a trial was published, but also whether its methods were distorted along the way, in a manner that would allow the results to be exaggerated (as described in Chapter 4).
Th
e first major paper to call for a registry of clinical trial protocols was published in 1986,53 and it was followed by a flood. In 1990 Iain Chalmers (we can call him Sir Iain Chalmers if you like*) published a classic paper called ‘Underreporting Research is Scientific Misconduct’,55 and he has traced the chequered history of trials registers in the UK.56 In 1992, as the Cochrane Collaboration began to gather influence, representatives of the Association of the British Pharmaceutical Industry (ABPI) asked to meet Chalmers.57 After explaining the work of Cochrane, and the vital importance of summarising all the trial results on a particular drug, he explained very clearly to them how biased under-reporting of results harms patients.
The industry’s representatives were moved, and soon they took action. Mike Wallace, the chief executive of Schering and a member of that ABPI delegation, agreed with Chalmers that withholding data was ethically and scientifically indefensible, and said that he planned to do something concrete to prevent it, if only to protect the industry from having the issue forced upon it in less welcome terms. Wallace stepped out of line from his colleagues, and committed to registering every trial conducted by his company with Cochrane. This was not a popular move, and he was reprimanded by colleagues, in particular those from other companies.
But soon GlaxoWellcome followed suit, and in 1998 its chief executive, Richard Sykes, wrote an editorial in the BMJ called ‘Being a modern pharmaceutical company involves making information available on clinical trial programmes’.58 ‘Programmes’ was the crucial word, because as we’ve seen, and as we shall see in greater detail later, you can only make sense of individual findings if you assess them in the context of all the work that has been done on a drug.
GlaxoWellcome set up a clinical trials registry, and Elizabeth Wager, the head of the company’s medical writers group, pulled together a group from across the industry to develop ethical guidelines for presenting research. The ABPI, seeing individual companies take the lead, saw the writing on the wall: it decided to commend GlaxoWellcome’s policy to the whole industry, and launched this initiative at a press conference where Chalmers – a strong critic – sat on the same side of the table as the industry. AstraZeneca, Aventis, MSD, Novartis, Roche, Schering Healthcare and Wyeth began registering some of their trials – only the ones involving UK patients, and retrospectively – but there was movement at last.
At the same time, there was movement in America. The 1997 FDA Modernization Act created clinicaltrials.gov, a register run by the US government National Institute of Health. This legislation required that trials should be registered, but only if they related to an application to put a new drug on the market, and even then, only if it was for a serious or life-threatening disease. The register opened in 1998, and the website clinicaltrials.gov went online in 2000. The entry criteria were widened in 2004.
But soon it all began to fall apart. GlaxoWellcome merged with SmithKline Beecham to become GlaxoSmithKline (GSK), and initially the new logo appeared on the old trials register. Iain Chalmers wrote to Jean-Paul Garnier, the chief executive of the new company, to thank him for maintaining this valuable transparency: but no reply ever came. The registry website was closed, and the contents were lost (though GSK was later forced to open a new register, as part of a settlement with the US government over the harm caused by its withholding of data on new drug trials just a couple of years later). Elizabeth Wager, the author of GSK’s Good Publication Practice guidelines for drug companies, was out of a job, as her writing department in the company was closed. Her guidelines were ignored.
From the moment that these registries were first suggested, and then opened, it was implicitly assumed that the shame of producing this public record, and then not publishing your study, would be enough to ensure that people would do the right thing. But the first problem for the US register, which could have been used universally, was that people simply chose not to use it. The regulations required only a very narrow range of trials to be posted, and nobody else was in a hurry to post their trials if they didn’t have to.
In 2004 the International Committee of Medical Journal Editors (ICMJE) – a collection of editors from the most influential journals in the world – published a policy statement, announcing that none of them would publish any clinical trials after 2005, unless they had been properly registered before they began.59 They did this, essentially, to force the hand of the industry and researchers: if a trial has a positive result, then people desperately want to publish it in the most prestigious journal they can find. Although they had no legal force, the journal editors did have the thing that companies and researchers wanted most: the chance of a major journal publication. By insisting on pre-registration, they were doing what they could to force researchers and industry sponsors to register all trials. Everyone rejoiced: the problem had been fixed.
If you think it seems odd – and perhaps unrealistic – that fixing this crucial flaw in the information architecture of a $600 billion industry should be left to an informal gathering of a few academic editors, with no legislative power, then you’d be right. Although everybody began to talk as if publication bias was a thing of the past, in reality it was continuing just as before, because the journal editors simply ignored their own threats and promises. Later we will see the phenomenal financial inducements on offer to editors for publishing positive industry papers, which can extend to millions of dollars in reprint and advertising revenue. But first we should look at what they actually did after their solemn promise in 2005.
In 2008 a group of researchers went through every single trial published in the top ten medical journals, every one of which was a member of the ICMJE, after the deadline for pre-registration. Out of 323 trials published during 2008 in these high-impact academic journals, only half were adequately registered (before the trial, with the main outcome measure properly specified), and trial registration was entirely lacking for over a quarter.60 The ICMJE editors had simply failed to keep their word.
Meanwhile, in Europe, there were some very bizarre developments. With enormous fanfare, the European Medicines Agency created a registry of trials called EudraCT. EU legislation requires all trials to be posted here if they involve any patients in Europe, and many companies will tell you that they’ve met their responsibilities for transparency by doing so. But the contents of this EU register have been kept entirely secret. I can tell you that it contains around 30,000 trials, since that figure is in the public domain, but that is literally all I know, and all anyone can know. Despite EU legislation requiring that the public should be given access to the contents of this register, it remains closed. This creates an almost laughable paradox: the EU clinical trials register is a transparency tool, held entirely in secret. Since March 2011, after heavy media criticism (from me at any rate), a subset of trials has slowly been made public through a website called EudraPharm. As of summer 2012, although the agency now claims that its register is accessible to all, at least 10,000 trials are still missing, and the search engine doesn’t work properly.61 It’s absolutely one of the strangest things I’ve ever seen, and nobody other than the EU even regards this peculiar exercise as a trials register: I certainly don’t, I doubt you do, and both the ICMJE and the World Health Organization have explicitly stated that EudraCT is not a meaningful register.
But new work was being done in the US, and it seemed sensible. In 2007 the FDA Amendment Act was passed. This is much tighter: it requires registration of all trials of any drug or device, at any stage of development other than ‘first-in-man’ tests, if they have any site in the US, or involve any kind of application to bring a new drug onto the market. It also imposes a startling new requirement: all results of all trials must be posted to clinicaltrials.gov, in abbreviated summary tables, within one year of completion, for any trial on any marketed drug that completes after 2007.
Once again, to great fanfare, everyone believed that the problem had been fixed. But it hasn’t been, for two very important reasons.
Firstly, unfortunat
ely, despite the undoubted goodwill, requiring the publication of all trials starting from ‘now’ does absolutely nothing for medicine today. There is no imaginable clinic, anywhere in the world, at which medicine is practised only on the basis of trials that completed within the past three years, using only drugs that came to market since 2008. In fact, quite the opposite is true: the vast majority of drugs currently in use came to market over the past ten, twenty or thirty years, and one of the great challenges for the pharmaceutical industry today is to create drugs that are anything like as innovative as those that were introduced in what has come to be known as the ‘golden era’ of pharmaceutical research, when all the most widely used drugs, for all the most common diseases, were developed. Perhaps they were the ‘low-lying fruit’, plucked from the research tree, but in any case, these are the tablets we use.