by Ben Goldacre
Sharing data of individual patients’ outcomes in clinical trials, rather than just the final summary result, has several significant advantages. Firstly, it’s a safeguard against dubious analytic practices. In the VIGOR trial on the painkiller Vioxx, for example, a bizarre reporting decision was made.83 The aim of the study was to compare Vioxx against an older, cheaper painkiller, to see if it was any less likely to cause stomach problems (this was the hope for Vioxx), and also if it caused more heart attacks (this was the fear). But the date cut-off for measuring heart attacks was much earlier than that for measuring stomach problems. This had the result of making the risks look less significant, relative to the benefits, but it was not declared clearly in the paper, resulting in a giant scandal when it was eventually noticed. If the raw data on patients was shared, games like these would be far easier to spot, and people might be less likely to play them in the first place.
Occasionally – with vanishing rarity – researchers are able to obtain raw data, and reanalyse studies that have already been conducted and published. Daniel Coyne, Professor of Medicine at Washington University, was lucky enough to get the data on a key trial for epoetin, a drug given to patients on kidney dialysis, after a four-year-long fight.84 The original academic publication on this study, ten years earlier, had switched the primary outcomes described in the protocol (we will see later how this exaggerates the benefits of treatments), and changed the main statistical analysis strategy (again, a huge source of bias). Coyne was able to analyse the study as the researchers had initially stated they were planning to in their protocol; and when he did, he found that they had dramatically overstated the benefits of the drug. It was a peculiar outcome, as he himself acknowledges: ‘As strange as it seems, I am now the sole author of the publication on the predefined primary and secondary results of the largest outcomes trial of epoetin in dialysis patients, and I didn’t even participate in the trial.’ There is room, in my view, for a small army of people doing the very same thing, reanalysing all the trials that were incorrectly analysed, in ways that deviated misleadingly from their original protocols.
Data sharing would also confer other benefits. It allows people to conduct more exploratory analyses of data, and to better investigate – for example – whether a drug is associated with a particular unexpected side effect. It would also allow cautious ‘subgroup analyses’, to see if a drug is particularly useful, or particularly useless, in particular types of patients.
The biggest immediate benefit from data sharing is that combining individual patient data into a meta-analysis gives more accurate results than working with the crude summary results at the end of a paper. Let’s imagine that one paper reports survival at three years as the main outcome for a cancer drug, and another reports survival at seven years. To combine these two in a meta-analysis, you’d have a problem. But if you were doing the meta-analysis with access to individual patient data, with treatment details and death dates for all of them, you could do a clean combined calculation for three-year survival.
This is exactly the kind of work being done in the area of breast cancer research, where a small number of charismatic and forceful scientists just happen to have driven a pioneering culture of easier collaboration. The summaries they are publishing represent real collaboration, between vast numbers of people, and on vast numbers of patients, producing highly reliable guidance for doctors and patients.
The process sheds a stark light on the reality of data collaboration on such a large scale. Here, for example, is the author list on an academic paper from the Lancet in November 2011: it’s reporting an immense, definitive, and incredibly useful meta-analysis of breast cancer treatment outcomes, using individual patient data pooled from seventeen different trials. The author list is printed in four-point font size (though I suspect that might go wrong in the e-book edition…) because there are seven hundred individual researchers named in it. I typed each of them in by hand for you.
Darby S, McGale P, Correa C, Taylor C, Arriagada R, Clarke M, Cutter D, Davies C, Ewertz M, Godwin J, Gray R, Pierce L, Whelan T, Wang Y, Peto R.Albain K, Anderson S, Arriagada R, Barlow W, Bergh J, Bliss J, Buyse M, Cameron D, Carrasco E, Clarke M, Correa C, Coates A, Collins R, Costantino J, Cutter D, Cuzick J, Darby S, Davidson N, Davies C, Davies K, Delmestri A, Di Leo A, Dowsett M, Elphinstone P, Evans V, Ewertz M, Gelber R, Gettins L, Geyer C, Goldhirsch A, Godwin J, Gray R, Gregory C, Hayes D, Hill C, Ingle J, Jakesz R, James S, Kaufmann M, Kerr A, MacKinnon E, McGale P, McHugh T, Norton L, Ohashi Y, Paik S, Pan HC, Perez E, Peto R, Piccart M, Pierce L, Pritchard K, Pruneri G, Raina V, Ravdin P, Robertson J, Rutgers E, Shao YF, Swain S, Taylor C, Valagussa P, Viale G, Whelan T, Winer E, Wang Y, Wood W, Abe O, Abe R, Enomoto K, Kikuchi K, Koyama H, Masuda H, Nomura Y, Ohashi Y, Sakai K, Sugimachi K, Toi M, Tominaga T, Uchino J, Yoshida M, Haybittle JL, Leonard CF, Calais G, Geraud P, Collett V, Davies C, Delmestri A, Sayer J, Harvey VJ, Holdaway IM, Kay RG, Mason BH, Forbes JF, Wilcken N, Bartsch R, Dubsky P, Fesl C, Fohler H, Gnant M, Greil R, Jakesz R, Lang A, Luschin-Ebengreuth G, Marth C, Mlineritsch B, Samonigg H, Singer CF, Steger GG, Stöger H, Canney P, Yosef HM, Focan C, Peek U, Oates GD, Powell J, Durand M, Mauriac L, Di Leo A, Dolci S, Larsimont D, Nogaret JM, Philippson C, Piccart MJ, Masood MB, Parker D, Price JJ, Lindsay MA, Mackey J, Martin M, Hupperets PS, Bates T, Blamey RW, Chetty U, Ellis IO, Mallon E, Morgan DA, Patnick J, Pinder S, Olivotto I, Ragaz J, Berry D, Broadwater G, Cirrincione C, Muss H, Norton L, Weiss RB, Abu-Zahra HT, Portnoj SM, Bowden S, Brookes C, Dunn J, Fernando I, Lee M, Poole C, Rea D, Spooner D, Barrett-Lee PJ, Mansel RE, Monypenny IJ, Gordon NH, Davis HL, Cuzick J, Lehingue Y, Romestaing P, Dubois JB, Delozier T, Griffon B, Mace Lesec'h J, Rambert P, Mustacchi G, Petruzelka, Pribylova O, Owen JR, Harbeck N, Jänicke F, Meisner C, Schmitt M, Thomssen C, Meier P, Shan Y, Shao YF, Wang X, Zhao DB, Chen ZM, Pan HC, Howell A, Swindell R, Burrett JA, Clarke M, Collins R, Correa C, Cutter D, Darby S, Davies C, Davies K, Delmestri A, Elphinstone P, Evans V, Gettins L, Godwin J, Gray R, Gregory C, Hermans D, Hicks C, James S, Kerr A, MacKinnon E, Lay M, McGale P, McHugh T, Sayer J, Taylor C, Wang Y, Albano J, de Oliveira CF, Gervásio H, Gordilho J, Johansen H, Mouridsen HT, Gelman RS, Harris JR, Hayes D, Henderson C, Shapiro CL, Winer E, Christiansen P, Ejlertsen B, Ewertz M, Jensen MB, Møller S, Mouridsen HT, Carstensen B, Palshof T, Trampisch HJ, Dalesio O, de Vries EG, Rodenhuis S, van Tinteren H, Comis RL, Davidson NE, Gray R, Robert N, Sledge G, Solin LJ, Sparano JA, Tormey DC, Wood W, Cameron D, Chetty U, Dixon JM, Forrest P, Jack W, Kunkler I, Rossbach J, Klijn JG, Treurniet-Donker AD, van Putten WL, Rotmensz N, Veronesi U, Viale G, Bartelink H, Bijker N, Bogaerts J, Cardoso F, Cufer T, Julien JP, Rutgers E, van de Velde CJ, Cunningham MP, Huovinen R, Joensuu H, Costa A, Tinterri C, Bonadonna G, Gianni L, Valagussa P, Goldstein LJ, Bonneterre J, Fargeot P, Fumoleau P, Kerbrat P, Luporsi E, Namer M, Eiermann W, Hilfrich J, Jonat W, Kaufmann M, Kreienberg R, Schumacher M, Bastert G, Rauschecker H, Sauer R, Sauerbrei W, Schauer A, Schumacher M, Blohmer JU, Costa SD, Eidtmann H, Gerber G, Jackisch C, Loibl S, von Minckwitz G, de Schryver A, Vakaet L, Belfiglio M, Nicolucci A, Pellegrini F, Pirozzoli MC, Sacco M, Valentini M, McArdle CS, Smith DC, Stallard S, Dent DM, Gudgeon CA, Hacking A, Murray E, Panieri E, Werner ID, Carrasco E, Martin M, Segui MA, Galligioni E, Lopez M, Erazo A, Medina JY, Horiguchi J, Takei H, Fentiman IS, Hayward JL, Rubens RD, Skilton D, Scheurlen H, Kaufmann M, Sohn HC, Untch M, Dafni U, Markopoulos C, Dafni D, Fountzilas G, Mavroudis D, Klefstrom P, Saarto T, Gallen M, Margreiter R, de Lafontan B, Mihura J, Roché H, Asselain B, Salmon RJ, Vilcoq JR, Arriagada R, Bourgier C, Hill C, Koscielny S, Laplanche A, Lê MG, Spielmann M, A'Hern R, Bliss J, Ellis P, Kilburn L, Yarnold JR, Benraadt J, Kooi M, van de Velde AO, van Dongen JA, Vermorken JB, Castiglione M, Coates A, Colleoni M, Collins J, Forbes J, Gelber RD, Goldhirsch A, Lindtner J, Price KN, Regan MM, Rudenstam CM, Senn HJ, Thuerlimann B, Bliss JM, Chilvers CE, Coombes RC, Hall E, Marty M, Buyse M, Possinger K, Schmid P, Untch M, Wallwiener D, Foster L, George WD, Stewart HJ, Stroner P, Borovik R, Hayat H, Inbar MJ, Robinson E,
Bruzzi P, Del Mastro L, Pronzato P, Sertoli MR, Venturini M, Camerini T, De Palo G, Di Mauro MG, Formelli F, Valagussa P, Amadori D, Martoni A, Pannuti F, Camisa R, Cocconi G, Colozza A, Passalacqua R, Aogi K, Takashima S, Abe O, Ikeda T, Inokuchi K, Kikuchi K, Sawa K, Sonoo H, Korzeniowski S, Skolyszewski J, Ogawa M, Yamashita J, Bastiaannet E, van de Velde CJ, van de Water W, van Nes JG, Christiaens R, Neven P, Paridaens R, Van den Bogaert W, Braun S, Janni W, Martin P, Romain S, Janauer M, Seifert M, Sevelda P, Zielinski CC, Hakes T, Hudis CA, Norton L, Wittes R, Giokas G, Kondylis D, Lissaios B, de la Huerta R, Sainz MG, Altemus R, Camphausen K, Cowan K, Danforth D, Lichter A, Lippman M, O'Shaughnessy J, Pierce LJ, Steinberg S, Venzon D, Zujewski JA, D'Amico C, Lioce M, Paradiso A, Chapman JA, Gelmon K, Goss PE, Levine MN, Meyer R, Parulekar W, Pater JL, Pritchard KI, Shepherd LE, Tu D, Whelan T, Nomura Y, Ohno S, Anderson A, Bass G, Brown A, Bryant J, Costantino J, Dignam J, Fisher B, Geyer C, Mamounas EP, Paik S, Redmond C, Swain S, Wickerham L, Wolmark N, Baum M, Jackson IM, Palmer MK, Perez E, Ingle JN, Suman VJ, Bengtsson NO, Emdin S, Jonsson H, Del Mastro L, Venturini M, Lythgoe JP, Swindell R, Kissin M, Erikstein B, Hannisdal E, Jacobsen AB, Varhaug JE, Erikstein B, Gundersen S, Hauer-Jensen M, Høst H, Jacobsen AB, Nissen-Meyer R, Blamey RW, Mitchell AK, Morgan DA, Robertson JF, Ueo H, Di Palma M, Mathé G, Misset JL, Levine M, Pritchard KI, Whelan T, Morimoto K, Sawa K, Takatsuka Y, Crossley E, Harris A, Talbot D, Taylor M, Martin AL, Roché H, Cocconi G, di Blasio B, Ivanov V, Paltuev R, Semiglazov V, Brockschmidt J, Cooper MR, Falkson CI, A'Hern R, Ashley S, Dowsett M, Makris A, Powles TJ, Smith IE, Yarnold JR, Gazet JC, Browne L, Graham P, Corcoran N, Deshpande N, di Martino L, Douglas P, Hacking A, Høst H, Lindtner A, Notter G, Bryant AJ, Ewing GH, Firth LA, Krushen-Kosloski JL, Nissen-Meyer R, Anderson H, Killander F, Malmström P, Rydén L, Arnesson LG, Carstensen J, Dufmats M, Fohlin H, Nordenskjöld B, Söderberg M, Carpenter JT, Murray N, Royle GT, Simmonds PD, Albain K, Barlow W, Crowley J, Hayes D, Gralow J, Green S, Hortobagyi G, Livingston R, Martino S, Osborne CK, Adolfsson J, Bergh J, Bondesson T, Celebioglu F, Dahlberg K, Fornander T, Fredriksson I, Frisell J, Göransson E, Iiristo M, Johansson U, Lenner E, Löfgren L, Nikolaidis P, Perbeck L, Rotstein S, Sandelin K, Skoog L, Svane G, af Trampe E, Wadström C, Castiglione M, Goldhirsch A, Maibach R, Senn HJ, Thürlimann B, Hakama M, Holli K, Isola J, Rouhento K, Saaristo R, Brenner H, Hercbergs A, Martin AL, Roché H, Yoshimoto M, Paterson AH, Pritchard KI, Fyles A, Meakin JW, Panzarella T, Pritchard KI, Bahi J, Reid M, Spittle M, Bishop H, Bundred NJ, Cuzick J, Ellis IO, Fentiman IS, Forbes JF, Forsyth S, George WD, Pinder SE, Sestak I, Deutsch GP, Gray R, Kwong DL, Pai VR, Peto R, Senanayake F, Boccardo F, Rubagotti A, Baum M, Forsyth S, Hackshaw A, Houghton J, Ledermann J, Monson K, Tobias JS, Carlomagno C, De Laurentiis M, De Placido S, Williams L, Hayes D, Pierce LJ, Broglio K, Buzdar AU, Love RR, Ahlgren J, Garmo H, Holmberg L, Liljegren G, Lindman H, Wärnberg F, Asmar L, Jones SE, Gluz O, Harbeck N, Liedtke C, Nitz U, Litton A, Wallgren A, Karlsson P, Linderholm BK, Chlebowski RT, Caffier H.
This is what medicine should look like. An honest list of all the people involved, free access to information, and all the data pooled together, giving the most accurate information we can manage, to inform real decisions, and so prevent avoidable suffering and death.
We are a very, very long way away from there.
What can be done?
We urgently need to improve access to trial data. In addition to the previous suggestions, there are small changes that would vastly improve access to information, and so improve patient care.
The results of all trials conducted on humans must be made available within one year of completion, in summary table form if academic journal publication has not occurred. This requires the creation of a body that is charged with publicly auditing whether or not trials have withheld results at twelve months; and primary legislation that is enforced, as a matter of urgency, internationally, with stiff penalties for transgression. In my view these penalties should include fines, but also prison terms for those who are found to be responsible for withholding trial data, as patients are harmed by this process.
All systematic reviews – such as Cochrane reviews – that draw together trial results on any clinical question should also include a section on the trials which they know have been conducted, but whose results are being withheld. This should state: which completed trials have not reported results; how many patients’ worth of information there is in each unreported trial; the names of the organisations and the individuals who are withholding the data; the efforts the reviewers have made to get the information from them. This is trivial extra work, as review teams already attempt to access this kind of data. Documenting this will draw attention to the problem, and make it easier for doctors and the public to see who is responsible for harming patient care in each area of medicine.
All clinical study reports should also be made publicly available, for all the trials that have ever been conducted on humans. This will be cheap, as the only costs are in finding one paper copy, scanning it and placing it online, perhaps with a check to remove confidential patient information. There is a vast mountain of highly relevant data on drugs that is currently being withheld, distorting what we know about treatments in widespread current use. Many of these documents will be sitting in the dry paper archives of drug companies and regulators. We need legislation compelling the industry to hand them over. Our failure to fix this is costing lives.
We need to work on new methods for academics to extract summary information from these documents, as they are more detailed than published academic papers. The Cochrane group working on Tamiflu have made great progress here, learning as they go, and this field will need manuals.
We should work towards all triallists having an obligation to share patient-level data wherever possible, with convenient online data warehouses,85 and streamlined systems whereby legitimate senior researchers can make requests for access, in order to conduct pooled analyses and double-check the results reported in published trials.
None of this is difficult, or impossible. Some of it is technical, for which I apologise. The field of missing data is a tragic and strange one. We have tolerated the emergence of a culture in medicine where information is routinely withheld, and we have blinded ourselves to the unnecessary suffering and death that follows from this. The people we should have been able to trust to handle all this behind the scenes – the regulators, the politicians, the senior academics, the patient organisations, the professional bodies, the universities, the ethics committees – have almost all failed us. And so I have had to inflict the details on you, in the hope that you can bring some pressure to bear yourself.
If you have any ideas about how we can fix this, and how we can force access to trial data – politically or technically – please write them up, post them online, and tell me where to find them.
2
Where Do New Drugs Come From?
Where do drugs come from?
That is the tale of hidden trial data. In the remainder of this book we will see how the pharmaceutical industry distorts doctors’ beliefs about drugs through misleading and covert marketing; we will also see how trials can be flawed by design, and how regulators fail to regulate. But first, we must see how drugs are invented in the first place, and how they come to be available for prescription at all. This is a dark art, and generally remains a mystery for both doctors and patients. There are hidden traps at every turn, odd incentives, and frightening tales of exploitation. This is where new drugs come from.
From laboratory to pill
A drug is a molecule that does something useful, somewhere in the human body,1 and luckily, there’s no shortage of such molecules. Some are found in nature, in particular from plants: this makes sense, because we share a lot of molecular make-up with plants. Sometimes you just extract the molecule, but more commonly you add a few bits onto it here and there, through some elaborate chemical process, or take a few bits away, in the hope of increasing the potency, or reducing the side effects.
&nb
sp; Often you’ll have some idea about the mechanism you’re targeting, and usually that’s because you’re copying the mechanism by which an existing drug works. For example, there’s an enzyme in the body called cyclooxygenase, which helps to make molecules that signal inflammation. If you stop that enzyme working, it helps to reduce pain. Lots of drugs work like this, including aspirin, paracetamol, ibuprofen, ketoprofen, fenoprofen and so on. If you can find a new molecule that stops cyclooxygenase working in a laboratory dish, then it’s probably going to stop it working in an animal, and if it does that, then it’s probably going to help reduce pain in a person. If nothing disastrous has happened to animals or humans in the past when they’ve taken a drug that stops that enzyme working, then your new drug is fairly likely (though not certain) to be safe.
A new drug that operates in a completely new way is much more of a development risk, because it’s unpredictable, and much more likely to fail at every step described above. But that kind of new drug would also be a more significant move forward in medical science. We’ll discuss the tension between copying and innovating later.
One way that drugs are developed is by a process called screening, one of the most boring jobs imaginable for a young laboratory scientist. Hundreds, maybe thousands, of molecules, all of slightly different shapes and sizes, will be synthesised in the hope that they will operate on a particular target in the body. Then you come up with a lab method that lets you measure whether the drug is inducing the change you hope for – stopping an enzyme from working properly, for example – and then you try every drug out, one after the other, measuring their effects until you come up with a good one. Lots of great data is created during this period, and then thrown away, or locked in one drug company’s vault.