2008 - Bad Science

by Ben Goldacre

  It is not a surprise to me that there are entrepreneurs and gurus—individuals—selling their pills and their ideas on the open market. In some strange sense I respect and admire their tenacity. But it strikes me that universities have a very different set of responsibilities, and in the field of nutrition there is a particular danger. Homeopathy degrees, at least, are transparent. The universities where it is taught are secretive and sheepish about their courses (perhaps because when the exam papers leak, it turns out that they’re asking questions about ‘miasma’—in 2008) but at least these degrees in alternative therapies are what they say on the tin.

  The nutritionists’ project is more interesting: this work takes the form of science—the language, the pills and the referenci-ness—making claims that superficially mirror the assertions made by academics in the field of nutrition, where there is much real science to be done. Occasionally there may be some good evidence for their assertions (although I can’t imagine the point of taking health advice from someone who is only occasionally correct). But in reality the work of ‘nutritionists’ is often, as we have seen, rooted in New Age alternative therapy, and while reiki quantum energy healing is fairly clear about where it’s coming from, nutritionists have adopted the cloak of scientific authority so plausibly, with a smattering of common-sense lifestyle advice and a few references, that most people have barely spotted the discipline for what it is. On very close questioning, some nutritionists will acknowledge that theirs is a ‘complementary or alternative therapy’, but the House of Lords inquiry into alternative medicines, for example, didn’t even list it as one.

  This proximity to real academic scientific work summons up sufficient paradoxes that it is reasonable to wonder what might happen in Teesside when Professor Holford begins to help shape young minds. In one room, we can only imagine, a full-time academic will teach that you should look at the totality of evidence rather than cherry-pick, that you cannot over-extrapolate from preliminary lab data, that referencing should be accurate, and should reflect the content of the paper you are citing, and everything else that an academic department might teach about science and health. In another room, will there be Patrick Holford, exhibiting the scholarship we have already witnessed?

  We can have one very direct insight into this clash from a recent Holford mailout. Periodically, inevitably, a large academic study will be published which finds no evidence of benefit from one of Patrick Holford’s favoured pills. Often he will issue a confused and angry rebuttal, and these critiques are highly influential behind the scenes: snippets of them frequently appear in newspaper articles, and traces of their flawed logic emerge in discussions with nutritionists.

  In one, for example, he attacked a meta-analysis of randomised controlled trials of antioxidants as being biased, because it excluded two trials he said were positive. In fact they were not trials, they were simply observational surveys, and so could never have been included. On the occasion we are interested in, Patrick Holford was angry about a meta-analysis on omega-3 fats (such as fish oils), co-authored by Professor Carolyn Summerbell: she holds the full-time academic chair in Nutrition at Teesside University, where she is also Assistant Dean of Research, with a long-standing track record of published academic research in the field of nutrition.

  In this case, Holford seems quite simply not to understand the main results statistics in the paper’s results blobbogram, which showed no benefit for the fish oils.*

  ≡ There is a more detailed explanation of his misunderstanding online, but for nerds, it seems he is amazed that several studies with a non-significant trend towards showing a benefit for fish-oil pills do not collectively add up to show a statistically significant benefit. This is in fact, as you know, rather commonplace. There are several other interesting criticisms to be made of the omega-3 research paper, as there always are of any research paper, but sadly this, from Holford, is not one of them.

  Furious at what he thought he had found, Professor Holford then went on to accuse the authors of being pawns of the pharmaceutical industry (you may be spotting a pattern). ‘What I find particularly deceptive is that this obvious skew is not even discussed in the research paper,’ he says. ‘It really makes me question the integrity of the authors and the journal.’ He is talking here, remember, about the Professor of Nutrition at Teesside University and Assistant Dean of Research. Things then deteriorate further. ‘Let’s explore that for a minute with a ‘conspiracy theory’ hat on. Last week pharmaceutical drug sales topped $600 billion. The number one best seller was Lipitor, a statin drug for lowering cholesterol. It brought in $12.9 billion…’

  Let us be clear: there is no doubt that there are serious problems with the pharmaceutical industry—I should know, I teach both medical students and doctors on the subject, I write about it regularly in national newspapers, and I am about to walk you through their evils in the next chapter—but the answer to this problem is not bad scholarship, nor is it another substitute set of pills from a related industry. Enough.

  How did Holford come to be appointed?

  David Colquhoun is Emeritus Professor in Pharmacology at UCL, and runs a magnificently shouty science blog at dcscience.net. Concerned, he obtained the ‘case’ for Professor Holford’s appointment using the Freedom of Information Act, and posted it online. There are some interesting finds. Firstly, Teesside accepts that this is an unusual case. It goes on to explain that Holford is director of the Food for the Brain Foundation, which will be donating funds for a PhD bursary, and that he could help in a university autism clinic.

  I am not going to dwell on Holford’s CV—because I want to stay focused on the science—but the one sent to Teesside makes a good starting point for a brief biography. It says that he was at York studying experimental psychology from 1973 to 1976, before studying in America under two researchers in mental health and nutrition (Carl Pfeiffer and Abram Hoffer), and then returning to the UK in 1980 to treat ‘mental health patients with nutritional medicine’. In fact 1975 was the first year that York ran a degree in psychology. Ho! ford actually attended from 1976 to 1979, and after getting a .2:2 degree he began his first job, working as a salesman for the supplement-pill company Higher Nature. So he was treating patients in 1980, one year out of this undergraduate degree. Not a problem. I’m just trying to get this clear in my mind.

  He set up the Institute of Optimum Nutrition in 1984, and he was director until 1998: it must therefoie have been a touching and unexpected tribute for Patrick in 1995 when the Institute conferred upon him a Diploma in Nutritional Therapy. Since he started but failed to complete his Mphil in nutrition at the University of Surrey twenty years ago, this Dip.ION from his own organisation remains his only qualification in nutrition.

  I could go on, but I find it unseemly, and also these are dreary details. OK, one more, but you’ll have to read the rest online:

  In 1986 he started researching the effects of nutrition on intelligence, collaborating with Gwillym Roberts, a headmaster and student at ION. This culminated in a randomised controlled trial testing the effects of improved nutrition on children’s IQ—an experiment that was the subject of a Horizon documentary and published in the Lancet in 1988.

  I have this Lancet paper in front of me. It does not feature Holford’s name anywhere. Not as an author, and not even as an acknowledgement.

  Let’s get back to his science, post haste. Could Teesside have easily discovered that there were reasons to be concerned about Patrick Holford’s take on science, without deploying any evidence, before they appointed him as a visiting professor? Yes. Simply by reading the brochures from his own company, Health Products for Life. Among the many pills, for example, they might have found his promotion and endorsement of the QLink pendant, at just £69.99. The QLink is a device sold to protect you from terrifying invisible electromagnetic rays, which Holford is eager to talk about, and it cures many ills. According to Holford’s catalogue:

  It needs no batteries as it is ‘powered’ by the weare
r—the microchip is activated by a copper induction coil which picks up sufficient micro currents from your heart to power the pendant.

  The manufacturers explain that the QLink corrects your ‘energy frequencies’. It has been covered in praise by The Times, the Daily Mail and ITV’s London Today, and it’s easy to see why: it looks a bit like a digital memory card for a camera, with eight contact pads on the circuit board on the front, a hi-tech electronic component mounted in the centre, and a copper coil around the edge.

  Last summer I bought one and took it to Camp Dorkbot, an annual festival for dorks held—in a joke taken too far—at a scout camp outside Dorking. Here, in the sunshine, some of the nation’s more childish electronics geeks examined the QLink. We chucked probes at it, and tried to detect any’frequencies’ emitted, but no luck. Then we did what any dork does when presented with an interesting device: we broke it open. Drilling down, the first thing we came to was the circuit board. This, we noted with some amusement, was not in any sense connected to the copper coil, and therefore it is not powered by the coil, as claimed.

  The eight copper pads did have some intriguing-looking circuit-board tracks coming out of them, but on close inspection these were connected to absolutely nothing. You might call them ‘decorative’. I should mention, in the name of accuracy, that I’m not clear if I can call something a ‘circuit board’ when there is no ‘circuit’.

  Finally, there is a modern surface-mount electronic component soldered to the centre of the device, prominently on display through the clear plastic cover. It looks impressive, but whatever it is, it is connected to absolutely nothing. Close examination with a magnifying glass, and experiments with a multimeter and an oscilloscope, revealed that this component on the ‘circuit board’ was a zero-ohm resistor. This is a resistor that has no resistance: a bit of wire in a tiny box. It sounds like a useless component, but they’re actually quite useful for bridging a gap between adjacent tracks on a circuit board. (I feel I should apologise for knowing that.)

  Now, such a component is not cheap. We must assume that this is an extremely high-quality surface-mount resistor, manufactured to very high tolerances, well calibrated, and sourced in small quantities. You buy them on paper tape in seven-inch reels, each reel containing about 5,000 resistors, and you could easily pay as much as £0.005 for such a resistor. Sorry, I was being sarcastic. Zero ohm resistors are extremely cheap. That’s the QLink pendant. No microchip. A coil connected to nothing. And a zero-ohm resistor, which costs half a penny, and is also connected to nothing.*

  ≡ I contacted qlinkworld.co.uk to discuss my findings. They kindly contacted the inventor, who informed me that they have always been clear that the QLink does not use electronics components ‘in a conventional electronic way’. Apparently the energy pattern reprogramming work is done by some finely powdered crystal embedded in the resin. I think that means it’s a New Age crystal pendant, in which case they could just say so.

  Teesside is only part of the story. Our main reason for showing an interest in Patrick Holford is his phenomenal influence on the nutritionist community in the UK. As I have mentioned, I have a huge respect for the people I am writing about in this book, and I am happy to flatter Holford by saying that the modern phenomenon of nutritionism which pervades every aspect of the media is in large part his doing, through the graduates of his phenomenally successful Institute for Optimum Nutrition, where he still teaches. This institute has trained the majority of self-styled nutrition therapists in the UK, including Vicki Edgson from Diet Doctors on Channel Five, and Ian Marber, owner of the extensive ‘Food Doctor’ product range. It has hundreds of students.

  We’ve seen some examples of the standard of Holford’s scholarship. What happens in his Institute? Are its students, we might wonder, being tutored in the academic ways of its founder?

  As an outsider, it’s hard to tell. If you visit the academic-sounding website, www.ion.ac.uk (registered before the current rules on academic .ac.uk web addresses), you won’t find a list of academics on the staff, or research programmes in progress, in the way that you would, say, for the Institute for Cognitive Neurosciences in London. Nor will you find a list of academic publications. When I rang up the press office once to get one, I was told about some magazine articles, and then when I explained what I really meant, the press officer went away, and came back, and told me that ION was ‘a research institute, so they don’t have time for academic papers and stuff’.

  Slowly, more so since Holford’s departure as head (he still teaches there), the Institute of Optimum Nutrition has managed to squeeze some respectability out of its office space in south-west London. It has managed to get its diploma properly accredited, by the University of Luton, and it now counts as a ‘foundation degree’. With one more year of study, if you can find anyone to take you—that is, the University of Luton—you can convert your JON diploma into a full BSc science degree.

  If, in casual conversation with nutritionists, I question the standards of the ION, this accreditation is frequently raised, so we might look at it very briefly. Luton, previously the Luton College of Higher Education, now the University of Bedfordshire, was the subject of a special inspection by the Quality Assurance Agency for Higher Education in 2005. The QAA is there to ‘safeguard the academic standards and quality of higher education in the UK’.

  When the QAA’s report was published, the Daily Telegraph ran an article about Luton titled: ‘Is this the Worst University in Britain?’ The answer, I suspect, is yes. But of particular interest to us is the way the report specifically singled out the slapdash approach of the university towards validating external foundation degrees (p. 12, para 45 onwards). It states outright that in the view of the auditing team, the expectations of the code of practice for the assurance of academic quality and standards in higher education, specifically with regard to accrediting foundation degrees, were simply not met. As they go—and I by not to read this kind of document too often—this report is pretty full-on. If you look it up online, I particularly recommend paragraphs 45 to 52.

  At the very moment that this book was going to press, it transpired that Professor Holford has resigned from his post as visiting professor, citing reorganisation in the university. I have time to add just one sentence, and it is this: It will not stop. He is now looking for academic credibility elsewhere. The reality is that this vast industry of nutritionism—and more importantly than anything, this fascinating brand of scholarship—is now penetrating, uncriticised, unnoticed, to the heart of our academic system, because of our desperation to find easy answers to big problems like obesity, our collective need for quick fixes, the willingness of universities to work with industry figures across the board, the admirable desire to give students what they want, and the phenomenal mainstream credibility that these pseudo-academic figures have attained, in a world that has apparently forgotten the importance of critically appraising all scientific claims.

  There are other reasons why these ideas have gone unexamined. One is workload. Patrick Holford, for example, will occasionally respond on an issue of evidence, but often, it seems to me, by producing an even greater cloud of sciencey material: enough to shoo off many critics, perhaps, and certainly reassuring for the followers, but anybody daring to question must be ready to address a potentially exponential mass of content, both from Holford, and also from his extensive array of paid staff. It’s extremely good fun.

  There is also the PCC complaint against me (not upheld, and not even forwarded to the paper for comment), the lengthy legal letters, his claims that the Guardian has corrected articles critical of him (which it most certainly has not), and so on. He writes long letters, sent to huge numbers of people, accusing me and others critical of his work of some rather astonishing things. These claims appear in mail-outs to the customers of his pill shop, in letters to health charities I’ve never heard of, in emails to academics, and in vast web pages: endless thousands of words, mostly revolving around his repeated and rather inco
ngruous claim that I am somehow in the pocket of big pharma. I am not, but I note with some delight that—as I may have mentioned—Patrick, who sold his own pill retail outfit for half a million pounds last year, now works for BioCare, which is 30 per cent owned by a pharmaceutical company.

  I am therefore speaking directly to you now, Professor Patrick Holford. If we disagree on any point of scientific evidence, instead of this stuff about the pharmaceutical industry being out to get you, or a complaint, or a legal letter, instead of airily claiming that queries should be taken up with the scientist whose valid work you are—as I think I have shown—overinterpreting, instead of responding on a different question than the one which was posed, or any other form of theatrics, I would welcome professorial clarification, simply and clearly.

  These are not complicated matters. Either it is acceptable to cherry-pick evidence on, say, vitamin E, or it is not. Either it is reasonable to extrapolate from lab data about cells in a dish to a clinical claim about people with AIDS, or it is not. Either an orange contains vitamin C, or it does not. And so on. Where you have made errors, perhaps you could simply acknowledge that, and correct them. I will always happily do so myself, and indeed have done so many times, on many issues, and felt no great loss of face.

  I welcome other people challenging my ideas: it helps me to refine them.

  10 The Doctor Will Sue You Now

  This chapter did not appear in the original edition of this book, because for fifteen months leading up to September 2008 the vitamin-pill entrepreneur Matthias Rath was suing me personally, and the Guardian, for libel. This strategy brought only mixed success. For all that nutritionists may fantasise in public that any critic is somehow a pawn of big pharma, in private they would do well to remember that, like many my age who work in the public sector, I don’t own a flat. The Guardian generously paid for the lawyers, and in September 2008 Rath dropped his case, which had cost in excess of £500,000 to defend. Rath has paid £220,000 already, and the rest will hopefully follow. Nobody will ever repay me for the endless meetings, the time off work, or the days spent poring over tables filled with endlessly cross-referenced court documents.