by Tim Noakes
We understand that the authors set isolcaloric as a criterion, which would mitigate the satiety advantage of low(er)-carbohydrate (CHO) diets.
We understand that the authors selected studies with an average CHO intake of 35% (35% fat, 30% protein) to represent ‘low CHO’ diets, which is substantially different from the 5% CHO (<50 g/day), moderate-protein and high-fat diet that is used for the therapeutic management of obesity and type 2 diabetes mellitus.
We understand that they set an inclusion criterion of 25–35% fat in the so-called balanced diet. This criterion was reiterated in Tables 2 and 3 and yet ignored by the authors, as they included two studies that failed their own criterion. These errors remain unaddressed by the authors in any of their responses.
We understand that they set the key outcome measure as ‘total weight change’. They then used end weight, with no reference to start weight, in two studies, which was in breach of their own protocol and absurd. These errors remain unaddressed by the authors.
In their response to the SAMJ, the authors have accepted only one of the numerous errors that we documented in their article – their admission that they included a duplicate study. This alone is grounds for retraction of the article.
The authors have not addressed any of the other numerous errors, material or otherwise, which we documented.
We then once more went through each of the material errors that the authors continued to ignore, at the same time addressing a set of red herrings that they had introduced in their response, and which only indicated that there were additional material errors that we had not originally identified.
It is understandable that the authors chose rather to raise new issues (ITT, standard mean difference and significance) and to ignore the numerous errors that we identified in our critique, presumably because they have no cogent answers. As a result, those material errors have remained unanswered since October 2016 and unless addressed, they render the article worthless, other than of retraction.
Given that only one error has been addressed and accepted (the duplication), we may never receive an answer to our research question: was this mistake or mischief? We may also never know if Prof. Noakes would have suffered for years in the way he has, had this article not made competence or conspiratorial errors.
Especially because ‘our repeat of the authors’ meta-analysis, using their methodology, but without the errors, produced a different result – a result that would not have given those keen to prosecute Prof. Noakes the ammunition they were “waiting for”’ – namely, that the ‘low-carbohydrate’ diet, even as inappropriately described by Naudé and her colleagues, outperformed the ‘balanced’ diet. Which was exactly what I had suspected might be the case when Harcombe and I began our re-analysis in July 2016.
This undignified academic tussle raises a number of tough questions. First, because Naudé et al. are so reluctant to explain the origin of the material errors in their paper, we may never know whether they were simple mistakes or part of something more sinister. Second, and related, if the meta-analysis had been performed properly and had shown that the ‘low-carbohydrate’ diet outperformed the low-fat ‘balanced’ diet, it would have been extraordinarily difficult for the HPCSA preliminary inquiry committee to initiate their action against me. These six authors therefore played a material role in bringing about most of what we describe in this book. Was this by design?
Third, the reluctance of the editors of PLoS One to properly investigate the nature of the material errors raises questions of who the journal is protecting, and why. If the journal wanted to make a statement but did not wish to investigate the authors for data manipulation, it could simply have withdrawn the paper on the basis that the analysis included duplicate publications. Fourth, the failure of the universities and the SAMRC to take action raises similar questions. The credibility of Cochrane organisations worldwide has also been tarnished, yet they, too, have done nothing.
So what effects did the publication of this paper have on me and the six authors?
I was saddled with a multimillion-rand hearing that lasted more than three years. Celeste Naudé, however, was appointed co-director of the newly established Cochrane Nutrition Field (CNF) on 23 August 2016. A Stellenbosch University Faculty of Medicine and Health Sciences press release stated:
The CNF will be under the leadership of Cochrane South Africa, the South African Medical Research Council, and the Centre for Evidence-based Health Care (CEBHC) at the FMHS [Faculty of Medicine and Health Sciences], along with international partners.
The field will be led by Co-Directors Solange Durão of Cochrane SA and Celeste Naude of the CEBHC, with guidance from an international advisory board comprising representatives from multiple stakeholder and partner groups.
‘The vision of Cochrane Nutrition is that Cochrane will be the independent, globally recognised go-to place for nutrition systematic reviews,’ said Solange Durão. ‘Cochrane Nutrition will support and enable evidence-informed decision-making for nutrition policy and practice by advancing the production and use of high-quality, globally relevant nutrition-related Cochrane reviews,’ she continued …
‘Cochrane Nutrition will aim to coordinate activities related to nutrition reviews within Cochrane; to ensure that priority nutrition reviews are conducted with rigorous methodological approaches; and, to promote the use of evidence from nutrition systematic reviews to inform healthcare decision-making,’ said Celeste Naude.
‘An exploratory meeting with interested stakeholders held in Cape Town in 2015 established that there is broad-based support for such a field from both Cochrane and external stakeholders,’ she continued.22
We can be certain of the exact nature of the nutrition advice that the public will be receiving from the CNF. My guess is that the South African food industry, in particular, would have been especially pleased by Naudé’s appointment.
For example, in another meta-analysis, Naudé proposes that ‘increasing vegetable and fruit intake in South Africa could potentially contribute to reducing the burden of nutrition-related conditions in this country. Increasing vegetable and fruit intake in preschool children could improve their vitamin A nutriture.’23
She is prepared to make this suggestion even though (a) the optimum sources of vitamin A are animal or fish, not plant, sources, and (b) she acknowledges that: ‘It should be kept in mind that the quality of the included systematic reviews ranged from low to high (AMSTAR), and most reviews did not assess the scientific quality of the studies.’24
If Naudé honestly wished to improve the vitamin A ‘nutriture’ of South African children, she did not need to undertake this meta-analysis of low-quality ‘evidence’. Rather, she could simply have studied a textbook of human nutrition. Or read the publication of one of the HPCSA’s expert witness, Muhammad Ali Dhansay, whose study found that the prevalence of vitamin A deficiency in undernourished South African schoolchildren was only 6 per cent in those who ate liver at least once a month, compared to the national deficiency prevalence of 64 per cent.25
But this would require that Naudé promote the eating of animal produce, a suggestion which, in my opinion, probably conflicts with her own dietary inclinations. So her final conclusion remains: ‘This evidence supports the need to promote greater vegetable and fruit intake in South Africa.’
It appears that according to Naudé’s logic, provided one acknowledges that the evidence is weak, once can still draw exactly the same definitive conclusion that one wished to derive, even before the meta-analysis has begun. This is known as confirmation bias and is unscientific.
When I was being trained in medicine in the 1970s, the standard joke was that orthopaedic surgeons use science in the same way that a drunk uses a lamp post: for support, not for illumination. Now I would argue, as perhaps does Professor John Ioannidis, discussed shortly, that nutritional epidemiologists are the modern drunkards of medical science.
I am sure many more similar meta-analyses will be forthcoming from the
Centre for Evidence-based Health Care over the next years and decades, and these will continue to promote the essentiality of ‘balanced, prudent, heart-healthy diets, in moderation’, the value of plant-based diets, the dangers of animal produce and the absolute innocence of sugar. So do we really need a CEBHC if we already know what they will conclude?
The sole UCT representative on the Naudé review, Marjanne Senekal, was promoted to full professor at the university’s Faculty of Health Sciences on 9 May 2017.
In November 2016, the dean of medicine at Stellenbosch University, Jimmy Volmink (who had moderated the Centenary Debate), received an SAMRC recognition award for ‘outstanding achievements in contributions to evidence-based healthcare in Africa’. The Faculty of Medicine and Health Sciences press release stated:
The African region’s ‘father of evidence-based healthcare’ is what the South African Medical Research Council (MRC) called Prof Jimmy Volmink …
‘Volmink is an internationally renowned researcher, leader, mentor, critic and teacher in clinical epidemiology and evidence-based healthcare,’ the MRC said in their commendation statement …
‘Volmink was never afraid to challenge the accepted norms and doctrines. For example, his systematic reviews of the evidence regarding Directly Observed Therapy for Tuberculosis and Prevention of Mother to Child Transmission of HIV directly contradicted South Africa health policy at the time, and highlighted the complexity of policymaking in spite of available best evidence,’ the statement continued.
‘His work was underscored by the mission of Cochrane South Africa which stated that health care decision-making on the African continent should be informed by best-available evidence.’26
Given his stated position at the end of the Centenary Debate, specifically that he had ‘heard nothing this evening that convinces me that we are on the wrong path in terms of these dietary guidelines’ and that ‘at all costs we must not do harm’, I am sure Volmink can be trusted not to challenge ‘the accepted norms and doctrines’ of our current dietary guidelines. This despite the fact that, as dean of the medical faculty at the epicentre of the South African diabetes epidemic, he must be exposed to the tragic personal and other costs of this plague daily.
I wonder if South African medical science should really be investing so much effort in developing its capacity to undertake yet more meta-analyses. Professor John Ioannidis of Stanford University thinks not. The title of his article on the topic signals his unconventional position: ‘The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses’.27 Figure 6.2 shows the massive increase in the number of systematic reviews and meta-analyses since 1986. If ever there was a growth industry in science, this is it.
Figure 6.2
Number of ‘systematic reviews’ or meta-analyses published between 1986 and 2014. Reproduced from Ioannidis, ‘The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses’, The Milbank Quarterly
His conclusions about the real value of such an explosion are not so sanguine:
Currently, there is massive production of unnecessary, misleading, and conflicted systematic reviews and meta-analyses. Instead of promoting evidence-based medicine and health care, these instruments often serve mostly as easily produced publishable units or marketing tools. Suboptimal systematic reviews and meta-analyses can be harmful given the major prestige and influence these types of studies have acquired. The publication of systematic reviews and meta-analyses should be realigned to remove biases and vested interests and to integrate them better with the primary production of evidence.
Figure 6.3 on page 132 shows Ioannidis’s analysis of the value of all those systematic reviews and meta-analyses. His conclusion is that only 3 per cent are decent and clinically useful; 17 per cent are decent, but not useful; 20 per cent are flawed beyond repair; and 27 per cent are redundant and unnecessary. Of the rest, 13 per cent were misleading and 20 per cent went unpublished.
Figure 6.3
A summary of the usefulness of currently produced meta-analyses. Reproduced from Ioannidis, ‘The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses’, The Milbank Quarterly
Will the CNF under the co-direction of Naudé really make any difference to the future nutritional health of South Africans? When will nutritional epidemiologists acknowledge that memory-based dietary assessment methods – their key measurement tools – are ‘fundamentally flawed owing to well-established scientific facts and analytic truths’?28 So, should we not perhaps rather focus on human biology and how it is influenced by what we eat? In my opinion, this is the only way we are ever going to return the majority of South Africans to some measure of improved health.
The overriding message of the Naudé affair is that, while you have a responsibility to report what you think is scientific fraud, never expect anyone to thank you for your troubles.
7
Responses of Official Bodies
‘The most difficult subjects can be explained to the most slow-witted man if he has not formed any idea of them already; but the simplest thing cannot be made clear to the most intelligent man if he is firmly persuaded that he knows already, without a shadow of doubt, what is laid before him.’
– Leo Tolstoy, Russian author1
Although most of the attacks on me and the Banting/LCHF/Noakes diet originated from within South African academic institutions, a number were instigated by official bodies interested in maintaining the status quo. Indeed, the very first public attack came from the Centre for Diabetes and Endocrinology (CDE) in Johannesburg.
Following the publication of my article in the Discovery magazine in 2011 and the updated edition of Challenging Beliefs a few months later, the CDE released a position statement on 4 April 2012 to counter what I was saying. They claimed that ‘“carb-free”, “low-carb”, “high-protein” and other such diets have shown no long-term benefit over conventionally balanced healthy eating plans’. The public, they warned, ‘should receive their guidance from practitioners trained in diabetes rather than from the media’.2
The CDE encourages a uniform approach to the management of diabetes in South Africa. It does this by training and certifying physicians in 280 centres across the country in a standard, CDE-certified method for managing diabetes. To its credit, the CDE monitors patient outcomes in an attempt to promote what it believes are best practices for the management of T2DM in South Africa. The problem is that, when looking at patient outcomes, the CDE sees what it has become programmed to see.
The CDE promotes high-carbohydrate diets and the liberal use of insulin to ‘control’ the predictably high blood glucose concentrations that occur in diabetic patients whenever they eat carbohydrates.3 In Chapter 17, I present evidence that long-term studies have not found that the prescription of insulin to patients with either type-1 diabetes mellitus (T1DM) or T2DM is the optimal form of management.
The CDE position statement makes three key points:
‘Diabetes UK recommends that for a 2000 kcal (8400 kJ) diet, 45%–60% of the total energy [for patients with either form of diabetes] should be supplied by carbohydrates (225–300 g per day).’
‘Not only would a low-carbohydrate diet not be recommended for those with type 1 diabetes, but also it could be considered to be absolutely contraindicated.’
‘The use of a low-carbohydrate diet in individuals with type 1 diabetes may well promote ketosis and predispose these individuals to either ketoacidosis or to severe hypoglycaemia following exercise.’
Predictably, because South African–trained dietitians are prohibited from prescribing LCHF diets, CDE-certified dietitians cannot understand the cardinal features of the pathophysiology of diabetes, or why prescribing high-carbohydrate diets to people with diabetes is, in my opinion, medically negligent.
My advice that those with either form of diabetes should eat carbohydrate-restricted diets is based on a number of scientific studies, whi
ch have established the benefits of this approach.4 In addition, the most basic understanding of human carbohydrate metabolism supports this position. Here is my counterargument to the CDE position statement:
1. Carbohydrates are not an essential component of the human diet. There is no known medical condition that is caused by a deficiency of dietary carbohydrate. As a result, the influential US National Academy of Medicine, formerly called the Institute of Medicine, concludes: ‘The lower limit of dietary carbohydrate compatible with life apparently is zero, provided that adequate amounts of protein and fat are consumed.’5 Hence, no human, and especially not anyone with an impaired capacity to metabolise carbohydrate, needs to eat carbohydrate.
2. This is to be expected, because, until the Neolithic period, which began with the agricultural revolution about 12 000 years ago, carbohydrates provided a minority of the food energy ingested by humans and human ancestors; most of the energy came from fat and protein.6 Only in the last 40 years has carbohydrate become the major source of food energy in Europe and North America, as is now the case in all countries swamped by the obesity/T2DM epidemic.7
3. It is conventionally argued that people with diabetes must eat carbohydrates in order to ensure that their brains receive sufficient glucose for optimal function. While the human brain does have an obligatory glucose requirement of about 25 grams per day, this can be adequately supplied by the liver, which produces glucose from fat and protein in the process known as hepatic gluconeogenesis. We see this in the traditional Arctic Inuit and the Masai of East Africa, who eat essentially no carbohydrate, other than that present in milk.
Importantly, but often overlooked, the key biological feature of T2DM is the uncontrolled overproduction of glucose by the liver. Logically, if diabetic patients are already in a state of glucose overproduction, then their brains, as well as all their bodily tissues, must already be wallowing in an excess of carbohydrate. Why would they need more? How will the excess carbohydrates they are commanded to eat reverse their biological abnormality? Logically, it will only increase their already too high blood glucose concentrations.