by Zoë Harcombe
Using this ovaries one as an interesting example... We can effectively study the same thing happening in humans without removing ovaries as an experiment. Some women do have ovaries removed as part of a hysterectomy or because of an ovarian cyst or other medical condition. We also see the mimicking of ovaries being removed when women reach menopause and eggs are no longer produced, as if the ovaries are no longer there. Having done the rat studies is then very useful because it helps us to observe similarities and to understand possible pathways. We observe rats with ovaries removed eating voraciously. Further studies can show hormonal changes that increase appetite hormones and impair metabolism hormones and we can then understand HOW the ovary changes are impacting weight. This does enable us to advise measures to counter the changes. Whereas we assume that people can largely trust appetite while eating real food/managed carbohydrate, post menopausal women may need to be aware that they may feel ‘inappropriately’ hungry much of the time and to develop strategies to overcome this.
One would need to be an expert in animal biology to know how comparable different animals are to humans. Here is an interesting general article about how useful animal studies are to tell us things about human health. (Link) We can easily research basics like – do rats have a pancreas? – which can tell us if they release insulin and may be of use in blood glucose studies. However, you don’t need to research far to see that the rat may have a pancreas, but it may not be an appropriate comparator for the human pancreas (Link). Rats don’t have gall bladders so will necessarily metabolise fat very differently to humans.
In conclusion for this factor, any studies done on rats or other animals should be treated with caution. A good study will raise the limitations of having done the study on animals and only make reasonable assertions for how this will translate to humans. However, that won’t get them in the Daily Mail, so expect this to happen less and less often.
3) Association vs. causation.
This is the huge one. Two things can be observed at the same time and this is association (at best). Those things can be entirely coincidental or they can be connected but with no causation (in either direction) or they can be causally related (although we can’t tell this from association) or they may be both caused by something else entirely. The best example I’ve found to use is the singing and bathing one. I can observe people in the bath singing, but I can no more say that being in the bath causes singing than I can say singing causes being in the bath! They are just two things that can be observed at the same time.
Epidemiological studies are entirely about association – they can suggest causation, but they cannot prove it. They can show us populations with below average incidence of a particular disease or illness and they can show us another differential that could be related, but they cannot tell us any more than this. They essentially help researchers to know what they should follow up with a clinical trial e.g. don’t bother looking at wine consumption, because this was similar in countries where condition X varied enormously. However, do look further at beer consumption, because an association was detected between beer consumption and condition X.
This introduces another very important factor and one where manipulation can occur. Epidemiological data is often available for most countries in the world. The World Health Organisation goes to great trouble to get health and lifestyle information for as many countries in the world as possible. If researchers then select a sub set of all the data available, this should immediately be challenged – why?!
The Seven Countries Study is the most classic example of this and the one that has had the most catastrophic consequences. In The Great Cholesterol Con, Dr Malcolm Kendrick analyses the World Health Organisation data for the seven countries with the lowest consumption of saturated fat (Georgia, Tajikistan, Azerbaijan, Moldova, Croatia, Macedonia and the Ukraine) vs. the seven countries with the highest consumption of saturated fat (Austria, Finland, Belgium, Iceland, Netherlands, Switzerland and France). He found that every single one of the seven countries with the lowest consumption of saturated fat had significantly higher heart disease than every single one of the countries with the highest consumption of saturated fat. This, of course, concludes the exact opposite of the Keys’ Seven Countries Study.
When I did my original research on cholesterol and heart deaths, I deliberately took all the WHO data for all 192 countries. Why would I pre select a certain number of countries? I wasn’t setting out to test any pre-judged theory. I was setting out to see if there were any relationship between cholesterol levels and heart deaths and was stunned to see the pattern. So stunned that I repeated it for cholesterol levels and all deaths and was even more stunned (Link).
Clinical trials can also claim causation when they have merely observed an association. A classic example would be the ‘study’ being done in the first episode of Food Hospital (Link) where a small group of people were randomly split into two and one half had some white chocolate daily and the other half had some dark chocolate daily. (You can test how good a study this is already – small group = not good; random allocation = good; test is not blind, let alone double blind = not good; I didn’t see any effort to establish if the group differed in any other way or if they were told to add the chocolate and make no other change whatsoever to their diet or lifestyle until the experiment ended = not good).
The voice over on the programme described the two versions of chocolate as nutritionally comparable – are they serious? I bet they have matched the same number of calories and maybe fat grams, but the carbs, sugar, protein, minerals, vitamins etc will be vastly different. Let us say that they find blood pressure drops in the dark chocolate group, they will jump to the conclusion – dark chocolate reduces blood pressure (immediate leap to causation). It could be that iron (of which dark chocolate is a rich source) reduces blood pressure and that the dark chocolate is merely a delivery mechanism. It could then be argued that the ‘how’ doesn’t matter, the fact is that dark chocolate reduces blood pressure. However, if the causal factor is iron (and all this is just an example), then iron needs vitamin C for its absorption. So, eating dark chocolate alone may not reduce blood pressure. Both the white and dark chocolate groups may have been diligent about their 5-a-day (ha ha) and have been getting plenty of vitamin C. We can’t then say that eating dark chocolate reduces blood pressure. We would have to say – eating dark chocolate can reduce blood pressure if vitamin C intake is sufficient. We can also say (again – just using this as an example) – eating anything containing iron, in the presence of sufficient vitamin C, will reduce blood pressure. We also don’t know if the caffeine in dark chocolate will raise blood pressure and if something else in dark chocolate will more than counter this. You can start to see how this ‘change only one thing’ in clinical trials is virtually impossible to achieve.
4) Get the definitions wrong.
This is also a classic error made – particularly with our dear friend saturated fat. It can happen in both clinical trial and epidemiological studies. It happened in the original Seven Countries Study, which led to the damnation of fat and the subsequent obesity epidemic, as we replaced natural fats with unnatural carbohydrates.
I go through the Seven Countries Study in great detail in Chapter 8 of The Obesity Epidemic, because it is the study that changed the way that the world eats. In the original publication about the study – in Circulation, April 1970, the claims that Keys made were three fold:
i) CHD (Coronary Heart Disease) tends to be directly related to serum cholesterol;
ii) Serum cholesterol tends to be directly related to saturated fat as a proportion of the diet;
iii) CHD is as closely related to saturated fat as it is cholesterol.
Please look at these assertions carefully. It should be noted at the outset that Keys did NOT say that saturated fat consumption causes anything; certainly not that it causes heart disease. The study did not assert that even a consistent association exists between saturated fat and heart disease and/o
r saturated fat and cholesterol and/or cholesterol and heart disease (although you really would not believe this when you see how entrenched diet advice has become since – and this study is the catalytic foundation of all views that saturated fat is bad for us).
The key mistake in the Seven Countries Study, however, was one of wrong definitions. I credit Dr Robert Lustig in Chapter 8 for finding this key passage in a 1980 publication by Keys “Seven Countries: a multivariate analysis of death and coronary heart disease”: “The fact that the incidence of coronary heart disease was significantly correlated with the average percentage of calories from sucrose in the diets is explained by the inter correlation of sucrose with saturated fat.”
There is only one real food on the planet containing sucrose and saturated fat and that is avocado. We know from this passage above, therefore, that Keys was measuring processed food in his study. Having read all twenty volumes of the study I can confirm that a) there is barely any information whatsoever about diet in a number of the cohorts and b) references to fat include cakes and ice cream and c) Keys has assumed that meat and eggs are saturated fat – they do contain saturated fat, but they contain more unsaturated fat.
Notwithstanding that the Seven Countries Study did not find or even allege causation between fat and anything, Keys was not measuring fat. He was measuring processed products – assuming that this was not a study about avocados! Hence, any association that Keys may have found cannot isolate saturated fat from unsaturated fat, let alone fat from sugar/flour carbohydrates. Is it the ultimate irony that Keys measured processed carbohydrates thinking that he was measuring fats and then demonised fat leading to us eating more carbohydrates – the real problem in the first place?!
The same can and does happen in clinical trials. In Chapter 11 in The Obesity Epidemic I go through the 17 studies relied upon by the Food Standards Agency in their continued demonisation of fat. Each of these studies claims to be a clinical trial looking at the impact of changing fat intake in a control group vs. an intervention group. Six studies need to be discarded because they are multi factorial trials. This means that several things were changed at the same time and therefore no difference can be attributed to any one change.
For example, you may have heard of a trial called MRFIT (1990) and this actually stands for Multiple Risk Factor Intervention Trial – bit of a clue there that many things were changed all at once! The things that changed in the intervention group were smoking cessation, diet, medication for hypertension and medication for water retention (diuretics). Even with all these interventions, 8.3% of the control group died and 7.7% of the intervention group died during the study. I would have expected stopping smoking alone to make more of a difference than that.
Even the studies that were not multiple factorial – ones that tried to change diet and diet alone – could not do anything with saturated fat alone because there is no food on this planet that is saturated fat alone. Hence, every single study that tried to change saturated or monounsaturated or polyunsaturated fat intake was actually changing all three real fats at the same time.
I think that people involved in these studies simply cannot know that all three fats are in all real foods and cannot know the proportions in different foods (e.g. meat and eggs being more unsaturated fat). You would not think that this could happen, and be perpetuated for almost 50 years. You would think that someone, at some stage, would know the composition of different foods. However, when I attended a conference in Cardiff in June called Physical Activity, Obesity and Health, Dr. Jason Gill gave a very interesting presentation on “Exercise, obesity and postprandial metabolism”. While talking about fat in food, the pictures on the slide behind were of cola, burgers and fries – processed food and primarily carbohydrates. Gill referred to a study in The Journal of the American Heart Association where Merrill et al allegedly tested lipids after a high-fat meal. (Ref 1) The study subjects were fed “a high-fat breakfast purchased from McDonald's restaurant consisting of approximately two sausage McMuffins with eggs, one order of hash brown potatoes, and a glass of ice water.” As soon as you know the food consumed, the conclusions are meaningless. This might tell us something about McDonald’s food, it might tell us something about processed food, it might tell us something about hash browns and McMuffins, but it tells us nothing about real fat or real saturated fat.
Ref 1: JR Merrill, RG Holly, RL Anderson, N Rifai, ME King and R DeMeersman, "Hyperlipemic response of young trained and untrained men after a high fat meal", Arteriosclerosis Thrombosis and Vascular Biology, (A Journal of the American Heart Association), (1989).
The price of ignorance has been immense.
5) The more obvious association.
Do you remember the recent “Eating just THREE eggs a week increases chance of men getting prostate cancer;”? (Their emphasis) (Link) I did a blog on this here exposing the many errors and mistakes in this study and the reporting of it (Link) This illustrates another error that can be made – ignoring a more obvious association. As the blog says: “Even if the eggs have any relevance at all – what else could be happening at the same time? Were the egg eating men Paleo dudes, or were they egg and soldier addicts (blame the bread), or egg and brown sauce addicts (blame the sugary gunge), or even egg and bacon addicts who hadn’t selected their bacon carefully enough (blame the processed meat).”
The same error is made in the opposite direction. When people find a weak association between, say, fruit and veg consumption and any health factor (because boy do they try to find these!) they make claims such as “eating more fruit and veg saves lives”. It happened just this last week with the “Eat like the English and 4,000 lives could be saved”. (Link) The claims went on to say that English people ate more fruit and veg and less saturated fat and salt than the Welsh, Scottish and Northern Irish. The error works both ways:
– On the fruit and veg side – people who do eat more fruit and veg are likely to be healthier people generally. The fruit and veg consumption is then a symptom not a cause. I said on a blog once – not many people put down an apple to light a cigarette! Many people may put down a beer to light a cigarette – it’s an indication of overall lifestyle.
The most staggering conclusion for me in the European published findings from the EPIC study was that the study grouped participants into five categories from the lowest intake of fruits and vegetables (0 to 226 grams a day) to the highest intake (more than 647 grams a day). Significantly, the cancer risk did not vary between the five groups. Given that those who eat virtually no fruit and veg are more likely to be McDonald’s and microwave consumers and given that those eating over 647 grams of fruit and veg a day (that will be some people eating a kilo of rabbit food) are more likely to be whole food health fans (sweeping generalisation, but I think it’s reasonable) – the fact that there was no discernible difference in cancer between these two groups was astonishing. So, a) the evidence is that fruit and veg is not helping English people and b) even if it were it would more likely be a symptom of a healthy lifestyle than a cause of good health.
– On the saturated fat and salt side – given that idiot powers that be think saturated fats are biscuits, cakes, pastries, pies, crisps, ice cream, sweets, confectionery, savoury snacks etc, if Wales, Scotland and N Ireland are eating more of these I would not be at all surprised if this is detrimental to health. We know that processed food, processed carbohydrates especially, are truly evil. However, for once and for all, will authorities please stop calling processed carbohydrates saturated fats?!
6) Making big numbers out of small numbers.
This one is also classic and also happens all the time. Kendrick gives a great example in his book The great Cholesterol Con. He says that he can double your chances of winning the lottery – who wouldn’t want that eh? Buy two tickets he says! If you have a one in fourteen million chance of winning the lottery, you now have a one in seven million chance. That’s still naff all chance in reality – but double the chance that you had before.<
br />
The same game is played with risk numbers. The headline screaming “30,000 lives will be saved” is nonsense to start with because no lives are going to be saved – we’re all going to die. However, the method by which they come up with this 30,000 number is equally nonsensical.
The gist of the game is that you ignore the denominator – that’s the bottom number in a fraction. (e.g. in 2/3 – 2 is the numerator and 3 is the denominator). If 20 people die in a control group and 10 die in the intervention group, the significance of this depends on the size of each group. If each group had 40 people in it, that would be a 50% chance of dying in the control group and a 25% dying in the intervention group. That’s pretty bad odds in both groups, but even worse in the control group. However, if the control and intervention groups are both 100,000 in size – that’s a one in 5,000 chance and a one in 10,000 chance. When the chances of dying in a car accident are one in 200, neither of those odds would worry me.
What they then do is to scale up and say – if there are 60 million people in the UK and 20 in 100,000 die in one group and 10 in 100,000 die in another group then if everyone did what the control group did, in theory 12,000 people would die and if everyone did what the intervention group died, in theory 6,000 people would die. Notwithstanding all the other factors and distortions that we have looked at already – this is quite simply how they apply the maths. This then gives the headline “6,000 lives could be saved.”
That’s the reality in virtually every headline that you see claiming thousands of lives could be saved.