The Olympic Year of Bad Science and Worst Practice

I was recently led to this piece in the Columbia Journalism Review via Twitter, where it was retweeted by @ProfTimNoakes. Tim Noakes, for those of you who may not have heard of him, is a South African sport scientist who used to be the guru of “carbo-loading” for major sports events like marathon-running, as chronicled in his once-iconic book, The Lore of Running. This year he caused a bit of a stir when he turned that lore on its head and declared that everything he had said about carbo-loading was false. Instead, he became the latest advocate for a low-carbohydrate, high-fat diet, which also means he became the latest poster-child for the deceased Robert Atkins and the very-alive-and-kicking Gary Taubes. (Read the Philosophe’s take on Noakes here)

These days you don’t have to actively seek out information about diets and eating to have come across some of the raging debates about what and how to eat, because even the Daily Mail will gladly fill you in on the latest while you’re catching up on Becks and Posh (just yesterday they reported on the possibility of the obesity crisis being “over” thanks to some amazing new research which can turn all our bad fat into good fat). And while reporting on science in popular media is in theory a good thing, in practice it too often turns out to be disastrous, because it too often is a) bad science reporting, b) a pissing contest or c) both. Cue the CJR article, which describes the “feud” between Gary Taubes and Gina Kolata, who had the apparent nerve to publish a Q&A with an esteemed scientist (that would be Dr. Jules Hirsch, ’emeritus professor and emeritus physician in chief at Rockefeller University’, and an obesity researcher for almost six decades) who maintained that there is no magic diet – not even the Atkins/Taubes/Noakes one.

Now, normally I wouldn’t have pegged the CJR and the Daily Mail in the same category, but on the evidence of this piece, I’m not so sure any more. Author Paul Scott is clearly more convinced by Taubes than he is by Kolata, and that is his prerogative. But the “evidence” for supporting that side is laughable. As an example of how the low-fat ideology has been left ‘in the rear-view mirror’, he tells us that ‘Michael Pollan readily goes to bat for butter’. Pollanites can sigh and help themselves to another slice of foie, but the rest of us should say, “so?”. Here’s another factoid “in support” of the low-carb, high-fat lifestyle: ‘Public health interventions are taking aim at Big Gulps, not Ben & Jerry’s’. Right. So because someone is trying to limit my freedom to consume a bucket of coke, I can happily chow down a bucket of Ben & Jerry’s? (I’ll have the Coffee Heath Bar Crunch, because I’m in dire need of caffeine, plus if you add an ‘l’, it becomes Health Bar Crunch’):

 (Thanks to Midlife Diplomatic Crisis blogger for the picture)

Almost worse, though, is the snarky tone of this piece which ends with a rather unsophisticated swipe at Kolata:

‘Poor Taubes. No one warned him that 600 pages of evidence were never going to be enough. The theory that weight gain boils down “calories-in, calories-out” is the last man standing in the diet wars. The principle anchors the comforting American belief that personal responsibility explains all of our ills. It validates all that wasted time on the treadmill that people like Kolata and others endorse. It keeps us watching shows like The Biggest Loser. It leaves the door open to low-calorie, high-carbohydrate food products that make the economy hum, are portable, do not require we learn to cook, make children stop crying, and taste good. Any efforts at reporting science to the contrary will always have a rough road.’

The person who originally tweeted the link to this piece – and who was retweeted by Tim Noakes – headlined it thus: ‘The diet wars at the NY Times: Who is winning and who smells of desperation?’

When, exactly, did this issue of public and private health turn into a competition? I could half understand a competition between scientists – because it would actually be cool to see who could come up with some real and ground-breaking science first – but between journalists? The only signs of desperation I smell here are from the ones who refuse to acknowledge that there is no magic marker that can wipe away the main culprit behind most people’s excess weight, which is (boring!) themselves.

Yes, I know that some people have metabolic disorders. I also know that some people are “pre-diabetic”, or “carb-resistant”, like Tim Noakes, which apparently means that he benefits particularly from the high-fat, low-carb diet. But that he benefits from it does not mean that you or I would, which also means that it is not automatically a public health serum, as he has repeatedly represented it as. I have read Taubes’ 600-page tome, and have been less impressed than bamboozled by the amount of evidence he presents for his cause. It was much the same at a Noakes talk I recently attended, where he threw study after study at the audience. Unfortunately throwing “evidence” at people isn’t a very solid approach (sceptics call this a Gish Gallop), particularly if you can’t answer the question (posed by me after the talk) of why he omits evidence that contradicts his thesis, or if a scientist like him admits in a public forum (as Noakes also did) to preferring observational studies to clinical trials. This should have all our alarm bells ringing, or rather the bullshit sirens wailing.

Some do. But they are quickly drowned out by the chorus of zombie voices who prefer the magic solutions. Ta-Nehisi Coates put it excellently in his comment on the Jonah Lehrer incident (is that too nice a word?) over at The Atlantic:

‘But we now live in a world where counter-intuitive bullshitting is valorized, where the pose of argument is more important than the actual pursuit of truth, where clever answers take precedence over profound questions. We have no patience for mystery. We want the deciphering of gods. We want oracles. And we want them right now.’

11 Replies to “The Olympic Year of Bad Science and Worst Practice”

  1. The focus of my talk was simple – to show that heart disease is due to abnormalities in carbohydrate not fat metabolism. Of course I threw many studies at you but they all show that there is only one conclusion. Why not report that the model of how dietary fat is meant to cause heart disease is simply not supported by the published evidence? It may be unacceptable to you but the evidence I presented is certainly not the derogatory word that you use to describe it.

    And incidentally I never said that I preferred observational studies to clinical trials. What I said was that you cannot afford to exclude any evidence that conflicts with the model you propose. I presented 7 predictions of the usual model of fat intake causing heart disease that are not supported by the published evidence. I used the published studies of clinical trials undertaken by established scientists to disprove the model that you advocate. If you were a scientist, your job would be to abandon the model that you propose, not to try to inure it to criticism by attacking those who present an alternate proof. This is known as refutation by denigration. It is not the best way to reach the truth and does not advance debate in a helpful way.

    What I find so interesting is that the rather obvious fact that humans are less healthy today than we were before the 1977 change to a high carbohydrate diet is so difficult for so many to see. But the evidence is abundantly clear as I also showed in my talk. Is there a reason why you ignore the obvious? Or do you have an alternate explanation to the one that I and Gary Taubes prefer?

  2. Actually, you did say that you prefer observational studies to clinical trials. A room full of people could verify that. Whether that’s what you meant to say is a different matter.

    And I don’t disagree that rising obesity levels are in part thanks to the easy and cheap abundance of energy-dense (and normally nutrient poor) foods, many of which were engineered in the second part of the twentieth, and many of which happen to be high in sugars/carbs. But that an excess of sugars/carbs can have an adverse effect on weight and health does not mean that all sugars and carbs are bad for you, nor does it mean that it’s automatically healthy for everyone to load up on steak and butter. It might be for you, but to sell this as a public health initiative is frankly irresponsible (and clearly not very practical either: what percentage of the South African population do you think can afford a low-carb, high-protein diet?).

    Finally, to this: ‘If you were a scientist, your job would be to abandon the model that you propose, not to try to inure it to criticism by attacking those who present an alternate proof. This is known as refutation by denigration. It is not the best way to reach the truth and does not advance debate in a helpful way.’ I am not trying to propose or advocate any particular model. I care about what the evidence supports. If new research uncovers statistically significant evidence, then I will pay attention. But if someone presents only evidence that supports their own conclusion and ignores existing evidence to the contrary, then I will not. This is known as confirmation bias, and most certainly does not advance debate in a helpful way.

  3. A further response from Tim Noakes (via email):

    So what exactly was the information that I ignored? Let me be bold enough to explain how science is meant to work.

    Biological scientists like myself develop models of how we think the body works. We then interpret that information according to that model. If the model predicts the findings that we make, then the model survives until the next experiment. But as soon as the model is unable to explain what we measure, then our job as reputable scientists is to say – sorry the model is wrong, we need to abandon it and to come up with a better model.

    The entire focus of my lecture was to show that there are 7 lines of simple argument that disprove the currently popular model which holds that fat in the diet impairs our health by raising our blood cholesterol levels leading to heart disease. On the basis of the evidence that I presented any respectable scientist should say – fine the model does not work so let us abandon it. Instead what has happened is that model has been inured from criticism largely by the commercial forces that benefit from its continued acceptance as the “truth”.

    But it really does not matter if there are 1 million pieces of information that “confirm” the model and which I did not present in the lecture (ie ignored). For the simple reason that ANY information IS ONLY TRUE ACCORDING TO THE MODEL USED TO INTERPRET IT. Humans cannot interpret any information without the use of a model with which to interpret the data. So the point is that if the model is incorrect then so must be the interpretation of any findings assessed according to that model.

    I presented evidence including a recent meta-analysis (considered the most comprehensive evaluation of the “truth”) which establish that fat in the diet is not a cause of heart disease. To me that means that that particular debate is now over. It is time to move on if we really want to understand what is the best diet for humans.

    I have used this style of argument to disprove the traditional model of fatigue during exercise and to advance our understanding of what you should drink during exercise. It is a very effective way – perhaps the most effective way – to advance knowledge.

    Because you do not understand this method of argument, you accuse me of “confirmational bias”. The point is that it matters not how much evidence supports a particular model. For that support is utterly dependant on its interpretation according to the very model that it is meant to support! That is, it has all the features of a circular argument. In fact the persons most guilty of confirmational bias are always those who fail to see that their beloved model has already been disproven. Are you one such?

    So please before accusing me of confirmational bias and perhaps of even worse in your original article, you need better to understand the scientific method.

    The sole reason why we originally began to believe that a high carbohydrate diet is especially “healthy” is because of the falsified evidence produced by Ancel Keys in the late 1960s (as presented in the lecture). Up until then, carbohydrates were considered fattening and there was no great rush to prove that they were incredibly healthy. In fact carbohydrates were considered the food of the poor (and hence undesirable if you could afford the high protein/fat alternatives).

    But Keys’ distortion produced the fear of fat (lipophobia) that has been driving global nutritional advice since 1977. I have researched and continue to research the entire literature and the only evidence I can find to support the lipophobia theory has been provided by scientists only AFTER we had been already been told to change our diets. But such science is very likely to product the wrong result because it begins with a bias which is more probable to be proven by whatever experiments are undertaken (after the bias has taken hold).

    So instead in my lecture I presented evidence that does not suffer from bias because it is not dependant on a pre-determined interpretation, of how a high carbohydrate diet produces abnormal metabolic responses that are associated with an increased risk of heart disease. Perhaps as you believe, these finding are specific to only a section of the population. But we do not know that at present. Instead the global mantra is that everyone should eat a high carbohydrate diet. The clear evidence that this advice is toxic for some, like myself, is glibly ignored in the rush to defend a universally popular dogma. There is incidentally a real condition called carbohydrate-resistance (intolerance); it can easily be detected when it is present and it is associated with a wide range of serious medical consequences. It is not bogus as you seem to imply in your original article.

    Your statement that I want everyone to follow this diet is false. At all times I have said and continue to say “if you are like me you will likely benefit from reducing the carbohydrate content of your diet”. The proportion of the South African population that is like me and who will benefit from this advice is unknown. But judging from the feedback I continue to receive on a daily basis from those who are overjoyed finally to be able effortlessly to reduce their weight without continual hunger, or to reduce or stop their anti-diabetic and anti-hypertensive drugs (amongst many other reported benefits) as a result of this simple dietary change, I suspect that I am not the only South Africa who will benefit from this dietary change.

    Your suggestion that we should not adopt this diet because it is too expensive for the poor does not make sense. Carbohydrates are cheap because their production began to be subsidized after 1971 by the Nixon administration for short-term political gain (as discussed in the lecture). Subsidizing the production of fat and protein-rich foods would bring down their cost. There are enormous wastages of high protein and fat foods in the processing of animal products. Kidneys for example are the cheapest source of high quality protein and eggs, sardines and liver, all cheaper than steak, are three of the most nutritious foods for humans. Members of UCT’s Health Economics Unit were present at the lecture and are actively trying to understand how foods with higher fat and protein content can be provided at reasonable cost to the poor. Had I not taken my stance 18 months ago, they would not now be working on this topic. The poor are locked into diets that cannot provide them with optimum health. The “carbohydrates are good for you” mantra is particularly convenient because it prevents any effort to improve dietary choices for the poor.

    Finally I note that the website of your (?) husband with whom you seem to collaborate contains the logo of a so-called “heart healthy” margarine and an advert for the Heart Foundation. This represents a conflict of interest when you discuss dietary advice. You need to acknowledge that conflict of interest which exists whether or not your husband receives any financial reward for promoting that product or service.

  4. Thanks, but I actually do understand the scientific method – though it’s less clear to me that you understand what’s meant by confirmation bias, and why that’s a problem for scientists.

    Also, I never suggested that there is no such thing as carb-resistance, and neither do I believe it to be true that ‘the global mantra is that everyone should eat a high-carbohydrate diet’. You seem to be missing the existence of a middle ground on which bread is neither eaten by the bucket load nor villified as ‘toxic’. Many of us live quite happily – and healthily – on this ground.

    And finally, no, my husband is not a shill for the margarine industry. I can’t even imagine in what universe such a connection would make sense.

  5. “I have read Taubes’ 600-page tome, and have been less impressed than bamboozled by the amount of evidence he presents for his cause.”

    This is a dismissive statement that adds no weight to your own stance and detracts nothing from Taubes’. Clarification is indicated: did you find that the evidence was too much to read (and if so, why would “too much” supporting evidence detract from a hypothesis)? Or did you find the evidence to disagree with or disprove Taubes’ hypothesis, and if so in what way? Perhaps there are logical or physiological errors that you found? If so, please clarify.

    I think, Dr. Rousseau, that when it comes to scientific method, I have substantially more confidence in the education, reputation in scientific research, experience and track record of Noakes than in a PhD earned in Food Media – particularly one with such an obvious sense of superiority and a fondness for nastiness that is so evident in many of your blogs.

  6. Dear “Havelock”,

    No, the evidence was not too much to read, and nor did I suggest that there was “too much” supporting evidence. I did suggest that there was a lack of consideration of existing conflicting evidence (particularly re. Noakes), for example this study, and this one, and this one. For the record, I do have respect for Taubes as a writer and researcher – and quote him at length on the fraught history of public health initiatives in my book – but not as an advocate of the low-carb lifestyle (because, at the risk of repeating myself, he is in the habit of ignoring evidence that contradicts his thesis, and also of misquoting or not quoting other scientists who disagree with him).

    But then I’m not sure why I’m explaining myself to a fictional character who thinks an appeal to authority is the yellow brick road to the Truth. If you find my tone unpalatable, you are welcome to take your attention elsewhere.

  7. Although having never read it, I suspect that Taubes’ 600 pages must surely be less bamboozling than rather disparaging Signe Rousseau’s. While nobody can always claim to be right, I guess Prof Noakes’ objectivity is supported by his willingness to change his view regarding his own carbohydrate theories of earlier years, and that he isn’t advertising Maragarine. Since I’ve been following this debate I have NEVER understood him to be saying that this is the diet for Everyone. He is language is somewhat more polite too. Just a few layman observations.

  8. “John”, nobody involved in this debate has advertised margarine, and the accusation that anyone did so is evidence against objectivity, if anything, as it was a self-serving inference based on selective reading of evidence. Your problem is perhaps more simple, and involves needing to read more carefully and critically.

  9. Dear Signe,

    Bamboozled is generally defined as a state of confusion, often implying a deliberate effort to confuse; please excuse me if I misunderstood you when you said “I… have been less impressed than bamboozled by the amount of evidence he presents”.

    “…a lack of consideration of existing conflicting evidence (particularly re. Noakes), for example this study, and this one, and this one…”

    Allow me then:

    First 2 links (de Souza et al and Sacks et al):
    Both of these studies are similar and are similarly irrelevant in their relation to the Taubes hypothesis, for multiple reasons – Taubes’ hypothesis is that carbohydrates drive fat accumulation through the action of insulin, and that a very low carbohydrate diet, that primarily replaces carbohydrate with fat, will reverse the process and allow effortless, sustainable weight loss without calorie restriction. Firstly, Taubes concedes throughout his work that a calorie-restriction diet will enable weight loss, and these studies are both conducted with a calorie restricted weight-loss diet, a factor that in itself limits its relevance to Taubes’ hypothesis. Secondly, both studies’ lowest carbohydrate level was 35% – hardly relevant to the very low carbohydrate levels that are indicated by Taubes’ hypothesis. This is particularly true if you consider that the hypothesis defines insulin as the driving factor in fat accumulation and insulin resistance as the key – meaning that even a low carbohydrate intake would produce disproportionately high insulin levels, and the resultant fat accumulation.

    Looking at Johansson et al, we find a similar problem – it clearly shows that at no stage, even at the lowest point, did carbohydrate percentage drop below about 40% of calories, which essentially means that the continuing upward trend shown in BMI is meaningless in reference to Taubes’ hypothesis on obesity. I also found no information in the study on the types of carbohydrates consumed – the Taubes hypothesis implies a probability that frequent concentrated blood sugar spikes, as produced by refined carbohydrates (especially sugar, white flour, white rice, corn syrup), is most likely the factor that triggers insulin resistance rather than the total energy value of carbohydrates consumed. Without this data, and considering that the minimum carbohydrate level is still above 40%, the study has no bearing on supporting or disproving Taubes’ hypothesis. Regarding the heart disease angle, it must be noted that the study uses total serum cholesterol measurement as its only indicator of heart disease risk, but there is now ever-increasing agreement that total serum cholesterol levels have no predictive relationship with heart disease whatsoever.

    So perhaps there has been no apparent “consideration of conflicting evidence” because it is not actually relevant to the hypothesis?

    As far as appeal to authority goes, I get your point, but my appeal was not only to the authority of Noakes’ qualifications, but more to his track record and reputation as an open-minded, impartial researcher who is willing to challenge popular consensus (and most importantly, not scared to admit a mistake).

    I would hope that you were not answering me for my own sake, but for others reading this, which is of course the reason that I took the time and trouble to question you (and why I don’t just “take my attention elsewhere”). Arguments seldom sway the opinions of the participants, but will hopefully influence the views of those following them.

  10. Dear “Havelock”,

    I apologise for perhaps a bad choice of word with ‘bamboozled’ – I don’t think Taubes is out to confuse or deceive, but I do think that his hypothesis ends up being a largely unfalsifiable one, in that describes a large-scale experiment which has never been executed, and the success of which is impossible to know, not to mention all the other factors not controlled for which contribute to good or poor health regardless of carbohydrate intake (eg. smoking, alcohol, exercise).

    And while you are correct in pointing out different levels of carbohydrates in these studies compared to those advocated by Taubes and Noakes, the fact remains that theirs is still generally translated as ‘low-carb’ in popular media, which, like it or not, remains the main source of information for most people. We can probably agree on the fact that it is a general problem that scientific research is routinely misrepresented and/or oversimplified by media reports, which is really the point of my post, and the crux of any problem I have with Taubes or Noakes, ie. that they have a responsibility to acknowledge how little the general public actually understand about science, and to make sure they don’t contribute to that in any way.

Leave a Reply