When an n=1 can actually teach us something

downloadThere’s been a good round of attention lately to various manifestations of bad science and hyperbolic (mis)representations of health claims (also don’t miss this excellent rant by a dietitian). Often the problem isn’t with the specific detail (however it may look to some people, the Philosophe actually isn’t out to “get” Professor Tim Noakes, nor does he have any reason to wish for the LCHF hypothesis to fail), but with the method of inquiry, and how people in positions of authority set poor examples of logical reasoning, and of how science works.

Specific to the LCHF-Noakes debate has also been the key issue of small sample sizes, beginning with the n=1 of Noakes’ big experiment on himself, and now “supported” by the publication of 127 cases of people who have lost weight and in some cases been apparently cured of Type-2 diabetes by eliminating carbs from their diets.

That’s great for them, but even if this had been an actual scientific study, these numbers are way too small to tell us anything useful, except that it might be worthy of further scrutiny. Yet once again, the problem is the foregrounding of an assumed hypothesis (“Low-carbohydrate and high-fat intake can manage obesity and associated conditions” declares the headline), rather than emphasising all this survey could ascertain, which is that there might be non-medical interventions worth exploring for managing diabetes/obesity, and this particular diet might be one of them.

But I don’t want to rehash what has already been (well) said on this matter. I want rather to commend a different n=1 experiment, this time focusing on how a very high-carbohydrate diet can improve non-HDL (“bad”) cholesterol. Actually it’s not the experiment I want to commend, or even the results (‘Transition from a very low carbohydrate diet to a very high carbohydrate diet resulted in a rapid and dramatic reduction in non-HDL cholesterol’), but rather the signs throughout this account that demonstrate that its author understands full well how science works, and therefore that even though this result cannot be considered statistically significant, it still sets an excellent example of how to do science. A few highlights:

From the Review of a few long-term dietary interventions

There is a rather telling comment in de Souza et al 2012: “despite the intensive behavioral counseling in our study, macronutrient targets were not fully met, which complicated the interpretation of our null result.” So they told different groups of people to eat different diets, but they all ate basically the same diet. Their outcome measures did not differ between groups at the end of the study (the “null result”), and therefore interpreting the data is “complicated.” Let me suggest, actually, that interpreting their data is “a waste of time.” (They published it anyway, of course.)

By the way, the Silberman study on the Ornish diet had 2,974 people in the intervention group (it was not a controlled trial). It is interesting that the Ornish researchers appear to be able to get people to actually eat very low fat diets, while other researchers seem to have more trouble getting participants to make such dramatic diet and lifestyle changes. I’m not commenting one way or the other on the Ornish plan, but it is a bit disappointing that the other research groups don’t seem to be able to effect such large changes in macronutrient intake in their study participants. This means the published studies are not especially helpful in evaluating diets at the extreme ends of the macronutrient spectrum.

It’s kind of a huge limitation (and one that doesn’t get nearly enough attention) that diet studies routinely misrepresent how (much) people are actually eating and/or don’t control for other factors which could lead to weight loss and improved health (much like one of the 127 cases reported by Noakes, who not only cut out carbs but changed his life quite dramatically in other ways too). This is a problem I have mentioned previously, but which keeps resurfacing in the way in which LCHF proponents assume that a) obesity and diabetes result from people actually eating according to dietary guidelines (no one ever recommended that cops should stand around eating donuts) and that b) there is no middle ground (“moderation” is evidently so last century, since we have to be able to eat as much fat as we damn well please).

From the section on Postprandial measurements

Some people are afraid to eat fruit these days because of concerns about blood sugar. My postprandial blood sugar after the oral cantaloupe tolerance test peaked at 107, so I’ll say with confidence that I am not likely to run my blood sugar up to unhealthy levels while eating real foods. Note that the protein consumed along with the cantaloupe likely triggered an insulin response that could have reduced the peak blood sugar level.

I highlighted “real foods” here, because what this author is not doing is saying it’s OK to go and stuff yourself with sugary, processed foods and drinks (as some have done in order to “prove” the ill effects of carbohydrates). He is saying, instead, that, well, if you’re a relatively healthy, non-diabetic person, eating fruits and rice will not kill you. They probably won’t even make you fat. (How sad is it that this should need saying?)

Anyway, I encourage you to go read about the experiment for yourself. Not only does it follow the exact model of what a scientific study should look like (complete with declarations of conflicts of interest – “none” – and the rather cute inclusion of ‘Approval of an Institutional Review Board was not required for this n=1 self-experiment. The author’s mother and girlfriend were informed of the study design in passing and they raised no ethical concerns. The study was conducted according to ad hoc human subjects research guidelines made up on the spot by the author, and reviewed and approved by the author as the sole human subject’).

But it also, and perhaps most importantly, represents at least one of the subjects for whom LCHF is not indicated, and therefore at least one instance of the falsely touted “proven” theory being falsified. Once again, that doesn’t bring down the whole LCHF hypothesis, but it does make it less magical, and therefore perhaps a bit more useful.