So there’s a “study” making the rounds claiming to have found evidence of a link between consuming “ultra-processed foods” and developing cancer. It’s not the first time we’ve heard that processed stuff like bacon and pastrami leads to cancer, but this one expands the range of “processed” to the more scary “ultra-processed” to include the following (handily summarised by the BBC):
Leaving aside what exactly even are “foods made mostly or entirely from sugar, oils and fats”, it’s an excellent example of the kind of rubbish headlines that lead to the worst outcomes of social media, and of the resulting issue of people being rightly confused about what, or what not, to eat, because it’s so beautifully tweetable, but mostly bullshit (scientifically speaking):
The main issue here is fairly simple to explain, but it unfortunately comes with consequences that are less simple to undo with a few words on Twitter.
Basically, there was a French study which involved more than 100,000 people who were asked to self-report on the amount of “ultra-processed” foods they consumed, and which then monitored how many of them got cancer. It was written up in the BMJ, and now here we are.
To the good bits:
- A sample of that size is immediately more persuasive than an n of 1!
To the troubling bits:
- People are typically rubbish at self-reporting, which is why this is not generally a scientifically respected method of collecting data. The poor design of the actual study seems to only exacerbate the fact that the data they collected is hardly trustworthy: ‘Participants were invited to complete a series of three non-consecutive, validated, web based 24 hour dietary records every six months (to vary the season of completion), randomly assigned over a two week period (two weekdays and one weekend day). To be included in the nutrition component of the NutriNet-Santé cohort, only two dietary records were mandatory. We did not exclude participants if they did not complete all optional questionnaires.’ 🤷♀️ So we’ll ask people to self-report every once in a while, but won’t exclude anyone who does so on a “once-in-a-when-I-feel-like-it/remember” basis.
- In collating data linking “ultra-processed” foods to cancer, the authors very curiously decide to eliminate one of the most common forms of that disease (which about 3.3 million Americans are diagnosed with every year): ‘In this study, we considered all first primary cancers diagnosed between the inclusion date and 1 January 2017 to be cases, except for basal cell skin carcinoma, which we did not consider as cancer.’ (This is just weird, as including it would probably have bumped their results significantly, but perhaps it would just have been too obvious a reminder of the possibility of confounding factors like that people who enjoy “foods made mostly or entirely from sugar, oils and fats” could also enjoy hanging out at the beach without sunscreen?)
- Here are the results as reported in the BMJ write-up: ‘During follow-up (426 362 person years, median follow-up time five years), 2228 first incident cases of cancer were diagnosed and validated, among which were 739 breast cancers (264 premenopausal, 475 postmenopausal), 281 prostate cancers, and 153 colorectal cancers. Among these 2228 cases, 108 (4.8%) were identified during mortality follow-up with the national CépiDC database.’ Sound scary? Here’s a breakdown which may be easier to understand: ‘The researchers documented about 2,200 cases of cancer among the 105,000 participants during 5 years — a rate of approximately 2.1%. Put in absolute terms, then, a 10% greater risk might translate to 0.2 percentage point increase in the overall cancer rate — from 2.1% to about 2.3%.’ (If you’d like to read more about putting numbers in context, I’d recommend Ben Goldacre’s “Putting a Number in its Context”.)
- Finally, because all of this is so much blah-blah until the authors themselves outline the “Strengths and Limitations of the Study”, key among which are the fact that (while large) the sample size ‘were more often women, with health conscious behaviours and higher socio-professional and educational levels than the general French population’ which ‘might limit the generalisability of the findings’, and ‘although we included a large range of confounding factors in the analyses, the hypothesis of residual confounding resulting from unmeasured behavioural factors and/or imprecision in the measure of included covariates cannot be entirely excluded owing to the observational design of this study’ (my emphases).
So this was an observational study of a non-representative sample of French (mostly middle-class female) eaters who, if they did end up with cancer, could attribute it to smoking, contraceptives, or something entirely unrelated to “ultra-processed” foods.
Association: 1- Causation: 0
As you were.