© 2011 Roy Benaroch, MD
The Lancet has published another terrible, worthless study guaranteed to confuse parents.
Back in 1998, the world-renowned British medical journal The Lancet published a study that singlehandedly created the entire MMR-autism “manufacturversy.” The study itself was an absolute fraud based on fake data, designed to make money for its lead author. Red flags about the study were ignored by The Lancet’s editorial board for years; but finally most of its authors retracted the study, and then The Lancet withdrew it. Still, the damage was done. Falling vaccine rates led to a return of measles and surging rates of pertussis. Fooled by an unscrupulous liar and a media relishing any opportunity to sensationalize garbage, many parents still distrust vaccines.
And now, The Lancet has done it again. A terrible, worthless study has been published, guaranteed to confuse parents. Maybe their motto ought to be “anything that’s fit to make headlines.”
The study, titled “Effects of a restricted elimination diet on the behaviour of children with attention-deficit hyperactivity disorder (INCA study): a randomised controlled trial”, was supposed to examine the relationship between diet and behavior in ADHD. It’s an important topic. Many children have trouble with focus and attention, and many parents feel that diet may play a role. Though many older studies have been unable to confirm a consistent effect of foods on behavior, a 2007 BMJ study did show at least a small effect of preservatives and dyes in worsening behavior in children.
What has made studying diet and behavior difficult is separating out what is called “confirmation bias.” Parents who are convinced that, say, a sugary meal will worsen their child’s behavior are very apt to notice when bad behavior follows a junk food meal. But those same parents probably don’t notice when relatively good behavior occurs after sugar, or when bad behavior doesn’t really come after a meal. This isn’t because those parents are dumb or delusional—it’s just human nature. We all subconsciously find evidence to support what we already believe, and ignore evidence to the contrary.
Good science seeks to minimize the effects of this kind of bias by using “blinded control groups”, where the observers don’t know if the child was exposed to a surgary diet or not. In the older 2007 BMJ study, the families were truly blinded: neither they nor the researchers knew which kids received a supplement that was a preservative-n-chemical cocktail, versus which ones got a supplement of “nothing”. Only after the parents made their behavioral observations, and after the researchers performed their statistics, were they allowed to know which kids got which diet. That’s good research. The BMJ study did show a statistically significant change in behavior, though the effect was fairly small. Still, it’s a tantalizing start, and the group is now pursuing a more-specific study trying to identify which chemicals and preservatives might be the culprit. It’s a slow process, but carefully-done, well-controlled research should give us a clear answer on this topic.
Unfortunately, the research group publishing in The Lancet didn’t feel the need to bother with these sorts of protocols. In the initial phase of their study, 100 kids were divided into two groups of 50. One group continued to get an ordinary diet (though they did receive counseling about healthy food choices), and the other group was put on a highly restrictive diet of mostly rice, meat, vegetables, pears, and water. But all of the parents knew exactly what group their child was in. At the end of this study period, about 60% of parents of children in the restricted diet group had improved, compared to “none” of the children in the non-restricted diet.
Wait a minute here. If something completely random happens—let’s say I ask parents to flip a coin, and tell me heads or tails—about 50% of the parents should report “tails.” In this behavioral study, if I ask parents to just decide, “did things get worse or better,” if there was just a random scatter of observations, 50% of the parents should say “worse”, and 50% should say “better.” How could “none” of the parents have seen any improvement? Surely at least some of the children had a few good weeks, even with no change in diet, no?
And if 60% improved in the restricted group, that means 40% didn’t improve, or got worse. A 60-40 split isn’t really that impressive, is it?
Besides, with no blinding whatsoever, what does it even mean?
The study gets worse. There was a phase 2 that took the “diet responders” and put them on even more restricted diets based on blood testing for allergies—but using an outdated, worthless test that’s been invalidated for years. This further phase found that the blood tests didn’t help guide parents to diets that would help, which is no surprise because those blood tests don’t work. We already knew that.
There you have it, another terrible study from The Lancet, which demonstrates nothing in a perniciously misleading way. Perhaps there is a link between diet, chemicals, preservatives, and behavior—and certainly, trying to put children on a diet that avoids these sorts of chemicals can’t do any harm. But these authors, and the editorial board of The Lancet, ought to be ashamed of publishing such a worthless study. Do you think the media, and the public, are ready to get duped again?