Thursday, September 4, 2014

What about the Other Weight Loss Diet Study??

The same day the low-fat vs low-carb study by Bazzano and colleagues was published, the Journal of the American Medical Association published a meta-analysis that compared the effectiveness of "named diet programs".  Many people have interpreted this study as demonstrating that low-carbohydrate and low-fat diets are both effective for weight loss, and that we simply need to pick a diet and stick with it, but that's not really what the study showed.  Let's take a closer look.

Johnston and colleagues sifted through PubMed for studies that evaluated "named diet programs", such as Ornish, Atkins, LEARN, Weight Watchers, etc (1).  In addition, the methods state that they included any study as low-carbohydrate that recommended less than 40% of calories from carbohydrate, was funded by the Atkins foundation, or was "Atkins-like".  These criteria weren't extended to the low-fat diet: only studies of name-brand low-fat diets like the Ornish diet were included, while the meta-analysis excluded low-fat diet studies whose guidelines were based on recommendations from government and academic sources, even though the latter group represents the majority of the evidence we have for low-fat diets.  The inclusion criteria were therefore extremely asymmetrical in how they represented low-carb and low-fat diets.  This fact explains the unusual findings of the paper.

The abstract immediately activated my skeptic alarm, because it states that at the one-year mark, low-carbohydrate diets and low-fat diets both led to a sustained weight loss of about 16 pounds (7.3 kg).  Based on my understanding of the weight loss literature, that number seems far too high for the low-fat diet, and also too high for the low-carbohydrate diet.

OK, I thought, where's Shai?  Shai et al. was, in my opinion, one of the best multi-diet studies, with low-carb, low-fat, and Mediterranean diet arms (2).  Follow-up went out to 2 years, and compliance was fairly good for the entire time.  Johnston and colleagues did include Shai, but they only included the low-carb arm!  The low-fat arm was excluded from the analysis presumably because it wasn't a "named diet program"; it was a calorie-restricted low-fat diet based on American Heart Association guidelines.  This arm lost about 8 lbs (3.6 kg) at the one-year mark, vs. approximately 12 lbs (5.4 kg) for the low-carb arm.

How about Brehm et al. (2003)?  This 6-month study compared a low-carb to a low-fat diet, finding that low-carb led to greater weight loss of 19 lbs (8.5 kg) vs. 9 lbs (3.9 kg) for low-fat (3).  Again, only the low-carb arm was included in the meta-analysis; the low-fat arm was excluded presumably because it wasn't a "named diet program"; it was an anonymous calorie-restricted low-fat diet.

How about Samaha et al?  Again, the study included a low-carb and a low-fat arm, but only the low-carb arm made it into the meta-analysis (4).  After 6 months, the low-carb arm had lost 13 lbs (5.7 kg) vs. only 4 lbs (1.9 kg) in the low-fat group.

What do we find when we look at other meta-analyses of low-fat and low-carb diets that had more balanced inclusion criteria?  Hession et al. (2009) conducted a systematic meta-analysis of studies that compared low-carb to low-fat weight loss diets in overweight and obese individuals (5).  This meta-analysis identified seven studies that met inclusion criteria and had a follow-up period of at least one year.  Weight loss in the low-carb arm ranged from 5-20 lbs (2-9 kg), with an average loss of 11 lbs (5 kg).  The low-fat arm also ranged from 5-20 lbs (2-9 kg), with an average loss of 9 lbs (4 kg).  The low-carb diet led to significantly greater weight loss, but the difference was very small, and neither diet produced very impressive weight loss at one year.  The difference in weight loss between diets was larger at 6 months.

Another detail that complicates interpretation of the Johnston meta-analysis is that many of the diet studies were actually multiple interventions that also included an exercise component-- and this was particularly true for the low-fat studies.  The Ornish diet, for example, has a physical activity component, whereas most of the low-carb studies didn't offer physical activity guidelines.


The meta-analysis by Johnston and colleagues is useful and adds to our body of knowledge on diets and health.  However, due to its unusual inclusion criteria, it must be interpreted with great caution-- which has not been the case in the media stories I've seen.  It doesn't actually have much to say about the relative effectiveness of low-fat vs. low-carbohydrate studies, because the inclusion criteria for each were so different that we can't directly compare them within the context of this paper.  Unfortunately, the paper doesn't make that particularly clear, which allowed it to be widely misinterpreted.

Previous meta-analyses, such as Hession et al, had balanced inclusion criteria that allow us to directly compare low-fat to low-carb diets.  They reported exactly what anyone would expect who is familiar with the weight loss diet literature:
  1. At 6 months, low-carb diets consistently lead to greater weight loss than low-fat diets. 
  2. At one year, the difference has all but disappeared. 
  3. Neither diet produces particularly impressive weight loss at one year or more.
  4. The weight loss effectiveness of typical low-fat diets tends to be modest at all time points.


Unknown said...


Have you seen this study

A meta-analysis of the past 25 years of weight loss research using diet, exercise or diet plus exercise intervention

CarbSane said...

When I discovered this other paper published on the Shai study, any notion that there was dietary compliance went out the window.

Self-reported free living diet accounts are suspect enough, but when the difference between reported averages using one method on the whole cohort and another method on a random subset of subjects differ by 300-600 cal/day and almost 1000 cal deficit from baseline at one year ...

I am also convinced that Shai did not report absolute dietary intake in the main study because to do so would have made it obvious that participants did not consume a typical Atkins diet. In addition to being advised to consume some veggie sources of fat and protein, it turned out that at carbohydrate intake at 6,12 and 24 months exceeded 150 grams/day.

raphi said...

Guyenet Comment

So, here you have it folks - just another case of meta-analyses having no magical powers and showing that it comes down to your data quality. 'Garbage-in = garbage-out'. Too bad!

Do you think these fat-loss meta-analyses are still worth conducting (for now)? It's been a while where conclusions are either muddled or redundant.

It seems to me that general research efforts would be best spent understanding healthy weight loss vs. unhealthy weight loss (for e.g. if we still want weight loss to remain as part of the equation).

Gretchen said...

Don't you mean JAMA?

Howard said...

I suppose I'm an outlier, because I lost 100 lbs on a low-carb diet, and kept it off for over 15 years now.

Except that I personally know several dozen such outliers. I have to wonder about some of these "studies." I have a theory about "stucies," namely that there is only one significant variable in all of the "studies" I have read since 2001.

That is, "Who paid for it?"

tomR said...

I'll repeat what I've already said - this is all obsolete stuff. Names like Atkins, Ornish - we are talking about 70s here. This was the time when the only known hormone was insulin...
Today we have 2014, and much more comprehensive, complete, and up-to date diets - the good examples for all-around one could be PHD by Jaminets, as therapeuthic - Wahls Paleo/Paleo+, or just a slimming diet as the Jonathan Bailors one. It's also unwise to not include any mention of gut-biome related stuff in papers published in 2014, since we know that it plays important role (eg. successes of fecal transplants).
Basically any study that doesn't include moder diets is WORTHLESS, as these were created because previous diets hadn't worked well. So basically testing the "old" diets, is testing the diets that we know were inferior to the new ones... Just make no sense. Tell the scientists to focus on modern diets, or shut up and stop publishing useless spam.

CarbSane said...

That's interesting novuelvoaengordar, I tried doing that calculation in my spread sheet at various time points for all of the diets and the results were inconsistent. By that I mean if you then tried to take the absolute data and reconstruct the percentages they were way off in some cases. I don't know why. You might be interested in the study I linked to as it provides absolute data.

Your point is still well taken, because even around 170 g carb/day they missed the LC boat.

Charlie said...

From reading the original diet books as an armchair nutritionist since around 1990: Broadly speaking, low carb generally means 20 to 100g of carbs minus fiber every day. Low fat means 10-15% of calories as fat, which for a 2000 calorie diet (if my math is right) works out to 20-30g. No matter what a researcher might personally believe is the right diet, if they don't use those definitions I gotta wonder if they even bothered to read the source material.

Unknown said...

Evelyn, it's good to state the author and title of a study when posting a link. The tandfonline site doesn't show the paper you were referring to.

CarbSane said...

My apologies that the link doesn't work a year later.
Adherence and Success in Long-Term Weight Loss Diets: The Dietary Intervention Randomized Controlled Trial (DIRECT) Greenberg, 2009