What about the Other Weight Loss Diet Study??

The same day the low-fat vs low-carb study by Bazzano and colleagues was published, the Journal of the American Medical Association published a meta-analysis that compared the effectiveness of "named diet programs".  Many people have interpreted this study as demonstrating that low-carbohydrate and low-fat diets are both effective for weight loss, and that we simply need to pick a diet and stick with it, but that's not really what the study showed.  Let's take a closer look.

Johnston and colleagues sifted through PubMed for studies that evaluated "named diet programs", such as Ornish, Atkins, LEARN, Weight Watchers, etc (1).  In addition, the methods state that they included any study as low-carbohydrate that recommended less than 40% of calories from carbohydrate, was funded by the Atkins foundation, or was "Atkins-like".  These criteria weren't extended to the low-fat diet: only studies of name-brand low-fat diets like the Ornish diet were included, while the meta-analysis excluded low-fat diet studies whose guidelines were based on recommendations from government and academic sources, even though the latter group represents the majority of the evidence we have for low-fat diets.  The inclusion criteria were therefore extremely asymmetrical in how they represented low-carb and low-fat diets.  This fact explains the unusual findings of the paper.

The abstract immediately activated my skeptic alarm, because it states that at the one-year mark, low-carbohydrate diets and low-fat diets both led to a sustained weight loss of about 16 pounds (7.3 kg).  Based on my understanding of the weight loss literature, that number seems far too high for the low-fat diet, and also too high for the low-carbohydrate diet.

OK, I thought, where's Shai?  Shai et al. was, in my opinion, one of the best multi-diet studies, with low-carb, low-fat, and Mediterranean diet arms (2).  Follow-up went out to 2 years, and compliance was fairly good for the entire time.  Johnston and colleagues did include Shai, but they only included the low-carb arm!  The low-fat arm was excluded from the analysis presumably because it wasn't a "named diet program"; it was a calorie-restricted low-fat diet based on American Heart Association guidelines.  This arm lost about 8 lbs (3.6 kg) at the one-year mark, vs. approximately 12 lbs (5.4 kg) for the low-carb arm.

How about Brehm et al. (2003)?  This 6-month study compared a low-carb to a low-fat diet, finding that low-carb led to greater weight loss of 19 lbs (8.5 kg) vs. 9 lbs (3.9 kg) for low-fat (3).  Again, only the low-carb arm was included in the meta-analysis; the low-fat arm was excluded presumably because it wasn't a "named diet program"; it was an anonymous calorie-restricted low-fat diet.

How about Samaha et al?  Again, the study included a low-carb and a low-fat arm, but only the low-carb arm made it into the meta-analysis (4).  After 6 months, the low-carb arm had lost 13 lbs (5.7 kg) vs. only 4 lbs (1.9 kg) in the low-fat group.

What do we find when we look at other meta-analyses of low-fat and low-carb diets that had more balanced inclusion criteria?  Hession et al. (2009) conducted a systematic meta-analysis of studies that compared low-carb to low-fat weight loss diets in overweight and obese individuals (5).  This meta-analysis identified seven studies that met inclusion criteria and had a follow-up period of at least one year.  Weight loss in the low-carb arm ranged from 5-20 lbs (2-9 kg), with an average loss of 11 lbs (5 kg).  The low-fat arm also ranged from 5-20 lbs (2-9 kg), with an average loss of 9 lbs (4 kg).  The low-carb diet led to significantly greater weight loss, but the difference was very small, and neither diet produced very impressive weight loss at one year.  The difference in weight loss between diets was larger at 6 months.

Another detail that complicates interpretation of the Johnston meta-analysis is that many of the diet studies were actually multiple interventions that also included an exercise component-- and this was particularly true for the low-fat studies.  The Ornish diet, for example, has a physical activity component, whereas most of the low-carb studies didn't offer physical activity guidelines.

Perspective

The meta-analysis by Johnston and colleagues is useful and adds to our body of knowledge on diets and health.  However, due to its unusual inclusion criteria, it must be interpreted with great caution-- which has not been the case in the media stories I've seen.  It doesn't actually have much to say about the relative effectiveness of low-fat vs. low-carbohydrate studies, because the inclusion criteria for each were so different that we can't directly compare them within the context of this paper.  Unfortunately, the paper doesn't make that particularly clear, which allowed it to be widely misinterpreted.

Previous meta-analyses, such as Hession et al, had balanced inclusion criteria that allow us to directly compare low-fat to low-carb diets.  They reported exactly what anyone would expect who is familiar with the weight loss diet literature:

1. At 6 months, low-carb diets consistently lead to greater weight loss than low-fat diets. 
2. At one year, the difference has all but disappeared. 
3. Neither diet produces particularly impressive weight loss at one year or more.
4. The weight loss effectiveness of typical low-fat diets tends to be modest at all time points.