Have you seen some of the criticisms of this study? I came across a couple of blog articles suggesting that the researchers are biased against low carb diets and manipulated the data to get results more inline with their beliefs. I would love to hear your thoughts on this.
Another Biased Study? Maybe...
The Diet Wars: The Saga Continues
I decided to comment in a separate post because the comments features here are limited (and no auto-save).
First Tom Naughton's analysis raises many of the same questions that I have over the actual dietary interventions in the study. Absent food diaries of what the participants were ACTUALLY consuming at the various time points, we have no idea what we're really comparing at each timepoint. While I, too, don't see where the low fat/calorie restriction group increased their intake, I simply cannot abide that they didn't. It would also have been instructive to know the caloric intakes of each group and the actual carb intake. Naughton concludes: "So in theory, we’re comparing a low-calorie weight-loss diet that lasted for two years with a low-carb diet that reached maintenance level within a year. Strange design for a study in which weight loss was listed as the primary outcome." I would have to disagree with this conclusion as I believe the actual dietary intake -- particularly in the second year -- probably differed from plan for both groups. It had to, or the participants in the LFCRD would have continued to lose. Naughton does say this very same thing later on. Naughton goes on scrutinize the potential bias of the researchers. I'll not go into the whole "it's not the calories" nonsense that permeates the low carb community. I think y'all know where I stand on that by now, but I don't see how this would have influenced the results of this study.
Which brings me to the statistics that are questioned by both Naughton and Bowden (w/Dr. Richard Feinman contributing in the comments). I'll admit to having been lazy not delving into their whole analytical methods in my first reading although that legend on the weight loss graph of "predicted" losses really struck me as "off". I can't for the life of me figure out why they chose the methods they did -- without at least reporting and comparing to the data for just those who stayed in the study AND reported for assessments. The authors presented some numbers in order to reach a certain statistical significance/power for their outcomes (weight and LDL), and with the dropout rate, they still seem to have the numbers with the remaining participants. Perhaps if they excluded all but those participants who attended each assessment for the entire study they fell short? Still, it appears that they not only included those no-shows who remained "in treatment" in their analysis, but also those who dropped out entirely. I think a case could be made for predicting a value for a missing timepoint for a person who otherwise remains in the study, but to include a prediction for a total drop-out??
In any case, where data were missing due to attrition, a statistical model was applied to predict what the value would be. I, too, would rather see the data for those that at least complied enough with the study to show up for five assessments. It will be interesting to see if/when actual data is published what the values are. It is MORE than suspicious to me that this was not included in the original publication. But it is premature to presume bias in the statistical model used to skew the predictions.
So, now onto just the commentary by Dr. Jonny Bowden.
First, since we just discussed the statistics, I would add that if we're going to discount the weight loss outcome on the basis of some bias in the model of biased researchers (who are biased because they believe in caloric balance), then we cannot hang our hats on any of the other predicted outcomes either. These were every bit as "predicted" as the weight based on a model constructed by the very same (biased??) researchers. Either their methods are valid, or they are not. There were 30% more predicted values in the LC group vs. the LF group at the two year mark, where values would have been predicted for 32% of the LF sample and 42% of the LC sample.
And yet Bowden makes the following statements:
And- hold on to your hats—at all time points throughout the 2 years,including at the finish line, the low-carb group had a significant increase in HDL (“good”) cholesterol, approximately 23% increase to be precise.
There’s not a drug on earth that’s been able to do that. ....
... Second, you may have noticed that those improved cardiovascular risk factors showed up for the low-carb group (only!) after six months, but that after that, there was no difference between the groups in those risk factors—both had improvements. (Except of course, for the very important improvement in HDL cholesterol, which was seen only in the low-carb group and was sustained throughout the two years!)As I've stated before, the percent game for HDL masks the fact that the changes in HDL are not all that impressive. The mean increase was 7.75 points for folks with starting value in the mid-40's. Considering that 2/3rds of the participants were women, their endpoint mean HDL hasn't improved into that "protective" range. Bowden is flat out wrong claiming that improvement in HDL was only seen for the LC'ers. In fact, the low fat group also saw an increase of 4.64 points -- a little over half the improvement seen in the LC'ers. The absolute difference being only 3 points -- is there any difference in terms of risk factors for this small a difference? Furthermore, HDL increased in the LF group during the second year while it declined somewhat in the LC group. Absent actual dietary intake records to assess compliance with the plans, what conclusion can we really draw from these data?
Bowden makes a mountain out of a mole hill regarding designating weight loss as a primary outcome and the CVD risk factors as secondary outcomes. So what? There was a difference in the secondary outcomes, and this HAS been reported in the headlines. The LC community seems to have jumped hook-line-and-sinker for these relatively modest improvements and claimed vindication for the healthfulness of long term low carbing. This study offers no such support.
Bowden goes on to state:
“Averages” often conceal real differences—for example within the low-carb group there were some folks who really stuck to the program, and I’m willing to bet that when the raw data are released, you’ll see a number of individuals who not only maintained their weight (or kept losing) but also maintained the significant gains in cardiovascular risk factors that were dramatically seen after six months.I have made this statement myself many times in response to comments made about a study where personal or anecdotal evidence counters the results of some study or other. So the fact that LDL went up almost 8 points on average for the first 3 months of induction doesn't mean that MY LDL would go up that much or wouldn't render a massive reduction impossible. But Bowden seems to imply that these averages only conceal differences for the low carbers. Surely within the low-fat group there were some folks who really stuck to the program and who not only maintained but lost more after the 6 mo/1yr mark!! In the end, the means are still our best comparison for the population at large.
Now Bowden skirts clear of coming out and blaming upping the carbs for the reversal of the 6 month improvements in triglycerides, but Naughton fell into that trap a bit as other LC bloggers have. Essentially there's that almost wistful air of "what if" they had NOT ramped up the carbs. Unfortunately, unless more information is forthcoming, we do not know how far they ramped up the carbs, but it is important to see that there was no change in the trigs from the 3 month to the 6 month assessment during which time they were presumably "climbing the rungs" on Atkins. Naughton mentions that most successful low carbers end up somewhere under 100g/day. I think that is high for the more hardcore wing of the "movement".
Low carb in the various studies comparing weight loss and other factors seems to follow a predictable pattern "on average" -- greater early losses and a greater regain. Does this mean everyone will follow this? No. But neither does it mean that low fat dieters necessarily regain either.
In the end I find myself annoyed at the "if only this or that, but look at the results anyway". If only we look at those who followed LC stringently. Yeah, well, if only. But in the real world -- where doctors and nutritionists are trying to advise strategies beneficial to the greatest proportion of the population -- we can't just say LC works better, you just have to stick to it. Because CRD's work when they're stuck to all the same.
Bowden starts to sum up with: "If you had your choice between two diets- both of which produced weight loss, but one of which did it with less hunger and better cardiovascular outcomes, which would you choose?"
Bias?
No comments:
Post a Comment