This article is a commentary on:
Studying a Possible Placebo Effect of an Imaginary Low-Calorie Diet
Stephanie L. Dickinson
Greyson Foote
David B. Allison- 1Department of Epidemiology and Biostatistics, School of Public Health, Indiana University Bloomington, Bloomington, IN, United States
- 2Department of Sociology, Indiana University Bloomington, Bloomington, IN, United States
A Commentary on
Studying a Possible Placebo Effect of an Imaginary Low-Calorie Diet
by Panayotov VS (2019). Front Psychiatry 10:550. doi: 10.3389/fpsyt.2019.00550
While Panayotov reported very strong effects in their study of a “placebo” diet (1), the statistical analysis was performed incorrectly. To evaluate the effect of a treatment in a randomized controlled trial (RCT), the difference between groups should be tested directly rather than basing conclusions on the changes within each group separately. Conclusions in Panayotov’s manuscript are based on what has been called a DINS error or difference in nominal significance (2). A significant change from baseline within the treatment group and a non-significant change from baseline within the control group does not necessarily mean that the treatment effect was significant, i.e. that the treatment group improved significantly more than the control group. Allison and colleagues have identified DINS errors in many papers in the field of nutrition calling for their corrections (3–5).
Panayotov’s RCT tests the differential effects of participants believing they are on a low-calorie diet compared to those who know they are on an isocaloric diet (n=7 per group). Outcomes measured were body mass (kg), fat mass (%), and BMI (kg/m2) in the treatment and control group at the beginning of the experiment and after 8 weeks of both groups being on an isocaloric diet, where the treatment group were told they were receiving a low calorie diet. The results of the study are based on paired t-tests comparing the outcomes at baseline and 8 weeks, separately within each group. This can make treatment effects appear to be significant where they otherwise would not be.
We commend the author for sharing his data at our request, and we re-analyze the data focusing on the between-group differences to isolate the treatment effect. When we conduct t-tests on the differences (post-pre) between groups, conclusions change for one of the three outcomes (Table 1). Panayotov reported a p-value of < .001 for decreased fat mass in the treatment group and non-significant change (p=.21) in the control group to conclude that the treatment is better than the control. However, when we compare the decrease in the treatment group to the decrease in the control group with an independent samples t-test, we obtain a non-significant p-value of p=.105. When analyzing the % change [(post-pre)/pre], the p-value for the treatment effect on fat mass is also not significant p=.156. Treatment effects on body mass and BMI remained significant in re-analysis.
TABLE 1
Table 1 Re-analysis of intervention effects.
Incidentally, in our re-analysis, we found that we were unable to replicate the results of the paired t-tests reported in Panayotov's Table 2 for the experimental group. We obtained more significant results than those reported with p=.002 and p=.001 instead of the reported p=.02 and p=.01 reported for body mass (kg) and BMI respectively.
Panayotov should re-analyze the data from his study with statistical comparisons of outcomes between groups rather than within-groups and publish corrected results.
Author Contributions
DA: identified the error in the Panayotov paper and co-wrote the paper. SD and GF: re-analyzed the data with the correct analyses and co-wrote the paper.
Funding
This work was funded in part by the National Institutes of Health (NIH): R25HL124208. The opinions expressed are those of the authors and not necessarily of the NIH or any other organization.
Conflict of Interest
In the last 12 months, DA has received personal payments or promises for same from: American Society for Nutrition; American Statistical Association; Biofortis; California Walnut Commission; Columbia University; Fish & Richardson, P.C.; Frontiers Publishing; Henry Stewart Talks; IKEA; Indiana University; Laura and John Arnold Foundation; Johns Hopkins University; Law Offices of Ronald Marron; MD Anderson Cancer Center; Medical College of Wisconsin; National Institutes of Health (NIH); Sage Publishing; The Obesity Society; Tomasik, Kotin & Kasserman LLC; University of Alabama at Birmingham; University of Miami; Nestle; WW (formerly Weight Watchers International, LLC). Donations to a foundation have been made on his behalf by the Northarvest Bean Growers Association. DA is an unpaid member of the International Life Sciences Institute North America Board of Trustees. The institution of DA, SD, and GF, Indiana University, has received funds to support their research or educational activities from: NIH; Alliance for Potato Research and Education; American Federation for Aging Research; Dairy Management Inc; Herbalife; Laura and John Arnold Foundation; National Cattlemen’s Beef Association, Oxford University Press, the Sloan Foundation, The Gordan and Betty Moore Foundation, and numerous other for-profit and non-profit organizations to support the work of the School of Public Health and the university more broadly. DA’s prior institution, the University of Alabama at Birmingham, received gifts, contracts, and grants from other organizations including the Coca-Cola Company, Pepsi, and Pepper/Snapple.
References
1. Panayotov VS. Studying a Possible Placebo Effect of an Imaginary Low-Calorie Diet. Front Psychiatry (2019) 10:550. doi: 10.3389/fpsyt.2019.00550
2. Allison DB, Brown AW, George BJ, Kaiser KA. Reproducibility: A tragedy of errors. Nat News (2016) 530(7588):27. doi: 10.1038/530027a
3. Allison DB, Antoine LH, George BJ. Incorrect statistical method in parallel-groups RCT led to unsubstantiated conclusions. Lipids Health Dis (2016) 15(1):77. doi: 10.1186/s12944-016-0242-3
4. Cassani RSL, Fassini PG, Silvah JH, Lima CMM, Marchini JS. Retraction Note: Impact of weight loss diet associated with flaxseed on inflammatory markers in men with cardiovascular risk factors: a clinical study. Nutr J (2015) 15(1):59. doi: 10.1186/s12937-016-0165-x
5. Kroeger CM, Brown AW, Allison DB. Differences in Nominal Significance (DINS) Error leads to invalid conclusions: Letter regarding,”Diet enriched with fresh coconut decreases blood glucose levels and body weight in normal adults”. J Complementary Integr Med (2019) 16(2):20180037. doi: 10.1515/jcim-2018-0037
Keywords: difference in nominal significance, reanalyzed data, error, correction, data analysis
Citation: Dickinson SL, Foote G and Allison DB (2020) Commentary: Studying a Possible Placebo Effect of an Imaginary Low-Calorie Diet. Front. Psychiatry 11:329. doi: 10.3389/fpsyt.2020.00329
Received: 11 January 2020; Accepted: 01 April 2020;
Published: 29 April 2020.
Edited by:
Stephan Zipfel, University of Tübingen, GermanyReviewed by:
Paul Enck, University of Tübingen, GermanyCopyright © 2020 Dickinson, Foote and Allison. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Stephanie L. Dickinson, sd3@indiana.edu