This article is a commentary on:
Commentary: Effect evaluation of Sahtak bi Sahnak, a Lebanese secondary school-based nutrition intervention: a cluster randomised trial
Liliane Said
Jessica S. Gubbels
Stef P. J. Kremers- 1Department of Nutrition and Food Sciences, Faculty of Arts and Sciences, Lebanese International University, Bekaa, Lebanon
- 2Department of Health Promotion, NUTRIM School of Nutrition and Translational Research in Metabolism, Faculty of Health, Medicine, and Life Sciences, Maastricht University, Maastricht, Netherlands
A Commentary on
Effect evaluation of Sahtak bi Sahnak, a Lebanese secondary school-based nutrition intervention: a cluster randomised trial
by Siddique, A. B., Brown, A. W., Golzarri-Arroyo, L., and Allison, D. B. (2023). Front Nutr. 10:1241165. doi: 10.3389/fnut.2023.1241165
In response to the commentary of Siddique et al. (1), some further clarifications are provided regarding the statistical analysis and description of the methods used. As the published paper focuses on the effect evaluation of the intervention rather than on describing the study protocol, we believe that the present response will add clarification to the article, especially for readers who may have a different perspective in approaching the analysis.
1 Randomization and sample size
In addition to the lines mentioned in “Study Design and Participants” in the original article, the randomization process and the sample size calculations are further elaborated in the text below:
1.1 Randomization
The target population in the current study included adolescents aged 15 to 18 years living in Beirut, Baalbeck, and Rayak. The Lebanese Ministry of Education and Higher Education (MEHE) divides secondary schools into two categories: (i) public schools and (ii) private schools. Each category is subdivided based on the location. Hence, secondary schools located in the target areas were selected and categorized as follows:
Next, schools from each cell (of the Table 1) were randomly chosen (by picking their name out of a basket) and contacted to obtain consent to participate. Once the school accepted to participate, it was randomly assigned to either intervention or control group with a toss of coin with 1:1 allocation.
Table 1
Table 1. Sample of the created school lists.
In Baalbeck, there are only three public secondary schools and they all agreed to participate, so we had to recruit three private schools to balance the sample. Similarly, in Rayak, there is only one public secondary school, so in parallel one private school was allocated in this town. As for Beirut, we aimed to reach eight schools (i.e., four public and four private ones) to obtain the same number of secondary schools in the urban and rural study locations. However, only two private schools accepted to participate, so we recruited six public schools, instead of four. This justifies the adjustment for the type of school later in the statistical analyses.
All students within the same school were assigned to be part of the same study group (i.e., intervention vs. control) to avoid information contamination between groups as all students from the same school met in the same recreation ground during recess.
It is important to note that the randomization did not take into consideration the gender, BMI z-score, and grade/class distribution within schools. However, these variables were adjusted for later in the statistical analyses. In addition, and as was mentioned in the paper, it is potentially not possible to generalize the obtained results to the rest of the country or the rest of Lebanese adolescents as the current sample was not representative.
1.2 Sample size
According to the CONSORT 2010 updated guidelines for reporting parallel group randomized trials (2), authors should indicate the sample size calculation including the estimated outcomes, the α error level, the statistical power, and the standard deviation of the measurements for continuous outcomes.
The following equation was used to calculate n (3):
• Zβ = 0.842 and β = 0.2.
• Zα=1.96 and α = 0.05.
• μ = 18.2% (4) representing the expected difference between the means of nutrition knowledge for the two groups being compared.
• σ = 0.54 (4) representing the standard deviation for the two groups.
This resulted in n = 139 per group, equalling n = 278 for both study groups. This method was similar to what other cluster randomized controlled trials (cRCT) adopted in their sample size calculation (5–7).
As the current study design is a cRCT, it is important to note that the coefficient of intracluster correlation (ICC) was not included in the calculation for the following reasons: (i) to our knowledge very few studies (8) focused on improving the dietary or nutrition knowledge among adolescents aged 15 to 18 years, and none were cRCT, hence evidence-based input for the ICC was lacking; (ii) second, we searched the literature for other cRCT including younger adolescents in Lebanon, but the only available study at that time did not indicate how the sample size was calculated, nor the ICC (9). The updated study published in 2020 did not indicate the details regarding the sample size nor the ICC value (10), which may reflect the lack of data regarding this issue (11); (iii) When looking at studies conducted in school settings among younger children, the ICC for children's nutritional knowledge was equal to 0 (12). Based on this, the calculated sample size remains the same as presented in the paper:
n = nI [1+(m−1)p] (13);
• n is the required sample per arm.
• nI is the required sample size per arm during an individually randomized trial.
• p is the ICC.
• m is the cluster size.
If ICC = 0, it means that: n = nI.
As the current paper was mainly focused on the evaluation of the intervention outcomes rather than on describing the study protocol, we decided not to present these underlying details in the original paper to focus more on the outcomes and remain within the journal's word limits. We are happy to have the opportunity to provide these details in the current response.
2 Chi-square and t-test analysis
According to the CONSORT guidelines (14), a table showing baseline demographic characteristics for each study group should be included. Even though the randomization process prevents selection bias, it does not automatically assume that the study groups are equivalent at baseline (2). Since the number of schools in each study group was the same (eight schools in the intervention group and eight schools in the control group) and as the number of schools per location was also equal (eight schools from the urban region and eight schools from the rural one), our focus turned to evaluating the demographic characteristics at the individual level. This was further proved by the statistical tests conducted by the authors of the commentary as there was no significant difference in any variable of Table 1 related to demographics (1).
To evaluate the appropriateness of the randomization process related to the demographic characteristics of the sample, both Chi-square and independent t-test were performed. As was mentioned in the paper (see Statistical Analyses), Chi-square tests were used to analyse the differences in frequency of the categorical background characteristics (such as gender) between study groups, and the independent t-test was applied to compare the variation between study groups at baseline for continuous variables (such as the BMI z-score). The same tests were also used for these purposes in other cRCT (5, 15–18).
3 Clustering and nesting
The current study took into consideration the clustered design of the study by applying either the multilevel analysis or other tests as indicated below.
In the current statistical analysis, clustering included adolescents, schools, and school location. The grade or class (grades 10 and 11) was considered as an independent variable and not a cluster for the following reasons: (i) class size: the number of students per class in grade 10 is higher compared to grade 11. This means that the intervention may be more effective in smaller classes compared to larger ones; (ii) age category: to be able to evaluate the effectiveness of the intervention among older vs. younger adolescents later on. This explains the adjustment of the analysis for grade/class in Table 2 and the inclusion of this particle variable in the subgroup analysis in Table 3 (as indicated in the Statistical Analyses on page 5).
Concerning the other tests, some references state that there are cases that do not require running a multilevel analysis, and a multiple regression test could be used instead (19). For instance, when the random intercept is non-significant, it makes sense not to run the multilevel analysis, especially when the intercept does not vary from one school to another.
It is worth noting that after running the multilevel analyses for the unhealthy items score, we obtained a B value of (−1.40) with a 95% Confidence Interval of (−1.88) to (−0.93), and p-value of < 0.001. The direction, magnitude, and significance of the results were similar as with the multi-regression analyses.
4 Reproduction of statistical analysis
As reproducibility is the base of any scientific work, we thank the authors for their time in reproducing the statistical analyses. The authors reproduced the results based on the shared data, the codebook, and the described methodology in the paper, which suggests that the methodology related to running the tests is thoroughly and sufficiently described allowing for a sound reproduction of the results.
It is important to note that adolescents, schools, and school location were considered in the multilevel analysis, whereas the authors of the commentary have only relied on adolescents and schools. Concerning the reproduced tests in Table 1 of the commentary, the small differences related to the confidence of interval are due to adding one additional covariate (i.e., age) which was not considered in the initial analysis. As mentioned in the footnote of Table 2, all models were adjusted for gender, class/grade, type of school, school location, BMI z-score, and obtained scores at baseline. Age was not among them because we already included “class/grade” which takes into consideration older and younger adolescents.
As for the reproduced tests in Table 2 of the commentary, the significant differences in the B values of the total dietary knowledge score based on the baseline score are due to dividing the sample according to the healthy items score at baseline, and not according to the knowledge score at baseline (pre-intervention), which is not what was done in the analysis shown in the paper. This is indicated in the attached code (see “if healthy_score_category_pre = = 1”).
It is also important to note that some variables were counted more than once in the reproduced results of Siddique et al. (1). For instance, in the initial analysis shown in the paper, when dividing the sample into two groups: grade 10 and grade 11, the grade/class variable was not added to the analysis, as it was already taken into consideration. The same goes for weight status and knowledge score at baseline. In other words, when dividing the sample based on the weight status, there is no need to adjust the model for BMI z-score, and when dividing the sample based on the baseline knowledge score, there is no need to adjust for the baseline knowledge score once again.
Additionally, the significant differences in the healthy items adherence score based on the location (Table 3 of the commentary) are also due to adding the wrong variable. The dependent variable is the healthy items score post-intervention, and not the unhealthy items score post-intervention.
Concerning the revised tests, the same comments apply as mentioned above.
Finally, we would like to thank the authors for engaging in the discussion and appreciate the opportunity to provide further details with regard to our original paper. We believe Siddique and colleagues have presented a different approach for the data analysis, presenting the reader with a valuable alternative analytical approach to consider. In this response, we have provided a thorough explanation of why certain choices in our analytical approach were made. We feel it is important to stress that both approaches can be argued for, and both result in the same conclusion.
Author contributions
LS: Writing – review & editing, Writing – original draft. JG: Writing – review & editing. SK: Writing – review & editing.
Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Siddique AB, Brown AW, Golzarri-Arroyo L, Allison DB. Commentary: Effect evaluation of Sahtak bi Sahnak, a Lebanese secondary school-based nutrition intervention: a cluster randomised trial. Front Nutr. (2023) 10:1241165. doi: 10.3389/fnut.2023.1241165
2. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. Int J Surg. (2012) 10:28–55. doi: 10.1016/j.ijsu.2011.10.001
3. Livingston EH, Cassidy L. Statistical power and estimation of the number of required subjects for a study based on the t-test: a surgeon's primer. J Surg Res. (2005) 126:149–59. doi: 10.1016/j.jss.2004.12.013
4. Oldewage-Theron WH, Egal A. The evaluation of a nutrition programme on the nutrition knowledge of children aged six and seven years. J Family Ecol Consumer Sci. (2009) 37:48945. doi: 10.4314/jfecs.v37i1.48945
5. Haerens L, Deforche B, Maes L, Brug J, Vandelanotte C, De Bourdeaudhuij I, et al. computer-tailored dietary fat intake intervention for adolescents: results of a randomized controlled trial. Ann Behav Med. (2007) 34:253–62. doi: 10.1007/BF02874550
6. Martens MK, Van Assema P, Paulussen TGWM, Van Breukelen G, Brug J. Krachtvoer: Effect evaluation of a Dutch healthful diet promotion curriculum for lower vocational schools. Public Health Nutr. (2007) 11:271–8. doi: 10.1017/S1368980007000298
7. Waldén M, Atroshi I, Magnusson H, Wagner P, Hägglund M. Prevention of acute knee injuries in adolescent female football players: Cluster randomised controlled trial. BMJ. (2012) 344:e3034. doi: 10.1136/bmj.e3042
8. Shah P, Misra A, Gupta N, Hazra DK, Gupta R, Seth P, et al. Improvement in nutrition-related knowledge and behaviour of urban Asian Indian school children: findings from the Medical education for children/Adolescents for Realistic prevention of obesity and diabetes and for healthy aGeing (MARG) intervention study. Br J Nutr. (2010) 104:427–36. doi: 10.1017/S0007114510000681
9. Habib-Mourad C, Ghandour LA, Moore HJ, Nabhani-Zeidan M, Adetayo K, Hwalla N, et al. Promoting healthy eating and physical activity among school children: findings from Health-E-PALS, the first pilot intervention from Lebanon. BMC Public Health. (2014) 14:1–11. doi: 10.1186/1471-2458-14-940
10. Habib-Mourad C, Ghandour LA, Maliha C, Awada N, Dagher M, Hwalla N. Impact of a one-year school-based teacher-implemented nutrition and physical activity intervention: main findings and future recommendations. BMC Public Health. (2020) 20:1–7. doi: 10.1186/s12889-020-8351-3
11. Parker K, Nunns M, Xiao ZM, Ford T, Ukoumunne OC. Characteristics and practices of school-based cluster randomised controlled trials for improving health outcomes in pupils in the United Kingdom: a methodological systematic review. BMC Med Res Methodol. (2021) 21:1–17. doi: 10.1186/s12874-021-01348-0
12. Kaufman-Shriqui V, Fraser D, Friger M, Geva D, Bilenko N, Vardi H, et al. Effect of a school-based intervention on nutritional knowledge and habits of low-socioeconomic school children: a cluster-randomized controlled trial. Nutrients. (2016) 8:234. doi: 10.3390/nu8040234
13. Hemming K, Girling AJ, Sitch AJ, Marsh J, Lilford RJ. Sample size calculations for cluster randomised controlled trials with a fixed number of clusters. BMC Med Res Methodol. (2011) 11:1–11. doi: 10.1186/1471-2288-11-102
14. Campbell MK, Piaggio G, Elbourne DR, Altman DG. Consort 2010 statement: extension to cluster randomised trials. BMJ (Online). (2012) 345:e5661. doi: 10.1136/bmj.e5661
15. Diddana TZ, Kelkay GN, Dola AN, Sadore AA. Effect of nutrition education based on health belief model on nutritional knowledge and dietary practice of pregnant women in Dessie town, northeast Ethiopia: a cluster randomized control trial. J Nutr Metab. (2018) 2018:6731815. doi: 10.1155/2018/6731815
16. Ariza C, Pérez A, Sánchez-Martínez F, Diéguez M, Espelt A, Pasarín MI, et al. Evaluation of the effectiveness of a school-based cannabis prevention program. Drug Alcohol Depend. (2013) 132:257–64. doi: 10.1016/j.drugalcdep.2013.02.012
17. Ardic A, Erdogan S. The effectiveness of the COPE healthy lifestyles TEEN program: a school-based intervention in middle school adolescents with 12-month follow-up. J Adv Nurs. (2017) 73:1377–89. doi: 10.1111/jan.13217
18. Mack I, Reiband N, Etges C, Eichhorn S, Schaeffeler N, Zurstiege G, et al. The kids obesity prevention program: cluster randomized controlled trial to evaluate a serious game for the prevention and treatment of childhood obesity. J Med Internet Res. (2020) 22:e15725. doi: 10.2196/15725
Keywords: cluster randomized controlled trial, reproducibility, randomization, transparency, schools
Citation: Said L, Gubbels JS and Kremers SPJ (2024) Response to Commentary: Effect evaluation of Sahtak bi Sahnak, a Lebanese secondary school-based nutrition intervention: a cluster randomised trial. Front. Nutr. 11:1396571. doi: 10.3389/fnut.2024.1396571
Received: 05 March 2024; Accepted: 03 April 2024;
Published: 26 April 2024.
Edited by:
Roberto Fernandes Da Costa, Autonomous University of Chile, ChileReviewed by:
Mariane De Oliveira, University of São Paulo, BrazilGerson Ferrari, University of Santiago, Chile
Copyright © 2024 Said, Gubbels and Kremers. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Liliane Said, bGlsaWFuZS5zYWlkJiN4MDAwNDA7aG90bWFpbC5jb20=