Reproducibility of Dietary Intake Measurement From Diet Diaries, Photographic Food Records, and a Novel Sensor Method

Objective: No data currently exist on the reproducibility of photographic food records compared to diet diaries, two commonly used methods to measure dietary intake. Our aim was to examine the reproducibility of diet diaries, photographic food records, and a novel electronic sensor, consisting of counts of chews and swallows using wearable sensors and video analysis, for estimating energy intake.

Method: This was a retrospective analysis of data from a previous study, in which 30 participants (15 female), aged 29 ± 12 y and having a BMI of 27.9 ± 5.5, consumed three identical meals on different days. Four different methods were used to estimate total mass and energy intake on each day: (1) weighed food record; (2) photographic food record; (3) diet diary; and (4) novel mathematical model based on counts of chews and swallows (CCS models) obtained via the use of electronic sensors and video monitoring system. The study staff conducted weighed food records for all meals, took pre- and post-meal photographs, and ensured that diet diaries were completed by participants at the end of each meal. All methods were compared against the weighed food record, which was used as the reference method.

Results: Reproducibility was significantly different between the diet diary and photographic food record for total energy intake (p = 0.004). The photographic record had greater reproducibility vs. the diet diary for all parameters measured. For total energy intake, the novel sensor method exhibited good reproducibility (repeatability coefficient (RC) of 59.9 (45.9, 70.4), which was better than that for the diet diary [RC = 79.6 (55.5, 103.3)] but not as repeatable as the photographic method [RC = 43.4 (32.1, 53.9)].

Conclusion: Photographic food records offer superior precision to the diet diary and, therefore, would be valuable for longitudinal studies with repeated measures of dietary intake. A novel electronic sensor also shows promise for the collection of longitudinal dietary intake data.

Introduction

Measurement of dietary intake is a necessary but difficult undertaking in clinical and research settings. Common methods used to measure dietary intake include 24-h diet recalls, diet diaries, photographic food records, and food frequency questionnaires (1). There are advantages and disadvantages to each method in terms of cost and participant burden, but all methods share the limitations of self-report. Studies using doubly labeled water have shown that underreporting of food intake is a common problem for self-report methods (2–6). Despite the limitations of these self-report methods, they remain the only validated methods available for measuring dietary intake in free-living situations.

Proper research practice requires that methods be validated against a standard: a previously validated method and/or a biomarker, such as doubly labeled water for energy expenditure (5). Validity refers to the accuracy of any measure; that is, how close the measured value is to the actual value. An equally important, and often overlooked, feature of a method is its reproducibility or precision. Reproducibility or precision is the extent to which a measure yields the same results under similar conditions.

The reproducibility of an instrument is especially important when dietary intake will be recorded longitudinally to assess habitual intake or changes over time. A study using repeated 24-h recalls showed total energy correlation of r = 0.59 between measurements (7). Reproducibility research conducted with food frequency questionnaires at two time points has shown that total energy correlations between repeat administrations of questionnaires range from r = 0.30–0.92 (7–13). Watson et al. (9) cited under- or over-reporting as a likely contributor to the low reproducibility for the food frequency questionnaire. This concept of systematic under- or over-reporting in dietary assessment was examined by Black and Cole (3). Their review of seven studies with repeated measurements of dietary intake revealed that some persons are more likely to underreport dietary intake than others, regardless of the assessment method used. This personal reporting bias is an issue that should not be ignored when examining dietary intake data and considering the necessity of repeated measures in such research.

Although two previous studies have looked at the reliability of diet diaries, neither used a gold standard reference method, such as a weighed food record, during the same period as the diet diary was recorded, thus limiting the general applicability of the data (14, 15). To our knowledge, no previous study has rigorously examined the reproducibility of the diet diary, which is one of the most commonly used methods to measure free-living dietary intake, or the photographic food record. Both instruments have been studied for accuracy, but there exists no data on their precision. The aim of this study was to examine the reproducibility of diet diaries, photographic food records, and a novel electronic sensor from three separate, identical meals using weighed food records as the gold standard reference method.

Methods

Participants

Thirty participants (15 females and 15 males) with a mean (±SD) age of 29 ± 12 y (range: 19–58 y) and body mass index (BMI) of 27.9 ± 5.5 kg/m² (range: 20.5–41.7) were recruited at the Clarkson University campus to participate in the study. The study was approved by the Institutional Review Board at Clarkson University, Potsdam, NY and all participants read and signed an informed consent form before participation. Participants with temporo-mandibular joint disease, dysphagia or other difficulties for chewing and/or swallowing were excluded from the study.

Each participant consumed three full meals at three different visits in the laboratory, ~1–4 weeks apart and at the same clock time at each visit. At the first visit, each participant was asked to select foods according to their own preferences (content and size) from the menu offered by one of the Clarkson University food courts. Any foods or amounts could be chosen by participants, with no restrictions. The initial meal selection was documented so that the selected meal was identical for all three study visits. Participants had no limitations on the quantity of consumed foods or order in which the foods had to be consumed.

Energy Intake Measurements

Four different methods were used to estimate total mass and energy intake: (1) weighed food record; (2) photographic food record; (3) diet diary; and (4) mathematical models based on counts of chews and swallows (CCS models) obtained via the use of electronic sensors (16).

To obtain the nutritional intake data from meals, records were deidentified and sent to the Colorado Clinical and Translational Sciences Institute's (CCTSI) Nutrition Core. A single operator assessed all deidentified photographic food records and logged consumed food amounts in a standard diet dairy format. A second blinded, independent operator entered all converted photo and original participant food diaries into the nutritional analysis program Nutrient Data System for Research (NDS-R; University of Minnesota, Minneapolis, MN). None of the data entry operators at the CCTSI Nutrition Core were involved in data collection. Using a single, trained operator at each step is the current operating procedure for all CCTSI protocols and reduces variation due to inter-operator differences in data entry. All weighed food records, photographic food records, and diet diaries were de-identified before operator entered nutritional intake into NDS-R. The novel method of using models to count chews and swallows to determine total mass and energy intake was blinded so that operator processing the data was not involved in the post-ingestion annotation of chews and swallow from the original videos.

Weighed Food Records

Before and after each meal, food was weighed by a trained member of the research team to calculate the total amount consumed. Each meal was documented and logged into a chart containing detailed information of each food item such as food name and description, mass at beginning and end of the meal, and total mass consumed. For items that could be deconstructed (e.g., a sandwich), each food item was weighed separately before and after consumption. The item was reassembled before being served to the participant. For items that could not be deconstructed (e.g., pizza or cookies) total energy intake was estimated using total weight consumed multiplied by the caloric density of the item. Weighed food records were used as the reference method for actual dietary intake. All other methods were compared to actual dietary intake measured by the weighed food records.

Photographic Records

Pre- and post-meal photographs were taken by study staff using a digital camera. The serving plate occupied the entire field of view, and photographs were taken at a 45° angle so that the depth of foods could be estimated (17). A picture of the selected meal was taken before serving and another picture was taken at the end of the eating period. A trained, validated CCTSI nutritionist used these pictures to estimate portion sizes, using the Portion Photos of Popular Foods guide (18) and entered consumed amounts into the food analysis program, NDS-R.

Diet Diary

At the beginning of the first visit, participants were trained to complete a diet diary. Oral and written instructions were given to participants for estimating portion sizes and recording foods in sufficient detail to obtain an accurate estimate of dietary intake. Examples of both fully complete and incomplete diaries were explained to demonstrate how to appropriately record intake. Participants also received a portion estimation guide that was used as a reference, but only during the first visit. All materials were supplied by the CCTSI Nutrition Core.

After each meal was finished, participants recorded the food items they just consumed in a blank food diary. Each food item was recorded on a single line indicating the type of food, preparation style, and amount consumed. Participants did not receive any help during this stage; however, the diet diary was reviewed to ensure that it was completed appropriately (i.e., all foods listed had a portion size and description assigned). Participants were not prompted to add any food items they had forgotten to record. Participants only filled out diet diaries for research meals, and no other meals consumed during the 3 days of the study.

Models Based on Counts of Chews and Swallows

Estimation of the mass and energy consumed during each meal was computed using participant-dependent models based on counts of chews and swallows. Before starting the experiments, participants were instrumented with a sensor system for monitoring ingestive behavior (19). The system consisted of: (1) a jaw motion sensor placed below the ear to capture chewing events; (2) a miniature microphone placed on the throat to capture swallowing sounds; and (3) a digital camera for video monitoring. Sensor data and video footage were used to compute the number of chews and swallows associated to each meal as previously described (16). The total mass and energy for a given meal was estimated using a counts of chews and swallows model created with the counts of chews and swallows observed in the remaining two meals consumed by the same participant (16).

Statistical Analysis

This was a retrospective data analysis of a previous study (16). The sensor method was only analyzed for total energy as this is an exploratory method, still under development and the form described in (16) was only able to estimate mass and energy intake during a meal. When the sensor method is further developed, it will be used to estimate energy, macronutrient, and micronutrient intakes.

Because the actual amount of food consumed varied between study visits, the percent difference from that assessed by the weighed food method serves as the outcome to compare reproducibility across diary, photographic, and sensor methods. The repeatability coefficient (RC) defined as RC $=1.96\times \sqrt{2\times }SDwithinsubject$ was used to assess the extent of reproducibility for each method. Within-participant variability (SDwithinsubject) of the outcome across three time points was assessed using the with-subject variance from a linear mixed effects model, where the fixed effect consists of intercept only and had a compound symmetry covariance structure. Five thousand Bootstrap samples were based to calculate the 95% confidence intervals for RC for each method and the difference in RC between methods as well as the p-values. SAS 9.4 software (SAS Institute Inc.) were used for all the analyses.

Results

Comparison of the weighed intake data from the three meals indicated that there were no differences in energy or macro- or micro-nutrient intake between the three meals (data not shown). The RC values for the percent difference from the weighed food records revealed that the photographic food record and sensor methods had greater reproducibility [RC = 43.4 (32.1, 53.9) and 59.9 (45.9, 70.4), respectively] than the diet diary [RC = 79.6 (55.5, 103.3)] for total energy intake over three separate meals (Table 1 and Figure 1). Differences in RC values between photographic food records and diet dairies were significantly different for total energy (p = 0.004), carbohydrate (p = 0.01), protein (p = 0.02), calcium (0.004) and iron (p = 0.02) intake (Table 2), with photographic food records having greater reproducibility for all nutrients measured (Table 1).

TABLE 1

Table 1. Repeatability coefficients (95% confidence interval) between measurement methods for percent deviation from weighed measurement.

FIGURE 1

Figure 1. RC and 95% CI of percent difference from weighed method for energy measurements over three time points. The photographic food record and sensor methods had greater reproducibility (lower RC values) than the diet diary for overall energy intake over three time points.

TABLE 2

Table 2. Difference in repeatability coefficient between methods over three time points.

Discussion

Participants completed diet diaries immediately following each meal under supervised conditions. The method of completing the diary immediately post-meal under controlled conditions, as in this study, gives the greatest chance for this method to perform at its best. However, the diet diary method displayed the lowest reproducibility of the three methods tested for total energy intake, and it was inferior to the photographic food record for macronutrients and micronutrients examined. It should be noted that the food photographs were taken by study staff, so this method was also performed under optimal conditions that are not normally present when photographic food records are used.

Reproducibility is an important factor to consider when designing longitudinal studies in which dietary intake is to be measured repeatedly. Under these circumstances, a tool that is more reproducible will decrease the variance in the data collected over time, thereby simplifying data interpretation. Considering the cost and time spent on such studies, as well as participant burden, any instrument which is highly reproducible would add value. Indeed, previous work showed that the photographic food record is as accurate as the diet diary in both energy intake and macronutrient composition but decreases participant burden (17). With the added benefit of higher reproducibility, the photographic food record offers increased utility over the traditional diet diary.

Although two previous studies have looked at the reliability of food records, both compared diet diaries recorded at different times, with no consistency of foods eaten during each recording period (14, 15). Therefore, any differences noted could have reflected actual differences in dietary intake rather than methodological issues. Putz et al. compared two diet diaries to a weighed food record as a reference method (15). However, the weighed food record was completed at separate time from the diet diaries so it is unclear if the dietary intake was similar across occasions and therefore, if the differences measured were due to the method used or actual differences in dietary intake on the different recording occasions. In these previous studies, for estimating total energy intake, the reproducibility of the diet diary was low to moderate [ICC of 0.49 and 0.69 for (14, 15), respectively], which compares well with our estimate of low reproducibility (RC = 43.4). Conversely, we found that the reproducibility for the sensor and photographic record methods was moderate to high, respectively.

Limitations of this study include small sample size, limited age range of participants, that energy intake was not matched between meals for each participant, and photographs in the photographic food records were taken by study staff and not participants. Whereas weighed food records are considered the gold standard and this method was used in our laboratory setting, in a free living situation doubly labeled water could be used to compare reported intake to total energy expenditure, albeit at greater expense. With regard to expense, studies have shown photographic food records to be similar in cost or less costly than self-report methods such as diet diaries and 24-h recalls (20–23). However, when compared to written diet diaries, it does take ~20 more mins per day of intake recording to analyze photos and convert the visual information to amounts for data entry, which is likely irrelevant for smaller studies but could create higher cost overall for large studies. This study had several strengths, however, including the use of a within subject repeated measures design, the large variety of foods for participants to choose from, the use of more than two repeated measures, and that the study took place in a controlled laboratory setting where the researchers had the ability to accurately determine energy intake using weighed food records, considered the gold-standard (24–26).

An interesting finding from this study is that the photographic food record was more precise/reproducible than the sensor method, even though previous work showed that the sensor was more accurate than photographic food records (19). Accuracy may be of greater concern when working with understudied or vulnerable populations where little data currently exist, or in studies that measure dietary intake at a single point. Under these circumstances, the sensor method displays promise, particularly for vulnerable populations such as children with developmental delay or the elderly who may not be able to complete any other method for the estimation of dietary intake. As the sensor can be placed on the participant's jaw and behind the ear, unobtrusively estimating energy intake via measurement of chewing, the need for participant literacy or cognizance of food choices is abolished. In all populations, this method would significantly reduce participant burden and negate some of the pitfalls of self-report. Our future work aims to combine the strengths of the photographic and sensor methods by enabling the sensor to automatically take images of food ingested during the day.

Conclusion

The higher reproducibility of the photographic food record warrants its use over the diet diary in longitudinal studies which aim to measure dietary intake repeatedly. The novel sensor method for estimating energy intake also shows promise as a dietary intake assessment tool for the future.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

This study was approved by the Institutional Review Board at Clarkson University, Potsdam, NY and all participants read and signed an informed consent form before participation.

Author Contributions

Conceptualization: JF, ZP, ES, MM, and JH. Data curation: JF and TM. Investigation: JF and ES. Project administration: ES, MM, JH, and JT. Supervision: ES, MM, and JH. Writing original draft: TM, JH, and KM. Writing—review and editing: JT, JF, ZP, KM, ES, MM, TM, and JH. All authors contributed to the article and approved the submitted version.

Funding

Research reported in this publication was supported by the National Institutes of Health through the National Institute of Diabetes and Digestive and Kidney Diseases award numbers R21DK085462 and R01DK122473 and National Center for Advancing Translational Sciences award number U01TR002535. The content was solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank the Colorado Clinical and Translational Sciences Institute Nutrition Core for assistance with data analysis.

Supplementary Material

Photographic Records: http://dx.doi.org/10.21227/5mjw-jz09.

References

1. Thompson FE, Subar AF. Dietary assessment methodology. In: Coulston A, editor. Nutrition in the Prevention and Treatment of Disease. Bethesda, MD: Academic Press. (2013) 16–23. doi: 10.1016/B978-0-12-391884-0.00001-9

CrossRef Full Text | Google Scholar

2. Kaczkowski CH, Jones PJ, Feng J, Bayley HS. Four-day multimedia diet records underestimate energy needs in middle-aged and elderly women as determined by doubly-labeled water. J Nutr. (2000) 130:802–5. doi: 10.1093/jn/130.4.802

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Black AE, Cole TJ. Biased over- or under-reporting is characteristic of individuals whether over time or by different assessment methods. J Am Dietetic Assoc. (2001) 101:70–80. doi: 10.1016/S0002-8223(01)00018-9

CrossRef Full Text | Google Scholar

4. Goris AH, Westerterp-Plantenga MS, Westerterp KR. Undereating and underrecording of habitual food intake in obese men: selective underreporting of fat intake. Am J Clin Nutr. (2000) 71:130–4. doi: 10.1093/ajcn/71.1.130

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Hill RJ, Davies PS. The validity of self-reported energy intake as determined using the doubly labelled water technique. Br J Nutr. (2001) 85:415–30. doi: 10.1079/BJN2000281

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Trabulsi J, Schoeller DA. Evaluation of dietary assessment instruments against doubly labeled water, a biomarker of habitual energy intake. Am J Physiol Endocrinol Metab. (2001) 281:E891–9. doi: 10.1152/ajpendo.2001.281.5.E891

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Sanjeevi N, Freeland-Graves J, George GC. Relative validity and reliability of a. 1-week, semiquantitative food frequency questionnaire for women participating in the supplemental nutrition assistance program. J Acad Nutr Diet. (2017) 117:1972–82.e2. doi: 10.1016/j.jand.2017.05.013

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Cullen KW, Watson K, Zakeri I. Relative reliability and validity of the block kids questionnaire among youth aged 10 to 17 years. J Am Diet Assoc. (2008) 108:862–6. doi: 10.1016/j.jada.2008.02.015

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Watson JF, Collins CE, Sibbritt DW, Dibley MJ, Garg ML. Reproducibility and comparative validity of a food frequency questionnaire for Australian children and adolescents. Int J Behav Nutr Phys Act. (2009) 6:62. doi: 10.1186/1479-5868-6-62

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Horn-Ross PL, Lee VS, Collins CN, Stewart SL, Canchola AJ, Lee MM, et al. Dietary assessment in the California teachers study: reproducibility and validity. Cancer Causes Control. (2008) 19:595–603. doi: 10.1007/s10552-008-9124-y

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Wakimoto P, Block G, Mandel S, Medina N. Development and reliability of brief dietary assessment tools for Hispanics. Prev Chronic Dis. (2006) 3:A95. Available online at: http://www.cdc.gov/pcd/issues/2006/jul/05_0117.htm

PubMed Abstract | Google Scholar

12. Kobayashi T, Kamimura M, Imai S, Toji C, Okamoto N, Fukui M, et al. Reproducibility and validity of the food frequency questionnaire for estimating habitual dietary intake in children and adolescents. Nutr J. (2011) 10:27. doi: 10.1186/1475-2891-10-27

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Morel S, Portolese O, Chertouk Y, Leahy J, Bertout L, Laverdière C, et al. Development and relative validation of a food frequency questionnaire for French-Canadian adolescent and young adult survivors of acute lymphoblastic leukemia. Nutr J. (2018) 17:45. doi: 10.1186/s12937-018-0355-9

CrossRef Full Text | Google Scholar

14. Yoshizawa K, Willett WC, Yuan C. Reliability of repeated measures of nutrient intake by diet records in residents in the Western region of Japan. Nutrients. (2019) 11:2515. doi: 10.3390/nu11102515

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Putz P, Kogler B, Bersenkowitsch I. Reliability and validity of assessing energy and nutrient intake with the Vienna food record: a cross-over randomised study. Nutr J. (2019) 18:9. doi: 10.1186/s12937-019-0431-9

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Fontana JM, Higgins JA, Schuckers SC, Bellisle F, Pan Z, Melanson EL, et al. Energy intake estimation from counts of chews and swallows. Appetite. (2015) 85:14–21. doi: 10.1016/j.appet.2014.11.003

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Higgins JA, LaSalle AL, Zhaoxing P, Kasten MY, Bing KN, Ridzon SE, et al. Validation of photographic food records in children: are pictures really worth a thousand words? Eur J Clin Nutr. (2009) 63:1025–33. doi: 10.1038/ejcn.2009.12

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Hess MA. Portions of Popular Foods. New York City: Culinary Nutrition Publishing (2014).

19. Fontana JM, Lopez-Meyer P, Sazonov ES. Design of a instrumentation module for monitoring ingestive behavior in laboratory studies. Conf Proc IEEE Eng Med Biol Soc. (2011) 2011:1884–7. doi: 10.1109/IEMBS.2011.6090534

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Martin CK, Han H, Coulon SM, Allen HR, Champagne CM, Anton SD. A novel method to remotely measure food intake of free-living individuals in real time: the remote food photography method. Br J Nutr. (2009) 101:446–56. doi: 10.1017/S0007114508027438

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Martin CK, Correa JB, Han H, Allen HR, Rood JC, Champagne CM, et al. Validity of the Remote Food Photography Method (RFPM) for estimating energy and nutrient intake in near real-time. Obesity. (2012) 20:891–9. doi: 10.1038/oby.2011.344

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Martin CK, Nicklas T, Gunturk B, Correa JB, Allen HR, Champagne C, et al. Measuring food intake with digital photography. J Hum Nutr Diet. (2014) 27(Suppl. 1):72–81. doi: 10.1111/jhn.12014

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Zhaoxing P, Marden T, Mande A, Higgins J. A novel approach to calculating energy density from food images reduces analysis time and cost. Food Nutr Sci. (2019) 10:235–47. doi: 10.4236/fns.2019.102018

CrossRef Full Text | Google Scholar

24. Carlsen MH, Lillegaard IT, Karlsen A, Blomhoff R, Drevon CA, Andersen LF. Evaluation of energy and dietary intake estimates from a food frequency questionnaire using independent energy expenditure measurement and weighed food records. Nutr J. (2010) 9:37. doi: 10.1186/1475-2891-9-37

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Abebe BT, Weiss M, Modess C, Roustom T, Tadken T, Wegner D, et al. A 24-h recall does not provide a valid estimate of absolute nutrient intakes for rural women in southern Ethiopia. Nutrition. (2011) 27:919–24. doi: 10.1016/j.nut.2010.10.015

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Nightingale H, Walsh KJ, Olupot-Olupot P, Engoru C, Ssenyondo T, Nteziyaremye J, et al. Validation of triple pass 24-hour dietary recall in Ugandan children by simultaneous weighed food assessment. BMC Nutr. (2016) 2:56. doi: 10.1186/s40795-016-0092-4

PubMed Abstract | CrossRef Full Text | Google Scholar