Tracy L. Burrows
Yan Yee Ho
Clare E. Collins- School of Health Sciences, Faculty of Health and Medicine, Priority Research Centre for Physical Activity and Nutrition, University of Newcastle, Callaghan, NSW, Australia
Accuracy in quantifying energy intake (EI) using common dietary assessment methods is crucial for interpreting the relationship between diet and chronic disease. The aim of this systematic review was to evaluate the validity of dietary assessment methods used to estimate the EI of adults in comparison to total energy expenditure (TEE) measured by doubly labeled water (DLW). Articles in English across nine electronic databases, published between 1973 and February 2019 were retrieved. Studies were included if participants were adults (≥18 years) and used the DLW technique to measure TEE compared to self-reported EI. A total of 59 studies were included, with a total of 6,298 free living adults and a mean of 107 participants per study. The majority of studies including 16 studies that included a technology based method reported significant (P < 0.05) under-reporting of EI when compared to TEE, with few over-reporting EI. Misreporting was more frequent among females compared to males within recall based dietary assessment methods. The degree of under-reporting was highly variable within studies using the same method, with 24 h recalls having less variation and degree of under-reporting compared to other methods.
Introduction
The accuracy of measuring food and nutrient intakes using various dietary assessment methods is crucial for interpreting the relationship between development of dietary related chronic diseases, including type 2 diabetes mellitus, cardiovascular disease, and some cancers (1). These chronic diseases contribute significantly to the global burden of disease (2). The validity of dietary assessment methods plays an important role in accurately describing the dietary patterns and nutrient intakes of populations, comparing dietary intakes to recommended dietary guidelines, and following trends in dietary intakes in populations over time (3–5). While self-report measures of EI have received criticism, recommendations have been made to minimize bias when collecting, analyzing, and interpreting dietary data assessed using self-reported methods (6).
The incorporation of technologies to assess dietary intake, including by way of smartphone and the Internet, has facilitated key developments in the collection, analysis and interpretation of dietary intake data (7). This includes reducing costs associated with data collection and analysis, lowering subject and researcher burden and facilitating more timely approaches to data analysis (7). However, the emergence of newer dietary assessment methods with technology assisted components, such as image-based methods and wearable devices (e.g., micro-camera) that incorporate technology for data collection, means that a review of the validity of technology based methods is also timely (8, 9).
A variety of established self-reported dietary assessment methods exist, including 24 h recalls, diet histories, food frequency questionnaires (FFQs) and food records. Many methods are subject to mis-reporting which is often classified as over- or under-reporting (10, 11), with an additional selection bias in terms of the type of people who volunteer to participate in these studies, due to high participant burden (12, 13). Other potential biases within assessment of dietary intake can stem from issues relating to memory, perception and conceptualization of portion sizes, knowledge and confidence with technology—all of which could impact adversely on accuracy of reported EI (14, 15).
Image-based methods require participants to capture digital images of food and beverages pre- and post-consumption with a camera device, and as such are similar to a food record (7). Image-based methods may be susceptible to mis-reporting due to reactivity bias, in that knowing one must take an image of the foods about to be eaten may influence what foods the person chooses to eat on that occasion (9). In addition, measurement using a technology based dietary intake method is dependent and subject to the inherent limitations of technology-based approaches; identification of the food and its components and accounting for intra- and inter-individual variability, and complexities (7) related to whether food is consumed from one's own plate or shared plates (16) and/or consumed with additional condiments.
Measuring the validity of dietary assessment tools requires an objective measure that does not face the same inherent errors found in the dietary assessment tool being assessed. The doubly labeled water (DLW) technique is an objective method of measuring total energy expenditure (TEE), and is considered a reference method for evaluating validity of self-reported EI in relatively weight stable individuals (3, 4, 12). It is also independent of self-reported error (17, 18). An initial DLW dose is determined by standardized equations according to body weight. Following consumption, urine samples are collected over a period of seven to 14 days to account for short-term day-to-day variation in physical activity (19).
A previous review (2001) provides valuable insight that EI is consistently under-reported compared against DLW, with the majority of studies at the time of publication using food records or diaries (17). An additional review by Hill and Davies in 2001 went further to describe characteristics associated with under-reporting which included: (1) Dietary restraint, (2) Socioeconomic status, and (3) Gender (under-reporting more common in women than in men) (20). An additional review by Livingstone and Black (21) detailed additional factors relating to low energy reporters, which included possible cultural influences. However, there have been no reviews in adults since that investigate the misreporting of energy intake. It is within this context that this review aims to evaluate the validity of self-reported dietary assessment methods in estimating the daily EI of adults (≥18 years) in comparison to TEE measured by DLW.
Materials and Methods
Search Strategy
Initially searches of online database were conducted in Cochrane, CINAHL, MEDLINE, EMBASE, Scopus, Cumulative Index to Nursing and Allied Health Literature, ProQuest, PubMed and Excerpta Medica Database. Keywords and combinations of keywords used included adult, dietary assessment, food frequency questionnaire, dietary recall, 24 h food recall, diet record, food record, food diary, energy intake, energy expenditure, doubly labeled water, valid*, accuracy*, precise* and combination of all above-mentioned, see Supplementary Material for example search strategy. Articles retrieved were limited to those published in English-language journals between 1973 and February 2019. The reference lists of articles that met the inclusion criteria were hand searched and key articles identified were used for further searches via the Web of Science database Cited Reference function. Authors were not contacted for any missing information and gray literature was not searched. The protocol for this review was developed and registered with PROSPERO—an international prospective register of systematic reviews, under the registration number CRD42017064545.
Study Selection
The flow of studies at each stage of the review is depicted in Figure 1. Following the initial database searches, titles and abstracts were screened to determine which studies required full text retrieval. The full-text articles retrieved were assessed for eligibility using inclusion criteria. The screening was done by two independent reviewers (Y.H and T.B). Articles were identified as relevant if they were studies that aimed to compare dietary intake with TEE, if they included adult participants (aged ≥18 years), if they reported EI measured by a dietary assessment method, if DLW was used to estimate TEE and if the primary purpose of the study was to validate the dietary assessment method. Full articles were retrieved if eligible for inclusion or if eligibility for inclusion was unclear after screening the abstracts. Articles were reviewed by two independent reviewers (YH and TB). Any disagreement between the two reviewers was resolved by discussion with a third independent reviewer (MR).
Figure 1. Flow diagram of method for determining studies to be included in this systematic review of evaluating dietary assessment methods against the gold standard doubly labeled water method (DLW).
Data Extraction and Quality Evaluation
All relevant articles were then independently assessed for quality using the American Dietetic Association quality checklist for primary studies as outlined in the Evidence Analysis Manual (22). A study was rated as ‘positive’ quality if it satisfied a majority of the quality criteria, including four priority criteria pertaining to (1) Selection of study participants, (2) Comparability of study groups, (3) Intervention description and (4) Outcomes. A study was rated as having “neutral” or “negative” quality based on the number of criteria that were met/ not met. No studies were excluded from the review based on quality assessments.
Data relevant to this review were extracted using a standardized tool which was initially piloted using four studies, with minor wording changes made for reviewer clarity. Data were then extracted by two independent reviewers (YH, TB), including study design participant characteristics, dietary assessment methods(s) used, and DLW results. Any discrepancy was resolved via discussion with a third reviewer (MR). Dietary assessment methods were categorized using the National Cancer Institutes of Health Dietary Assessment Primer definitions (23). Dietary assessment methods with technology components were also recorded if any form of communication and/or information technology was used, such as mobile or smartphone, the Internet or sensors collecting image, movement or auditory data. The technology could be utilized in either the collection, analysis or interpretation of the dietary method.
Results
Population
The search strategy identified 572 records (Figure 1). After review of full text papers, 59 articles were included and underwent critical appraisal and data extraction. Major reasons for exclusion were: the study did not report dietary validation results (n = 12), not a study (n = 3) or not conducted in an adult population (n = 1). Table 1 summarizes study details including number of participants and anthropometry, dietary assessment methods used and DLW reporting period. Across the 59 included studies there was a total of 6,298 adults. The majority of studies were conducted in free-living settings with one conducted in a military population (78), one in clinical population group with short bowel syndrome (36), one in obese pregnant women (60) and one in wrestlers (71). The mean number of participants per study was 107 (ranging from 6 to 1075) with the age of participants ranging from 18 to 96 years.
Table 1. Characteristics of studies identified in a systematic review of the validity of dietary assessment methods used in adults (≥18 years) when compared with the method of doubly labeled water (DLW).
The majority of studies were conducted in the United States of America (n = 25) (5, 25, 27, 28, 31–33, 38, 41, 42, 44, 50, 54, 55, 58–60, 63, 67, 72, 74, 75, 79, 80) in adults of Caucasian ethnicity. Ten studies included participants from a range of ethnicities including; African American (n = 6) (25, 41, 50, 54, 55, 73), Native American (n = 1) (41) Hispanic (n = 3) (31, 41, 59), Asian (n = 6) (31, 39, 41, 50, 54, 73), Swedish (n = 1) (61), Nordic (n = 1) (61), Maori (n = 1) or unspecified (n = 5) (31, 50, 54, 59, 67). The majority of studies included both male and female participants (n = 26), with 23 studies having female participants only and four studies with male participants only (47, 68, 71, 78). Two studies did not report the sex of participants (48, 55). The majority of studies (>70%) measured body weight pre- and post-study, 13 studies measured participant body weight at baseline only (25, 28, 29, 33, 34, 37, 49, 51, 53, 66, 67, 70, 77), and body weight was unclear or not reported for five studies (38, 46, 60, 74, 75). The majority of studies reported minor weight change, while the degree of weight change was not statistically significant in 22 studies.
Quality Appraisal
Forty-three of the 59 studies were evaluated as having a positive study quality, with 16 rated as being of neutral quality (Table 2). The main reasons for being rated as neutral quality were a lack of detail in describing the intervention/therapeutic regimens/exposure factors and/or procedures or comparators (n = 9) (25, 35, 38, 46, 49, 50, 54, 64, 71), statistical analyses not adequately described (n = 6) (35, 38, 48, 49, 59, 71), possible bias in participant selection (n = 5) (38, 64, 71, 74, 79), possible bias due to funding and sponsorship (n = 5) (25, 42, 43, 47, 50), conclusion not supported by results or lack of description of limitations (n = 3) (27, 54, 74).
Table 2. Quality assessment of included studies.
Study Design
The reporting period for DLW measurement of TEE ranged from 7 to 22 days (Supplementary Material) 24 h. Five studies collected additional saliva samples for DLW purposes (31, 33, 42, 45, 56) and two also collected blood samples (5, 64).
A total of five different dietary assessment methodologies were used across 59 studies. The most commonly used dietary assessment method was a food record (FR) (n = 36), 12 of which were weighed food records (WFR) (26, 27, 29, 30, 33, 41, 51, 56, 69, 71, 72, 76). The range of recording days were 2 and 16 days with the majority (n = 12) of studies had a reporting period of 7 days. The next most frequently used method were 24 h recalls (n = 24) with the multi-pass method (MPR) used in 13 studies with recall days ranging from two to seven. Seven of the MPR studies had a reporting period of 2 days and an additional six studies reported for 3 days. Of the studies that used a 24 h recall approach (n = 24), the range was from two (42) to 14 recalls (41). A total of 18 studies clearly described that they used non-consecutive days for recalls (5, 25, 27, 31, 38, 39, 42, 44, 49, 50, 52, 53, 59, 66, 67, 72, 74, 79, 82).
The next most used method was the FFQ (n = 21) (5, 24, 25, 27, 31, 34, 37, 38, 48–50, 54, 61, 62, 66, 72–74, 76, 77, 81) with a reporting period ranging from 1 month to 1 year, with the most frequent reporting period being 1 year (n = 8) (5, 24, 25, 48, 50, 66, 72, 77). For studies that utilized the diet history method (n = 5) (26, 30, 35, 40, 70) the reporting period was 1 month in two studies (40, 70), 1 year in one study (30), and the other two studies did not specify the reporting period (26, 35). One study utilized a short dietary questionnaire (SDQ) with a reporting period of 3 months (77). Twenty seven studies used one method of dietary assessment, 25 studies utilized two dietary methods within the same study and an additional seven studies (27, 30, 31, 63, 72, 73, 81) used at least three dietary assessment methods.
Eighteen studies used a technology component within a dietary assessment method to estimate EI, with food records the most common (n = 10) (43, 45, 55, 58, 60, 65, 67–69, 71), followed by 24 h recalls (n = 3) (25, 31, 48) and FFQs (n = 3) (34, 39, 57). The technology components included a wearable camera (n = 4) (39, 65, 68, 71), digital photos, food photography (n = 4) (55, 60, 67, 69), computer/web assisted recalls (n = 4) (25, 31, 43, 48) and a handheld personal digital assistant (PDA) (n = 1) (58). An additional two studies (29, 45) did not use a technology based method as defined in this review but the dietary assessment method was recorded onto a cassette which was then transcribed. Two studies directly compared a traditional dietary assessment method against one with a technology component. Study results are reported in Table 3.
Table 3. Results and outcomes of studies included in a systematic review of the validity of dietary assessment methods used in adults (≥18 years) when compared with the method of doubly labeled water (DLW)A.
Outcomes by Dietary Assessment Category
Food Record
Of the studies that reported the accuracy of food records at the group level, the majority of studies (n = 19) found significant under-reporting of EI, by 11 to 41% (26, 35, 42, 43, 46, 53, 54, 61, 65, 73, 78, 80) with over-reporting found in only one study by 8% (64). Three studies found no significant difference between absolute EI estimated by food record and TEE measured by DLW (31, 47, 58).
Six studies using food records reported outcomes by sex (26, 29, 41, 53, 76, 83), with three studies (26, 29, 53) reporting no significant difference between sexes while one study each for males (76) and females (83) identified as having a lower degree of misreporting. One study (41) found that females under-reported while males slightly over-reported.
Two additional studies reported a negative correlation (35, 46) between EI reporting accuracy and BMI while no association with BMI was reported in two studies (56, 72). Two studies found that individuals with overweight and obesity were more likely to under-report compared to normal weight individuals (54, 80), although only one study reported this difference to be statistically significant (p = 0.032) (80).
Food Record with technology component
Technology was applied to the food record method most commonly using a digital camera (n = 4) (45, 67, 68, 71), a mobile phone (image based) (n = 3) (55, 60, 69), a wearable camera (n = 1) (65), the Internet (n = 1) (43), and a PDA (n = 1) (58). Of the studies that used a digital camera, three studies reported under-reporting of 6, 17, and 24%, respectively (45, 68, 71) while one study found no significant difference between EI and TEE (67). However, those with overweight or obesity were more likely to over-report EI. Image based methods using a smart phone to estimate EI were under-reported compared to DLW between 20 and 37% (54) and in one study where a wearable camera was used in addition to a food record compared with food record alone, the use of the wearable camera reduced level of under-reporting from 34 to 30% (65).
24-Hour Recall
EI was found to be under-reported by 8–30% (5, 25, 38, 39, 44, 50, 53, 59, 66, 73, 79) across seven studies that evaluated EI reporting by sex. Females tended to under-report more than males in all studies (5, 38, 39, 50, 53, 59, 66, 74). Two studies found a relationship between EI reporting accuracy and weight status, with greater EI under-reporting when expressed as a percentage by overweight/obese adults than normal weight adults (50, 73). One study found EI was over-reported in a clinical group of individuals with short bowel syndrome (36).
24 h MPR with technology component
Technology was mostly added to 24 h recalls through use of a web-based system to assist in standardizing the multiple-pass approach (25, 31, 63). In one study, the 24 MPR method was compared with the same method but with the addition of a wearable camera (39). While both methods were found to under-report EI in comparison with DLW, the camera-based method had a lower degree of under-reporting (13 and 7% for females and 17 and 9% for males for the 24 MPR and 24 MPR with camera, respectively) (39). The camera used in this study was a wearable camera worn around the neck with movement, heat, and light sensors.
Studies Using Multiple Methods
Seven studies utilized and reported outcomes of EI mis-reporting using three different dietary methods in one study. The combination of dietary assessment methods most often used were a 24 h recall, FFQ and food records (n = 5) (27, 31, 63, 72, 73). Three studies reported that under-reporting was lowest for the MPR method (31, 63, 73), while one reported that food record was lowest (72) and one reported that FFQ was lowest (27).
Food Frequency Questionnaire
Significant under-reporting of EI was found at the group level in all studies using an FFQ when compared to the DLW method. EI under-reporting ranged from 4.6 to 42% (5, 24, 25, 27, 31, 34, 37, 38, 48, 50, 54, 61, 62, 66, 72–74, 76, 77). One study showed no significant difference between reported EI and TEE on average when using an adapted version of FFQ from a validated FFQ among low income women in Brazil, however, at the individual level significant misreporting remained (49).
Three studies compared the validity of different FFQs (i.e., Block FFQ vs. National Cancer Institute's Diet History Questionnaire (DHQ) (72) and a full vs. brief FFQ i.e., Meal-Q vs. MiniMeal-Q (28, 34, 72, 77). No significant difference in validity was found between the Block FFQ and DHQ, with both having similar, significant EI under-reporting, by ~27% in 20 female adults (72). The other study found significant (P < 0.001) under-reporting of 30 and 36% by both Meal-Q and MiniMeal-Q, respectively. The difference between EI estimated by Meal-Q and MiniMeal-Q was found to be significant (P < 0.001) (34). In the study by Sawaya et al. (72), both FFQs were also found to under-report EI in young females.
Sex differences in EI for FFQs were reported in seven studies (5, 37, 38, 50, 62, 66, 76). Three studies reported males misreported to a lesser extent when compared to females (50, 66, 76), two studies reported females misreported to a lesser extent (37, 62), while two studies reported similar amounts or no significant differences (5, 38).
One study using an FFQ identified that individuals with obesity under-reported to a greater extent than their non-obese counterparts (50). Another study indicated that the difference between the EI from a FFQ and the DLW method were significantly correlated with BMI (r = 0.50) (48). One study used an FFQ, known as the Short Dietary Questionnaire (SDQ), and identified that EI was significantly (P < 0.001) under-reported by ~26%, and that females with overweight/obesity under-reported more than normal weight females (77).
Diet History
Four out of the five studies found EI was under-reported by 1.3–47% (26, 30, 35, 70). One study found females under-report to a greater extent than males by 47 and 1.3% respectively (26).
Discussion
The aim of the current review was to evaluate the validity of self-reported dietary assessment methods used to estimate EI of adults in comparison to TEE measured by the DLW method. A total of 59 studies were included, which utilized a number of dietary assessment methods, of which food records were the most commonly used method (n = 36). The main finding from the review is that EI was underestimated for the majority of dietary assessment methods, in the range of 11–41% for food records, 1.3–47% for diet histories and 4.6–42% for FFQs. The method with lowest total amount and lowest level of variation was found to be 24 h recalls, with underestimations of EI ranging between of 8–30%. This variation could be attributed to recall bias, length of reporting period and use of visual aids to estimate portion size.
Methods utilizing a technology component are relatively new compared to traditional methods. They are often more appropriate for some population groups when compared to more traditional methods, such as individuals with language barriers (84). They can also help assist in reducing reliance on respondents' memory and with estimating portion size by capturing intakes in real time via images and/or on audio recordings (85). The current review included 15 studies that used a technology component, with only two studies making direct comparisons with traditional methods. The Handheld PDA and the Remote Food Photography Method (RFPM), both categorized as food records, were found to have a lower degree of misreporting, however, these technologies were only supported by one study each (55, 58). For many studies in the current review, the technology component was primarily utilized in the collection phase (31, 34, 39, 43), however, it was unclear in many studies. To date, research estimating EI using wearable devices has been limited to small samples sizes, a limited variety of foods and controlled environments (8, 86). Objective measurement of intake in larger sample sizes and free-living individuals is required to determine the performance of technology based methods, including those that utilize sensors or wearable devices (7).
The current review also identified sex-differences in the validity of EI, with females having a greater tendency than males to misreport EI when using MPR (38, 39, 53), diet history (26) and FFQ. However, for food records and FFQ the differences by sex on self-reported EI were inconsistent (37, 76). In study populations of adults with overweight or obesity, under-reporting of EI was identified to a greater degree compared to adults with normal weight when comparing MPR (50, 73), diet history and food record to TEE using DLW. These results could be reflective of a range of reasons including: difficulty to capture dietary intake using the aforementioned methods in this population group such as differences in portion size or frequency of consumption, as well as dieting practices in these individuals, which has been reported previously (87).
In this systematic review, 32 studies used the method of triads (i.e., 2+ measures of diet + DLW) to evaluate the validity of dietary assessment methods (e.g., FFQ, 24h recall, DLW). Nine of these studies used a technology assisted method (25, 31, 39, 48, 57, 65, 67, 69, 71). The method of triads is a statistical approach occasionally used in dietary assessment research (88–90). This method began to be utilized for validation of dietary assessment methods in the twentieth century and involves three separate methods to measure dietary intake. These could include a primary method and a reference method and a biomarker (90). The method assumes the linearity between the three measurements and the true intake and independence between the three measurement errors. There are several limitations and systemic errors known to affect this approach including the occurrence of correlation coefficients >1 or negative coefficients which limits the application (90).
Interestingly, FFQ was the most common method used in the included validation studies (n = 12) (5, 25, 38, 48–50, 54, 61, 66, 74, 76, 77). Similar to other methods, FFQs significantly underestimated EI and its reliability is low due to degree of variation in underestimation across studies with under-reporting ranging from 4.6 to 42%. This may be driven by variation within the FFQ method itself, such as length of reporting period and number of foods and beverages on the questionnaire. Despite this, other dietary assessments, including diet history, FR, WFR, 24 h recall, 24 h MPR and SDQ also underestimated EI. Investigating ways to improve accuracy of estimations of EI are needed and technology-based methods may help to better capture portion size and reduce participant burden (84).
The limitations of using self-reported EI from dietary assessment methods have been previously reported (6, 91). This includes the timeframe of DLW measurements do not necessarily overlap with the period of time covering EIs measurement. If the total EI of participants were atypical during DLW measurement period, the degree of the under or overestimation would be greater than usual. It should also be acknowledged that TEE measured by DLW is not always equal or nearly equal to energy intake in non-weight stable individuals (92, 93). True mis-reporting of EI may have occurred in the included studies. A lack of agreement between methods may be the result of reporter bias or reactivity which occurs when individuals change their dietary behavior due to greater awareness of the measurement of their dietary intake. Reactivity may stem from an individual's desire to reduce burden by simplifying the reporting process (e.g., consuming single foods rather than combination foods) or to comply with socially desirable norms (i.e., to appear to have a healthy diet by reporting intake as per recommended in dietary guidelines).
Conclusion
The majority of dietary assessment methods included in the current review were found to significantly under-estimate EI when compared to TEE measured using the DLW technique. The degree of under-reporting was highly variable across all methods, however, 24 h recalls were associated with a lower degree of mis-reporting and less variation in degree of under-reporting compared to other dietary assessment methods.
Author Contributions
TB, MR, and CC designed the review. YH completed a large proportion of the work as part of her final year honors project. All authors contributed to all stages of the title screening, data extraction, and critical review of the manuscript.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fendo.2019.00850/full#supplementary-material
References
1. Afshin A, Forouzanfar MH, Reitsma MB, Sur P, Estep K, Lee A, et al. Health Effects of Overweight and Obesity in 195 Countries over 25 Years. N Engl J Med. (2017) 377:13–27. doi: 10.1056/NEJMoa1614362
2. Forouzanfar M, Alexander L, Anderson HR, Bachman VF, Biryukov S, Brauer M, et al. Global, regional, and national comparative risk assessment of 79 behavioural, environmental and occupational, and metabolic risks or clusters of risks in 188 countries, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. (2015) 386:2287–323. doi: 10.1016/S0140-6736(15)00128-2
3. Livingstone MB, Robson P, Wallace J. Issues in dietary assessment in children and adolescents. British Journal of Nutrition. (2004) 92:S213–22. doi: 10.1079/BJN20041169
4. Burrows T, Martin R, Collins C. A systematic review of the validity of dietary assessment methods in children when compared with the method of doubly labelled water. J Am Diet Assoc. (2010) 110:1501–10. doi: 10.1016/j.jada.2010.07.008
5. Subar AF, Kipnis V, Troiano RP, Midthune D, Schoeller DA, Bingham S, et al. Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN Study. Am J Epidemiol. (2003) 158:1–13. doi: 10.1093/aje/kwg092
6. Freedman LS, Commins JM, Moler JE, Willett W, Tinker LF, Subar AF, et al. Pooled results from 5 validation studies of dietary self-report instruments using recovery biomarkers for potassium and sodium intake. Am J Epidemiol. (2015) 181:473–87. doi: 10.1093/aje/kwu325
7. Rollo M, Williams RL, Burrows TL, Kirkpatrick SI, Bucher T, Collins C. What are they really eating? A review on new approaches to dietary intake assessment and validation. Curr Nutr Rep. (2016) 5:307–14. doi: 10.1007/s13668-016-0182-6
8. Vu T, Lin F, Alshurafa N, Xu W. Wearable food intake monitoring technologies: a comprehensive review. Computers. (2017) 6:4. doi: 10.3390/computers6010004
9. Gemming L, Utter J, Mhurchu CN. Image-assisted dietary assessment: a systematic review of the evidence. J Acad Nutr Diet. (2015) 115:64–77. doi: 10.1016/j.jand.2014.09.015
10. Harrison GG, Galal OM, Ibrahim N, Khorshid A, Stormer A, Leslie J, et al. Underreporting of food intake by dietary recall is not universal: a comparison of data from egyptian and american women. J Nutr. (2000) 130:2049–54. doi: 10.1093/jn/130.8.2049
11. Johansson G, Wikman A, Ahrén AM, Hallmans G, Johansson I. Underreporting of energy intake in repeated 24-hour recalls related to gender, age, weight status, day of interview, educational level, reported food intake, smoking habits and area of living. Public Health Nutr. (2001) 4:919–27. doi: 10.1079/PHN2001124
12. Gersovitz M, Madden J, Smiciklas-Wright H. Validity of the 24-hr. dietary recall and seven-day record for group comparisons. J Am Diet Assoc. (1978) 73:48–55.
13. Thompson F, Subar A. Nutrition in the Prevention and Treatment of Disease. 3rd ed. Bethesda, MD: Academic Press (2013).
14. Frobisher S, Maxwell M. The estimation of food portion sizes: a comparison between using descriptions of portion sizes and a photographic food atlas by children and adults. J Hum Nutr Diet. (2003) 16:181–8. doi: 10.1046/j.1365-277X.2003.00434.x
15. Thompson FE, Subar AF, Loria CM, Reedy JL, Baranowski T. Need for technological innovation in dietary assessment. J Am Diet Assoc. (2010) 110:48–51. doi: 10.1016/j.jada.2009.10.008
16. Burrows T, Collins CE, Adam M, Duncanson K, Rollo M. Dietary assessment of shared plate eating: a missing link. Nutrients. (2019) 11:789. doi: 10.3390/nu11040789
17. Trabulsi J, Schoeller DA. Evaluation of dietary assessment instruments against doubly labeled water, a biomarker of habitual energy intake. Am J Physiol Endocrinol Metab. (2001) 281:E891–9. doi: 10.1152/ajpendo.2001.281.5.E891
18. Black AE, Cole TJ. Biased over- or under-reporting is characteristic of individuals whether over time or by different assessment methods. J Am Diet Assoc. (2001) 101:70–80. doi: 10.1016/S0002-8223(01)00018-9
19. Black AE, Cole TJ. Within- and between-subject variation in energy expenditure measured by the doubly-labelled water technique: implications for validating reported dietary energy intake. Eur J Clin Nutr. (2000) 54:386–94. doi: 10.1038/sj.ejcn.1600970
20. Hill R, Davies P. The validity of self-reported energy intake as determined using the doubly labelled water technique. Br J Nutr. (2001) 85:415–30. doi: 10.1079/BJN2000281
21. Livingstone B, Black AE. Markers of the validity of reported energy intake. J Nutr. (2003) 133:895S–920. doi: 10.1093/jn/133.3.895S
22. American Dietetic Association. Evidence Analysis Manual: Steps in the ADA Evidence Analysis Process. Chicago, IL: Academy of Nutrition and Dietetics (2016).
23. Thompson FE, Kirkpatrick SI, Krebs-Smith SM, Reedy J, Schap TE, Wilson MM, et al. The National Cancer Institute's Dietary Assessment Primer: a resource for diet research. J Acad Nutr Diet. (2015) 115:1986–95. doi: 10.1016/j.jand.2015.08.016
24. Andersen LF, Tomten H, Haggarty P, Lovo A, Hustvedt BE. Validation of energy intake estimated from a food frequency questionnaire: a doubly labelled water study. Eur J Clin Nutr. (2003) 57:279–84. doi: 10.1038/sj.ejcn.1601519
25. Arab L, Tseng CH, Ang A, Jardack P. Validity of a multipass, web-based, 24-hour self-administered recall for assessment of total energy intake in blacks and whites. Am J Epidemiol. (2011) 174:1256–65. doi: 10.1093/aje/kwr224
26. Barnard JA, Tapsell LC, Davies PS, Brenninger VL, Storlien LH. Relationship of high energy expenditure and variation in dietary intake with reporting accuracy on 7 day food records and diet histories in a group of healthy adult volunteers. Eur J Clin Nutr. (2002) 56:358–67. doi: 10.1038/sj.ejcn.1601341
27. Bathalon GP, Tucker KL, Hays NP, Vinken AG, Greenberg AS, McCrory MA, et al. Psychological measures of eating behavior and the accuracy of 3 common dietary assessment methods in healthy postmenopausal women. Am J Clin Nutr. (2000) 71:739–45. doi: 10.1093/ajcn/71.3.739
28. Beasley JM, Jung M, Tasevska N, Wong WW, Siega-Riz AM, Sotres-Alvarez D, et al. Biomarker-predicted sugars intake compared with self-reported measures in US Hispanics/Latinos: results from the HCHS/SOL SOLNAS study. Public Health Nutr. (2016) 19:3256–64. doi: 10.1017/S1368980016001580
29. Black AE, Bingham SA, Johansson G, Coward WA. Validation of dietary intakes of protein and energy against 24 hour urinary N and DLW energy expenditure in middle-aged women, retired men and post-obese subjects: comparisons with validation against presumed energy requirements. Eur J Clin Nutr. (1997) 51:405–13. doi: 10.1038/sj.ejcn.1600425
30. Black AE, Welch AA, Bingham SA. Validation of dietary intakes measured by diet history against 24 h urinary nitrogen excretion and energy expenditure measured by the doubly-labelled water method in middle-aged women. Br J Nutr. (2000) 83:341–54. doi: 10.1017/S0007114500000441
31. Blanton CA, Moshfegh AJ, Baer DJ, Kretsch MJ. The USDA automated multiple-pass method accurately estimates group total energy and nutrient intake. J Nutr. (2006) 136:2594–9. doi: 10.1093/jn/136.10.2594
32. Boushey CJ, Spoden M, Delp EJ, Zhu F, Bosch M, Ahmad Z, et al. Reported energy intake accuracy compared to doubly labeled water and usability of the mobile food record among community dwelling adults. Nutrients. (2017) 9:22. doi: 10.3390/nu9030312
33. Champagne CM, Bray GA, Kurtz AA, Monteiro JB, Tucker E, Volaufova J, et al. Energy intake and energy expenditure: a controlled study comparing dietitians and non-dietitians. J Am Diet Assoc. (2002) 102:1428–32. doi: 10.1016/S0002-8223(02)90316-0
34. Christensen SE, Moller E, Bonn SE, Ploner A, Wright A, Sjolander A, et al. Two new meal- and web-based interactive food frequency questionnaires: validation of energy and macronutrient intake. J Med Internet Res. (2013) 15:e109. doi: 10.2196/jmir.2458
35. Farooqi N, Slinde F, Haglin L, Sandstrom T. Assessment of energy intake in women with chronic obstructive pulmonary disease: a doubly labeled water method study. J Nutr Health Aging. (2015) 19:518–24. doi: 10.1007/s12603-014-0575-4
36. Fassini PG, Das SK, Pfrimer K, Suen VMM, Sérgio Marchini J, Ferriolli E. Energy intake in short bowel syndrome: assessment by 24-h dietary recalls compared with the doubly labelled water method. Br J Nutr. (2018) 119:196–201. doi: 10.1017/S0007114517003373
37. Ferriolli E, Pfrimer K, Moriguti JC, Lima NK, Moriguti EK, Formighieri PF, et al. Under-reporting of food intake is frequent among Brazilian free-living older persons: a doubly labelled water study. Rapid Commun Mass Spectrom. (2010) 24:506–10. doi: 10.1002/rcm.4333
38. Freedman LS, Midthune D, Carroll RJ, Krebs-Smith S, Subar AF, Troiano RP, et al. Adjustments to improve the estimation of usual dietary intake distributions in the population. J Nutr. (2004) 134:1836–43. doi: 10.1093/jn/134.7.1836
39. Gemming L, Rush E, Maddison R, Doherty A, Gant N, Utter J, et al. Wearable cameras can reduce dietary under-reporting: doubly labelled water validation of a camera-assisted 24 h recall. Br J Nutr. (2015) 113:284–91. doi: 10.1017/S0007114514003602
40. Hagfors L, Westerterp K, Skoldstam L, Johansson G. Validity of reported energy expenditure and reported intake of energy, protein, sodium and potassium in rheumatoid arthritis patients in a dietary intervention study. Eur J Clin Nutr. (2005) 59:238–45. doi: 10.1038/sj.ejcn.1602064
41. Hise ME, Sullivan DK, Jacobsen DJ, Johnson SL, Donnelly JE. Validation of energy intake measurements determined from observer-recorded food records and recall methods compared with the doubly labeled water method in overweight and obese individuals. Am J Clin Nutr. (2002) 75:263–7. doi: 10.1093/ajcn/75.2.263
42. Howat PM, Mohan R, Champagne C, Monlezun C, Wozniak P, Bray GA. Validity and reliability of reported dietary intake data. J Am Diet Assoc. (1994) 94:169–73. doi: 10.1016/0002-8223(94)90242-9
43. Hutchesson MJ, Truby H, Callister R, Morgan PJ, Davies PS, Collins CE. Can a web-based food record accurately assess energy intake in overweight and obese women? A pilot study. J Hum Nutr Diet. (2013) 26 (Suppl. 1):140–4. doi: 10.1111/jhn.12094
44. Johnson RK, Soultanakis RP, Matthews DE. Literacy and body fatness are associated with underreporting of energy intake in US low-income women using the multiple-pass 24-hour recall: a doubly labeled water study. J Am Diet Assoc. (1998) 98:1136–40. doi: 10.1016/S0002-8223(98)00263-6
45. Kaczkowski CH, Jones PJ, Feng J, Bayley HS. Four-day multimedia diet records underestimate energy needs in middle-aged and elderly women as determined by doubly-labeled water. J Nutr. (2000) 130:802–5. doi: 10.1093/jn/130.4.802
46. Koebnick C, Wagner K, Thielecke F, Dieter G, Hohne A, Franke A, et al. An easy-to-use semiquantitative food record validated for energy intake by using doubly labelled water technique. Eur J Clin Nutr. (2005) 59:989–95. doi: 10.1038/sj.ejcn.1602200
47. Koehler K, Braun H, De Marees M, Fusch G, Fusch C, Mester J, et al. Parallel assessment of nutrition and activity in athletes: validation against doubly labelled water, 24-h urea excretion, and indirect calorimetry. J Sports Sci. (2010) 28:1435–49. doi: 10.1080/02640414.2010.513482
48. Kroke A, Klipstein-Grobusch K, Voss S, Moseneder J, Thielecke F, Noack R, et al. Validation of a self-administered food-frequency questionnaire administered in the European Prospective Investigation into Cancer and Nutrition (EPIC) Study: comparison of energy, protein, and macronutrient intakes estimated with the doubly labeled water, urinary nitrogen, and repeated 24-h dietary recall methods. Am J Clin Nutr. (1999) 70:439–47. doi: 10.1093/ajcn/70.4.439
49. Lins IL, Bueno NB, Grotti Clemente AP, Pfrimer K, Sawaya AL, de Menezes Toledo Florencio TM. Energy intake in socially vulnerable women living in Brazil: assessment of the accuracy of two methods of dietary intake recording using doubly labeled water. J Acad Nutr Diet. (2016) 116:1560–7. doi: 10.1016/j.jand.2016.02.023
50. Lissner L, Troiano RP, Midthune D, Heitmann BL, Kipnis V, Subar AF, et al. OPEN about obesity: recovery biomarkers, dietary reporting errors and BMI. Int J Obes. (2007) 31:956–61. doi: 10.1038/sj.ijo.0803527
51. Livingstone MB, Prentice AM, Strain JJ, Coward WA, Black AE, Barker ME, et al. Accuracy of weighed dietary records in studies of diet and health. BMJ. (1990) 300:708–12. doi: 10.1136/bmj.300.6726.708
52. Lof M, Forsum E. Validation of energy intake by dietary recall against different methods to assess energy expenditure. J Hum Nutr Diet. (2004) 17:471–80. doi: 10.1111/j.1365-277X.2004.00554.x
53. Lopes TS, Luiz RR, Hoffman DJ, Ferriolli E, Pfrimer K, Moura AS, et al. Misreport of energy intake assessed with food records and 24-h recalls compared with total energy expenditure estimated with DLW. Eur J Clin Nutr. (2016) 70:1259–64. doi: 10.1038/ejcn.2016.85
54. Mahabir S, Baer DJ, Giffen C, Subar A, Campbell W, Hartman TJ, et al. Calorie intake misreporting by diet record and food frequency questionnaire compared to doubly labeled water among postmenopausal women. Eur J Clin Nutr. (2006) 60:561–5. doi: 10.1038/sj.ejcn.1602359
55. Martin CK, Correa JB, Han H, Allen HR, Rood JC, Champagne CM, et al. Validity of the Remote Food Photography Method (RFPM) for estimating energy and nutrient intake in near real-time. Obesity. (2012) 20:891–9. doi: 10.1038/oby.2011.344
56. Martin LJ, Su W, Jones PJ, Lockwood GA, Tritchler DL, Boyd NF. Comparison of energy intakes determined by food records and doubly labeled water in women participating in a dietary-intervention trial. Am J Clin Nutr. (1996) 63:483–90. doi: 10.1093/ajcn/63.4.483
57. Medin AC, Carlsen MH, Hambly C, Speakman JR, Strohmaier S, Andersen LF. The validity of a web-based FFQ assessed by doubly labelled water and multiple 24-h recalls. Br J Nutr. (2017) 118:1106–17. doi: 10.1017/S0007114517003178
58. McClung HL, Sigrist LD, Smith TJ, Karl JP, Rood JC, Young AJ, et al. Monitoring energy intake: a hand-held personal digital assistant provides accuracy comparable to written records. J Am Diet Assoc. (2009) 109:1241–5. doi: 10.1016/j.jada.2009.04.015
59. Moshfegh AJ, Rhodes DG, Baer DJ, Murayi T, Clemens JC, Rumpler WV, et al. The US department of agriculture automated multiple-pass method reduces bias in the collection of energy intakes. Am J Clin Nutr. (2008) 88:324–32. doi: 10.1093/ajcn/88.2.324
60. Most J, Vallo PM, Altazan AD, Gilmore LA, Sutton EF, Cain LE, et al. Food photography is not an accurate measure of energy intake in obese, pregnant women. J Nutr. (2018) 148:658–63. doi: 10.1093/jn/nxy009
61. Nybacka S, Berteus Forslund H, Wirfalt E, Larsson I, Ericson U, Warensjo Lemming E, et al. Comparison of a web-based food record tool and a food-frequency questionnaire and objective validation using the doubly labelled water technique in a Swedish middle-aged population. J Nutr Sci. (2016) 5:e39. doi: 10.1017/jns.2016.29
62. Okubo H, Sasaki S, Rafamantanantsoa HH, Ishikawa-Takata K, Okazaki H, Tabata I. Validation of self-reported energy intake by a self-administered diet history questionnaire using the doubly labeled water method in 140 Japanese adults. Eur J Clin Nutr. (2008) 62:1343–50. doi: 10.1038/sj.ejcn.1602858
63. Park Y, Dodd K, Kipnis V, Thompson F, Potischman N, Schoeller D, et al. Comparison of self-reported dietary intakes from the automated self-administered 24-h recall, 4-d food records, and food-frequency questionnaires against recovery biomarkers. Am J Clin Nutr. (2018) 107:80–93. doi: 10.1093/ajcn/nqx002
64. Persson M, Elmstahl S, Westerterp KR. Validation of a dietary record routine in geriatric patients using doubly labelled water. Eur J Clin Nutr. (2000) 54:789–96. doi: 10.1038/sj.ejcn.1601092
65. Pettitt C, Liu J, Kwasnicki RM, Yang GZ, Preston T, Frost G. A pilot study to determine whether using a lightweight, wearable micro-camera improves dietary assessment accuracy and offers information on macronutrients and eating rate. Br J Nutr. (2016) 115:160–7. doi: 10.1017/S0007114515004262
66. Pfrimer K, Vilela M, Resende CM, Scagliusi FB, Marchini JS, Lima NK, et al. Under-reporting of food intake and body fatness in independent older people: a doubly labelled water study. Age Ageing. (2015) 44:103–8. doi: 10.1093/ageing/afu142
67. Ptomey LT, Willis EA, Honas JJ, Mayo MS, Washburn RA, Herrmann SD, et al. Validity of energy intake estimated by digital photography plus recall in overweight and obese young adults. J Acad Nutr Diet. (2015) 115:1392–9. doi: 10.1016/j.jand.2015.05.006
68. Rafamantanantsoa HH, Ebine N, Yoshioka M, Yoshitake Y, Tanaka H, Saitoh S. The effectiveness of three-day dietary records with advanced photo system camera for measuring energy intake in Japanese men as determined by doubly labeled water technique. J Clin Biochem Nutr. (2003) 33:33–8. doi: 10.3164/jcbn.33.33
69. Rollo ME, Ash S, Lyons-Wall P, Russell AW. Evaluation of a mobile phone image-based dietary assessment method in adults with Type 2 diabetes. Nutrients. (2015) 7:4897–910. doi: 10.3390/nu7064897
70. Rothenberg E, Bosaeus I, Lernfelt B, Landahl S, Steen B. Energy intake and expenditure: validation of a diet history by heart rate monitoring, activity diary and doubly labeled water. Eur J Clin Nutr. (1998) 52:832–8. doi: 10.1038/sj.ejcn.1600655
71. Sagayama H, Kondo E, Shiose K, Yamada Y, Motonaga K, Ouchi S, et al. Energy requirement assessment and water turnover in Japanese college wrestlers using the doubly labeled water method. J Nutr Sci Vitaminol. (2017) 63:141–7. doi: 10.3177/jnsv.63.141
72. Sawaya AL, Tucker K, Tsay R, Willett W, Saltzman E, Dallal GE, et al. Evaluation of four methods for determining energy intake in young and older women: comparison with doubly labeled water measurements of total energy expenditure. Am J Clin Nutr. (1996) 63:491–9. doi: 10.1093/ajcn/63.4.491
73. Scagliusi FB, Ferriolli E, Pfrimer K, Laureano C, Cunha CS, Gualano B, et al. Underreporting of energy intake in Brazilian women varies according to dietary assessment: a cross-sectional study using doubly labeled water. J Am Diet Assoc. (2008) 108:2031–40. doi: 10.1016/j.jada.2008.09.012
74. Schulz LO, Harper IT, Smith CJ, Kriska AM, Ravussin E. Energy intake and physical activity in Pima Indians: comparison with energy expenditure measured by doubly-labeled water. Obes Res. (1994) 2:541–8. doi: 10.1002/j.1550-8528.1994.tb00103.x
75. Shook RP, Hand GA, O'Connor DP, Thomas DM, Hurley TG, Hébert JR, et al. Energy intake derived from an energy balance equation, validated activity monitors, and dual X-ray absorptiometry can provide acceptable caloric intake data among young adults. J Nutr. (2018) 148:490–6. doi: 10.1093/jn/nxx029
76. Svendsen M, Tonstad S. Accuracy of food intake reporting in obese subjects with metabolic risk factors. Br J Nutr. (2006) 95:640–9. doi: 10.1079/BJN20051662
77. Svensson A, Renstrom F, Bluck L, Lissner L, Franks PW, Larsson C. Dietary intake assessment in women with different weight and pregnancy status using a short questionnaire. Public Health Nutr. (2014) 17:1939–48. doi: 10.1017/S1368980013003042
78. Tanskanen M, Uusitalo AL, Hakkinen K, Nissila J, Santtila M, Westerterp KR, et al. Aerobic fitness, energy balance, and body mass index are associated with training load assessed by activity energy expenditure. Scand J Med Sci Sports. (2009) 19:871–8. doi: 10.1111/j.1600-0838.2008.00857.x
79. Tran KM, Johnson RK, Soultanakis RP, Matthews DE. In-person vs telephone-administered multiple-pass 24-hour recalls in women: validation with doubly labeled water. J Am Diet Assoc. (2000) 100:777–83. doi: 10.1016/S0002-8223(00)00227-3
80. Weber JL, Reid PM, Greaves KA, DeLany JP, Stanford VA, Going SB, et al. Validity of self-reported energy intake in lean and obese young women, using two nutrient databases, compared with total energy expenditure assessed by doubly labeled water. Eur J Clin Nutr. (2001) 55:940–50. doi: 10.1038/sj.ejcn.1601249
81. Yuan C, Spiegelman D, Rimm EB, Rosner BA, Stampfer MJ, Barnett JB, et al. Relative validity of nutrient intakes assessed by questionnaire, 24-hour recalls, and diet records as compared with urinary recovery and plasma concentration biomarkers: findings for women. Am J Epidemiol. (2018) 187:1051–63. doi: 10.1093/aje/kwx328
82. Scagliusi FB, Ferriolli E, Pfrimer K, Laureano C, Cunha CSF, Gualano B, et al. Under-reporting of energy intake is more prevalent in a healthy dietary pattern cluster. Br J Nutr. (2008) 100:1060–8. doi: 10.1017/S0007114508971300
83. Livingstone MB, Prentice AM, Coward WA, Ceesay SM, Strain JJ, McKenna PG, et al. Simultaneous measurement of free-living energy expenditure by the doubly labeled water method and heart-rate monitoring. Am J Clin Nutr. (1990) 52:59–65. doi: 10.1093/ajcn/52.1.59
84. Coates JC, Colaiezzi BA, Bell W, Charrondiere UR, Leclercq C. Overcoming dietary assessment challenges in low-income countries: technological solutions proposed by the International Dietary Data Expansion (INDDEX) Project. Nutrients. (2017) 9:E289. doi: 10.3390/nu9030289
85. Ashman AM, Collins CE, Brown LJ, Rae KM, Rollo ME. Validation of a smartphone image-based dietary assessment method for pregnant women. Nutrients. (2017) 9:73. doi: 10.3390/nu9010073
86. Hassannejad H, Matrella G, Ciampolini P, De Munari I, Mordonini M, Cagnoni S. Automatic diet monitoring: a review of computer vision and wearable sensor-based methods. Int J Food Sci Nutr. (2017) 68:656–70. doi: 10.1080/09637486.2017.1283683
87. Ballard-Barbash R, Graubard I, Krebs-Smith S, Schatzkin A, Thompson F. Contribution of dieting to the inverse association between energy intake and body mass index. Eur J Clin Nutr. (1996) 50:98–106.
88. Kaaks R. Biochemical markers as additional measurements in studies of the accuracy of dietary questionnaire measures : conceptual issues. Am J Clin Nutr. (1997) 65(Suppl.):1232s–9. doi: 10.1093/ajcn/65.4.1232S
89. Kabagambe E, Allan D, Siles X, Spiegelman D, Campos H. Application of the method of triads to evaluate the performance of food frequency questionnaires and biomarkers as indicators of long term dietary intake. Am J Epidemiol. (2001) 154:1126–35. doi: 10.1093/aje/154.12.1126
90. Yokota RTdC, Miyazaki ES, Ito MK. Applying the triads method in the validation of dietary intake using biomarkers. Cad Saúde Pública. (2010) 26:2027–37. doi: 10.1590/S0102-311X2010001100004
91. Subar A, Freedman L, Tooze J, Kirkpatrick S, Boushey C, Neuhouser M, et al. Addressing current criticism regarding the value of self-report dietary data. J Nutr. (2015) 145:2639–45. doi: 10.3945/jn.115.219634
92. Schoeller DA. Limitations in the assessment of dietary energy intake by self-report. Metabolism. (1995) 44(Suppl. 2):18–22. doi: 10.1016/0026-0495(95)90204-X
Keywords: dietary assessment, doubly labeled water, validation, adults, energy intake
Citation: Burrows TL, Ho YY, Rollo ME and Collins CE (2019) Validity of Dietary Assessment Methods When Compared to the Method of Doubly Labeled Water: A Systematic Review in Adults. Front. Endocrinol. 10:850. doi: 10.3389/fendo.2019.00850
Received: 05 September 2019; Accepted: 21 November 2019;
Published: 17 December 2019.
Edited by:
Negar Naderpoor, Monash University, AustraliaReviewed by:
William Wong, Baylor College of Medicine, United StatesJulio Sergio Marchini, São Paulo University, Brazil
Copyright © 2019 Burrows, Ho, Rollo and Collins. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Tracy L. Burrows, tracy.burrows@newcastle.edu.au