Structural Validation of a French Food Frequency Questionnaire of 94 Items

Background: Food frequency questionnaires (FFQs) are used to estimate the usual food and nutrient intakes over a period of time. Such estimates can suffer from measurement errors, either due to bias induced by respondent’s answers or to errors induced by the structure of the questionnaire (e.g., using a limited number of food items and an aggregated food database with average portion sizes). The “structural validation” presented in this study aims to isolate and quantify the impact of the inherent structure of a FFQ on the estimation of food and nutrient intakes, independently of respondent’s perception of the questionnaire.

Methods: A semi-quantitative FFQ (n = 94 items, including 50 items with questions on portion sizes) and an associated aggregated food composition database (named the item-composition database) were developed, based on the self-reported weekly dietary records of 1918 adults (18–79 years-old) in the French Individual and National Dietary Survey 2 (INCA2), and the French CIQUAL 2013 food-composition database of all the foods (n = 1342 foods) declared as consumed in the population. Reference intakes of foods (“REF_FOOD”) and nutrients (“REF_NUT”) were calculated for each adult using the food-composition database and the amounts of foods self-reported in his/her dietary record. Then, answers to the FFQ were simulated for each adult based on his/her self-reported dietary record. “FFQ_FOOD” and “FFQ_NUT” intakes were estimated using the simulated answers and the item-composition database. Measurement errors (in %), spearman correlations and cross-classification were used to compare “REF_FOOD” with “FFQ_FOOD” and “REF_NUT” with “FFQ_NUT”.

Results: Compared to “REF_NUT,” “FFQ_NUT” total quantity and total energy intake were underestimated on average by 198 g/day and 666 kJ/day, respectively. “FFQ_FOOD” intakes were well estimated for starches, underestimated for most of the subgroups, and overestimated for some subgroups, in particular vegetables. Underestimation were mainly due to the use of portion sizes, leading to an underestimation of most of nutrients, except free sugars which were overestimated.

Conclusion: The “structural validation” by simulating answers to a FFQ based on a reference dietary survey is innovative and pragmatic and allows quantifying the error induced by the simplification of the method of collection.

Introduction

In nutritional intervention studies, reliable dietary data are essential to avoid misleading conclusions. Food intakes are generally assessed using short-term instruments (i.e., 24 h-recall, dietary records) or long-term instruments such as food frequency questionnaires (FFQs) (1). A FFQ is a retrospective instrument where the respondent has to report the frequency of consumption on a predefined list of food items (an item could be an individual food or an aggregation of same kind of foods), during a more or less long period of time (1 month to 1 year) (1). A FFQ may be qualitative when it does not collect information on the quantity consumed, semi-quantitative if it contains standard portions sizes or quantitative when it includes questions about portion sizes consumed. FFQs require a short time to complete, are less burdensome for respondents, and are less expensive to setup than short-term instruments (2, 3). FFQs were shown to be valuable tools to estimate dietary changes in nutritional intervention studies (4) and they remain one of the most common dietary measurement tools in dietary intervention studies to capture the usual food and nutrient intakes over a period of time (5–7).

However, the accuracy of the nutritional intakes estimated using FFQs has been fully in debate (8–12). Dietary estimation relies on a difficult cognitive task for the respondent, to remember the frequency and, when necessary, portion sizes of foods consumed in the past. Moreover, food and nutrient estimates could be biased by errors inherent to the FFQ (3, 13, 14). Nutrient intakes from a FFQ are estimated by multiplying the frequency of consumption of each food item by its respective portion size and nutritional content. Food consumption is collected for a closed list of items and, therefore, cannot fully capture in detail an individual’s diet. Quantification of the food consumed does not account for the variability of portion sizes across eating occasions or between specific foods related to the same item. Each item composing the FFQ is an aggregation of different foods with different nutrient contents. The nutrient content of an item is, in general, a weighted mean nutritional composition of all foods represented in the item, taking into account the amount consumed, to reflect the foods actually eaten in the population of interest (15). Therefore, the error in the estimation of food and nutrient intakes could either be due to imprecision in the nutritional content of food items or to imprecision in the portion sizes used. Because of known systematic errors in a FFQ and because the “true” intake is unknown, the validation of a FFQ is necessary. It is usually done by comparing the estimation of food and nutrient intakes against “a gold standard,” the latter often being a short-term open-ended instrument (16). Such comparison allows identifying the sources and magnitudes of the measurement error but cannot distinguish between the error due to the inherent structure of the FFQ or due to the differential in the respondent perception of the two instruments. This work describes a new method called “structural validation,” which allows isolating and quantifying the impact of the inherent structure of a given questionnaire on the estimation of food and nutrient intakes, independently of respondent’s perception of the questionnaire.

Materials and Methods

Survey Design and Estimation of “REF_FOOD” and “REF_NUT” Intakes

Food intakes were derived from the French Individual and National Dietary Survey 2 (INCA2) conducted in 2006–2007 by the French Agency for Food, Environmental and Occupational Health Safety, performed on nationally representative samples of children (3–17 years) and adults (18–79 years). This survey was approved by the CNIL [French authority of data protection (“Commission Nationale Informatique et Libertés” No. 2003X727AU)] and the CNIS [French national council for statistical information (“Conseil National de l’Information Statistique”)]. INCA2 remains the most recent version of an available French population-based survey providing dietary data. A detailed survey methodology is available elsewhere (17, 18). This study focused on the adult population (n = 2,624). Individuals who completed the report less than 7 days were excluded as well as under-reporters identified using Black equations (19), leading to a final sample of 1,863 individuals (1,111 women and 752 men).

Individual socioeconomic variables were collected using a self-reported questionnaire and a face-to-face questionnaire. Food intakes were collected using a 7-day record in which each individual reported all foods and beverages consumed at home or outside on seven consecutive days, during three meals and three snacking occasions. “REF_FOOD” intakes were the amount in g/day of each individual food consumed by each individual from their dietary record. The CIQUAL 2013 French food composition database (20) was used to estimate “REF_NUT” intakes.

Development of the FFQ

The FFQ used in this study was designed to assess food and nutrient intakes of adults during the previous month. The list of items was developed by experts, according to the nutritional content of foods and the type of foods (raw or cook, liquid, etc.) using data from the national dietary survey INCA2. Each item of the FFQ is a combination of individual foods (e.g., the item “fatty fish” is the combination of “cooked salmon,” “cooked trout,” “sardine in vegetal oil,” and others) from the list of foods consumed in the INCA2 study. The quantitative French FFQ contains 94 items. Portion sizes are requested for 50 items using units (one egg, two eggs), manufacture’s containers (one can of soft drink, etc.), or household measures (one teaspoon, two teaspoons, etc.). The number of different portion sizes proposed varies across items. For the items “raw vegetables,” “cooked vegetables,” “pasta/rice/semolina,” “whole grains starches,” and “legumes,” the respondent can choose a frequency for each different portion size proposed (1/4 of plate, 1/2 plate, a whole plate) to take into account within-person variation in portion sizes if respondent consumed these foods as a side dish as well as a main dish. For breads, frequencies and portion sizes are requested by moment of consumption. Portion sizes are not requested for 44 items, for which there are no simple units or household measures, or, for which portion sizes varied slightly in the population of interest (e.g., yogurt). Frequencies and portion sizes of beer, wine, and strong alcohol are also requested. An additional question asks whether the respondent adds salt in his/her plate for each meal.

Sex-specific item-composition databases were developed as follows. The nutritional composition of each item was derived from a wider list of corresponding foods from the food composition database associated with the INCA2 survey. The list of foods used to derive the nutrient content of each item was selected, according to the number of foods related to each item, and the frequency of consumption of each food, avoiding to take into account too peculiar and rarely consumed foods. For the items which were related to more than 25 different foods, half of foods related to this item were selected, as being the most frequently consumed foods. For the items related between 8 and 25 foods, 75% of the related foods were taken into account to derive the nutrient content of the item, and 100% for the items which were related to less than 8 foods. The most frequently consumed foods were identified using the percentage of consumers among adults from the INCA2 survey. For each item and sex, the nutritional composition was calculated as a mean weighted by the intake of its related foods by adults from the INCA2 survey. Portion sizes were assigned based on the manufacturer’s weights or household measures for items for which portion sizes were requested. For the others items, a unique sex-specific portion size was assigned as the median quantity eaten daily among French adults in the INCA2 survey.

“FFQ_FOOD” and “FFQ_NUT” Intakes

For each INCA2 individual and each item, “FFQ” frequency was simulated by calculating the number of times an item was declared in his/her INCA2 dietary record. For instance, the frequency of consumption of the item “all-season fresh fruits” for an individual who has declared to have consumed every day an apple, and two times a banana during the week of data collection, was nine times a week.

“FFQ_FOOD” intakes in g/day have been estimated by multiplying the simulated “FFQ” frequencies by portion sizes. Individual portion sizes were used for the 50 items for which portion sizes were requested. The individual portion size was chosen as the closest portion size that an individual could chose in the FFQ, based on his/her own individual median portion size. For instance, an individual for whom 50% of his/her reported intake of eggs was 125 g during the week of interview was attributed a portion size of 120 g in the “eggs” item of the FFQ (twice the weight of a standard egg at 60 g).

“FFQ_NUT” intakes were calculated by multiplying the “FFQ_FOOD” intakes by the sex-specific item-composition database.

“ITEM_NUT” Intakes

In order to assess only the impact of the item-composition database on nutrient intakes, “ITEM_NUT” intakes were calculated for each individual by multiplying the exact amounts consumed of items, estimated from the self-reported dietary record (i.e., sum of the intake of each individual foods related to the item) by the sex-specific item-composition database.

Statistical Analysis

Each food and item were categorized into 8 food groups and 34 food subgroups. Food categorization is presented in Table S1 in Supplementary Material.

For each food group and subgroup, “REF_FOOD” and “FFQ_FOOD” mean intakes (with the exclusion of non-consumers within food groups and subgroups) were estimated and compared, to assess the impact of using portion sizes instead of real quantities, using mixed generalized linear model with repeated measures. Measurement errors were quantified through calculating the variations in absolute values between “FFQ_FOOD” and “REF_FOOD,” expressed in percentage of “REF_FOOD” intakes by food groups and subgroups. A threshold of 5% of variation (in %) was chosen to identify consumers with an underestimation (i.e., variation below −5%) or overestimation (i.e., variation above 5%), for each food group and subgroup. Measurement errors were compared between individuals with over- and underestimation by generalized linear model by food groups and subgroups. The direction of measurement error was visualized for each food group by plotting mean food intake variations (in %) against deciles of “FFQ_FOOD” intakes (excluding non-consumers). Relative agreements between “FFQ_FOOD” and “REF_FOOD” intakes were assessed by food groups and subgroups, using cross-classification into quartiles of food intakes, weighted Kappa coefficients, and Spearman correlation.

“REF_NUT,” “ITEM_NUT,” and “FFQ_NUT” mean daily total energy and macronutrients in % energy, as well as the intakes of water, fiber, docosahexaenoic acid (DHA), eicosapentaenoic acid (EPA), α-linolenic and linoleic acids, 11 vitamins, and 10 minerals (with the exclusion of alcoholic beverages) were estimated. Pairwise comparisons were performed between “REF_NUT,” “ITEM_NUT,” and “FFQ_NUT” intakes using mixed generalized linear model with repeated measures, first to identify the impact on nutrient intakes of using the item-composition database by comparing “REF_NUT” and “ITEM_NUT” intakes and then to identify the impact of the use of average portion sizes by comparing “ITEM_NUT” with “FFQ_NUT” intakes. Measurement errors in nutrient intakes were assessed for each nutrient by calculating mean variations in absolute values between “ITEM_NUT” or “FFQ_NUT” and “REF_NUT” intakes, in percentage of “REF_NUT” intakes. For each nutrient, measurement errors were compared, using generalized linear models, between individuals with over- and underestimations identified as described earlier. Variations (in %) between “FFQ_NUT” and “REF_NUT” energy and macronutrient intakes were plotted against deciles of “FFQ_NUT” intakes. The relative agreements between “REF_NUT” and “FFQ_NUT” intakes were assessed using cross-classification and weighted Kappa coefficients. Weighted Kappa coefficients (one per nutrient) were plotted in descending order. For each nutrient, the association between “REF_NUT” intakes and the two other estimates was also tested using Spearman correlation coefficient.

All analyses were adjusted on “REF_NUT” total energy intakes, age and gender, and were performed with SAS Version 9.4. An α level of 1% was used for all statistical tests.

Results

Comparison between “REF_FOOD” and “FFQ_FOOD” Intakes

On average, “FFQ_FOOD” total food intake was lower than “REF_FOOD” total food intake (−198 g/day), with a measurement error of 10.7% (Table 1). “FFQ_FOOD” total food intake was considered as underestimated for 56.8% of consumers and overestimated for 12.7%, with measurement errors of 15.0% and 11.1%, respectively.

TABLE 1

Table 1. “REF_FOOD” and “FFQ_FOOD” mean intakes,^a and measurement errors between “REF_FOOD” and “FFQ_FOOD” intakes among all consumers and among consumers with over- or underestimation.

On average, “FFQ_FOOD” mean intakes of all food groups were significantly different from “REF_FOOD” mean intakes, except for fruits and vegetables. Measurement error ranged from 9.6% for starches to 22.3% for sweet products and water and other beverages food groups (Table 1). “FFQ_FOOD” tended to overestimate fruits and vegetables intakes (48.9% of individuals with an overestimation and 32.2% with an underestimation) and underestimate the other food group intakes, except starches for which the percentage of individuals with an error measurement lower than 5% was the highest (34.7%) and the variation (in %) was close to 0 for almost all deciles of “FFQ_FOOD” intakes (p for trend not significant) (Figure 1). Water and other beverages food groups had the highest percentage of individuals with underestimation (71.9%), with a mean measurement error of 27.8%.

FIGURE 1

Figure 1. Variations between “FFQ_FOOD” and “REF_FOOD” intakes (in %) by deciles of “FFQ_FOOD” intakes^a, among consumers of each food group^b (A–H). ^aA negative variation indicates an underestimation, the symbol ϕ means a significant p. for linear trend and the symbol * means a variation significantly different from 0. ^bThe maximum y-axis has been set to 200% for fruits and vegetables, dairy products, and sweet products because of extreme values.

At food subgroup level, no significant differences were found between “FFQ_FOOD” and “REF_FOOD” intakes for protein substitutes, milk, water, light drinks, sweet drinks, and cold sauces subgroups (Table 1). Measurement errors between “FFQ_FOOD” and “REF_FOOD” intakes were above 40% for nuts and oilseeds, cereals for breakfast, fish, offal, mixed dishes, milk, ice cream and dairy desserts, sweet drinks, and hot sauces subgroups. However, SD values were high and medians were much lower than the means, indicating that the high measurement errors were steered by the values reached by specific consumers. For salt, yogurt, breads, dairy substitutes, and starches and legumes, measurement error was below 5% for more than 30% of consumers (68.3, 54, 43.8, 41, 38, and 34.7% of consumers, respectively). Vegetables, eggs, vegetal fats, potatoes, sweet drinks, and cold sauces subgroups were mainly overestimated (percentage of consumers with overestimation greater than percentage of consumers with underestimation or with a low measurement error), whereas the remaining food subgroups were mainly underestimated. The highest mean measurement errors among food groups with a high proportion of overestimation were 120% for sweet drinks, followed by eggs (43.1%) and vegetables (36.1%). Among food subgroups with high proportion of underestimation, the highest mean measurement errors were for nuts and oilseeds (43%), followed by hot sauces (40.8%) and fish (37.9%). Average measurement errors were not different between consumers with an overestimation and those with an underestimation for bread, eggs, protein substitutes, yogurt, ice cream and dairy desserts, biscuit and sweets, water, light drink, and fruit juices subgroups.

Spearman correlation coefficients and cross-classification into quartiles between “REF_FOOD” and “FFQ_FOOD” food group and subgroup intakes are presented in Table S2 in Supplementary Material. The lowest Spearman correlation coefficient was for meat/fish/eggs and substitutes and water and other beverages (0.82). The percentage of individuals with an “exact agreement” was above 60% for 28 food subgroups (out of 34) and for most food groups (dairy products, fats and condiments, fruits and vegetables, mixed dishes and sandwiches, and sweet products). The percentage of individuals with “extreme disagreement” was low for all food groups and subgroups with the highest values for fats and condiment food group (4.3%) and hot sauces subgroup (1.4%).

Comparison “REF_NUT,” “ITEM_NUT,” and “FFQ_NUT” Intakes

Impact on Nutrient Intakes Estimates of Using the Item-Composition Database

For all nutrients except free sugars (% energy), “ITEM_NUT” intakes were not significantly different from “REF_NUT” intakes (Table 2). Variations in absolute values between “ITEM_NUT” and “REF_NUT” intakes (in %) ranged from 1.2% for water to 42.5% for EPA. The highest measurement error (above 15%) were found for EPA (42.4%), DHA (39.7%), vitamin A (36.4%), free sugars (in % energy) (33%), vitamin B-12 (27.8%), vitamin D (23.1%), α-linolenic acids (20.3%), vitamin C (18.9%), copper (18.3%), and iodine (16.8%).

TABLE 2

Table 2. “REF_NUT,” “ITEM_NUT,” and “FFQ_NUT” mean daily total energy and nutrient intakes and measurement errors^a between “ITEM_NUT” or “FFQ_NUT” with “REF_NUT” nutrient intakes.

Impact on Nutrient Intakes Estimates of Using Portion Sizes

For total fat (in % energy) and monounsaturated fatty acids (in % energy), “ITEM_NUT” intakes were not significantly different from “FFQ_NUT” intakes (Table 2). For remaining nutrients, “FFQ_NUT” mean intake was always lower than “ITEM_NUT” mean intake, except for carbohydrates (in % energy) and poly-unsaturated fatty acids (in % energy).

Overall Impact on Nutrient Intakes Estimates of the Inherent Structure of the Questionnaire

Mean energy intake was 8699 and 8033 kJ/d (2,075 and 1,917 kcal/day) for “REF_NUT” and “FFQ_NUT,” respectively, leading to an underestimation of 666 kJ/d (158 kcal) (Table 2). “FFQ_NUT” energy intake was underestimated for 55% individuals, with a mean measurement error of 14.7% (Table 3). “FFQ_NUT” energy intake was underestimated whatever the decile of “FFQ_NUT” energy intake, with a negative variation which came closer to 0 with increasing “FFQ_NUT” intakes (p for trend < 0.01) (Figure 2).

TABLE 3

Table 3. Measurement errors^a between “FFQ_NUT” and “REF_NUT” total energy and nutrient intakes among individuals with over- and underestimation (N total = 1,863).

FIGURE 2

Figure 2. Variations between “FFQ_NUT” and “REF_NUT” total energy intake (A) and macronutrients (in % energy) (B–D) by decile of “FFQ_NUT” intakes^a. ^aA negative variation indicates an underestimation, the symbol ϕ means a significant p. for linear trend and the symbol * means a variation significantly different from 0.

For the other nutrients, no significant differences were found between “FFQ_NUT” and “REF_NUT” intakes for carbohydrates (in % energy), total fat (in % energy), and monounsaturated fatty acids (in % of energy). Measurement errors ranged from 5.4% for carbohydrate (in % energy) to 47.3% for DHA, with 15 nutrients with a measurement error above 15%. For carbohydrates (in % energy), total fat (in % energy), saturated fat (in % energy), proteins (in % energy), and monounsaturated fatty acids (in % energy), proportion of individuals with a low measurement error (i.e., below 5%) is lower than percentage of individuals with a higher one (56.4, 52.5, 43.4, 38.4, and 37.8% of consumers, respectively) (Table 3). Figure 2 shows that for carbohydrates and total fats, mean variation (in %) was significantly different from 0 only for higher deciles (and the first decile for total fat), and no difference was noticed for proteins. More individuals were considered to have an overestimation for free sugars (in % energy), poly-unsaturated fatty acids (in % energy), and vitamin A rather an underestimation or low measurement error, whereas a higher percentage of individuals were identified with an underestimation for the other nutrients. The highest measurement error among nutrients which were mostly overestimated was for free sugar (52.4%). Among nutrients with underestimation, the highest measurement errors were for α-linolenic acid (35.6%), DHA (34.4%), and vitamin D (27.4%). Average measurement errors were not different between individuals with an overestimation and those with an underestimation, for total sugars (in % energy), fiber, linoleic acids, iodine, calcium, zinc, iron, vitamins E, niacin, vitamin B-6, and folates. The variations by deciles of “FFQ_NUT” intakes for vitamins and minerals are presented in Figures S1 and S2 in Supplementary Material: they show an overall trend to underestimate micronutrient nutrient intakes, except for vitamin A, vitamin C, and vitamin B-12.

Spearman correlation coefficients between “REF_NUT” and “FFQ_NUT” ranged from 0.66 to 0.90 for vitamin A intake and free sugars (in % energy), respectively (Table S3 in Supplementary Material). The percentage of individuals with an “exact agreement” was above 60% for 18 nutrients, with a minimum at 49.2% for vitamin A. The percentage of individuals with “extreme disagreement” was not above 2.1% (for vitamin A). The highest weighted Kappa coefficients between quartiles of “FFQ_NUT” and “REF_NUT” were observed for vitamin E (0.72), water (0.71), and total sugars (in % energy) and lowest for copper and vitamin B-12 (0.53), selenium and vitamin D (0.52), and vitamin A (0.47) (Figure 3). Twenty-three nutrients had a coefficient considered as “substantial agreement” (between 0.61 and 0.80) and the 13 others had a coefficient considered as “moderate agreement” (between 0.41 and 0.60).

FIGURE 3

Figure 3. Weighted Kappa coefficients between quartiles of REF_NUT and FFQ_NUT intakes by nutrient.

Discussion

This paper describes a new method for validating a FFQ, independently to the bias induced by respondent’s answers. This method was named “structural validation,” because it aims to assess the impact—on food and nutrient intakes estimates—of the inherent structure of a FFQ, especially the impact of using an aggregated food database and of using average and/or standard portion sizes. In this paper, the method was applied to a French medium-length quantitative FFQ. Results indicated an overall good structural validity, although an overall tendency to underestimate most of food groups, subgroups, and nutrient intakes was noticed. Overestimation was observed for certain food groups such as vegetables and sweet drinks, as well as certain nutrients such as free sugars. However, it was noticeable that, for some food groups, intakes were correctly estimated, notably for starches.

Measurement errors can be due to the estimation of food quantities, based on the use of portion sizes associated with each item instead of real and precise amounts. The use of portion sizes was shown to induce an overall underestimation of food intakes compared to “REF_FOOD” intakes, but on average, the magnitude of the underestimation was acceptable. The highest positive variation between “FFQ_FOOD” and “REF_FOOD” subgroups intakes was observed for the vegetables subgroup, with an average variation of +16.7 g/day, and the highest negative variation was for the hot drinks (−125.7 g/day) subgroup. The variation was above ±10 g for only six subgroups. The overall underestimation of total quantity was led by an underestimation of beverages (especially hot drinks), which are known to be difficult to assess, even with an open-ended instrument (21). To improve the accuracy of food and nutrient estimation, individual portion sizes were requested for 50 items in the present FFQ. The choice of using of a quantitative questionnaire or a qualitative questionnaire is a subject of long controversy. Some authors think that asking the respondent to report their own portion sizes does not improve significantly the validity of the questionnaire (22–25), whereas others argue that the individual portion sizes can take into account the inter-individual variability of portion sizes, which could highly differ according to gender and age (22). Nonetheless, taking into account individual portion sizes for certain items seemed to improve the estimation. In our data, the average variation in absolute values between “FFQ_FOOD” and “REF_FOOD” intakes (in %) was 25% among items for which an individual portion size was taken into account, instead of 46% among the others (data not shown).

The use of an aggregated food database can lead to measurement errors due to the dilution of the nutritional information of specific foods explaining between-person variance in nutrient intakes. But, less the aggregation is, longer the questionnaire will be. In a review, Cade et al. found that the number of food items in existing FFQs ranged from 5 to 350. There is currently no consensus about the optimal length of a questionnaire (2). Whereas the accuracy was greater using less aggregation of foods (22, 26), a food list of more than 100 items induced overestimation (6, 27, 28). In this study, the length was closed to the median identified by Cade (median length at 79 items). Without taking into account the respondent’s perception, this study showed that the use of aggregated food items did not impact the estimation of most of the nutrients except free sugars.

The whole impact of the inherent structure of the questionnaire seemed to be acceptable given the validation measurements (mean differences, cross-classification, and correlation coefficients). Yet, the FFQ showed an overall tendency to underestimate food intakes compared to REF_FOOD intakes. Positive and high measurement errors between “FFQ_FOOD” and “REF_FOOD” intakes (i.e., measured by the variation in absolute values, expressed in percentage of “REF_FOOD” intakes) were observed for specific food groups, such as vegetables, nuts and oilseed, milk, sweet drinks, or fishes. After investigation, these results were steered by some individuals who actually declared a very small intake, compared to the average portion size assigned to each item after simulation of FFQ answers. For instance, the individual food “concentrated fruit syrup” from the INCA2 dietary survey was related to the item “sweet beverages.” An overestimation of “sweet beverages” was observed for all individuals who declared “concentrated fruit syrup” in a very small amount, because of assigning a too large portion size. Similarly, some individuals declared a low intake of milk, which was found to be milk added in hot drinks (in a small portion), difficult to take into account into the simulation. This fact will be taken into account in the FFQ by using two independent questions about milk, one about milk as a drink and the other about milk added into the coffee with specific portion sizes. However, validation measures for food intakes (Table S3 in Supplementary Material) were high (high correlation coefficient and high percentage of individuals classified in the same quartile) compared to values found in the literature for French FFQs (6, 29). Regarding nutrient intakes, results indicated also an overall trend to underestimate nutrient intakes, except for some macronutrients expressed in % of energy for which “FFQ_NUT” intakes were higher than “REF_NUT” intake. Even if significant differences were observed for almost all nutrients in pairwise comparisons between nutrient intake estimates and “REF_NUT” intakes, differences between the two estimates were small for most of the nutrients, with respect to “REF_NUT” intakes. Indeed, the large sample size could partly explained a higher sensitivity for the statistical tests. Nevertheless, the numerous measures of validity for nutrient intakes (weighted Kappa values, Spearman correlation coefficient, and cross-classification into quartiles of nutrient intakes presented in Table S3 in Supplementary Material) showed a good performance of ranking individual based on their nutritional intakes, with good correlation coefficients (ranged from 0.67 to 1) compared with the range 0.5–0.8 proposed by Willett et al. (30). Finally, the comparison of results obtained in this paper with other validation studies is difficult because of different statistical methods and because we did not consider respondents’ bias. Nevertheless, this study pointed out that the inherent structure of the questionnaire (use of average portion sizes and of an aggregated food database) induced on average an underestimation of nutritional and food intakes.

This study presents limitations. First, methods based on self-reporting of food consumption, as the self-reported weekly dietary record used in this study, are prone to multiple bias, but, they are still widely used in epidemiological research. The design of a FFQ must be chosen according to the target population, which determines the source of the data to use, to build the questionnaire. In this study, we used the most recent national food consumption survey (INCA2), which dates from 2006. Some items should be added in the future to represent more closely today’s consumption patterns. The 94 items were based on an aggregation of the individual foods declared as consumed by adults in the INCA2 survey. The choice of the food aggregation was done by expertise, but another way to aggregate the foods could lead to different food and nutrient intake estimates. Another limitation of this study is the different time frame over which food intake was assessed by the reference method (dietary record on seven consecutive days) and will be assessed by the FFQ (aimed at assessing food intakes for the previous month). FFQs are typically designed to measure long-term food intake, conversely to dietary records which measure short-term intake. But, it could be assumed that the dietary data collection on day 7 is representative of the habitual consumption pattern of the individual. Moreover, the data used for the construction and validation were from the same study. It would be necessary to apply this approach with another open-ended food consumption survey. Despite the fact that the French FFQ showed an acceptable validity against the dietary survey used as a reference, the error estimated in this study did not represent the overall error when the FFQ will be used in practice with individuals (31). Indeed, self-report of food intakes could be biased by social desirability, which usually tend to overestimate intakes of foods considered “healthy” and underestimate less “healthy” foods (32–35). Validation of the individual’s perception of this questionnaire should be investigated further in the future.

The novelty of this study was to explore the impact on food and nutrient intakes estimates of the inherent structure of a FFQ. Usually, validation is done by comparing food and nutrient intakes estimated from the FFQ and a reference method (i.e., 24 h-recall or dietary record), completed by the same respondent under the same period. The reference method is supposed to quantify the same measure (i.e., food intakes) and should be independent of the FFQ, to avoid an interdependence of errors (36). However, measurement errors in validity measurement can also be attributable to the reference method. A better option would be to validate the nutrient intakes questionnaire estimates against biomarkers, but it is often too expensive and difficult to implement. This new “structural validation” method provides a first insight into validity of a FFQ by decomposing the measurement error according to its source (the use of an aggregated food database or the estimation of food quantity), independently of respondent induced bias and possible correlation errors. To date, the few FFQs which have been developed in French are either not recent (37, 38), longer than we needed (6, 29, 38, 39), or designed for a specific population or food group (40–42) and often not freely accessible. The FFQ presented in this study will be a useful tool to assess the usual food and nutrient intakes of French individuals. In a near future, a web-based version of this questionnaire will be used for French adults. Such tool will enable to assess easily the habitual diet of individuals to be used in ongoing studies focusing on monitoring usual behavior. Web-based versions have shown similar accuracy when compared to printed version (5, 43). They were also recognized to facilitate the collection of data (immediate storage), to reduce errors via automatic control, and are less burdensome for respondent than paper versions (44, 45). Moreover, the questionnaire can be personalized (adding complementary questions or removing one) according to the previous responses of the respondents.

Conclusion

The “structural validation” presented in this study demonstrated that, without taking into account the respondent induced bias, the FFQ of 94 items designed for French adults provides reliable estimates of food and nutrient intakes for average consumers but with an overall trend to underestimated food and nutrient intakes. Further work would be required to validate the reproducibility and understanding of the questionnaire by respondents.

Author Contributions

RG contributed to the design of the study, performed the statistical analysis, interpreted the results, wrote the manuscript, and was responsible for the final content of the manuscript; MM and FV contributed to the design of the study, help to interpret the results, and to produce the final draft of the manuscript; ND helped to produce the final draft of the manuscript; and all authors: read and approved the final version of the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer LD and handling editor declared their shared affiliation.

Funding

RG was financially supported by MS-Nutrition and ANRT (Agence Nationale de la Recherche et de la Technology).

Supplementary Material

The Supplementary Material for this article can be found online at http://www.frontiersin.org/articles/10.3389/fnut.2017.00062/full#supplementary-material.

Abbreviations

FFQ, Food frequency questionnaire; INCA2, French Individual and National Dietary Survey 2; ANSES, French Agency for Food, Environmental and Occupational Health Safety; CNIL, French authority of data protection (“Commission Nationale Informatique et Libertés”); CNIS, French national council for statistical information (“Conseil National de l’Information Statistique”); EPA, eicosapentaenoic acid; DHA, docosahexaenoic acid.

References

1. Thompson FE, Byers T. Dietary assessment resource manual. J Nutr (1994) 124:2245S–317S.

Google Scholar

2. Cade JE, Burley VJ, Warm DL, Thompson RL, Margetts BM. Food-frequency questionnaires: a review of their design, validation and utilisation. Nutr Res Rev (2004) 17(1):5–22. doi:10.1079/NRR200370

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Hulshof K, Ovesen L, Amorim JA. Selection of methodology to assess food intake. Eur J Clin Nutr (2002) 56:25–32. doi:10.1038/sj.ejcn.1601426

CrossRef Full Text | Google Scholar

4. Thomson CA, Giuliano A, Rock CL, Ritenbaugh CK, Flatt SW, Faerber S, et al. Measuring dietary change in a diet intervention trial: comparing food frequency questionnaire and dietary recalls. Am J Epidemiol (2003) 157(8):754–62. doi:10.1093/aje/kwg025

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Forster H, Fallaize R, Gallagher C, O’Donovan CB, Woolhead C, Walsh MC, et al. Online dietary intake estimation: the Food4Me food frequency questionnaire. J Med Internet Res (2014) 16(6):e150. doi:10.2196/jmir.3105

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Verger EO, Armstrong P, Nielsen T, Chakaroun R, Aron-Wisnewsky J, Gøbel RJ, et al. Dietary assessment in the MetaCardis Study: development and relative validity of an online food frequency questionnaire. J Acad Nutr Diet (2017) 117:878–88. doi:10.1016/j.jand.2016.10.030

CrossRef Full Text | Google Scholar

7. Apovian CM, Murphy MC, Cullum-Dugan D, Lin P, Gilbert KM, Coffman G, et al. Validation of a web-based dietary questionnaire designed for the DASH (dietary approaches to stop hypertension) diet: the DASH online questionnaire. Public Health Nutr (2010) 13(5):615–22. doi:10.1017/S1368980009991996

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Willett WC, Hu FB. Not the time to abandon the food frequency questionnaire: point. Cancer Epidemiol Biomarkers Prev (2006) 15(10):1757–8. doi:10.1158/1055-9965.EPI-06-0388

CrossRef Full Text | Google Scholar

9. Kristal AR, Potter JD. Not the time to abandon the food frequency questionnaire: counterpoint. Cancer Epidemiol Biomarkers Prev (2006) 15(10):1759–60. doi:10.1158/1055-9965.EPI-06-0727

CrossRef Full Text | Google Scholar

10. Kristal AR, Peters U, Potter JD. Is it time to abandon the food frequency questionnaire? Cancer Epidemiol Biomarkers Prev (2005) 14(12):2826–8. doi:10.1158/1055-9965.EPI-12-ED1

CrossRef Full Text | Google Scholar

11. Freedman LS, Schatzkin A, Thiebaut ACM, Potischman N, Subar AF, Thompson FE, et al. Abandon neither the food frequency questionnaire nor the dietary fat-breast cancer hypothesis. Cancer Epidemiol Biomarkers Prev (2007) 16(6):1321–2. doi:10.1158/1055-9965.EPI-07-0179

CrossRef Full Text | Google Scholar

12. Kelemen LE. Food frequency questionnaires: not irrelevant yet. Cancer Epidemiol Biomarkers Prev (2006) 15(5):1054. doi:10.1158/1055-9965.EPI-06-0077

CrossRef Full Text | Google Scholar

13. Thompson FE, Subar AF. Nutrition in the Prevention and Treatment of Disease. Elsevier (2013). p. 5–46. Available from: http://www.sciencedirect.com/science/article/pii/B9780123918840000019

Google Scholar

14. Subar AF. Developing dietary assessment tools. J Am Diet Assoc (2004) 104(5):769–70. doi:10.1016/j.jada.2004.02.007

CrossRef Full Text | Google Scholar

15. Pennington JA, Stumbo PJ, Murphy SP, McNutt SW, Eldridge L, McCabe-Sellers BJ, et al. Food composition data: the foundation of dietetic practice and research. J Am Diet Assoc (2007) 107(12):2105–13. doi:10.1016/j.jada.2007.09.004

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires – a review. Public Health Nutr (2002) 5(4):567–87. doi:10.1079/PHN2001318

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Lioret S, Dubuisson C, Dufour A, Touvier M, Calamassi-Tran G, Maire B, et al. Trends in food intake in French children from 1999 to 2007: results from the INCA (étude Individuelle Nationale des Consommations Alimentaires) dietary surveys. Br J Nutr (2010) 103(4):585–601. doi:10.1017/S0007114509992078

CrossRef Full Text | Google Scholar

18. Dubuisson C, Lioret S, Touvier M, Dufour A, Calamassi-Tran G, Volatier J-L, et al. Trends in food and nutritional intakes of French adults from 1999 to 2007: results from the INCA surveys. Br J Nutr (2010) 103(7):1035–48. doi:10.1017/S0007114509992625

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Black AE. The sensitivity and specificity of the Goldberg cut-off for EI:BMR for identifying diet reports of poor validity. Eur J Clin Nutr (2000) 54(5):395–404. doi:10.1038/sj.ejcn.1600971

PubMed Abstract | CrossRef Full Text | Google Scholar

20. The French Information Center on Food Quality. French Food Composition Table Ciqual. (2013). Available from: https://pro.anses.fr/tableciqual/

Google Scholar

21. Gandy J. Water intake: validity of population assessment and recommendations. Eur J Nutr (2015) 54(Suppl 2):11–6. doi:10.1007/s00394-015-0944-8

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Molag ML, de Vries JHM, Ocké MC, Dagnelie PC, van den Brandt PA, Jansen MCJF, et al. Design characteristics of food frequency questionnaires in relation to their validity. Am J Epidemiol (2007) 166(12):1468–78. doi:10.1093/aje/kwm236

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Nöthlings U, Hoffman K, Bergmann MM, Boeing H. Portion size adds limited information on variance in food intake of participants in the EPIC-Potsdam study 1. J Nutr (2003) 133(2):510–5.

Google Scholar

24. Nöthlings U, Hoffman K, Bergmann MM, Boeing H. Fitting portion sizes in a self-administered food frequency questionnaire. J Nutr (2007) 137:2781–6.

Google Scholar

25. Schlundt DG, Buchowski MS, Hargreaves MK, Hankin JH, Signorello LB, Blot WJ. Separate estimates of portion size were not essential for energy and nutrient estimation: results from the Southern Community Cohort food-frequency questionnaire pilot study. Public Health Nutr (2007) 10(3):245–51. doi:10.1017/S1368980007258574

CrossRef Full Text | Google Scholar

26. Thompson FE, Subar AF, Brown CC, Smith AF, Sharbaugh CO, Jobe JB, et al. Cognitive research enhances accuracy of food frequency questionnaire reports: results of an experimental validation study. J Am Diet Assoc (2002) 102(2):212–25. doi:10.1016/S0002-8223(02)90050-7

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Fernández-Ballart JD, Piñol JL, Zazpe I, Corella D, Carrasco P, Toledo E, et al. Relative validity of a semi-quantitative food-frequency questionnaire in an elderly Mediterranean population of Spain. Br J Nutr (2010) 103(12):1808–16. doi:10.1017/S0007114509993837

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Krebs-Smith SM, Heimendinger J, Subar AF, Patterson BH, Pivonka E. Using food frequency questionnaires to estimate fruit and vegetable intake: association between the number of questions and total intakes. J Nutr Educ (1995) 27(2):80–5. doi:10.1016/S0022-3182(12)80346-3

CrossRef Full Text | Google Scholar

29. Kesse-Guyot E, Castetbon K, Touvier M, Hercberg S, Galan P. Relative validity and reproducibility of a food frequency questionnaire designed for French adults. Ann Nutr Metab (2010) 57(3–4):153–62. doi:10.1159/000321680

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Willett WC, Sampson L, Stampfer M, ROSNER B, BAIN C, WITSCHI J, et al. Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol (1985) 122(1):51–65. doi:10.1093/oxfordjournals.aje.a114086

CrossRef Full Text | Google Scholar

31. van Staveren WA, Ocké MC, de vries JHM. Estimation of dietary intake. 10th ed. In: Erdman JW, Macdonald IA, Zeisel SH, editors. Present Knowledge in Nutrition. Oxford, UK: Wiley-Blackwell (2012). p. 1012–26. Available from: http://doi.wiley.com/10.1002/9781119946045.ch59

Google Scholar

32. Schoch AH, Raynor HA. Social desirability, not dietary restraint, is related to accuracy of reported dietary intake of a laboratory meal in females during a 24-hour recall. Eat Behav (2012) 13:78–81. doi:10.1016/j.eatbeh.2011.11.010

CrossRef Full Text | Google Scholar

33. Hebert JR, Hurley TG, Peterson KE, Resnicow K, Thompson FE, Yaroch AL, et al. Social desirability trait influences on self-reported dietary measures among diverse participants in a multicenter multiple risk factor trial. J Nutr (2008) 138:226S–34S.

PubMed Abstract | Google Scholar

34. Fallaize R, Forster H, Macready AL, Walsh MC, Mathers JC, Brennan L, et al. Online dietary intake estimation: reproducibility and validity of the Food4Me food frequency questionnaire against a 4-day weighed food record. J Med Internet Res (2014) 16(8):e190. doi:10.2196/jmir.3355

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Steinemann N, Grize L, Ziesemer K, Kauf P, Probst-Hensch N, Brombach C. Relative validation of a food frequency questionnaire to estimate food intake in an adult population. Food Nutr Res (2017) 61(1):1305193. doi:10.1080/16546628.2017.1305193

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Lombard MJ, Steyn NP, Charlton KE, Senekal M. Application and interpretation of multiple statistical tests to evaluate validity of dietary intake assessment methods. Nutr J (2015) 14:40. doi:10.1186/s12937-015-0027-y

PubMed Abstract | CrossRef Full Text | Google Scholar

37. van Liere MJ, Lucas F, Clavel F, Slimani N, Villeminot S. Relative validity and reproducibility of a French dietary history questionnaire. Int J Epidemiol (1997) 26(Suppl 1):S128–36. doi:10.1093/ije/26.suppl_1.S128

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Bonifacj C, Gerber M, Scali J, Daures JP. Comparison of dietary assessment methods in a southern French population: use of weighed records, estimated-diet records and a food-frequency questionnaire. Eur J Clin Nutr (1997) 51(4):217–31. doi:10.1038/sj.ejcn.1600387

CrossRef Full Text | Google Scholar

39. Deschamps V, de Lauzon-Guillain B, Lafay L, Borys J-M, Charles MA, Romon M. Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr (2009) 63(2):282–91. doi:10.1038/sj.ejcn.1602914

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Shatenstein B, Nadon S, Godin C, Ferland G. Development and validation of a food frequency questionnaire. Can J Diet Pract Res (2005) 66(2):67–75. doi:10.3148/66.2.2005.67

CrossRef Full Text | Google Scholar

41. Ross AB, Pineau N, Kochhar S, Bourgeois A, Beaumont M, Decarli B. Validation of a FFQ for estimating whole-grain cereal food intake. Br J Nutr (2009) 102(11):1547–51. doi:10.1017/S0007114509990845

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Giovannelli J, Dallongeville J, Wagner A, Bongard V, Laillet B, Marecaux N, et al. Validation of a short, qualitative food frequency questionnaire in French adults participating in the MONA LISA-NUT study 2005–2007. J Acad Nutr Diet (2014) 114(4):552–61. doi:10.1016/j.jand.2013.07.002

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Lo Siou G, Csizmadi I, Boucher B, Akawung A, Whelan H, Sharma M, et al. The comparative reliability and feasibility of the past-year Canadian diet history questionnaire II: comparison of the paper and web versions. Nutrients (2017) 9(2):133. doi:10.3390/nu9020133

PubMed Abstract | CrossRef Full Text | Google Scholar

44. van Gelder MMHJ, Bretveld RW, Roeleveld N. Web-based questionnaires: the future in epidemiology? Am J Epidemiol (2010) 172(11):1292–8. doi:10.1093/aje/kwq291

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Ngo J, Engelen A, Molag M, Roesle J, García-Segovia P, Serra-Majem L. A review of the use of information and communication technologies for dietary assessment. Br J Nutr (2009) 101(Suppl):S102–12. doi:10.1017/S0007114509990638

CrossRef Full Text | Google Scholar