Validity and reproducibility of a food frequency questionnaire assessing food group intake in the PERSIAN Cohort Study

Purpose: A semi-quantitative food frequency questionnaire (FFQ) was developed for use in the Prospective Epidemiological Research Studies in IrAN (PERSIAN Cohort), investigating non-communicable disease risk factors. This study aimed to assess the validity and reproducibility of this FFQ, through food group intake.

Methods: Participants, recruited from seven PERSIAN cohort centers, completed the FFQ at the beginning of the study (FFQ1) and at the end (FFQ2), with a 12-month interval in between, during which two 24-h dietary recalls (24 h) were completed each month. Correlation coefficients of the median intake of food groups recorded by the FFQs were compared to those of the 24 h to assess validity, and the two FFQs were compared to assess reproducibility of findings.

Results: Overall, data from 978 participants were included in this validation analysis. Of the 26 food groups assessed, Tea, Sugars, Whole/Refined Grains, and Solid Fats/Oils, had the strongest correlations (0.6–0.79), while Red Meat, Chicken and Eggs showed moderate correlations (0.42–0.59). The weakest correlations observed belonged to Fresh fruit Juice and Other Meats (0.23–0.32). Reproducibility was assessed among those who completed both FFQ1 and FFQ2 (n = 848), revealing moderate to strong correlations in all food groups, ranging from 0.42 in Legumes to 0.72 in both Sugar and Sweetened Drinks.

Conclusion: The PERSIAN Cohort FFQ is appropriate to rank individuals based on food group intake.

Introduction

The Prospective Epidemiological Research Studies in IrAN (PERSIAN Cohort) is the largest cohort study in Iran, aiming to investigate risk factors of common non-communicable diseases (NCDs) in different geographical areas and among various ethnic populations of Iran. Among many questionnaires completed for participants to obtain baseline information on lifestyle, environmental, and social exposures, a semi-quantitative food frequency questionnaire (FFQ) was developed for use in the PERSIAN Cohort Study, to assess diet’s role in NCD development.

Two semi-quantitative FFQs had been previously developed and validated in Iran, but were of limited use in the PERSIAN Cohort Study, because they were validated in specific populations, not best depicting the PERSIAN Cohort population. The FFQ used in the Golestan Cohort Study (GCS) was validated among the Turkmen ethnic population (2% of Iran’s total population), who have specific dietary habits and local food items, while the questionnaire used in the Tehran Lipid and Glucose Study (TLGS), was validated among the capital city’s population, whose dietary habits are again different from those of many smaller cities and rural areas included in the PERSIAN Cohort Study (1, 2). Besides the population differences, both of these FFQs were long, with the GCS FFQ including 150 and the TLGS including 168 food items. Given that multiple questionnaires are completed for each individual who enrolls in the PERSIAN Cohort, a shorter FFQ was desired, to reduce participant fatigue, which can subsequently affect response accuracy. A simplified FFQ, on the other hand, including 48 items was also validated for use in the Isfahan Healthy Heart Program (IHHP). The items in this questionnaire were chosen with a focus on foods affecting cardiovascular diseases and thus, it was not comprehensive enough to be used for assessing diet-NCD relationships in the PERSIAN Cohort population (3).

The aim in the development of the PERSIAN Cohort FFQ was therefore, to develop a comprehensive, yet shorter FFQ for possibility of use in different populations of Iran with varying dietary habits. To assess the validity and reproducibility of this questionnaire, a multi-center study was designed and executed in seven different PERSIAN Cohort centers, in order to better capture the dietary variations of the PERSIAN Cohort participants.

Given that individual nutrients, foods, food groups and dietary patterns can influence disease development, FFQ validation at all levels is recommended (4, 5). In this manuscript, we report the validity and reproducibility of the PERSIAN Cohort FFQ in assessing food group intake.

Materials and methods

We conducted this study, parallel to the pilot phase of the PERSIAN Cohort Study, the methodology and rationale of which have been previously published (6, 7). Briefly, PERSIAN started in 2014 in 18 locations of Iran. Individuals aged 35–70 years were invited to participate and those who agreed, reported to the cohort center on their appointment date, when laboratory tests, anthropometric measurements and interviewer-administered questionnaires were completed, including an FFQ. All participants are currently being followed annually to record the occurrence of common NCDs or death.

Study participants

We chose this study’s participants from those enrolling in the pilot phase of the PERSIAN Cohort study. Our inclusion criteria parallels that of PERSIAN’s, which enrolled men and women of Iranian descent, who were 35–70 years of age, and who resided in the designated cohort areas. The only exclusion included having a physical or psychological disability that hinders participation in the study by interfering with accurate data collection (6).

Given that the pilot phase at different PERSIAN Cohort centers started at various times, this study stretched over approximately three years, from January 2015 to November 2017. During this time, 1,260 individuals who enrolled in the PERSIAN Cohort at the Fasa, Rafsanjan, Azar, Yazd, Ravansar, Zahedan, and Tabari cohort centers (180 from each center), were also invited to participate in this validation study. Of these individuals, 1,097 agreed to participate. Sample collection for the validation study relied on invitations in the main cohort and when the desired sample size was reached at each center, enrollment ceased. These seven cohort centers were chosen in order to include major ethnic populations of Iran as well as geographical areas, with varying lifestyles and eating habits. This study was approved by the ethics committee of the Digestive Diseases Research Institute, Tehran University of Medical Sciences (IR.TUMS.DDRI.REC.1398.001). Written informed consent was obtained from all participants.

FFQ development and completion

The PERSIAN Cohort FFQ was developed by modifying the GCS FFQ, which included 150 single food items, about 90 of which were common foods used throughout Iran and 10, local to Golestan province (1). The remaining items were either variations of the same foods included, or foods neither local to Golestan, nor commonly used elsewhere in Iran. We also evaluated foods included in the TLGS FFQ and finally selected 113 food items categorized in 9 major groups, as the standard FFQ items (2). These items were chosen by nutrition experts, and based on their frequency of use in the Iranian diet, their energy-contribution, as well as access to the items throughout Iran. Local experts at each cohort center were also consulted and if food items not included in the standard items were identified that were either used frequently in that population, or were nutrient and/or calorie-dense, these items were also added to the FFQ for that center only, as local food items. These mostly consisted of local breads, sweets, or few fruits and vegetables and varied between five to ten items per center. In some centers, the interviewers were instructed to add the amount of a specific local item consumed to one of the standard items, if the two items were very close in composition. In many cases however, to limit data collection mistakes, information on the local food items were recorded as separate items and later equated to the standard items by nutritionists based on their major ingredients.

We chose to include food items in this FFQ, rather than dishes, because while many dishes in the Persian cuisine are well-known and made throughout Iran, the ingredients used in those dishes sometimes differs from one area to another. Also, Persian dishes are very ingredient-rich and individual variations and preferences put into recipes also make a dish-based FFQ that is reflective of all the variations, difficult to design and analyze.

Our FFQ was designed as a semi-quantitative, interviewer-administered questionnaire, enquiring about individuals’ usual intake of each food item over the year prior to the interview date. Participants reported their daily, weekly, monthly or yearly use of each item, as well as the portion consumed each time, based on portion sizes pertaining to each item. Actual dish, cups and utensils, as well as several portion size models were used for a more precise portion size estimation. In addition, a 64-picture album including standard portions for selected items was used whenever needed (8). All tools were centrally purchased and distributed to cohort centers to ensure consistency and all interviewers were trained by the same person, using the same study protocol.

Given that all individuals aged 35–70 years were invited to participate in the cohort study, most participants enrolled along with and on the same day as other family members (spouses or parents). While all procedures were completed for each individual separately, the FFQ of spouses were completed at the same time and by the same interviewer, since women predominantly cook in the Iranian culture and information regarding many ingredients used in cooking is not well-known by men. Women reported the frequency of use and overall amount of these items they typically use in cooking, and then each person’s share was determined and recorded in their questionnaire. If individuals did not enroll with their spouses or were single, information on these items was asked from pertinent family members, by phone.

Reference method and data collection timeline

The 24-h dietary recall (24 h) method was used as the reference method for FFQ validation. These recalls were also interviewer-administered and were completed in person. The United States Department of Agriculture (USDA) multiple-pass method was used to complete the 24 h (9). The same tools used to record FFQ portion sizes were also used when obtaining the 24 h and again pertinent family members were consulted in the completion of the 24 h, if the participant was not involved in cooking.

Upon entering the validation study, an FFQ was completed for each participant (FFQ1). Then, 24 h were completed twice monthly for 12 months, followed by another FFQ at the end of the study (FFQ2). To assess validity, data obtained from the 24 h were compared to those recorded by the FFQs and the two FFQs were compared in the reproducibility assessment of the study.

Missing data

Missing data was not observed in the FFQs, since all questionnaires were completed on a smart electronic questionnaire that alarmed missing values upon completion. Missing an entire 24 h or FFQ2 did on the other hand occur, as sometimes participants did not meet their scheduled appointment to complete the questionnaires or were no longer interested to cooperate. When a visit to the cohort center was not possible, interviewers were instructed to complete the 24 h by phone to limit missing 24 h. Although two 24 h were to be obtained from each participant each month, when it was not possible to obtain two, having one recall per month was also considered adequate. However, participants with either more than 12 recalls missing, or those missing all 24 h in one season, were excluded from the analysis.

As for FFQ2, participants were invited to the cohort center three times to complete the questionnaire at the end of the study, and afterwards were considered missing and were excluded from any analysis requiring data from FFQ2.

Data processing

Frequency data obtained for each food item on the FFQs were converted to daily intake, then multiplied by the weight (in grams) of the portion size consumed each time to obtain the grams consumed from each food item per day (grams/day). For the 24 h, the grams/day was calculated by adding the amount of each food item consumed in all 24 h, then dividing the sum by the number of 24 h obtained.

The USDA Food Composition Tables (USDA-FCT) were used to obtain daily energy intake of food items (10). Standard, non-branded foods in the USDA-FCT, checked by four nutritionists to be the best equivalent of the Iranian food items in regards to ingredients and macronutrients were chosen for energy estimations. For several foods native to Iran, not included in the USDA-FCT, the weighted average of major ingredients was used to equate that food item. The local food items were also, as previously stated, equated to the standard FFQ items, based on their major ingredients.

For the purpose of the food group analysis, food items were first grouped based on the USDA MyPlate groups, then, further narrowed based on major and important ingredients. Total food group intake was obtained by adding the grams/day consumption of all food items within each group.

Statistical analysis

Kolmogorov–Smirnov test and Q-Q normal plot were used to test the normality assumption for all food groups. Since the distribution of most food groups were skewed, medians with the first and third quartiles [interquartile range (IQR)] were used to describe the food group intakes in the questionnaires examined. Crude (C), energy-adjusted (EA) and de-attenuated energy-adjusted (DEA) Spearman’s rank correlation coefficients (SCC) were obtained to assess the validity of FFQ1 and FFQ2 relative to the 24 h. EA-SCC were calculated using the nutrient density approach (11). The DEA-SCC, which was corrected for intra-person variability in the 24 h, was calculated through the following formula:

$energy-adjustedSCC\times \left[1+\lambda /n\right]1/2$

where n is the number of 24 h replicates (24 in this study), and λ is the ratio of within-person and between-person variance (4). Food groups were categorized into tertiles to examine agreement between the questionnaires. Agreement was described as the proportion of individuals classified in the same, adjacent and extreme categories.

To assess reproducibility, crude and energy-adjusted Intraclass Correlation Coefficients (C-ICC and EA-ICC, respectively) and their 95% confidence intervals (CI) were calculated between FFQ1 and FFQ2. Cross-classification analysis was also conducted. All statistical analyses were performed using the statistical software STATA 12 (StataCorp, College Station, TX, United States). p < 0.05 was considered as statistically significant for all tests.

Results

A total of 1,097 individuals entered this study; 76.5% completed more than 20 recalls (53.9% completed all 24), while 10.8% completed less than 12 and were excluded from all analysis, leaving 978 individuals as the final study population (Figure 1). Age, gender and BMI of those excluded was not significantly different from the remaining participants (data not shown). Baseline characteristics of participants are shown in Table 1. Mean age was 46.6 ± 8.25 years and 58% were female. While over 90% of individuals had some formal education, 42.8% had only primary education or were illiterate.

FIGURE 1

Figure 1. Participant recruitment and retention in the PERSIAN Cohort FFQ validation study.

TABLE 1

Table 1. Baseline characteristics of the study population.

Comparing the median intake of food groups across the three questionnaires (Table 2), FFQ1 recorded higher intake in 14 of the 26 food groups while the 24 h recorded greater intake in 5 groups compared to the FFQs. The median intake of Fresh Fruit Juice, Oils, Salty Snacks and Salt were the same in FFQ1 and 2, while Pizza and Olives had zero median intake in all questionnaires.

TABLE 2

Table 2. Median (IQR) intake of food groups in each questionnaire (g/day).

Validity assessment

C-SCC, EA-SCC and DEA-SCC are shown in Table 3 comparing FFQ1, FFQ2 and mean of FFQ1 and 2 (FFQ1&2), vs. the 24 h. C-SCC and DEA-SCC ranged from 0.23 to 0.70, and 0.22 to 0.70, respectively, in comparing FFQ1 and 24 h, from 0.25 to 0.74, and 0.27 to 0.76 in FFQ2 vs. 24 h, and finally from 0.28 to 0.79 and 0.30 to 0.79 when the mean of FFQ1&2 and 24 h were compared, respectively.

TABLE 3

Table 3. Crude, energy-adjusted and de-attenuated energy-adjusted Spearman correlation coefficients comparing FFQ1, FFQ2 and mean of FFQ1 and FFQ2 with the 24 h.

At least seven groups had strong DEA-SCC (>0.6) in all three comparisons, including Refined Grains, Solid Fats and Oils. The Fruits, Vegetables, Cheese, and Dairy groups had moderate correlations (0.3–0.6) when FFQ1 was compared to the 24 h, and strong correlations with FFQ2 and FFQ1&2. Red Meat, Chicken and Eggs showed moderate correlations in all three comparisons. Legumes had a weak DEA-SCC (<0.3) when FFQ 1 and the 24 h were compared, but this group had moderate correlations in the other comparisons made. Fresh fruit juice and Other Meats showed weak correlations in two of the three comparisons.

Gender-specific SSC comparing the various questionnaires were also calculated (Supplementary Tables 1, 2). Correlation values observed in men and women for the various food groups, as well as patterns of food groups having strong and weak SCC, were similar to those observed for the entire study population.

On average, 54.3% [median (IQR): 50.2% (46.7 to 53.6%)], 51.6% [median (IQR): 50.7% (47.2 to 55.3%)], and 51.7% [median (IQR): 51.6% (47.3 to 54.3)] of participants were correctly classified into the same tertiles for all food groups in FFQ1 vs. 24 h, FFQ2 vs. 24 h, and mean of FFQ1&2 vs. 24 h, respectively (Table 4). The highest mismatch occurred for Pizza, in all comparisons [28.3% (FFQ1), 27.3% (FFQ2), 26.7% (FFQ1&2)], then for Fresh Fruit Juice, Processed Meat, Olives and Salty Snacks, with about one in four individuals being misclassified in these groups.

TABLE 4

Table 4. Percent agreement for tertiles between FFQ1, FFQ2 and Mean of FFQ1&2 with 24 h.

Reproducibility assessment

Of the 978 study participants, 848 (87%) completed FFQ2 and were included in the reliability assessment. Crude and energy-adjusted ICC (95% CI) for food group intake between the two FFQs are shown in Table 5. The C-ICC ranged from 0.4 (Fresh fruit juice) to 0.77 (Refined grains) and the EA-ICC from 0.42 (Legumes) to 0.72 (both Sugar and Sweetened Drinks). Strong correlations (>0.6) were observed in half of the 26 food groups, and moderate correlations (0.3–0.6) in the other half. Same category agreement ranged from 46.3 to 76%, averaging 54.6% of participants [median (IQR): 54.6 (51.5–57%)]. Gender-specific reproducibility also yielded similar results as that of the entire population (Supplementary Table 3).

TABLE 5

Table 5. Reproducibility assessed by intraclass correlation coefficients (ICC) comparing FFQ1 and FFQ2 (N = 848).

Discussion

FFQs are commonly used in epidemiological studies to collect dietary information (4, 12, 13). While different FFQ designs—qualitative vs. quantitative or dish-based vs. item-based—have been used in various studies, the ultimate importance is for the FFQ to accurately capture what it was intended to measure so that diet-disease associations can be correctly made (14). In this study, we evaluated the validity and reproducibility of the PERSIAN Cohort FFQ in seven locations across Iran and found it to be appropriate to rank individuals based on their food group intake.

Questionnaire design and administration

We designed this FFQ by modifying the validated GCS questionnaire, making it more concise and less detailed, as extensive FFQs lead to fatigue and decreased accuracy (15, 16). Also, given that a common error in self-reported questionnaires, including FFQs, is overestimation of foods consumed (5, 14, 17), and that inclusion of multiple foods or varieties of a food from the same group increase overestimation (5), we limited the number of food items in our FFQ, to foods with the highest frequency of consumption in our study population and only included enough detail to capture major dietary intakes and to avoid overlap between items. For example, the GCS questionnaire records chicken intake in ten separate items, distinguishing between various parts consumed, which makes reporting difficult and also may result in overlap and overestimation in the reported intakes; but we reduced the ten items to one item only, enquired about the overall frequency and amount of chicken intake. A direct comparison of correlations in chicken intake or other similar modifications between our FFQ and the GCS FFQ is not possible since we assessed food group intake and they evaluated nutrient intakes.

A similar comparison, however, may be made between the PERSIAN and the TLGS FFQs, which with 168 items, also recorded varieties of several foods. We asked about red meat use in one item—lamb or beef, as ground meat or cubes—while TLGS recorded beef, lamb and ground meat as three separate items. We reported red meat intake as a separate group in our analysis, while TLGS grouped all animal proteins together. Nonetheless, the DEA-SCC obtained in our study for Red Meat (0.52 to 0.59), Chicken (0.42 to 0.54), Eggs (0.46–0.54) and Fish (0.35–0.42) were higher than those reported for the TLGS Meats group (0.37–0.39 in men and 0.36–0.37 in women). Similar groupings of a single food item, or different items with similar nutrients were also performed throughout our FFQ. While this may decrease accuracy in the estimation of some nutrients, we believe that it limits overestimation of energy intake, while at the same time being easier for participants to report.

Another common problem seen with many dietary data collection methods, especially FFQs, is energy misreporting, most frequently seen as underreporting of nutrient-dense foods by participants (18, 19). Previous studies have found the following individuals to be most prone to underreport their intake: women, those with higher body mass index, lower literacy and education, as well as individuals of the lower socioeconomic status (18–22). While underreporting is sometimes intentional, especially by overweight/obese individuals, not all underreporting is intended, and participant fatigue, memory problems, as well as misperception of portion sizes can also lead to it (18). Strategies to limit underreporting have been suggested, some of which were used in our study. For example, we designed a shorter questionnaire compared to those previously validated to reduce participant fatigue and used common household measures, pictures and food models for a better estimation of portion sizes. Some interviewing techniques were also employed such as repeating participants’ responses back to them for various food items. Hearing their reported intake from the interviewers sometimes made participants realize they had misreported and corrected their responses. In addition, meal counting for grain intake was also used to limit under and over reporting of the most energy-contributing foods in the Iranian diet (described in greater detail in the following sections).

Our FFQ was interviewer-administered because some participants in smaller cities and villages were illiterate or with low education. But in general, interviewer-administered questionnaires result in systematically more desirable responses to lifestyle-related topics (23). In addition, interviewers trained on the same administration protocols can guide participants the same way and limit individual variations in interpretation of questions.

Interviewer-administered 24 h were chosen as the reference method in this study. Diet records, however, are considered more precise than 24 h and are suggested as the first reference method of choice in validation studies. This is so, because they share the least correlated errors with the FFQs, compared to other methods including the 24 h (4). For example, the FFQ relies on memory, whereas diet records do not, as foods are recorded at the same time they are consumed. Also, portion sizes are estimated when completing FFQs, but they are measured and exact amounts are written in diet records. The 24 h, on the other hand, shares these errors with the FFQ, and therefore its use as the reference method in validation studies yields to higher correlations that are a result of correlated errors. Nevertheless, the 24 h are most commonly used across validation studies due to their feasibility (24) and are considered the primary alternative to diet records, especially in instances when low participant cooperation/motivation for the completion of the diet records is expected or when participants have low literacy levels (4). In our study too, the 24 h seemed as the most reasonable option and most suitable for our population, given their low literacy levels (about 42% being illiterate of with only primary education). The USDA multiple-pass method was used to conduct the 24 h, which has been previously validated in different populations (25, 26).

Validity

Our results showed that our FFQ is moderate-to-highly acceptable in estimating intakes of major energy-contributing food groups in the Iranian diet. The DEA-SCC between FFQ1, FFQ2, and FFQ1&2 vs. 24 h ranged from 0.23–0.7, 0.27–0.76 and 0.3–0.79, respectively, with most values being between 0.4–0.7 in all three comparisons. Previous validation studies of food group intakes have reported correlations between 0.3–0.8 (2, 4, 5, 13, 16, 27). To our knowledge, only the TLGS and the IHHP FFQs have been validated by assessing food group intakes in the Iranian population, however, the IHHP simplified FFQ, being focused on food habits related to cardiovascular diseases, is different in questionnaire design, foods included and validation groupings than the TLGS FFQ and ours, and therefore, its findings are not discussed in this manuscript. The median DEA-SCC observed by TLGS for FFQ1 and FFQ2 were 0.43 and 0.44 in men, and 0.43 and 0.37 in women, respectively, compared to the median DEA-SCC of 0.52 (FFQ1), 0.52 (FFQ2) and 0.58 (FFQ1&2) in our overall population (2).

We observed stronger SCC in food groups consumed at greater frequencies. The strongest correlations belonged to simple sugars, tea, grains, oils/fats, followed by dairy, vegetables, fruits, and animal proteins. Grains are the main staple foods of Iranians, used daily as bread and rice and for most individuals at every meal. We therefore placed great emphasis on the grains section of the FFQ and interviewing protocol. We ensured that all major grains consumed are included in the questionnaire and that local breads are also added, to not miss a major energy-contributing food item. Also, we tried to limit over/underreporting in grain consumption, by having the interviewers count the frequency of grain use per week based on the reported use of all grains, and enquire about patterns of grain use if over/under-reporting was observed. For example, if more than 21 uses of all grains combined was counted (the typical number of meals consumed/week), interviewers asked if grains are used in between meals as well, or if multiple types of grains are used simultaneously in one meal, to make sure over-reporting is limited. Likewise, if less than 21 meals were counted, interviewers asked participants if they routinely omit meals or not eat any grains at meals—not often customary with the Iranian cuisine—to make sure the amount recorded is not underreported. Necessary changes were then made, if needed. Therefore, we believe the correlations obtained in Refined/Whole Grains are closer to participants’ true intake than expected from an FFQ.

Tea consumption also showed strong correlations, because of its frequency of use, often drunk multiple times per day by most individuals. Interestingly, correlations of tea and sugar intake were very close, showing that the FFQ may also capture certain repetitive dietary habits, as many Iranians use sugar/sugar cubes daily to sweeten tea. The strongest correlations observed in TLGS also belonged to tea and sugar (2).

Correlations regarding solid fat and oil intake were also strong (0.65–0.78), given that they are also used predominantly daily in cooking. With the high rate of obesity and other NCDs related to high calorie and fat intake, these results are acceptable for use in future association studies. Our findings for fat intake differ from those observed in TLGS, where SCC ranged from 0.03–0.32 in men and 0.33–0.51 in women. Hosseini Esfahani et al. explained the weak associations observed in men, to be due to their lack of culinary knowledge, as women mostly cook in the Iranian culture (2). We tried to overcome this in our study by completing the questionnaire of spouses simultaneously. As explained, families enrolled in the PERSIAN Cohort on the same day and their FFQs were completed at the same time. Much emphasis was made on each individual reporting their own usual intake and spouses were not allowed to respond on behalf of one another except in the case of food items referred to as “hidden items” in the study protocol, such as salt, oil, tomato paste, etc. where the amount used in cooking is often not known by men who do not engage in cooking, and not visibly seen in their plate while eating. For these items, women reported the frequency and overall amount used in cooking, then each individual would report the portion of the total dish they would typically eat each time, and that proportion was used to estimate how much of the “hidden item” was consumed by each individual. This method may have influenced the stronger accuracy of fat/oil intake observed in our study.

Our FFQ was less valid at estimating Legume intake, with both C-SCC and DEA-SCC being below 0.3 in FFQ1 vs. 24 h and below 0.4 in the other two comparisons. Other Meat, Pizza and Fresh Fruit Juice also followed similar correlation patterns in the comparisons made. SCC related to legume intake was weak in TLGS as well (0.26–0.43 in men and 0.1–0.18 in women), possibly because legumes are mostly used in mixed dishes and stews in Persian cuisine, making their portion size difficult to report (2). The weak correlations observed in our study for Other Meat, Pizza and Fresh Fruit Juice were expected however, given their low median intake, ranging from 0 to 2.5 grams per day.

On average, 51–54% of individuals were classified correctly in the agreement analysis between the data collection methods. These findings are acceptable and compare to those observed by previous studies (2, 15, 28).

Reproducibility

When assessing reproducibility, EA-ICC ranged from 0.42 to 0.72; correlations between 0.4–0.8 are typically seen in studies evaluating reproducibility of food group intake (4, 5, 29). Given that our second FFQ was administered one year after the first, real changes in dietary habits may have affected the lower correlations observed.

The complexity of a questionnaire also affects its reproducibility (30). Typically, questionnaires recording portion sizes tend to produce lower reproducibility due to higher variations in responses (5). Our FFQ, not only recorded portion sizes, but also gave individuals a choice for portion size reporting, using various tools, as they were also free to choose any time interval for the frequency of food consumption, not being limited by pre-determined frequency intervals. Therefore, our reproducibility results are more susceptible to random errors in comparison to qualitative FFQ or other, simpler methods.

Interestingly, foods groups with low median intake and weak validity, such as Fresh Fruit Juice, Pizza and Other Meats, had acceptable reproducibility, showing that they are consistently not eaten frequently in our study population and may possibly even be omitted from the FFQ in future uses.

Strengths and limitations

• Perhaps one important strength of our study is the diversity of the study population. Our sample size exceeds typical recommendations for a validation study (between 100–200 individuals) (4). We exceeded this sample size not to increase precision—as increases over 200 do little for precision (4)—but to include an adequate number of individuals from each study location and have the diversity needed to use this FFQ in different Iranian populations.

• Repeating the 24 h twice monthly for a total of 24 records is another strength, trying to account for variations in foods consumed over one year.

• All interviewers were trained by the same individual and tools used for portion size estimation were centrally purchased and distributed to cohort centers to ensure consistency. The fact that our FFQ must be administered by an interviewer increases precision, while at the same time can be seen as a limitation because it may influence underreporting of foods perceived as unhealthy and over-reporting of healthy foods. It also adds to the personnel cost of studies wanting to use this questionnaire. But having a self-administered questionnaire was not possible in the PERSIAN Cohort due to a considerable proportion of the population having low literacy.

• Addition of the local food items (mostly breads and sweets) to the FFQ for each center is another strength of our questionnaire, making it appropriate for use in various populations of Iran by taking into account their different local foods and dietary habits. As previously described, grains (various breads and rice) are the staple food in Iran and the most energy-contributing foods, being consumed at all meals. And while the three main breads used across Iran (Lavash, Barbari and Sangak) were included in our questionnaire as standard food items, some areas included in the PERSIAN Cohort did not use any of these breads and not including the local breads would have led to inaccurate recording of their energy intake as no bread consumption would have been recorded. But in order to make sure all FFQs, despite the different local items, are analyzed the same, the local food items for each center were equated to the standard items by nutritionists, after data collection and therefore analyzed data from the FFQs in one PERSIAN Cohort site is not different from the others.

• We tried to limit biases in reporting by having the same interviewers who completed the cohort FFQs, complete the 24 h, using the same tools. This may have, on the other hand, caused an overestimation in correlations between methods, further increasing the correlated errors previously described.

• Correlations between FFQ2 and the 24 h were higher in comparison to those of FFQ1 and the 24 h. This was expected, however, as FFQ1 measured food intake 1 year prior to the start of the study, while the time of data collection in both FFQ2 and the 24 h coincided, both recording the intake of foods during the 1-year study period (the 24 h, recording food intake each month for one year, and FFQ2 recording food intake at the end of that same year, retrospectively). Another reason however for the higher correlations, may be that individuals had become more aware of their food intake during the study period, due to the monthly questionnaire completions and the fact that they knew they would have to complete another FFQ at the end of the study, and therefore it is possible that FFQ2 was actually completed with greater precision. This is an unavoidable limitation that is seen in validation study designs. We tried to provide better means of comparison for the validity and reproducibility evaluation of our questionnaire, however, by presenting correlations with FFQ1 and also with the mean of the two FFQs as well.

• Because our FFQ is shorter than those previously validated in Iran, a food item commonly consumed by a participant may have been included in the 24 h, but not the FFQ. Also, for food group or food item analysis, items recorded in the 24 h must be combined to correspond items on the FFQ, which adds sources of error (4).

Conclusion

The PERSIAN Cohort FFQ is appropriate to rank individuals by their food group intake. Validity and reproducibility of the questionnaire in assessing dietary patterns and nutrient intakes must be further evaluated.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Digestive Diseases Research Institute, Tehran University of Medical Sciences (IR.TUMS.DDRI.REC.1398.001). The patients/participants provided their written informed consent to participate in this study.

Author contributions

SE, AH, HP, WW, and RM have contributed to the design of the research study. SE, HP, EF, RH, HH, MM, ZM, YP, and AD have contributed to study execution and data collection. SE and MS performed the data cleaning and statistical analysis, respectively. SE, AH, and MS prepared the manuscript. All other authors reviewed, commented on, and approved the final text.

Funding

This study was supported by the Digestive Diseases Research Institute, Tehran University of Medical Sciences through Grant no. 97-03-37-39,212. The Iranian Ministry of Health and Medical Education has contributed to the funding used in the PERSIAN Cohort Study through Grant no. 700/534.

Acknowledgments

The authors would like to thank the participants of this study from the Fasa, Rafsanjan, Azar, Yazd, Ravansar, Zahedan, and Tabari PERSIAN cohort centers, without whom this study would not have been possible. We would also like to thank the hardworking personnel at these cohort centers for their contribution to data collection.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnut.2023.1059870/full#supplementary-material

References

1. Malekshah, A, Kimiagar, M, Saadatian-Elahi, M, Pourshams, A, Nouraie, M, Goglani, G, et al. Validity and reliability of a new food frequency questionnaire compared to 24 h recalls and biochemical measurements: pilot phase of Golestan cohort study of esophageal cancer. Eur J Clin Nutr. (2006) 60:971–7. doi: 10.1038/sj.ejcn.1602407

CrossRef Full Text | Google Scholar

2. Esfahani, FH, Asghari, G, Mirmiran, P, and Azizi, F. Reproducibility and relative validity of food group intake in a food frequency questionnaire developed for the Tehran lipid and glucose study. J Epidemiol. (2010) 20:150–8. doi: 10.2188/jea.je20090083

CrossRef Full Text | Google Scholar

3. Mohammadifard, N, Sajjadi, F, Maghroun, M, Alikhasi, H, Nilforoushzadeh, F, and Sarrafzadegan, N. Validation of a simplified food frequency questionnaire for the assessment of dietary habits in Iranian adults: Isfahan healthy heart program. Iran ARYA Atheroscler. (2015) 11:139–6.

Google Scholar

4. Willett, WC, and Lenart, E. Reproducibility and validity of food frequency questionnaires In: W. Willett, editor. Nutritional epidemiology. New York: Oxford University Press (2012).

Google Scholar

5. Bohlscheid-Thomas, S, Hoting, I, Boeing, H, and Wahrendorf, J. Reproducibility and relative validity of food group intake in a food frequency questionnaire developed for the German part of the EPIC project. European prospective investigation into cancer and nutrition. Int J Epidemiol. (1997) 26:S59–70. doi: 10.1093/ije/26.suppl_1.S59

CrossRef Full Text | Google Scholar

6. Poustchi, H, Eghtesad, S, Kamangar, F, Etemadi, A, Keshtkar, A-A, Hekmatdoost, A, et al. Prospective epidemiological research studies in Iran (the PERSIAN cohort study): rationale, objectives, and design. Am J Epidemiol. (2018) 187:647–55. doi: 10.1093/aje/kwx314

CrossRef Full Text | Google Scholar

7. Eghtesad, S, Mohammadi, Z, Shayanrad, A, Faramarzi, E, Joukar, F, Hamzeh, B, et al. The PERSIAN cohort: providing the evidence needed for healthcare reform. Arch Iran Med. (2017) 20:691–5.

Google Scholar

8. Ghafarpour, M, Kianfar, H, Hoshyarrad, A, and Banieghbal, B. Food album. Tehran, National Nutrition and food technology research institute: (2007).

Google Scholar

9. Moshfegh, AJ, Rhodes, DG, Baer, DJ, Murayi, T, Clemens, JC, Rumpler, WV, et al. The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes. Am J Clin Nutr. (2008) 88:324–2. doi: 10.1093/ajcn/88.2.324

CrossRef Full Text | Google Scholar

10. United States Department of Agriculture (2021) Food data central. Available at: https://fdc.nal.usda.gov/ (Accessed December 2021).

Google Scholar

11. Willett, WC, Howe, GR, and Kushi, LH. Adjustment for total energy intake in epidemiologic studies. Am J Clin Nutr. (1997) 65:1220S–8S. doi: 10.1093/ajcn/65.4.1220S

CrossRef Full Text | Google Scholar

12. Cade, JE, Burley, VJ, Warm, DL, Thompson, RL, and Margetts, BM. Food-frequency questionnaires: a review of their design, validation and utilisation. Nutr Res Rev. (2004) 17:5–22. doi: 10.1079/NRR200370

CrossRef Full Text | Google Scholar

13. Ocké, MC, Bueno-de-Mesquita, HB, Goddijn, HE, Jansen, A, Pols, MA, van Staveren, WA, et al. The Dutch EPIC food frequency questionnaire. I. Description of the questionnaire, and relative validity and reproducibility for food groups. Int J Epidemiol. (1997) 26:S37–48. doi: 10.1093/ije/26.suppl_1.s37

CrossRef Full Text | Google Scholar

14. Burrows, TL, Ho, YY, Rollo, ME, and Collins, CE. Validity of dietary assessment methods when compared to the method of doubly Labeled water: a systematic review in adults. Front Endocrinol. (2019) 10:850. doi: 10.3389/fendo.2019.00850

CrossRef Full Text | Google Scholar

15. Egashira, EM, Aquino, RC, and Philippi, ST. Técnicas e métodos para a avaliação do consumo alimentar. In: Tirapegui J, Ribeiro SML, organizadores. Avaliação nutricional: teoria e prática. Rio de Janeiro: Editora Guanabara Koogan. (2009):13–23.

Google Scholar

16. Martinez, MF, Philippi, ST, Estima, C, and Leal, G. Validity and reproducibility of a food frequency questionnaire to assess food group intake in adolescents. Cad Saude Publica. (2013) 29:1795–04. doi: 10.1590/0102-311x00055512

CrossRef Full Text | Google Scholar

17. Rutishauser, IH. Dietary intake measurements. Public Health Nutr. (2005) 8:1100–7. doi: 10.1079/PHN2005798

CrossRef Full Text | Google Scholar

18. Castro-Quezada, I, Ruano-Rodríguez, C, Ribas-Barba, L, and Serra-Majem, L. Misreporting in nutritional surveys: methodological implications. Nutricion Hospitalaria. (2015) 31:119–7. doi: 10.3305/nh.2015.31.sup3.8760

CrossRef Full Text | Google Scholar

19. Magalhães, V, Severo, M, Torres, D, Ramos, E, and Lopes, C, by IAN-AF Consortium. Characterizing energy intake misreporting and its effects on intake estimations, in the Portuguese adult population. Public Health Nutr. (2020) 23:1031–40. doi: 10.1017/S1368980019002465

CrossRef Full Text | Google Scholar

20. Mattisson, I, Wirfält, E, Aronsson, CA, Wallström, P, Sonestedt, E, Gullberg, B, et al. Misreporting of energy: prevalence, characteristics of misreporters and influence on observed risk estimates in the Malmö diet and cancer cohort. Br J Nutr. (2005) 94:832–2. doi: 10.1079/bjn20051573

CrossRef Full Text | Google Scholar

21. Lara, JJ, Scott, JA, and Lean, ME. Intentional mis-reporting of food consumption and its relationship with body mass index and psychological scores in women. J. Hum Nutr Diet. (2004) 17:209–8. doi: 10.1111/j.1365-277X.2004.00520.x

CrossRef Full Text | Google Scholar

22. Grech, A, Hasick, M, Gemming, L, and Rangan, A. Energy misreporting is more prevalent for those of lower socio-economic status and is associated with lower reported intake of discretionary foods. Br J Nutr. (2021) 125:1291–8. doi: 10.1017/S0007114520003621

CrossRef Full Text | Google Scholar

23. Okamoto, K, Ohsuka, K, Shiraishi, T, Hukazawa, E, Wakasugi, S, and Furuta, K. Comparability of epidemiological information between self-and interviewer-administered questionnaires. J Clin Epidemiol. (2002) 55:505–1. doi: 10.1016/s0895-4356(01)00515-7

CrossRef Full Text | Google Scholar

24. De Keyzer, W, Huybrechts, I, De Vriendt, V, Vandevijvere, S, Slimani, N, Van Oyen, H, et al. Repeated 24-hour recalls versus dietary records for estimating nutrient intakes in a national food consumption survey. Food Nutr Res. (2011):55. doi: 10.3402/fnr.v55i0.7307

CrossRef Full Text | Google Scholar

25. Conway, JM, Ingwersen, LA, and Moshfegh, AJ. Accuracy of dietary recall using the USDA five-step multiple-pass method in men: an observational validation study. J Am Diet Assoc. (2004) 104:595–3. doi: 10.1016/j.jada.2004.01.007

CrossRef Full Text | Google Scholar

26. Conway, JM, Ingwersen, LA, Vinyard, BT, and Moshfegh, AJ. Effectiveness of the US Department of Agriculture 5-step multiple-pass method in assessing food intake in obese and nonobese women. Am J Clin Nutr. (2003) 77:1171–8. doi: 10.1093/ajcn/77.5.1171

CrossRef Full Text | Google Scholar

27. Wong, JE, Parnell, WR, Black, KE, and Skidmore, PM. Reliability and relative validity of a food frequency questionnaire to assess food group intakes in New Zealand adolescents. Nutr J. (2012) 11:65–73. doi: 10.1186/1475-2891-11-65

CrossRef Full Text | Google Scholar

28. Sasaki, S, Kobayashi, M, and Tsugane, S. Validity of a self-administered food frequency questionnaire used in the 5-year follow up survey of the JPHC study cohort I: comparison with dietary records for food groups. J Epidemiol. (2003) 13:57–63. doi: 10.2188/jea.13.1sup_57

CrossRef Full Text | Google Scholar

29. Järvinen, R, Seppänen, R, and Knekt, P. Short-term and long-term reproducibility of dietary history interview data. Int J Epidemiol. (1993) 22:520–7. doi: 10.1093/ije/22.3.520

CrossRef Full Text | Google Scholar

30. Block, G, and Hartman, AM. Issues in reproducibility and validity of dietary studies. Am J Clin Nutr. (1989) 50:1133–8. doi: 10.1093/ajcn/50.5.1133

CrossRef Full Text | Google Scholar