You are on page 1of 7

Supplemental Material can be found at: 1.


Nutritional Epidemiology

Relative Validity of Food Intake Estimates Using a Food Frequency Questionnaire Is Associated with Sex, Age, and Other Personal Characteristics1,2
Geoffrey C. Marks,*3 Maria Celia Hughes,y and Jolieke C. van der Pols*y
* School of Population Health, University of Queensland, Herston, Qld 4006, Australia and y Queensland Institute of Medical Research, Herston, Qld 4029, Australia
ABSTRACT We investigated the validity of food intake estimates obtained by a self-administered FFQ relative to weighed food records (WFR) and the extent to which demographic, anthropometric, and social characteristics explain differences between these methods. A community-based sample of 96 Australian adults completed a FFQ and 12 d of WFR over 12 mo. The FFQ was adapted to the Australian setting from the questionnaire used in the US Nurses’ Health Study. Spearman rank correlation coefficients ranged from 0.08 for ‘‘other vegetables’’ to 0.88 for tea. Exact agreement by quartiles of intake ranged from 27% (eggs) to 63% (tea). Differences between FFQ and WFR regressed on personal characteristics were significantly associated with at least 1 characteristic for 20 of the 37 foods. Sex was significantly associated with differences for 17 food groups, including 5 specific vegetable groups and 2 ‘‘total’’ fruit and vegetable groups. Use of dietary supplements and the presence of a medical condition were associated with differences for 5 foods; age, school leaving age, and occupation were associated with differences for 1–3 foods. BMI was not associated with differences for any foods. Regression models explained from 3% (wholemeal bread) to 37% (for all cereals and products) of variation in differences between methods. We conclude that the relative validity of intake estimates obtained by FFQ is different for men and women for a large number of foods. These results highlight the need for appropriate adjustment of diet-disease relations for factors affecting the validity of food intake estimates. J. Nutr. 136: 459–465, 2006. KEY WORDS:  food frequency questionnaire  weighed diet records  validation  relative validity  dietary assessment

Downloaded from by guest on July 1, 2012

There is an increasing need for reliable measurements of foods that are consumed as part of the usual diet. Valid estimates of the consumption of food items are the premise of dietary pattern characterizations, and intervention trials such as the CARET study showed that whole foods rather than individual nutrients may best indicate the potential role of the diet in disease prevention (1). FFQ are widely used to investigate customary food intake over extended periods of time. Like all dietary methods, estimates derived from FFQ data suffer from random and systematic error and may not represent the ‘‘true’’ usual diet. Numerous factors may compromise the validity of food consumption estimates (2), but the effects of these on measurement error are generally assessed for nutrient intake estimates rather than food intake per se. As a result, the validity of food intake estimates derived from FFQ data is not well docu-

1 M.C.H. was funded by the Public Health Research and Development Committee of the National Health and Medical Research Council of Australia and the World Cancer Research Fund International. J.C.P. was funded by a NHMRC Capacity Building Grant in Population Health Research (grant number: 252834). 2 Supplemental Tables 1–2 are available as Online Supporting Material with the online posting of this paper at 3 To whom correspondence should be addressed. Email: g.marks@sph.uq.

mented. Further, experience with validation of nutrient intake estimates showed that specific subject characteristics are frequently associated with measurement error (3,4). Knowledge of such factors may help improve the design of food intake instruments and provide a basis for more appropriate modeling of diet-disease relations. Here we evaluate the relative validity of food intake estimates derived from an FFQ administered to participants in the Nambour Skin Cancer Prevention Trial (5). This field trial was conducted in an unselected adult population in Australia. One of the central objectives of the project was to examine the relation of dietary factors to development of actinic skin and eye disease (6). This study compares estimates of intake of food from the FFQ with those based on 12 d of weighed food records (WFR) over a 12-mo period for a randomly selected subsample of the Nambour study population. We estimated the relative bias and imprecision of food intake estimates, and assessed the extent to which selected demographic, anthropometric, and social characteristics of participants explain any difference between the 2 dietary methods. SUBJECTS AND METHODS
Selection of study subjects. The Nambour Skin Cancer Prevention Trial is a community-based randomized trial of the effects of

0022-3166/06 $8.00 Ó 2006 American Society for Nutrition. Manuscript received 23 August 2005. Initial review completed 19 September 2005. Revision accepted 22 November 2005. 459

1/d 5 1. In the first phase. 0.’’ For seasonal fruits and vegetables. participants were asked to indicate how often these foods were eaten in season. nonprofessionals). occupation. 1/wk 5 0. R2 was calculated to quantify the extent to which the explanatory variables accounted for total variation in the difference in intakes. cereals. Statistical analyses were performed in 2 phases. These analyses compared the ranking of individuals by food intake. margarine.g. further revisions were made to improve estimates of intake of antioxidant-rich foods (inclusion of major food sources. Participants were considered to have a medical condition if they answered ‘‘yes’’ to any of the conditions listed in the question ‘‘Have you ever been told by a doctor/nurse that you have: glaucoma. education. Statistical analyses. organ meats other than liver. recording days were advanced by 1 d.10 and LOA of 0. A research dietitian collected the food diaries and reviewed the records with the participants to check for errors. For this comparison we used nonparametric methods including Spearman rank correlations and the Wilcoxon signed rank test for difference between paired observations.. and use of dietary supplements (yes.10.40 indicate that. multivariable regression analysis was performed with the difference in log-transformed food intakes between dietary methods as the dependent variable and personal characteristics of participants as explanatory variables. Thus. stroke. (8. Seasonal foods were weighted according to the proportion of the year that each food was available. 1 tablespoon (15 mL). To identify factors associated with the validity of FFQ intake estimates. . We compared the classification of intakes into quartiles by the 2 methods. For this reason. trained research staff measured body weight and height using standard protocols. The difference in intakes (FFQlog 2 WFRlog) was plotted against the mean (FFQlog-adj 1 WFRlog-adj)/2 to determine whether the difference between the methods varied across the range of intakes. agreement in estimating absolute estimates of intakes was assessed using parametric tests for which food group intake data were log-(natural) transformed (FFQlog and WFRlog) to achieve a normal distribution. Data from WFR were entered into a database using Xyris Diet 1 Software (7. Revisions were made to ensure that the list of foods reflected the Australian diet according to the 1983 National Dietary Survey of Adults (10. or 1 cup (250 mL). The amounts were in household or common measures such as 1 slice. 2–4/wk 5 0. they consumed a given amount of each food during the past 6 mo (judged appropriate for this population). The formula log(gram intake 1 1) was used if food groups were not consumed by all participants. and the proportion of grossly misclassified individuals (disagreement by 3 quartiles) were calculated. The 8 nonmatching food groups included water (drinking or used in food preparation). and take-out foods eaten. meal replacements (e. trained research assistants explained the objective of the FFQ and instructed the participants on how to complete it. Mean differences and LOA were exponentiated to provide a ratio of the gram intakes estimated by FFQ relative to the WFR (13). Collection of health indicators.e. Participants were eligible to participate if they had completed the FFQ and remained an active participant in the trial (7).12). All statistical tests were two-sided and a significance level of Downloaded from jn. The intended use of the FFQ includes the association of food groups with skin cancer. All participants provided written informed consent and the institutional ethics committee approved the study.460 MARKS ET AL. medical condition (yes. and 41/d 5 4. Information on age. The intake estimates were not normally distributed. no). The LOA is calculated as follows: mean difference 6 tn21. representing 1 standard serve for each food. WFR participants completed 2 nonconsecutive days of food weighing every 2 mo over a period of 12 mo. SustagenÒ). All FFQ data were double entered. Administration of the weighed food records. Participants used 2-kg capacity digital scales in 2-g gradations to weigh all food and beverages consumed for the 2 recording days. we compared agreement between the 2 methods by using the intake estimates in the form in which they would be used in future dietdisease analyses. Respondents were requested to recall how often. a mean ratio of 1.5. 2–3/d 5 2. high blood pressure/hypertension. on average. Initial start days for data collection were randomly allocated among participants to ensure that each day of the week was equally represented and that the records for the sample were spaced evenly over the initial 2-mo block..0:05 SDr ð111=nÞ: The lower and upper boundaries of the LOA present the range in which 95% of the differences between the dietary methods were expected to lie. The proportion of exact agreement.02.05) and limits of agreement (LOA) described by Bland and Altman (13) with correction for a small sample size (n . dosage. deviation by 1 quartile. Mean. particularly vegetables and fruits). At the start of the trial. The selfadministered semiquantitative FFQ was adapted from the questionnaire used in the US Nurses’ Health Study developed by Willett et al.or underestimated the WFR. occupation (professionals. condiments. FFQ overestimates WFR by 10% and that 95% of the differences range from 15% below to 40% above. Daily grams of consumption of each food group were calculated by summing foods in each food group per day of WFR and obtaining the mean of all weighing days. If the days specified were unsuitable for participants. daily consumption of a dietary supplement of b-carotene and application of sunscreen in the prevention of skin cancer (6).14. i. Slopes significantly different from 0 identified cases in which the difference between methods increased or decreased across the range of intakes. If any factors are associated with the differences. cancer?’’ Calculation of food intakes. the 129 FFQ food items were reclassified into 37 food groups. we calculated the median grams of food intake as determined using WFR for each FFQ quartile. Personal characteristics assessed included age (y).nutrition. Additional information collected included cooking methods and specific types of oil. Frequency of consumption of each food item in the FFQ was converted to intake in grams per day by multiplying the standard serving size of each food as specified in the FFQ by the following values for each frequency option: Never 5 0. omissions. 1–3/mo 5 0. a random sample of 168 participants was invited to participate in a validation study involving WFR.85–1. sex. Results of paired t tests indicate whether on average. angina. and medical condition was obtained by questionnaires (6). Daily grams of intake for individual FFQ items were summed to obtain daily intake of each food group.1/mo 5 0. meat pastes. For subsequent blocks. and meat replacement foods. A regression line was fitted and the slope was tested for significant difference from 0 (P 5 0. sauces. Finally. including those that reflect dietary patterns that are hypothesized to modify the risk of skin cancers (see Supplemental Table 1). Mean intakes were included in the model as a predictor of difference in intake estimates if the preliminary analyses (see above) showed that mean intakes were associated with the difference in intakes at P . Because of the dissimilarity in construct and error source of the 2 dietary methods. on average. diabetes/high blood sugar. no). Individual food items were classified into 45 food groups. Response options ranged from ‘‘Never’’ to ‘‘41 times a day.79.11).9). it would indicate that these factors are associated with the relative validity of the measures.43.0. (untransformed) grams of intake and quartiles of grams of intakes. 2012 . high cholesterol. education (schoolleaving age). and any discrepancies resolved by reference to the original forms. Coding decisions were made by the research dietitians who checked all decisions for open-ended questions in the FFQ and checked a random 10% subsample of daily records for the WFR (error rate 0.05). Information on recipes and dietary supplements used was also recorded. gallstones.7%). The FFQ also collected information on brand. 5–6/ wk 5 by guest on July 1. drinking chocolate. sex. For the Nambour Trial. butter. and frequency of use of dietary supplements. alternative days were determined to ensure an overall balance of week and weekend days. The Nambour Trial began in 1992 with 1621 participants aged 25–75 y who were previously randomly selected from the Nambour electoral roll for a skin cancer prevalence survey in 1986 (5). SD and median food group intakes (grams) were calculated for each dietary method. the FFQ consistently over. Administration of the food frequency questionnaire. 100) by Ludbrook (14) were used to quantify the degree of agreement/disagreement between the 2 dietary methods. high triglycerides. BMI. At baseline. one would generally expect differences to be random relative to other factors. Also in 1992. or doubtful entries. The FFQ collected consumption information for 129 food items or food groups.07. heart attack. Paired t test (P 5 0. In the second phase. Only the 37 food groups that matched the FFQ were included in this analysis.

0* 48.9 144.7 64.05 was used.7 82.0 62.7* 445. Thus. There was no gross misclassification (differing by 3 quartiles) for unmodified dairy.3 69.1 76. compared with expected proportions using random allocation.5 14.3* 18. Of the 115 participants in the WFR. rice.1 20.4 75.9* 94.3 30. and coffee.1 12.0 273.6 360.4 15. and nuts.2 126.1 212.1 31. jams Nuts Mean 6 SD 17.3 10. Random allocation would result in 25% exact agreement.7 25.5 9. tea.3 77.4* 336.3* 15.0 25.2 13.5 4.6 54.9* 378.6 2.9* 44.05).9 72.6 183.7 69.7* 45.4 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 10.0 52.9 17. 12 completed . 5 omitted consumption frequencies for .3 209.9 14.0 16.9 3.8 227.0 0.2 21.3 27.4 10.nutrition.7 10. 0.6 15.8 61. Table 2 presents the extent of agreement in assignment by quartiles of intake for each food group. 2012 TABLE 1 Intakes from food groups in a semiquantitative FFQ and 12-d WFR kept by a sample of Nambour residents aged 25 to 75 y.6 394. the sample size for the validity study was 96. alcoholic beverages.0 36.7* Median g/d 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 15.4 4.2* 163.8 44.9 15.70 included all dairy products (unmodified and modified).6 158.8 113.3 6.9 8.88 for tea.9* 43.9 52.9 20. and the 1621 Nambour Trial population did not differ in age.6 20.2 429. 0. whole-meal Bread.2 114.5 48. white Rice Pasta and noodles Breakfast cereals Other cereals All cereals and products Cakes and biscuits Snacks Juice Alcohol Tea Coffee Soft drinks Sugars.4 14.9 6.8 55.0 62. All analyses were performed using SAS v8.1* 430.10% of items in the FFQ. The 96 subjects in the validation study.2 43. P .5 58.5 24. the 115 subjects who completed the WFR.2 293.0 40. 0.8 829.0 130.7 49.1 22.0 9. whereas 11 groups did not differ.4* 168.3 by guest on July 1.2 software(15).5 243.0 202.05 level (Wilcoxon signed rank test for difference between paired observations).8 819.8 1. .5 48.5 23.08 for ‘‘other vegetables’’ to 0.4 67.4 3.5 69.4 232.2 75.6* 8.4 187.3 46.2 28.5 460.0 250.4 39.9* 105.9 162.0 141.8* 186.2 472. There were significantly more regular users of dietary supplements in the validation study (45.7 18.2 6. sweets. consisting of 37 men and 59 women.7* 21. RESULTS A total of 1447 participants (89%) completed the FFQ.4 126.4 203.2 181.2 39. sex. education.0* 216.4 21. alcoholic beverages.1 WFR Mean 6 SD Median * Different from WFR.2 17.6 41.4* 368.FACTORS ASSOCIATED WITH VALIDITY OF FFQ FOOD ESTIMATES 461 P .3 6.7 625.10% were grossly misclassified (differing by 3 quartiles) by the FFQ for all vegetables (no potato).4 28.3 186.0* 18.6 9.3 21.8 340.4 487. WFR consumption estimates were significantly higher for 5 groups.0 246. 19 were omitted from analyses.4* 230.5 171.0* 32.2 42. estimated intakes by the FFQ were significantly higher than those by WFR.6 21.2 30.0 19.1 2.8 25.4 46.1 0. Exact agreement ranged from 27% (eggs) to 63% (tea).7 158.5 112.7 68.1 19.7 2.8 84.3 5.0 53.8 57. It is notable that mean FFQ intakes of all fruit and vegetable food groups (except legumes) were significantly higher.3 356.5 10.6 4. and coffee.7 117.9 24.0 20.2 72.7%) (P .7* 21.1 29.1 358. The mean and median grams of food group intakes are presented in Table 1. The proportion of professionals or participants with a medical condition did not differ.4 6. BMI.7 25.7* 222.8%) compared with the trial participants (32.6 381. .3 724.9 13.5 455.0 48.6 14.7 19.4 37.7 3.or C-rich fruits Other fruits All fruits All vegetables and fruit All vegetables and fruit (no potato) Bread.0 129.1 21.7 12.7 34.6 140.7 932.4 167.5 137.8 13. and 1 man was excluded because his energy intake exceeded limits suggested by Willett (9).0 27.6 50.4 17.2 20.5 365.4* 81.8 404.6 181. tea.5 34.10 d of the WFR.8 37.6 486.5 235.9 113.9 133. There were 10 food groups Downloaded from jn. or smoking status.1 15.6* 46. whereas a final sample of 115 (69% acceptance rate) completed the WFR. 1 woman became pregnant early in the study. Food groups with correlations $0.3 131.9 24.7 79.9 315. and the SD for most was also much larger.5 22.1 28.8 329.4 75.4 14.1 41.5 133.8 23.3 217. For 21 of the 37 food groups. Of the participants. FFQ Food group Fats and oils Unmodified dairy Modified dairy Meat Poultry Processed meats Eggs Seafood Green leafy vegetables Cruciferous vegetables Red and yellow vegetables Legumes Potato Other vegetables All vegetables All vegetables (no potato) Vitamin A.8 91.1 59.6 83.5 201. Spearman rank correlation coefficients ranged from 0.0 170.1 50.1 413.3 168. n 5 96.

i.50. The most marked underestimation was of white bread. FFQ estimations of 3 fruit food groups. other cereals and cakes.34 0. modified and unmodified dairy.75 0. on average.21 0. 0.or underestimation by the FFQ for the entire study population. whereas consumption of soft drinks was overestimated by 300%. median WFR intakes increased over increasing FFQ quartiles. 5 of these were vegetable food groups.13NS 0.28 0.85 0.47 0.22 Median intake from WFR by quartile of FFQ intake Q1 15 52 0 23 13 4 10 0 5 18 61 1 52 53 225 160 18 68 113 370 299 12 4 5 0 8 2 146 21 0 0 0 0 0 0 7 1 Q2 g 17 79 18 38 21 22 20 8 13 19 65 3 53 73 248 177 55 116 181 444 389 53 14 7 13 38 7 147 37 0 27 2 345 89 5 20 2 Q3 24 181 113 58 24 30 15 18 13 20 83 6 69 43 253 205 78 151 222 453 366 57 53 12 9 45 12 202 46 3 40 29 544 290 82 35 2 Q4 22 294 229 67 35 37 21 20 15 29 74 13 77 63 251 160 103 156 277 564 460 107 64 7 31 72 185 87 110 385 710 413 104 49 5 7 0 3 5 9 2 9 4 3 6 6 8 8 7 9 11 4 5 6 3 4 3 2 14 3 1 NA4 3 1 NA 1 0 0 0 2 3 13 13 Downloaded from jn. jams Nuts Random expected 1 2 3 4 Exact 39 51 58 40 36 42 27 47 40 29 31 41 35 28 30 30 39 45 44 35 36 48 49 30 35 42 41 31 41 50 41 54 63 54 42 48 38 25 Adjacent % 39 42 35 42 36 48 47 31 40 38 45 33 36 29 39 41 44 32 35 43 33 32 40 30 31 48 49 41 47 34 49 38 33 38 45 39 33 38 GM 1 Spearman correlation coefficient2 0. with correlations #0.35 0.46 0.40 0. with a more pronounced increase in intakes observed for correlations $0. disagreement by 3 quartiles.51 0. the differences ranged from underestimation at 0.42 0. and all cereals and products increased significantly with the . The strength of correlations was reflected in the comparison of median WFR intakes per FFQ quartile.462 MARKS ET AL. Overall.30 0.22 of the WFR value to a 9-fold overestimation.65 by guest on July 1.2. For 14 food groups.62 0.60 0..65 0.08NS 0.88 0. the mean ratio for meat showed overestimation by 41% (95% CI 17–71%). white Rice Pasta and noodles Breakfast cereals Other cereals3 All cereals and products Cakes.32 0. In 9 food groups. the differences in intakes varied significantly with the magnitude of the intakes. but the LOA indicated that for 95% of individuals. whole-meal Bread. and biscuits were underestimated by the FFQ.00). rice. biscuits Snacks3 Juice Alcohol Tea Coffee Soft drinks Sugars. and pasta and noodles were double the WFR estimates (mean ratio .68 0.48 0.31 0. Differences in estimated intakes of fats and oils. not applicable. For example.26 0.73 0. 2012 Gross misclassification. All correlation coefficients are significant (P .or C-rich fruits Other fruits All fruits All vegetables and fruit All vegetables and fruit (no potato) Bread. Allocation by quartiles Food group Fats and oils Unmodified dairy Modified dairy Meat Poultry Processed meats Eggs Seafood Green leafy vegetables Cruciferous vegetables Red and yellow vegetables Legumes Potato Other vegetables All vegetables All vegetables (no potato) Vitamin A. Based on tertiles only as frequency distribution of intakes of these items precluded the formation of quartiles.17 0.e. The mean ratio (and 95% CI) of FFQ to WFR and LOA was similar to that presented in Table 1.62 0. sweets. Intakes of fats and oils. Although the population mean ratio of FFQ to WFR provides an estimate of over. individuals who do not consume these food groups regularly showed more error in their reporting of small amounts of intake compared who those who ate large amounts of the same food group. the LOA provides information on the variability of estimates between the 2 methods at the individual level.56 0. there was a negative association in which the differences were larger at lower levels of intake.14NS 0.30.05) except those identified with NS (¼ not significant).43 0. n 5 96. whole-meal and white bread. with higher estimates for the FFQ than the WFR in a majority of the food groups (see Supplemental Table 2). 2 vegetable food groups. NA. TABLE 2 Comparison of daily grams of food intake obtained from a semiquantitative FFQ and the mean of 12-d WFR kept by a sample of Nambour residents aged 25 to 75 y.81 0.49 0.39 0.nutrition.30 0.44NS 0. the FFQ estimates were 75% below the WFR.61 0.

33711 0.288 0. Age was significantly associated with green leafy vegetables. Downloaded from jn.019 0. Table 3 shows the associations between personal characteristics of individuals and the difference in estimated intake between the 2 methods.370 0.819111 ÿ0.23 0.151 0. and all cereals and products.003 ÿ0.2551 0.029 0.005 0.007 ÿ0.3441 ÿ0.17 0. white Rice Pasta and noodles Breakfast cereals Other cereals All cereals and products Cakes.014 Sex2 0.011 0.002 0.028 0.48811 ÿ0.312 ÿ0.168 0.07 0.004 0.171 0.175 ÿ0.005 ÿ0.18 0.065 ÿ0. 6 Included only for multivariable models in which the mean of the intake estimate was significantly associated with difference in intakes (see Table 3).230 0. cruciferous vegetables.92211 0.51 times greater (exponential of 0.584 ÿ0.009 0.026 ÿ0.002 0.45411 0.146 ÿ0. 0.235 ÿ0. n 5 96.007 ÿ0.0191 ÿ0.198 ÿ0.4521 0.129 Occupation3 0.030 School leaving age1 0.107 0.009 ÿ0.011 ÿ0.267 0.051 0.161 ÿ0.003 0.004 0.048 0.2711 ÿ0.042 ÿ0.143 ÿ0.222 ÿ0. 0.2991 ÿ0.221 ÿ0. 3 Reference category: nonprofessionals.3811 0.361 0.180 ÿ0.32 0.124 ÿ0.12 0. 11P .014 0.004 0.000 ÿ0.26 0.057 ÿ0.30011 0.380111 0.628111 ÿ0.108 ÿ0.274 0.046 ÿ0.05 0.015 0.5291 0.075 0.018 0.6711 0.014 0. all vegetables and fruits.152 0.32911 0.165 0.092 BMI1 0.260 0.006 0.034 0.243 ÿ0.060 0.027 ÿ0. and tea and decreased difference for all cereals and products. with each factor adjusted for all others.118 ÿ0.10 0.205 0.5871 0.006 ÿ0. the greatest error occurred in subjects who reported high levels of intake compared with nonregular consumers of the same food.412) in participants with medical condition than in those with no medical condition.163 0.346 Medical condition4 0.142 0.102 ÿ0.31 0. including green leafy vegetables.008 0.620 0.14 0. BMI was not significantly associated with difference TABLE 3 Factors associated with difference in (log-transformed) daily grams of food intakes from a semi-quantitative FFQ and 12-d WFR: regression models were multivariable.098 ÿ0.230 0.70011 0.114 ÿ0.12 0.087 0.335 0.031 0. For these food groups. all vegetables.001.000 ÿ0.006 0.230 0.060 ÿ0.15 0.026 0. whereas differences were larger for alcoholic beverages and nuts.304 0.059 0.023 ÿ0.119 ÿ0.485111 0.040 0. biscuits Snacks Juice Alcohol Tea Coffee Soft drinks Sugars.050 0.076 0.290 0.4121 ÿ0.004 0. .263 0.25 0.17 ÿ0.001 0.020 0.080 ÿ0.052 ÿ0.009 0.089 0.28 0.268 0.126 ÿ0. 111P .242 0.037 0.177 0.209 ÿ0.057 0.295 ÿ0.571 0.009 0.541 0.097 ÿ0.093 0.165 0.263 ÿ by guest on July 1. the association between dietary supplements and intake differences in almost all of the other vegetable and fruit food groups was in the same direction.3731 ÿ0.028 ÿ0.5831 0.518 ÿ0.12 0.07 0.20 0.011 0. pasta and noodles.08 0.045 ÿ0.054 ÿ0.009 0. The difference in reported intakes was larger among women than men for fats and oils.097 0.2181 0.071 ÿ0.112 ÿ0.372 0.012 ÿ0.175 0.050 ÿ0.037 ÿ0.206 0.058 ÿ0.012 0.126 ÿ0.055 ÿ0.12 0. However.203 ÿ0.069 0. 1 Continuous variable. the direction of the associations was not consistent. the ratio of intakes of meat estimated by FFQ to WFR was 1.23 0. 5 Reference category: no use dietary supplement use.005 0.08 0.414 0.004 0.76511 Mean of FFQ and WFR6 0. 2012 Association between factors and difference in intake estimates (regression coefficients) Food group Fats and oils Unmodified dairy Modified dairy Meat Poultry Processed meats Eggs Seafood Green leafy vegetables Cruciferous vegetables Red and yellow vegetables Legumes Potato Other vegetables All vegetables All vegetables (no potato) Vitamin A.010 ÿ0.324 0.01.09 0.312 0.126 0.000 ÿ0.306 0.75911 ÿ0.004 0. and all cereals and products.468111 R2 0.19 0.621 ÿ0.3371 ÿ0. For example. poultry and seafood.062 0.052 0.37 0. whole-meal Bread.9051 ÿ0.08 0.040 0.228 ÿ0.120 ÿ0.139 ÿ0.10 0. and other fruits were noted for participants who reported taking dietary supplements.054 ÿ0. whereas the difference was smaller for the intake of tea.0551 ÿ0.372 0.188 ÿ0.nutrition.013 ÿ0.438 0.015 ÿ0.380 ÿ0.010 ÿ0.041 0.072 0.058 0. and all vegetables and fruit decreased significantly with the use of dietary supplements.038 ÿ0.106 ÿ0.4851 ÿ0.4421 0. sweets. meat (including processed). 4 Reference category: no medical condition. rice.003 0.107 ÿ0.071 ÿ0.046 ÿ0.336 0.53511 0.FACTORS ASSOCIATED WITH VALIDITY OF FFQ FOOD ESTIMATES 463 magnitude of intakes.175 ÿ0.003 ÿ0.027 0.052 0.011 0.446x Use of dietary supplements5 0.049 ÿ0.001 ÿ0. pasta and noodles.1751 ÿ0.153 0.005 0.067 ÿ0.237 ÿ0.712111 ÿ0.045 ÿ0.018 ÿ0.020 0.13 0.28 0.038 ÿ0.25 0. Although only the difference in intakes of other vegetables.0101 0.229 0.111 0.213 ÿ0. a number of vegetable food groups.0211 ÿ0. other vegetables.002 ÿ0.115 ÿ0.053 0.221 ÿ0.037 ÿ0.002 0.06 0.05.042 ÿ0.059 0.15 0.062 0.7461 0.2221 1P .312 ÿ0.44311 0.1651 ÿ0.1931 ÿ0.10 0.4521 0. all vegetables. 0.011 0.011 ÿ0.042 0.033 0.03 0.274 0.525111 ÿ0.or C-rich fruits Other fruits All fruits All vegetables and fruit All vegetables and fruit (no potato) Bread.135 ÿ0.147 ÿ0.077 ÿ0.058 0. 2 Reference category: men.086 ÿ0. jams Nuts Age1 0.259 0.001 0.061 ÿ0.155 ÿ0.064 ÿ0.127 0.400 ÿ0. modified dairy.498111 0.008 ÿ0.5921 0.139 ÿ0.065 0.129 0.230 0.450 0. The presence of any medical condition was significantly associated with increased difference in reported intakes of meat.077 ÿ0.200 0.7861 0.15 0.20 0. Smaller differences in estimated intakes of other vegetables.

with agreement for these food groups little different from that expected in random allocation. The correlation coefficients in our study were better than those reported for Chinese miners by Forman et al. sugars/sweets/jams. the results from other studies are more varied.. Finland (23) (pregnant women). Sex was significant for 17 of 37 food groups. after some small adaptations. For the other foods with poor agreement.e.23.20% of the variance for fats and oils. rice. There was no clear pattern concerning which food groups had a negative. (29) compared the validity in estimation of nutrient intakes for 3 FFQ formats and found that the Willett instrument tends to underestimate the nutrient intakes of men and overestimate Downloaded from jn.3 in our study were for poultry. BMI was not associated with intake for any food group. Key findings were that on average the FFQ overestimated the intake of most foods. to obtain valid estimates of food intake in an Australian adult population. . but the majority of associations was positive. but poor agreement and gross misclassification were not as extensive as suggested by the correlation results. DISCUSSION We tested whether a widely used questionnaire that was developed for American nurses could be used. after inclusion of energy intake in the model (mean energy estimated from WFR and FFQ). and France (21) (nursing staff). but a general trend for overestimation. seafood. all vegetables (no potato) and all vegetables and fruit (not shown). the effect of sex on measures of agreement is illustrated with the observation above that FFQ validations in single-sex studies tend to have higher correlations. and unlikely to be correlated (correlated errors can lead to overestimation of validity by some measures). with large overestimations for intake of the fruit and vegetable groups. Agreement in the estimation of absolute intake was assessed in the limits of agreement analysis. Mali (17). other vegetables.24). explaining 20% or more of the variance for 12 of the 37 food groups. a major advantage is that the main sources of errors are different for the 2 methods.4 in the United States (16) and Guatemala (18). We reported previously that there were sex differences in the extent of underreporting of energy intakes for FFQ and WFR in this study group (7). which was also reported in other validation studies (17.2 times that observed for women using the FFQ. Bingham et al. and Finland (23) (pregnant women). all vegetables. sex. and all cereals and products. the extent of underreporting was highest among women using the WFR. although this was not monotonic for all food groups. (26). The poorest agreement was observed for eggs. supplement usage. or no association with magnitude of intake. the WFR is still regarded as the method of choice to use as the reference method in validation studies of this type. R2 ranged from as low as 3% up to 37%. The poor agreement found for vegetables. explaining that misreporting of vegetables can occur for a number of reasons including double counting of items and social desirability bias. We expanded a straightforward evaluation of the questionnaire’s validity in this setting with a more in-depth investigation of the characteristics of participants that are associated with the differences between the FFQ and the reference method. and Shanghai/ China (24) (women). and particularly that the treatment of portion sizes in data collection contributes to both the sex differences observed between FFQ and WFR and the underreporting. For example. Using cut-off values based on the ratio of energy intake to basal metabolic rate. with the exception of the United States (Nurses’ Health Study). However. 2012 . The authors concluded that WFR ‘‘remain the most accurate measure of dietary intake. all samples included both men and women. and 12 having upper limits . This is reflected also at the individual level. medical condition. and Japan (19). Guatemala (18). In spite of recognized weaknesses. all vegetables and fruits.e. studies have tended to report higher correlations for eggs than we observed.’’ rice. such as for whole-meal bread. Compared with food-based validation studies in other countries.05. Subar et al. BMI was positively associated with differences for intake estimates of green leafy vegetables. Guatemala (18). positive. and age and school leaving age together explained .. Mali (17). rice. pasta and noodles. China (20) (miners). results comparable to those reported in other food-based studies (17. snacks. The differences between the FFQ and WFR intake estimates varied significantly with magnitude of intake estimates for 14 food groups. None of the personal characteristics were significant in the models for those with lowest R2. pasta and noodles. poultry. as described by Goldberg et al. assessed as a food group or as individual foods.3 is a level at which attenuation is so severe that it would be difficult to detect associations (25). (25) noted in their review of validation studies for FFQ that mean correlations between FFQ and reference methods are usually lowest for vegetables. Importantly. Nevertheless.24). (27) showed in an evaluation of 7 dietary assessment methods in comparison with several biomarkers of dietary intake that WFR were consistently more strongly associated with the biological markers than were the other methods. but not by others. cruciferous vegetables. Japan (19).g. i. all vegetables (no potatoes). the pattern of results was similar to that of the correlations. medical condition. It was suggested that the FFQ format contributes to sex differences.20. an increasing difference between the 2 dietary methods with increasing age or BMI.0. Median WFR intake estimates generally showed increasing trends over increasing FFQ quartiles. The R2 statistic gives an indication of the extent to which the personal characteristics of participants explained the variation in difference in nutrient intakes between the FFQ and WFR. or for men using either method. there were very broad ranges for many by guest on July 1. and agreement within 1 quartile ranged from 60 to 93%. with 9 showing a negative association and 5 showing a positive association.464 MARKS ET AL. To our knowledge. Thus.’’ Cade et al. the higher correlation coefficients were generally observed in single-sex studies. coffee. 5 vegetable groups. A correlation coefficient of 0. eggs. and nuts. there is no other Australian study that presented food-based validity of FFQ to which we can compare our results. other vegetables. Studies on diet and disease associations frequently divide food intake into quartiles. (20). i. the strength of the correlations obtained in this study is comparable to studies in the United States (16). and nuts. was also reported by studies in the United States (16). ‘‘other vegetables. However. and breakfast cereals. and were generally not as good as correlations observed in France (21) (nursing staff). The correlations #0. . and age and school leaving age (3 food groups each). Sweden (22) (population based). measurement errors in the WFR likely contributed to the results observed in our study.. Sweden (22) (population based). Cade et al. in intakes of any food group. with 4 of the 37 food groups having 95% lower limits #0. Exact agreement in the allocation of food intakes by the FFQ and WFR ranged from 26 to 63%. and supplement usage (5 food groups each). all vegetables.nutrition. Nevertheless. e. The ranking of individuals in terms of quartiles was assessed by comparing the quartile allocation of intake estimates from FFQ with that of the WFR. Overall.19. (28) in a recent review of FFQ also suggest that WFR should be the first method of choice in validation studies.

Public Health Nutr. 2001. Australia: Australian Government Publishing Service. 1987. validation and utilisation. B.53: 53–60. Neuhouser by guest on July 1. Jebb SA. editors. Slesinski MJ. Witschi J. Stein AD. 1999. 1999. Stat Methods Med Res. Of the other food groups with particularly poor performance. In press. Rodriguez MM. National dietary survey of adults: 1983. No. et al.73:531–50. Measurement of fruit and vegetable consumption with diet questionnaires and implications for analyses and interpretation. 1985. 16.5:691–9. Warm DL.1. 12. Karppinen M. Shu XO. 1997 4. rice. various vegetable groups. Forman MR. Validity and reproducibility of a food frequency questionnaire for pregnant Finnish women. and all cereals and products explained $25% of the variation in difference between FFQ and WFR. Speizer FE. 9. Cade JE. 1991. Public Health Nutr. Kipnis V. Johansson I. New York:Oxford University Press. Omenn GS. Comparison of dietary assessment methods in a southern French population: use of weighed records. Margetts BM. Hart V. Rosner B. the presence of a medical condition and dietary supplement intake were also associated for some food groups. Bates C. for which the performance of the FFQ is otherwise poor. Design concepts in nutritional epidemiology. Parsons P. 2nd ed. Relative validity of a food frequency questionnaire among tin miners in China: 1992/93 and 1995/96 diet validation studies. Day NE. 1998. Cade J. Parsons P. Green A. Williams G. Derivation of cut-off limits to identify under-recording. Hartman AM. Public Health Nutr. Measuring agreement in method comparison studies.12:350–8. Eur J Clin Nutr. Burley V. Watanabe Y. 14.. SAS Institute Inc. Thurnham D. 1996. Solvoll K. Willett WC. 2005. McNutt S. The significant association between personal characteristics and difference for most food groups raises the possibility of differential bias and misclassification. seafood. 3. Leslie D. Am J Epidemiol.nutrition. Torun B. 5:567–87. Sampson L. Qiao YL. Green. Aust J Public Health. Knip M. 1989. 29. Foodfrequency questionnaires: a review of their design. Bingham SA. Reproducibility and validity of a semiquantitative food frequency questionnaire. Schroeder D. Liu D. Br J Nutr. Marks GC. Warm D. McIntosh A. King IB. Biessy C. Patterson RE. For all vegetable groups. Bland JM. Sex was a significant explanatory variable for most of these. SAS system for Windows. Validation of a quantitative food-frequency questionnaire for use in Western Mali. sex was most commonly associated with intake estimate errors for food groups. Ludbrook J. et al. 2. Khaw KT. Cassidy A. pasta and noodles. Bonifacj C. Compliance with dietary goals in a Queensland community. 11. Stampfer MJ. Nelson M. and further that correlations in validity studies tended to be highest when subjects were able to describe their own portion sizes. This would also suggest that different subgroups of the study population may need different FFQ to accurately measure dietary intake.122:51–65. Maher M. Ross S. Willett. 1999. 28. Ohkubo T. Neale R. Department of Community Services and Health. Williams G. Reasons for this are not well established. Subar AF. Marks. 2001. The findings highlight the need to assess FFQ validity in a sample that is representative of the overall population in which the FFQ will be used. and The Nambour Study Group.50:1133–8. Tsubono Y. Hart V. Harvey P. Validity and reproducibility of the food frequency questionnaire used in the Shanghai Women’s Health Study. Runswick SA. as applied in this study. Daures JP. Midthune D. Eur J Clin Nutr.18:858–67. Diarra MM. It is widely acknowledged that a number of factors such as gender. Goodman GE. Nutritional epidemiology. In: Margetts BM. Canberra. 7. Validation and calibration of food-frequency questionnaire measurements in the Northern Sweden Health and Disease cohort.15:512–22.2:301–15. Validation of a semi-quantitative food-frequency questionnaire for use among adults in Guatemala. Nelson M. Bingham SA. 1989. Am J Epidemiol. Cole TJ. Radimer K. Green A.58:17–23. and utilization of FFQ.51:217–31. Hughes MC. Jin F. 19. Riboli E. Sampson L. Nebeling L. Hunter DJ. 1999–2001. Watanabe T. 8. (28) reported that in studies in which portion sizes are self-defined. The multivariable modeling in the limits of agreement analysis shows that the models for fats and oils. Xyris Software Diet 1: nutrient calculation software. The poor performance of vegetables across measures of agreement is a matter of particular concern because of our interest in examining their potential role in cancer etiology. Nakatsuka H. Food-based validation of a dietary questionnaire: the effects of weekto-week variation in food consumption. Keith S. Eur J Clin Nutr. Luben R. 1995. 2004. the model explained . Scali J. Hallmans G. and a reasonably large proportion of variation in difference is explained by the models. Gerber M. with a sample size that is large enough to assess differences among subgroups.2 times those found with WFR.08 to 0. They attribute it to the same portion sizes being assigned to men and women. Oxford: Oxford University Press. Leslie D. Javanainen J. age. Kawamura M. 2012 . Nutr Res Rev. 2002. 13. Cary.10% of the variation in difference. Zhang J. Torheim LE. 2001. and National Cancer Institute food frequency questionnaires: the Eating at America’s Table Study. 1994. 22. Kushi L. Cole TJ. Colditz GA.40. Hatloy A. Thornquist MD. Barikmo I. Prentice AM. Gaffney P. Erkkola M.8:135–60. Battistutta D. 25. 6. Statistical techniques for comparing measurers and methods of measurement: a critical review. Thompson FE. Rasanen L. et al. Lancet. Black AE. 27. Oshaug A. Public Health Nutr. Zheng W.29:527–36. Thompson RL.02. Release 8. et al. Marks G. These findings have important implications for modeling dietdisease relations. Battistutta. Thompson R. Takahashi N. Hennekens CH. Mendez H. 10. In a recent review of the design. Willett WC. Gao YT. The effect of personal characteristics on the validity of nutrient intake estimates using a food frequency questionnaire. Stampfer MJ. 2003. ranging from 0. 24. there tended to be differences in portion size between men and women. Comparative validation of the Block. Control Clin Trials. 2002. validation and utilisation of food-frequency questionnaires—a review. Am J Clin Nutr. Hurwitz P. and were generally stronger for fruits. Michels KB.161:987–94. 2nd ed. Marks GC. this remains to be confirmed by further investigation. Green A. Rosenfeld S. Am J Epidemiol. 18. 21. NC: SAS Institute. 2002. Willett WC. 2006. Development. Orrell E. Cancer Epidemiol Biomarkers Prev. Fruits and vegetables are associated with lower lung cancer risk only in the placebo arm of the beta-carotene and retinol efficacy trial (CARET). The Nambour Skin Cancer and Actinic Eye Disease Prevention Trial: design and baseline characteristics of participants. 1991. 1997. Clin Exp Pharmacol Physiol. Ashton. estimated-diet records and a food-frequency questionnaire. Version 3. Burley VJ. 26. Public Health Nutr.FACTORS ASSOCIATED WITH VALIDITY OF FFQ FOOD ESTIMATES 465 those of women. 20. This is consistent with other studies in which levels of agreement between the FFQ and other dietary assessment methods were generally found to be poor for vegetables and fruits (30). Welch AA. 17.154:1089–99. Nishino Y.16:277–81. Am J Epidemiol. Yang G. Issues in reproducibility and validity of dietary studies. Battistutta D. et al. and socioeconomic factors may be associated with the validity of dietary estimates (3). Daily sunscreen application and betacarotene supplementation in prevention of basal-cell and squamous-cell carcinomas of the skin: a randomised controlled trial. G. 15. This is the first study we know of that directly assessed the association between personal characteristics and measurement errors in FFQ food intake estimates. Salvini S. 2002. Altman DG.. One might expect this to be particularly appropriate for the vegetable groups.4:1267–77. Goldberg GR. Validation of a food-frequency questionnaire for cohort studies in rural Japan. 2 Nutrient intakes. Wen W.. validation. Diakite M. Downloaded from jn. Validation of weighed records and other methods of dietary assessment using the 24 h urine nitrogen technique and other biological markers. Int J Epidemiol. Highgate Hill. Yao SX. Queensland: Xyris Software (Australia) Pty Ltd. Public Health Nutr.45:569–81. Public Health Nutr. 1992. Aust J Nutr Diet.154:466–76. Adjusting intake estimates for these characteristics will improve the validity of the model. 5. Our study assessed the extent to which they affect measurement error in the Nambour study population. 23. Ogawa K. Of all the personal characteristics studied. A. Rosner B. The validation of dietary assessment. Correlations were modest. intakes were estimated by the FFQ to be . Cade et al. Murgatroyd PR.6:147–57. D. Block G. poultry. Welch A.17:5–22. 2003. Hirst L. Bain C. Gaffney P. ACKNOWLEDGMENTS Bronwyn Ashton contributed significantly to early work related to this study and Adrian Barnett was a discussant on methods of analysis. Virtanen SM. Kaaks R. 30. van der Pols JC. Black AE. 2004. LITERATURE CITED 1. Critical evaluation of energy intake data using fundamental principles of energy physiology: 1.5:487–96. Coward WA. Wikman A. Under-reporting of energy intake in two methods of dietary assessment in the Nambour Trial.354:723–9.