American Journal of Epidemiology Vol. 154, No.

12
Copyright © 2001 by the Johns Hopkins University Bloomberg School of Public Health Printed in U.S.A.
All rights reserved

Validation of FFQs by Using the Method of Triads Kabagambe et al.

Application of the Method of Triads to Evaluate the Performance of Food
Frequency Questionnaires and Biomarkers as Indicators of Long-term
Dietary Intake

Edmond K. Kabagambe,1 Ana Baylin,1 Damon A. Allan,1 Xinia Siles,2 Donna Spiegelman,3,4 and Hannia
Campos1,2

Downloaded from https://academic.oup.com/aje/article/154/12/1126/64377 by guest on 26 June 2021

Little is documented about the performance of the food frequency questionnaire (FFQ) in US minority groups
and in populations in developing countries. The authors applied a novel technique, the method of triads, to
assess the validity and reproducibility of the FFQ among Hispanics. The subjects were men (n = 78) and women
(n = 42) living in Costa Rica. Seven 24-hour dietary recalls and two FFQ interviews (12 months apart) were
conducted between 1995 and 1998 to estimate dietary intake during the past year. Plasma and adipose tissue
samples were collected from all subjects. Validity coefficients, which measure the correlation between observed
and “true” dietary intake, were also estimated. The median validity coefficients for tocopherols and carotenoids
estimated by dietary recall, the average of the two FFQs, and plasma were 0.71, 0.60, and 0.52, respectively.
Compared with adipose tissue, plasma was a superior biomarker for carotenoids and tocopherols. Adipose
tissue was a poor biomarker for saturated and monounsaturated fatty acids but performed well for
polyunsaturated fatty acids (validity coefficients, 0.45–1.01) and lycopene (validity coefficient, 0.51). This study
also showed that biomarkers did not perform better than the FFQ and that they should be used to complement
the FFQ rather than substitute for it. Am J Epidemiol 2001;154:1126–35.

biological markers; diet; epidemiologic methods; Hispanic Americans; methods; questionnaires; recall;
reproducibility of results

High intake of certain nutrients and total energy is a rec-
ognized risk factor for conditions such as diabetes, stroke,
and cardiovascular disease (1, 2), whereas intake of others
(e.g., tocopherols, carotenoids, folic acid, vitamin C, and
dietary fiber) is thought to be protective (3–5). Assessment
of long-term dietary intake, the essential exposure factor for
disease, is complicated by a lack of accurate methods (6, 7).
Traditionally, intake of micro- and macronutrients has been
estimated by the use of semiquantitative food frequency
questionnaires (FFQs), dietary records, and dietary recalls
(7, 8). Use of biomarkers of dietary intake is a recent addi-
tion to these techniques (3, 9, 10). Assessment of true long-
term dietary intake is fraught with a number of problems
Received for publication February 13, 2001, and accepted for
publication September 13, 2001.
Abbreviations: FFQ, food frequency questionnaire; USDA, US
Department of Agriculture; VC, validity coefficient.
1
Department of Nutrition, Harvard School of Public Health,
Harvard University, Boston, MA.
2 lem attending this approach is that the same factors that
Salud Coronaria, Instituto de Investigaciones en Salud,
Universidad de Costa Rica, San Pedro, Costa Rica.
3
Department of Epidemiology, Harvard School of Public Health,
Harvard University, Boston, MA.
4
Department of Biostatistics, Harvard School of Public Health,
Harvard University, Boston, MA.
Correspondence to Dr. Hannia Campos, Department of Nutrition,
Room 353A, Harvard School of Public Health, Harvard University,
Boston, MA 02115 (e-mail: hcampos@hsph.harvard.edu).
(11–13), and biomarkers may not be better than traditional
methods because they can be affected by factors other than
diet and are not available for all nutrients (10, 14).
Regardless of the technique used, actual food intake and its
reporting are affected by factors such as culture, education,
type of food consumed, age, gender, and body weight (11,
12, 15–17). Furthermore, menus and recipes tend to vary
across sociocultural groups.
Although a questionnaire is an excellent tool for assessing
long-term dietary intake, an FFQ valid for one population
may be invalid when applied to another. For this reason, a
validation study is necessary whenever an FFQ is used. The
performance of FFQs in describing food consumption pat-
terns (18) and intake of individual nutrients has been evalu-
ated extensively in European and US Caucasian populations
(8, 14, 19). Little is known about the performance of the
FFQ in other ethnic groups, particularly in developing coun-
tries and in US minority populations. In addition, FFQs have
been validated mainly against one dietary assessment
method such as dietary recalls or dietary records. The prob-
affect the reference method may also affect the FFQ. This
problem would make it impossible to assume independent
random errors in the two methods, which could lead to over-
estimation of the correlation between the reference method
and the FFQ (20).
Even if this error did not exist, a bias leading to underes-
timation of this correlation could occur as a result of corre-

1126

lated random errors in repeated measurements when the ref-
erence method is used (14, 20). Furthermore, in most FFQ
evaluation studies, additional information on biomarkers of
intake is usually not available to augment data from the ref-
erence method. When such information is available, Kaaks
(6) has recommended that the correlation of the estimate by
using the dietary assessment method and a person’s “true”
long-term intake (referred to as the validity coefficient
(VC)) be estimated from three pairwise correlations
between the FFQ, the reference method, and the biomarker
(figure 1). The technique for this estimation, named the
“method of triads,” is an application of a factor analysis
model and does correct for bias due to correlated errors in
the repeated measurements from the reference method (6,
20). Briefly, if Q, R, and M are the measurements from the
FFQ, the reference method, and the biomarker, respectively,
VCs can be estimated as follows (6, 14, 20):
VCQT  211rQR  rQM 2>rRM 2, VCRT  211rQR  rRM 2>rQM 2,
and VCMT  211rQM  rRM 2>rQR 2,
where r is the correlation corrected for within-subject varia-
tion and T is the true, but unknown long-term dietary intake.
This technique assumes that random errors in the dietary
assessment methods are uncorrelated and that positive linear
correlations exist between estimates obtained by the dietary
assessment methods and true intake (6, 14, 20). Existence of
negative correlations and high random variation in the sample
correlation could cause VCs to be inestimable or >1, a condi-
tion referred to as Heywood case (6). The authors of the
method of triads have applied it in a few validation studies
(14, 20), but we are not aware of other studies in which this
novel approach has been used.
Here, we apply the method of triads to validate the FFQ
and the biomarkers used in our ongoing study of myocardial
FIGURE 1. Diagrammatic representation of the method of triads used to estimate the correlation (?) between true long-term nutrient intake
and intake estimated using dietary assessment methods. Modified from Ocké and Kaaks (20).
Am J Epidemiol Vol. 154, No. 12, 2001
Validation of FFQs by Using the Method of Triads 1127
infarction and gene-diet interaction in Costa Rica. We also
report the reproducibility of the FFQ in assessing dietary
intake of selected nutrients in Costa Rica.
MATERIALS AND METHODS
Study population and design
The subjects in this 1995–1998 validation study were
recruited from the control group of an ongoing case-control
study of myocardial infarction in Costa Rica. The 78 men and
42 women in this study are of Mestizo background and cul-
turally are Hispanic Americans. All subjects had no history of
myocardial infarction or physical or mental disabilities prior
Downloaded from https://academic.oup.com/aje/article/154/12/1126/64377 by guest on 26 June 2021
to enrolling in the study. The study was approved by the Use
of Human Subjects in Research committees at the Harvard
School of Public Health (Boston, Massachusetts) and the
University of Costa Rica (San Pedro, Costa Rica). Subjects
randomly selected from the control group were asked to par-
ticipate in the FFQ validation study. All subjects were visited
at their homes by a dietitian and were interviewed about their
dietary intake during the past year (FFQ1). In addition, seven
24-hour dietary recall assessments were conducted at each
participant’s home and served as a reference dietary assess-
ment method. To account for seasonal variations in nutrient
intake, the dietary recall assessments for each subject were
distributed across 7 months of the year. For each subject, the
months and days on which dietary recall assessments were
conducted were chosen at random. All 7 days of the week
were represented, except for a few subjects (n  9) who could
not be contacted on certain days. All dietary recall assess-
ments were completed within 1 year of the FFQ1 interview.
To assess the reproducibility of FFQ1, we conducted a
second interview (FFQ2) among all validation study sub-
jects about a year later. The same 135-item FFQ (modified
from that of Willett et al. (19)) was used for both interviews.
Each subject provided a plasma sample (for assessment of
tocopherol and carotenoid intake) and an adipose tissue
1128 Kabagambe et al.
biopsy (for assessment of tocopherol, carotenoid, and fatty
acid intake).
Intake of energy and nutrients was computed by multi-
plying the consumption frequency of each food by the nutri-
ent content of the specific portion; food composition values
from the US Department of Agriculture (USDA) database
(19, 21) and data from manufacturers and published reports
were used. Carotenoid intake (including α-carotene, β-
carotene, β-cryptoxanthin, lycopene, and zeaxanthin 
lutein) was calculated by using the new carotenoid database
that includes more than 2,400 fruits, vegetables, and
selected multicomponent foods (22, 23). The carotenoid
content of tomato-based products was updated with values
from the USDA, which were derived recently by reverse-
phase high-performance liquid chromatography (24).
“Food Processor” nutrient analysis software (ESHA
Research, Salem, Oregon), which is based on USDA food
composition data, was used to compute nutrient intakes
from dietary recalls. This database was enhanced with nutri-
ent composition data for foods and recipes specific to Costa
Rica.
Sample collection and biochemical analyses
All biologic specimens were collected at the subjects’
homes on the morning after an overnight fast. A subcuta-
neous adipose tissue biopsy was collected from the upper
buttock with a 16-gauge needle and disposable syringe, as
described previously (25). Blood samples were collected
into tubes containing 0.1 percent ethylenediaminetetraacetic
acid (EDTA). Both samples were stored in a cooler with ice
packs at 4˚C and were transported to the fieldwork station
within 4 hours. Blood was then centrifuged at 2,500 rpm for
20 minutes at 4˚C to obtain plasma. Plasma and adipose tis-
sue samples were stored at –80˚C and, within 6 months of
collection, were transported over dry ice to the Harvard
School of Public Health for analysis.
Concentrations of α- and γ-tocopherol were measured
with a dual wavelength Hitachi high-performance liquid
chromatography system and data station (26). The L-4200
detector was set at a wavelength of 300 nm, and injections
were performed by a programmable AS-4000 Auto-
Sampler. One large sample of adipose tissue was maintained
as a stock for quality assurance in subsequent runs. A por-
tion of the same weight (20–60 mg) as the study sample was
taken from the core of the quality assurance sample and was
included in each run to adjust for between-run variation.
Every run also included a pooled plasma sample. The coef-
ficients of variation for plasma α- and γ-tocopherol were 9.4
and 10.1 percent, respectively. The respective coefficients of
variation were 20.4 and 17.8 percent for adipose tissue α-
and γ-tocopherol. The minimum detection limits in a 30-mg
adipose tissue sample were 1.189 and 0.718 µg/g for α- and
γ-tocopherol, respectively, while they ranged from 0.023 to
0.035 µg/g for the carotenoids. Samples (n ≤ 4) below the
detection limits were set to missing. The same procedure
was used to measure concentrations of α-carotene, β-
carotene, β-cryptoxanthin, lycopene, and zeaxanthin 
lutein in adipose tissue and plasma samples, except that the
L-4200 detector was set at a wavelength of 445 nm. Lutein
and zeaxanthin were grouped because they coelute on the
chromatogram. The between-run coefficients of variation
for α-carotene, β-carotene, β-cryptoxanthin, lycopene, and
zeaxanthin + lutein were 7.2, 7.5, 12.6, 8.1, and 8.6 percent,
respectively, in plasma samples and 18.6, 21.8, 26.4, 16.7,
and 19.6 percent in adipose tissue samples.
The proportion of total fat in adipose tissue contributed
by each fatty acid was determined by gas-liquid chromatog-
raphy (27). In brief, about 2 mg of adipose tissue was added
to a hexane:isopropanol (3:2) mixture. To 200 µl of this
mixture was added methanol and acetyl chloride, which
formed methyl esters. After esterification, the adipose tissue
was evaporated and the fatty acid methyl esters were redis-
Downloaded from https://academic.oup.com/aje/article/154/12/1126/64377 by guest on 26 June 2021
solved in iso-octane. The fatty acid concentrations were
determined according to the following specifications: fused-
silica capillary cis/trans column, SP2560, 100 m × 250 µm
internal diameter × 0.20 µm film (Supelco, Belefonte,
Pennsylvania); splitless injection port at 240˚C; hydrogen
carrier gas at 1.3 ml/minute, constant flow; Hewlett-Packard
(now Agilent, Palo Alto, California) Model GC 6890 flame
ionization detector (FID) gas chromatograph with 7673
autosampler injector; 1 µl of sample injected; and a temper-
ature program of 90–170˚C at 10˚/minute, 170˚C for 5 min-
utes, 170–175˚C at 5˚/minute, 175–185˚C at 2˚/minute,
185–190˚C at 1˚/minute, 190–210˚C at 5˚/minute, 210˚C for
5 minutes, 210–250˚C at 5˚/minute, and 250˚C for 10 min-
utes. We used known standards (NuCheck Prep, Elysium,
Minnesota) and Agilent Technologies ChemStation A.08.03
software to identify peak retention times and the relative
quantity of each fatty acid. For the 16 blind duplicates
included with the test samples, the coefficients of variation
were 24.9 percent for myristic, 5.4 percent for palmitic, 14.0
percent for palmitoleic, 16.0 percent for stearic, 3.0 percent
for oleic, 5.5 percent for linoleic, and 8.5 percent for
linolenic acids.
Plasma triacylglycerol, cholesterol, and high density
lipoprotein cholesterol concentrations were measured with
enzymatic reagents (Boehringer-Mannheim, Indianapolis,
Indiana) and a Cobas Mira Plus autoanalyzer (Roche
Diagnostics, Somerville, New Jersey). Cholesterol measure-
ments were standardized to guidelines from the Centers for
Disease Control (Atlanta, Georgia) and the National Heart,
Lung, and Blood Institute (Bethesda, Maryland) (28, 29).
Statistical analysis
SAS software (SAS Institute, Inc., Cary, North Carolina)
was used for all statistical analyses. The mean of seven
dietary recalls was computed and was compared with intakes
estimated from FFQs, plasma, and adipose tissue. We also
determined the mean of the two FFQ (average FFQ) assess-
ments in an attempt to obtain a better estimate of long-term
dietary intake (30), and we compared it with the mean of
dietary recalls. The significance of the differences between
the mean of dietary recalls and the average from the two
FFQs was assessed by using Wilcoxon’s signed rank test
(31). Descriptive statistics for the validation study population
and for the overall control population were computed.
Am J Epidemiol Vol. 154, No. 12, 2001
 Validation of FFQs by Using the Method of Triads 1129

Because there were high correlations between total energy
intake and individual nutrient intakes estimated by dietary
recalls and the FFQs, variables were adjusted for total energy
intake by regressing the nutrient against total energy intake,
as described previously (7, 30). Briefly, a nutrient was loge or
square-root transformed to improve normality. The trans-
formed nutrient was then regressed on total energy intake
obtained from the same dietary assessment method. The
regression coefficient, the intercept, and residuals were
obtained. Although by definition the residuals are indepen-
dent of the explanatory variable, some are negative and dif-
ficult to interpret. Therefore, we multiplied the regression
coefficient by the mean of energy intake for the whole study
population; to this product we added the intercept and the
der, and county of residence in Costa Rica) as numeric
regressors and used them as partial variables in the CORR
procedure of SAS (32). Because of the established negative
relation between smoking and amounts of carotenoids and
tocopherols in the plasma and diets of smokers (30, 33), and
the observed associations between these nutrients, smoking
status, and body mass index in this study, we used regression
methods to compute Pearson’s partial correlation coeffi-
cients for these nutrients (32).
Random within-subject error in estimating the intake of a
nutrient tends to attenuate correlations toward zero (7).
Therefore, we performed analysis of variance on the data
from dietary recalls to estimate within- and between-subject
variation and used the ratio of the two variances to correct

Downloaded from https://academic.oup.com/aje/article/154/12/1126/64377 by guest on 26 June 2021

residual (7). The result was back-transformed to obtain
energy-adjusted nutrient intake for each subject. Plasma
tocopherol and carotenoid concentrations correlated highly
with plasma triacylglycerol and cholesterol, respectively.
Accordingly, plasma tocopherol and carotenoid measure-
ments were adjusted for triacylglycerol and cholesterol con-
centrations by the methods used for energy adjustment. The
same approach was used to adjust adipose tissue tocopherols
and carotenoids for the quantity of the adipose biopsy.
Because some individual fatty acids are somewhat related to
the total area of the chromatogram, we adjusted adipose tis-
sue fatty acids for the total area of the lipid chromatogram by
using the method described for energy adjustment.
All correlations among dietary assessment methods were
computed from intakes adjusted for total energy intake,
plasma triacylglycerol, cholesterol, quantity of adipose tis-
sue, or area of the chromatogram. Since these data are from
a matched case-control study, we computed Pearson’s par-
tial correlation coefficients on normalized nutrient intake
estimates by coding the matching variables (i.e., age, gen-
correlations for day-to-day variation (7). Briefly, the follow-
ing formula was used: Cr  r211  1λx>nx 22 , where Cr is
the corrected correlation, r is the observed correlation, λx is
the ratio of within to between subject variation, and nx is the
number of dietary recall replicates for each subject. Only
subjects whose data were not missing were included in the
computation of λx, leaving at least 110 subjects for whom
data on each of the nutrients were complete.
We used the method of triads (6, 14, 20) to estimate the
VCs between “true” nutrient intakes and those estimated
from using dietary assessment methods. Because data on
biomarkers were available for carotenoids, tocopherols,
and fatty acids only, VCs were computed for these nutri-
ents only. The correlation between the FFQ and the bio-
marker, and the deattenuated correlations of the dietary
recalls with the biomarker and the FFQ, were computed
and were used to estimate the VCs (14, 20). Three mea-
surements, a biomarker (e.g., plasma or adipose tissue), the
mean of dietary recalls, and the estimate from the FFQ

TABLE 1. Characteristics of the 120 controls interviewed to validate the semiquantitative food
frequency questionnaire used in a study of myocardial infarction, Costa Rica, 1995–1998
% or mean (standard deviation)
Variable (unit) Validation study population Total control population
(n = 120) (n = 503)

Females in the population 35 27

Current smokers* 28 27
Age (years) 59 (10) 57 (11)
Weight (kg) 67 (13) 69 (13)
Height (m) 1.63 (0.09) 1.63 (0.09)
Body mass index (kg/m2) 25 (4) 26 (4)
Waist-to-hip ratio 0.93 (0.08) 0.93 (0.08)
Physical activity† 1.75 (0.87) 1.81 (0.91)
Plasma
Total cholesterol (mg/dl) 201 (40) 201 (40)
HDL‡ cholesterol (mg/dl) 43 (12) 42 (12)
LDL‡ cholesterol (mg/dl) 120 (37) 120 (35)
Triacylglycerol (mg/dl) 212 (122) 209 (114)
Retinol concentration (mg/l) 558 (190) 564 (180)
Adipose tissue
Retinol concentration (mg/g) 0.80 (0.52) 0.81 (0.54)
* Defined as ≥1 cigarette/day.
† Expressed as kcal/kg per waking hour.
‡ HDL, high density lipoprotein; LDL, low density lipoprotein.

Am J Epidemiol Vol. 154, No. 12, 2001

1130 Kabagambe et al.

(e.g., Q1 or Q2 or their average, Q3) were used to estimate RESULTS

the VCs (figure 1). The VCs obtained through the various
dietary assessment methods were then compared by
assuming that they should be about equal if the assessment
method was reproducible (e.g., FFQ1 vs. FFQ2) or if all
methods were relatively valid. We used bootstrap sampling
to construct confidence intervals around the VCs (14, 20).
A total of 1,000 bootstrap samples of equal size (n  120)
were obtained from 120 study subjects by random sam-
pling with replacement. For each bootstrap sample, VCs
for the FFQ, dietary recalls, and the biomarker were
obtained and were merged into a single data set. The UNI-
VARIATE procedure of SAS software, which uses the
empirical distribution function (32), was applied to com-
Descriptive statistics
The subjects in this validation study were similar to the
overall control group with respect to most variables, includ-
ing biologic parameters (table 1). However, this study
included a larger proportion of women (35 percent) than did
the total control population (27 percent). The mean absolute
intakes estimated by dietary recalls and the FFQs are shown
in table 2. For most nutrients, intake estimates obtained by
FFQ tended to be higher than those obtained by dietary
recall. The average FFQ values also tended to be higher than
the mean of the dietary recalls, and the differences were sig-
nificant (p < 0.05) for all nutrients but vitamin K, iron, and

Downloaded from https://academic.oup.com/aje/article/154/12/1126/64377 by guest on 26 June 2021

pute the 5th and 95th percentiles that we used as the non- caffeine. On average, the FFQ overestimated daily energy
parametric confidence interval for the VC. intake by 477 kcal.

TABLE 2. Comparison of mean daily energy and absolute nutrient intakes estimated by using a
semiquantitative food frequency questionnaire and seven 24-hour dietary recalls for 120 controls
in a study of myocardial infarction, Costa Rica, 1995–1998
Mean (standard deviation)
Variable (unit) Mean of seven
FFQ1* FFQ2 Average FFQ
24-hour recalls

Total energy (kcal) 1,872 (585) 2,370 (728) 2,328 (724) 2,349 (611)
Protein (g) 65 (22) 76 (24) 75 (25) 76 (21)
Total fat (g) 59 (24) 86 (30) 92 (40) 89 (29)
Saturated fat (g) 20 (9) 30 (10) 32 (13) 31 (10)
Monounsaturated fatty acids (g) 22 (10) 32 (14) 34 (17) 34 (15)
Polyunsaturated fatty acids (g) 10 (4) 15 (6) 15 (6) 15 (5)
Total carbohydrate (g) 275 (88) 332 (118) 312 (101) 322 (92)
Sucrose (g) 101 (43) 69 (34) 61 (31) 65 (28)
Total fiber (g) 20 (8) 26 (11) 24 (9) 25 (8)
Cholesterol (mg) 231 (111) 272 (164) 279 (194) 276 (152)
Vitamin B6 (mg) 1.5 (0.6) 2.4 (1.0) 2.3 (0.9) 2.3 (0.8)
Vitamin B12 (mg) 3.6 (3.6) 5.3 (3.9) 4.6 (3.2) 4.9 (3.1)
Vitamin C (mg) 143 (80) 260 (144) 213 (116) 237 (114)
Vitamin K (mg) 119 (115) 104 (77) 93 (69) 99 (66)†
Folic acid (mg) 265 (100) 377 (139) 345 (131) 361 (113)
Calcium (mg) 636 (258) 891 (384) 820 (368) 856 (319)
Iron (mg) 15 (5) 15 (4) 15 (5) 15 (4)†
Sodium (mg) 2,267 (911) 2,054 (619) 1,892 (673) 1,973 (544)
Zinc (mg) 8.5 (3.0) 9.5 (3.2) 9.6 (3.5) 9.6 (2.9)
Caffeine (mg) 303 (173) 342 (182) 295 (167) 319 (162)†
Myristic (14:0) acid (g) 1.4 (0.8) 1.9 (1.0) 1.9 (1.2) 1.9 (1.0)
Palmitic (16:0) acid (g) 11 (6) 20 (6) 21 (7) 20 (6)
Stearic (18:0) acid (g) 3.9 (1.9) 6.0 (2.3) 6.3 (2.9) 6.1 (2.3)
Palmitoleic (16:1) acid (g) 0.95 (0.52) 1.3 (0.6) 1.3 (0.7) 1.3 (0.6)
Oleic (18:1) acid (g) 17 (8) 30 (13) 32 (16) 31 (13)
Linoleic (18:2) acid (g) 7.5 (3.0) 13 (5) 13 (6) 13 (5)
Linolenic (18:3) acid (g) 0.88 (0.40) 1.2 (0.5) 1.3 (0.5) 1.2 (0.4)
a-Tocopherol (mg) 3.0 (1.2) 9.0 (4.1) 9.5 (5.6) 9.3 (4.2)
g-Tocopherol (mg) 3.0 (1.1) 15 (8) 15 (8) 15 (7)
a-Carotene (mg) 118 (70) 640 (792) 459 (442) 550 (547)
b-Carotene (mg) 596 (351) 4,239 (3,344) 3,604 (2,283) 3,921 (2,442)
b-Cryptoxanthin (mg) 262 (172) 610 (603) 457 (498) 533 (482)
Lycopene (mg) 916 (798) 5,180 (3,709) 5,249 (3,637) 5,215 (3,108)
Zeaxanthin + lutein (mg) 599 (292) 2,922 (2,892) 2,423 (2,265) 2,672 (2,308)

* FFQ, food frequency questionnaire.

† Not significantly different (p > 0.05) from the mean of seven assessments by 24-hour dietary recall.

Am J Epidemiol Vol. 154, No. 12, 2001

 Validation of FFQs by Using the Method of Triads 1131

TABLE 3. Pearson’s partial correlation coefficients* for nutrients estimated from two food frequency
questionnaires and seven 24-hour dietary recalls for 120 subjects from Costa Rica, 1995–1998

FFQ1† FFQ2 Average FFQ FFQ1

vs. vs. vs. vs.
Nutrient dietary recalls dietary recalls dietary recalls FFQ2
r† Cr† r Cr r Cr r
Protein 0.31 0.37 0.37 0.44 0.39 0.46 0.54
Total fat 0.55 0.69 0.54 0.68 0.59 0.74 0.46
Saturated fat 0.47 0.58 0.57 0.70 0.58 0.71 0.60
Monounsaturated fatty acids 0.45 0.58 0.49 0.64 0.49 0.64 0.47
Polyunsaturated fatty acids 0.55 0.77 0.46 0.64 0.54 0.75 0.59
Total carbohydrate 0.41 0.44 0.40 0.43 0.47 0.50 0.48
Sucrose 0.38 0.41 0.39 0.43 0.40 0.43 0.50
Total fiber 0.57 0.62 0.58 0.64 0.65 0.72 0.52

Downloaded from https://academic.oup.com/aje/article/154/12/1126/64377 by guest on 26 June 2021

Total cholesterol 0.44 0.52 0.38 0.45 0.44 0.53 0.53
Vitamin E 0.03‡ 0.03 0.18‡ 0.24 0.11‡ 0.15 0.66
Vitamin A 0.35 0.77 0.34 0.76 0.39 0.86 0.59
Vitamin B6 0.40 0.46 0.32 0.37 0.41 0.47 0.46
Vitamin B12 0.30 0.72 0.47 1.00§ 0.42 1.00§ 0.46
Vitamin C 0.57 0.64 0.60 0.68 0.63 0.71 0.62
Folic acid 0.45 0.50 0.54 0.59 0.55 0.60 0.51
Calcium 0.63 0.69 0.64 0.70 0.71 0.78 0.54
Iron 0.31 0.34 0.45 0.49 0.42 0.46 0.40
Sodium 0.20 0.25 0.13‡ 0.16 0.20 0.24 0.49
Zinc 0.30 0.36 0.32 0.38 0.32 0.38 0.55
Caffeine 0.72 0.75 0.75 0.78 0.80 0.83 0.77
Myristic (14:0) acid 0.49 0.58 0.60 0.72 0.59 0.70 0.51
Palmitic (16:0) acid 0.25 0.30 0.50 0.61 0.42 0.51 0.34
Stearic (18:0) acid 0.34 0.47 0.54 0.55 0.46 0.64 0.38
Palmitoleic (16:1) acid 0.31 0.50 0.34 0.75 0.37 0.59 0.53
Oleic (18:1) acid 0.24 0.33 0.42 0.57 0.37 0.51 0.33
Linoleic (18:2) acid 0.43 0.63 0.48 0.71 0.50 0.73 0.57
Linolenic (18:3) acid 0.35 0.51 0.36 0.53 0.40 0.58 0.39
a-Tocopherol¶ 0.37 0.43 0.39 0.45 0.41 0.48 0.64
g-Tocopherol¶ 0.15‡ 0.19 0.25 0.31 0.23 0.29 0.66
a-Carotene¶ 0.25 0.33 0.36 0.46 0.28 0.36 0.61
b-Carotene¶ 0.37 0.50 0.40 0.53 0.40 0.54 0.58
b-Cryptoxanthin¶ 0.27 0.46 0.25 0.42 0.28 0.47 0.65
Lycopene¶ 0.25 0.37 0.27 0.40 0.30 0.44 0.40
Zeaxanthin + lutein¶ 0.37 0.49 0.46 0.60 0.43 0.57 0.54
* All correlations are adjusted for total energy intake, gender, age, and county of residence.
† FFQ, food frequency questionnaire; r, Pearson’s partial correlation coefficient; Cr, r corrected for day-to-day
variation and, where applicable, used to compute validity coefficients.
‡ Not significant at p = 0.05; the rest of the noncorrected correlations are significant.
§ Corrected correlation exceeds 1 and thus was set to 1.00.
¶ Correlation also adjusted for smoking and body mass index by using regression methods.

Correlations plasma measurements correlated fairly well with estimates

Pearson’s partial correlation coefficients between the
mean of the dietary recalls and FFQs and between the two
FFQs are shown in table 3. It is notable that the correlations
between the mean of the dietary recalls with FFQ1 and those
with FFQ2 were similar for most nutrients. Given the error
inherent in the reporting of food intake, these correlations (r 
0.40) (6) show that the FFQ has good validity. Furthermore,
correlations between FFQ1 and FFQ2 were high, indicating
that the questionnaire has high reproducibility.
Correlations between nutrient concentrations measured
in biomarkers and those estimated by dietary recall and the
FFQ are presented in table 4. Except for α-tocopherol,
from dietary recalls. Overall, the correlations between adi-
pose tissue concentrations and traditional dietary assess-
ment methods were lower than those with plasma. Good
correlations with dietary recalls were observed for adipose
tissue β-carotene (r  0.43) and lycopene (r  0.42). It is
noteworthy that the correlations between measurements of
saturated fatty acids and monounsaturated fatty acids in
adipose tissue and the estimates obtained by dietary recall
and FFQ were low. However, the correlations between
measurements of adipose tissue polyunsaturated fatty
acids and those obtained by dietary recall and FFQ were
high.

Am J Epidemiol Vol. 154, No. 12, 2001

1132 Kabagambe et al.

TABLE 4. Pearson’s partial correlation coefficients* for carotenoids, tocopherols, and fatty acids
estimated from plasma, adipose tissue, seven 24-hour dietary recalls, and two semiquantitative food
frequency questionnaires for 120 subjects from Costa Rica, 1995–1998

Dietary recalls Average Adipose

Nutrient FFQ1† FFQ2
r† Cr† FFQ tissue

Plasma samples
a-Tocopherol‡ 0.18§ 0.21 0.03§ 0.04§ 0.06§ 0.41
g-Tocopherol‡ 0.24 0.30 0.28 0.30 0.32 0.45
a-Carotene‡ 0.25 0.33 0.32 0.28 0.36 0.01§
b-Carotene‡ 0.26 0.35 0.33 0.36 0.38 0.50
b-Cryptoxanthin‡ 0.25 0.43 0.53 0.55 0.58 0.49
Lycopene‡ 0.30 0.45 0.24 0.18§ 0.26 0.43
Zeaxanthin + lutein‡ 0.32 0.43 0.28 0.26 0.27 0.36
Adipose tissue samples

Downloaded from https://academic.oup.com/aje/article/154/12/1126/64377 by guest on 26 June 2021

a-Tocopherol‡ 0.15§ 0.17 0.15§ 0.09§ 0.13§
g-Tocopherol‡ 0.10§ 0.12 0.15§ 0.25 0.22
a-Carotene‡ 0.12§ 0.15 0.08§ 0.06§ 0.01§
b-Carotene‡ 0.32 0.43 0.21 0.11§ 0.16§
b-Cryptoxanthin‡ 0.12§ 0.21 0.29 0.33 0.33
Lycopene‡ 0.28 0.42 0.30 0.17§ 0.27
Zeaxanthin + lutein‡ 0.25 0.33 0.09§ 0.20 0.15§
Myristic (14:0) acid 0.01§ 0.01 0.00§ 0.04§ 0.02§
Palmitic (16:0) acid 0.00§ 0.01 0.06§ 0.06§ 0.07§
Stearic (18:0) acid 0.28 0.39 0.05§ 0.15§ 0.10§
Palmitoleic (16:1) acid 0.08§ 0.12 0.06§ 0.02§ 0.06§
Oleic (18:1) acid 0.25 0.34 0.07§ 0.03§ 0.07§
Linoleic (18:2) acid 0.37 0.55 0.51 0.55 0.59
Linolenic (18:3) acid 0.31 0.45 0.27 0.23 0.27

* All correlations, except those between plasma and adipose tissue, are adjusted for total energy intake,
gender, age, and county of residence.
† FFQ, food frequency questionnaire; r, Pearson’s partial correlation coefficient; Cr, r corrected for day-to-day
variation and used to compute validity coefficients.
‡ Correlation also adjusted for smoking and body mass index.
§ Not significant at p = 0.05; the rest of the noncorrected correlations are significant.

Validity coefficients
The VCs for each dietary assessment method are pre-
sented in tables 5 and 6. Although a few Heywood cases
were evident, especially when adipose tissue was used as a
biomarker, most VCs were within the range of 0–1. For
γ-tocopherol and carotenoids in plasma and for adipose tis-
sue polyunsaturated fatty acids, at least two of the three
VCs were similar, suggesting that at least two of the meth-
ods yielded similar measurements of the nutrient. When
the average FFQ was used together with the mean of
dietary recalls, plasma (VC, 0.16–0.72; median, 0.52) was
found to be a better biomarker for carotenoids and γ-
tocopherol than adipose tissue (VC, 0.05–0.51; median,
0.30). VCs for the average FFQ were similar to or higher
than those for plasma and adipose tissue biomarkers and
were in some cases (e.g., β-cryptoxanthin) higher than
those for dietary recall.
VCs for adipose tissue palmitic acid could not be esti-
mated because of a negative correlation (tables 4 and 6).
However, adipose tissue was a good biomarker for polyun-
saturated fatty acids; VCs ranged from 0.45 to 1.01. For
polyunsaturated fatty acids, the VCs from the FFQ were
always higher than those from adipose tissue and, for
linoleic acid, were similar to those from dietary recalls,
again suggesting that the three methods are comparable.
DISCUSSION
Unlike most validation studies, in which data from only
one FFQ interview and one reference method are available,
our study estimated nutrient intake by using three methods
(FFQ, dietary recalls, and biomarkers). The sample size of
120 subjects, coupled with seven dietary recall replicates per
subject, allowed for reasonable precision in estimating the
correlations (7). In addition, potential seasonal variation in
food intake, especially of fruits and vegetables, was consid-
ered in the study design. The analysis also produced a better
estimation of long-term intake because the two FFQs were
averaged.
In this study, the correlations between estimates obtained
by the FFQ and the dietary recall are similar to those
observed between FFQs and dietary records in other studies
(8, 19). The median of the correlations between the dietary
recalls and average FFQ measurements was 0.55, again
indicating good validity of the FFQ. Relative to the dietary
recalls, the FFQ also performed well in estimating intake of
fatty acids, irrespective of whether they were saturated or

Am J Epidemiol Vol. 154, No. 12, 2001

 Validation of FFQs by Using the Method of Triads 1133

TABLE 5. Validity coefficients (95% bootstrap confidence intervals) for carotenoids and tocopherols estimated from plasma,
adipose tissue, seven 24-hour dietary recalls, and two semiquantitative food frequency questionnaires for 120 subjects from
Costa Rica, 1995–1998

Variable* a-Tocopherol g-Tocopherol a-Carotene b-Carotene b-Cryptoxanthin Lycopene Zeaxanthin + lutein

Plasma as a biomarker
Q1†
DR† vs. T† 1.00 (0.39, 1.00) 0.44 (0.18, 0.74) 0.58 (0.23, 0.95) 0.74 (0.43, 1.00) 0.61 (0.41, 1.00) 0.84 (0.48, 1.00) 0.87 (0.74, 1.00)
Q1 vs. T 0.25 (0.15, 0.97) 0.42 (0.18, 0.85) 0.57 (0.29, 0.99) 0.68 (0.50, 0.95) 0.75 (0.30, 0.84) 0.44 (0.21, 0.59) 0.57 (0.31, 0.69)
PL† vs. T 0.12 (0.05, 0.38) 0.67 (0.33, 1.00) 0.56 (0.22, 0.72) 0.48 (0.31, 0.71) 0.70 (0.42, 0.93) 0.53 (0.35, 0.90) 0.49 (0.30, 0.73)
Q2†
DR vs. T 1.00 (0.41, 1.00) 0.55 (0.28, 0.78) 0.73 (0.39, 1.00) 0.72 (0.43, 0.97) 0.57 (0.34, 1.00) 0.98 (0.64, 1.00) 1.00 (0.79, 1.00)
Q2 vs. T 0.29 (0.16, 1.00) 0.56 (0.35, 0.94) 0.63 (0.34, 1.00) 0.74 (0.55, 1.00) 0.74 (0.34, 0.88) 0.40 (0.16, 0.60) 0.60 (0.38, 0.77)
PL vs. T 0.14 (0.06, 0.35) 0.54 (0.28, 0.80) 0.45 (0.12, 0.56) 0.49 (0.33, 0.66) 0.75 (0.55, 1.00) 0.45 (0.23, 0.73) 0.43 (0.30, 0.70)
Q3†
DR vs. T 1.00 (0.45, 1.00) 0.52 (0.26, 0.74) 0.57 (0.24, 0.91) 0.71 (0.43, 0.99) 0.59 (0.38, 1.00) 0.87 (0.57, 1.00) 0.95 (0.77, 1.00)
Q3 vs. T 0.37 (0.22, 1.00) 0.55 (0.31, 0.92) 0.63 (0.40, 1.00) 0.76 (0.55, 1.00) 0.80 (0.37, 0.88) 0.51 (0.28, 0.66) 0.60 (0.38, 0.73)

Downloaded from https://academic.oup.com/aje/article/154/12/1126/64377 by guest on 26 June 2021

PL vs. T 0.16 (0.07, 0.36) 0.57 (0.32, 0.83) 0.57 (0.22, 0.71) 0.50 (0.35, 0.69) 0.72 (0.51, 0.98) 0.52 (0.35, 0.81) 0.45 (0.32, 0.71)
Adipose tissue as a
biomarker
Q1
DR vs. T 0.71 (0.27, 1.00) 0.39 (0.09, 1.00) 0.78 (0.24, 1.00) 1.00 (0.76, 1.00) 0.57 (0.22, 1.00) 0.71 (0.36, 1.00) 1.00 (0.80, 1.00)
Q1 vs. T 0.61 (0.29, 1.00) 0.48 (0.14, 1.00) 0.42 (0.12, 1.00) 0.50 (0.22, 0.73) 0.80 (0.42, 1.00) 0.52 (0.30, 0.75) 0.37 (0.12, 0.59)
AD† vs. T 0.25 (0.06, 0.47) 0.31 (0.05, 0.76) 0.20 (0.05, 0.47) 0.42 (0.16, 0.63) 0.37 (0.12, 0.64) 0.58 (0.35, 0.91) 0.25 (0.09, 0.54)
Q2
DR vs. T 0.92 (0.35, 1.00) 0.39 (0.11, 0.75) NA† 1.00 (0.96, 1.00) 0.52 (0.18, 0.94) 1.00 (0.55, 1.00) 0.99 (0.66, 1.00)
Q2 vs. T 0.49 (0.23, 1.00) 0.80 (0.44, 1.00) NA 0.38 (0.13, 0.61) 0.81 (0.39, 1.00) 0.40 (0.17, 0.63) 0.61 (0.34, 0.86)
AD vs. T 0.19 (0.04, 0.42) 0.31 (0.08, 0.54) NA 0.30 (0.10, 0.52) 0.40 (0.15, 0.70) 0.42 (0.17, 0.69) 0.33 (0.16, 0.57)
Q3
DR vs. T 0.81 (0.36, 1.00) 0.40 (0.11, 0.82) 1.00 (0.23, 1.00) 1.00 (0.89, 1.00) 0.55 (0.22, 1.00) 0.82 (0.43, 1.00) 1.00 (0.76, 1.00)
Q3 vs. T 0.59 (0.25, 1.00) 0.73 (0.33, 1.00) 0.12 (0.11, 1.00) 0.45 (0.16, 0.70) 0.86 (0.44, 1.00) 0.53 (0.31, 0.79) 0.51 (0.27, 0.70)
AD vs. T 0.21 (0.05, 0.43) 0.30 (0.08, 0.55) 0.05 (0.03, 0.41) 0.36 (0.12, 0.57) 0.38 (0.12, 0.67) 0.51 (0.26, 0.75) 0.30 (0.13, 0.56)

* Validity coefficients and confidence limits >1 (Heywood cases) were set to 1.00.
† Q1, food frequency questionnaire (FFQ) 1; DR, 24-hour dietary recalls; T, “true” dietary intake; PL, plasma; Q2, FFQ2; Q3, average FFQ;
AD, adipose tissue; NA, not applicable because of a negative correlation.

unsaturated. The highest correlations were found for linoleic
(r  0.73) and myristic (r  0.70) acids. The FFQ was
highly reproducible, as shown by Pearson’s partial correla-
tion coefficients ranging from 0.33 to 0.77. Intraclass corre-
lations were also high (data not shown).
In comparison with dietary recall estimates, lycopene
seemed to be equally well estimated from both plasma (r 
0.45) and adipose tissue (r  0.42). Plasma was also a good
biomarker for β-cryptoxanthin (r  0.43) and zeaxanthin 
lutein (r  0.43), and adipose tissue was a good biomarker
for β-carotene (r  0.43). The correlation between plasma
and adipose tissue α-tocopherol measurements (r  0.41)
was similar to the one (r  0.39) reported in the European
Community Multicenter Study on Antioxidants, Myocardial
Infarction and Cancer (EURAMIC) of the Breast (34).
However, our correlations between plasma and adipose tis-

TABLE 6. Validity coefficients (95% bootstrap confidence intervals) for selected fatty acids estimated from adipose tissue, seven
24-hour dietary recalls, and two semiquantitative food frequency questionnaires for 120 subjects from Costa Rica, 1995–1998

Adipose tissue used as a Fatty acid

biomarker and DR*, Q1*,
Q2*, or Q3* as methods Myristic (14:0) Palmitic (16:0) Stearic (18:0) Palmitoleic (16:1) Oleic (18:1) Linoleic (18:2) Linolenic (18:3)
for dietary assessment† acid acid acid acid acid acid acid
Q1
DR vs. T* 1.00‡ NA* 1.00 (0.88, 1.00) 0.99 (0.35, 1.00) 1.00 (0.46, 1.00) 0.82 (0.55, 1.00) 0.92 (0.56, 1.00)
Q1 vs. T 0.40‡ NA 0.24 (0.10, 0.55) 0.51 (0.18, 1.00) 0.26 (0.07, 0.51) 0.77 (0.60, 0.91) 0.55 (0.35, 0.81)
AD* vs. T 0.01‡ NA 0.21 (0.09, 0.50) 0.12 (0.03, 0.36) 0.27 (0.08, 0.62) 0.67 (0.53, 0.80) 0.49 (0.27, 0.72)
Q2†
DR vs. T 0.49 (0.31, 1.00) NA 1.00 (0.90, 1.00) 1.00 (0.34, 1.00) 1.00 (0.79, 1.00) 0.84 (0.57, 1.00) 1.00 (0.70, 1.00)
Q2 vs. T 1.00 (0.27, 1.00) NA 0.53 (0.26, 0.78) 0.33 (0.14, 1.00) 0.23 (0.10, 0.69) 0.84 (0.71, 1.00) 0.52.(0.34, 0.76)
AD vs. T 0.03 (0.02, 0.27) NA 0.28 (0.11, 0.45) 0.07 (0.02, 0.30) 0.14 (0.05, 0.38) 0.65 (0.50, 0.79) 0.45 (0.23, 0.64)
Q3
DR vs. T 0.64 (0.30, 1.00) NA 1.00 (0.93, 1.00) 1.00 (0.39, 1.00) 1.00 (0.70, 1.00) 0.82 (0.57, 1.00) 0.99 (0.66, 1.00)
Q3 vs. T 1.00 (0.23, 1.00) NA 0.40 (0.15, 0.71) 0.55 (0.21, 1.00) 0.31 (0.11, 0.62) 0.89 (0.77, 1.00) 0.59 (0.40, 0.81)
AD vs. T 0.02 (0.02, 0.26) NA 0.25 (0.09, 0.45) 0.11 (0.03, 0.35) 0.21 (0.06, 0.42) 0.67 (0.53, 0.78) 0.46 (0.25, 0.67)

* DR, 24-hour dietary recalls; Q1, food frequency questionnaire (FFQ) 1; Q2, FFQ2; Q3, average FFQ; T, “true” dietary intake; NA, not applic-
able because of a negative correlation; AD, adipose tissue.
† Validity coefficients and confidence limits >1 (Heywood cases) were set to 1.00.
‡ 38.8% of the bootstrap samples produced negative correlations leading to inaccurate confidence intervals, which therefore were omitted.

Am J Epidemiol Vol. 154, No. 12, 2001

1134 Kabagambe et al.
sue β-carotene (r  0.50) and lycopene (r  0.43) were
higher than those reported in the EURAMIC study (β-
carotene, r  0.39; lycopene, r  0.24).
Measurements of saturated fatty acids and monounsatu-
rated fatty acids from adipose tissue correlated poorly with
those estimated by the FFQ. This poor correlation may
reflect the fact that other than dietary sources, saturated fatty
acids can be synthesized endogenously; furthermore, other
physiologic processes may influence adipose tissue concen-
trations. Compared with plasma, higher coefficients of vari-
ation were observed for nutrients measured from adipose
tissue. We determined that this high variation is due to
nonuniform distribution of the nutrient in adipose tissue and
could in part explain the poor correlations between adipose
tissue measurements and those obtained by the FFQ and
dietary recalls. Estimates of saturated fatty acids from
dietary recalls correlated better with those from the FFQ
than the measurements from adipose tissue, suggesting that
although biomarkers are informative, there are nutrients for
which the FFQ remains a better dietary assessment method.
The VCs estimated for the FFQ were high, suggesting
that the FFQ is a valid dietary assessment method. We also
estimated VCs for the dietary recall and the biomarker.
Doing so enabled us to correct correlations for potential
attenuating effects of correlated errors in repeated dietary
recalls (6) and therefore to simultaneously compare the rel-
ative performance of the three methods. For some of the
nutrients from the average FFQ (e.g., β-cryptoxanthin and
lycopene), the VCs for the questionnaire were higher or sim-
ilar to those for the biomarker, suggesting that the FFQ and
the biomarker are comparable. Published data on VCs are
scarce, especially those from studies in which dietary recalls
are used as a reference method. In one study (n  61) in
which serum was used as a biomarker for β-carotene and
dietary recall as a reference dietary assessment method, the
VCs for the FFQ, dietary recall, and biomarker were 0.44,
0.58, and 0.32, respectively; in our study, the VCs were
0.76, 0.71, and 0.50, respectively (20). In another study
(n  87) in which dietary records were used as a reference
method (14), the VCs for the FFQ, dietary records, and
plasma β-carotene were 0.39, 0.52, and 0.85, respectively.
The disparity between estimates obtained from these studies
could be due to the different reference methods used, ran-
dom error resulting from differences in sample sizes, differ-
ences in the populations studied, or the structure and size of
the questionnaires used in these studies.
The VCs for FFQ1 and FFQ2 were very similar, further
suggesting that the FFQ is a reproducible dietary assessment
method. Random error could explain the Heywood cases
found in our analysis. Thus, sample sizes of >120 may be
necessary to stabilize the variances in validation studies.
Efforts to correct for random error and to constrain VC esti-
mates between 0 and 1 are needed to perfect this method.
We cannot rule out the potential for correlated errors
between the FFQ and dietary recalls; thus, the VCs reported
in this study could have been overestimated and should be
regarded as upper limits of the true VC.
This study has shown that the FFQ is a valid and repro-
ducible instrument for assessing dietary intake among
Hispanics in Costa Rica. Furthermore, the VCs suggest that
plasma could be used as a biomarker for γ-tocopherol, β-
carotene, β-cryptoxanthin, lycopene, and zeaxanthin  lutein,
while adipose tissue could be used as a biomarker for
lycopene and polyunsaturated fatty acids. This study also
shows that biomarkers did not perform better than the FFQ
and that they should be used to complement the FFQ rather
than substitute for it.
ACKNOWLEDGMENTS
This study was supported by National Institutes of Health
Downloaded from https://academic.oup.com/aje/article/154/12/1126/64377 by guest on 26 June 2021
grant HL49086.
The authors thank the field workers of Proyecto Salud
Coronaria, San José, Costa Rica, for their help with data
collection.
REFERENCES
1. Hannah JS, Howard BV. Dietary fats, insulin resistance, and
diabetes. J Cardiovasc Risk 1994;1:31–7.
2. He J, Ogden LG, Vupputuri S, et al. Dietary sodium intake and
subsequent risk of cardiovascular disease in overweight adults.
JAMA 1999;282:2027–34. (Comments published in JAMA
2000;283:1957–8).
3. Kardinaal AF, Kok FJ, Ringstad J, et al. Antioxidants in adipose
tissue and risk of myocardial infarction: the EURAMIC Study.
Lancet 1993;342:1379–84. (Comments published in Lancet
1994;343:595–6).
4. Werler MM, Shapiro S, Mitchell AA. Periconceptional folic acid
exposure and risk of occurrent neural tube defects. JAMA
1993;269:1257–61. (Comments published in JAMA 1993;269:
1292–3.)
5. Salmeron J, Ascherio A, Rimm EB, et al. Dietary fiber, glycemic
load, and risk of NIDDM in men. Diabetes Care 1997;20:
545–50.
6. Kaaks RJ. Biochemical markers as additional measurements in
studies of the accuracy of dietary questionnaire measurements:
conceptual issues. Am J Clin Nutr 1997;65:1232S–9S.
7. Willett WC. Nutritional epidemiology. New York, NY: Oxford
University Press, 1998.
8. Andersen LF, Solvoll K, Johansson LR, et al. Evaluation of a
food frequency questionnaire with weighed records, fatty
acids, and alpha-tocopherol in adipose tissue and serum. Am J
Epidemiol 1999;150:75–87.
9. Bingham SA, Day NE. Using biochemical markers to assess the
validity of prospective dietary assessment methods and the effect
of energy adjustment. Am J Clin Nutr 1997;65:1130S–7S.
10. Hunter D. Biochemical indicators of dietary intake. In: Willett
WC, ed. Nutritional epidemiology. New York, NY: Oxford
University Press, 1998:174–243.
11. Frank GC. Methodological issues regarding eating behavior of
high-risk adolescents. Ann N Y Acad Sci 1997;817:66–82.
12. Singhal S, Goyle A, Gupta R. Quantitative food frequency
questionnaire and assessment of dietary intake. Natl Med J
India 1998;11:268–75. (Comments published in Natl Med J
India 1999;12:138–9).
13. Bathalon GP, Tucker KL, Hays NP, et al. Psychological mea-
sures of eating behavior and the accuracy of 3 common dietary
assessment methods in healthy postmenopausal women. Am J
Clin Nutr 2000;71:739–45.
14. Daures JP, Gerber M, Scali J, et al. Validation of a food-
frequency questionnaire using multiple-day records and bio-
chemical markers: application of the triads method. J
Am J Epidemiol Vol. 154, No. 12, 2001

Epidemiol Biostat 2000;5:109–15.
15. Forsythe HE, Gage B. Use of a multicultural food-frequency
questionnaire with pregnant and lactating women. Am J Clin
Nutr 1994;59:203S–6S.
16. Bingham SA, Cassidy A, Cole TJ, et al. Validation of weighed
records and other methods of dietary assessment using the 24
h urine nitrogen technique and other biological markers. Br J
Nutr 1995;73:531–50.
17. Heerstrass DW, Ocke MC, Bueno-de-Mesquita HB, et al.
Underreporting of energy, protein and potassium intake in rela-
tion to body mass index. Int J Epidemiol 1998;27:186–93.
18. Hu FB, Rimm E, Smith-Warner SA, et al. Reproducibility and
validity of dietary patterns assessed with a food-frequency
questionnaire. Am J Clin Nutr 1999;69:243–9. (Comments
published in Am J Clin Nutr 1999;70:422).
19. Willett WC, Sampson L, Stampfer MJ, et al. Reproducibility
and validity of a semiquantitative food frequency question-
naire. Am J Epidemiol 1985;122:51–65.
20. Ocké MC, Kaaks RJ. Biochemical markers as additional mea-
surements in dietary validity studies: application of the method
of triads with examples from the European Prospective
Investigation into Cancer and Nutrition. Am J Clin Nutr 1997;
65:1240S–5S.
21. US Department of Agriculture. Composition of foods: raw,
processed and prepared. USDA nutrient database for standard
reference, release 13. Beltsville, MD: USDA, Agricultural
Research Service, Beltsville Human Nutrition Research
Center, Nutrient Data Laboratory, 1999:1–37.
22. Chug-Ahuja JK, Holden JM, Forman MR, et al. The develop-
ment and application of a carotenoid database for fruits, veg-
etables, and selected multicomponent foods. J Am Diet Assoc
1993;93:318–23.
23. Mangels AR, Holden JM, Beecher GR, et al. Carotenoid con-
tent of fruits and vegetables: an evaluation of analytic data. J
Am Diet Assoc 1993;93:284–96. (Erratum published in J Am
Diet Assoc 1993;93:527).
24. Tonucci LH, Holden JM, Beecher GR, et al. Carotenoid con-
Am J Epidemiol Vol. 154, No. 12, 2001
Validation of FFQs by Using the Method of Triads 1135
tent of thermally processed tomato-based food products. J
Agric Food Chem 1995;43:579–86.
25. Beynen AC, Katan MB. Rapid sampling and long-term storage
of subcutaneous adipose-tissue biopsies for determination of
fatty acid composition. Am J Clin Nutr 1985;42:317–22.
26. Hess D, Keller HE, Oberlin B, et al. Simultaneous determina-
tion of retinol, tocopherols, carotenes and lycopene in plasma
by means of high-performance liquid chromatography on
reversed phase. Int J Vitam Nutr Res 1991;61:232–8.
27. Lillington JM, Trafford DJ, Makin HL. A rapid and simple
method for the esterification of fatty acids and steroid carboxylic
acids prior to gas-liquid chromatography. Clin Chim Acta 1981;
111:91–8.
28. Myers GL, Cooper GR, Winn CL, et al. The Centers for
Disease Control–National Heart, Lung and Blood Institute
Lipid Standardization Program. An approach to accurate and
precise lipid measurements. Clin Lab Med 1989;9:105–35.
Downloaded from https://academic.oup.com/aje/article/154/12/1126/64377 by guest on 26 June 2021
29. Myers GL, Kimberly MM, Waymack PP, et al. A reference
method laboratory network for cholesterol: a model for stan-
dardization and improvement of clinical laboratory measure-
ments. Clin Chem 2000;46:1762–72.
30. Michaud DS, Giovannucci EL, Ascherio A, et al. Associations
of plasma carotenoid concentrations and dietary intake of spe-
cific carotenoids in samples of two prospective cohort studies
using a new carotenoid database. Cancer Epidemiol Biomarkers
Prev 1998;7:283–90.
31. Pagano M, Gauvreau K. Principles of biostatistics. 2nd ed.
Pacific Grove, CA: Duxbury Press, 2000.
32. SAS Institute, Inc. Selected SAS documentation for BIO 226ab:
applied longitudinal analysis (fall 1998). Cary, NC: SAS
Institute, Inc, 1998.
33. Wei W, Kim Y, Boudreau N. Association of smoking with
serum and dietary levels of antioxidants in adults: NHANES
III, 1988–1994. Am J Public Health 2001;91:258–64.
34. Su LC, Bui M, Kardinaal A, et al. Differences between plasma
and adipose tissue biomarkers of carotenoids and tocopherols.
Cancer Epidemiol Biomarkers Prev 1998;7:1043–8.