See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/51469065

Respondents' evaluation of the 24-h dietary recall method (EPIC-Soft) in the

EFCOVAL project

Article  in  European Journal of Clinical Nutrition · July 2011

DOI: 10.1038/ejcn.2011.85 · Source: PubMed

CITATIONS READS

17 378

17 authors, including:

Inge Huybrechts Jeanne H M de Vries

International Agency for Research on Cancer Wageningen University & Research
568 PUBLICATIONS   40,417 CITATIONS    285 PUBLICATIONS   12,254 CITATIONS   

SEE PROFILE SEE PROFILE

Willem De Keyzer Inger Theres Lillegaard Lillegaard

Hogeschool Gent Norwegian Scientific Committee for Food Safety
57 PUBLICATIONS   1,855 CITATIONS    87 PUBLICATIONS   2,430 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

ADEPS Project View project

Longitudinal project - Total Diet Study in the Czech Republic View project

All content following this page was uploaded by Sandra P Crispim on 29 October 2017.

The user has requested enhancement of the downloaded file.

 European Journal of Clinical Nutrition (2011) 65, S29–S37
& 2011 Macmillan Publishers Limited All rights reserved 0954-3007/11
www.nature.com/ejcn

ORIGINAL ARTICLE
Respondents’ evaluation of the 24-h dietary recall
method (EPIC-Soft) in the EFCOVAL Project
I Huybrechts1,2, A Geelen3, JH de Vries3, C Casagrande2, G Nicolas2, W De Keyzer1,4, IT Lillegaard5,
J Ruprich6, L Lafay7, EC Wilson- van den Hooven8, EM Niekerk8, I Margaritis7,9, I Rehurkova6,
SP Crispim2, H Freisling2, S De Henauw1,4 and N Slimani2, on behalf of the EFCOVAL Consortium

1
Department of Public Health, Faculty of Medicine and Health Sciences, Ghent University, Ghent, Belgium; 2Dietary Exposure
Assessment Group, International Agency for Research on Cancer, Lyon, France; 3Division of Human Nutrition, Wageningen University,
Wageningen, The Netherlands; 4Department of Nutrition and Dietetics, Faculty of Health Care, University College Ghent, Ghent,
Belgium; 5Department of Nutrition, Institute of Basic Medical Sciences, University of Oslo, Oslo, Norway; 6National Institute of Public
Health, Brno, Czech Republic; 7Food intake and Nutritional Epidemiology Unit, French agency for food, environmental and
occupational health safety (ANSES), Maisons-Alfort, France; 8National Institute for Public Health and the Environment (RIVM),
Bilthoven, The Netherlands and 9INSERM U907 (Laboratoire Dysfonctions Métaboliques et Diabetes, Mécanismes et Approches
Thérapeutiques), Faculte de Médecine, Université de Nice Sophia-Antipolis, Nice, France

Background: To improve participation rate, accuracy and respondents’ compliance, it is important to know the respondents’
viewpoint.
Objective: To evaluate respondents’ preferences and perception about the EPIC-Soft (the software developed to conduct 24-h
dietary recalls (24-HDRs) in the European Prospective Investigation into Cancer and Nutrition (EPIC) Study) 24-HDR interviews
and to compare these preferences and perception between population groups (for example, between genders).
Design: Data were collected in Belgium, Czech Republic, France, the Netherlands and Norway in 2007. Two 24-HDRs (face-to-
face and telephone administered) were conducted using EPIC-Soft. An evaluation questionnaire on different study aspects was
completed by the respondents.
Setting: Data were collected in the European Food Consumption Validation Study.
Subjects: A convenience sample of 600 apparently healthy men and women, 45–65 years old and including all educational
levels, were recruited (120 subjects per country). Differences among population groups were compared by means of the w2-test.
Results: A total of 585 respondents completed the evaluation questionnaire. In all, 88% experienced problems only to a low
degree when answering face-to-face and telephone-administered 24-HDR using EPIC-Soft. A total of 15% would have preferred
help of another person during the face-to-face interview in the study center (mainly men: Po0.001). Significantly, more subjects
in the Netherlands and in Norway preferred two telephone (instead of face-to-face) interviews compared with the other
countries (Po0.001).
Conclusion: Most subjects only experienced problems to a low degree during the EPIC-Soft interviews. Differences in
preferences and capabilities to answer the EPIC-Soft interviews were identified between population groups (for example, gender
differences). Therefore, the methods and the design to be used in a survey should be adapted according to the study
population, so as to optimize response rate and compliance.
European Journal of Clinical Nutrition (2011) 65, S29–S37; doi:10.1038/ejcn.2011.85

Keywords: respondent evaluation; 24-h dietary recall; telephone; face-to-face

Correspondence: Dr I Huybrechts, Dietary Exposure Assessment (NME/DEX),
International Agency for Research on Cancer, 150 cours Albert Thomas,
F-69372 Lyon Cedex 08, France.
E-mail: Inge.Huybrechts@Ugent.be
Contributors: IH, NS, CC, GN, AG and ITL contributed to the development and
concept of the respondent evaluation questionnaires. IH was responsible for
the statistical analyses. IH, AG, JHdV, CC, GN, WDK, ITL, JR, LL, CW, EMN, IM,
IR, SPC, HF, SDH and NS contributed to the conception of the paper. All
authors gave input on the drafting of the manuscript, and read and approved
the final version.
Introduction
Although different studies have focused on the (relative) validity
and reproducibility of dietary intake assessment methods, the
respondent’s viewpoint on these methods is often lacking.
However, in large-scale surveys in which representativity of the
study sample could importantly influence the results, it is
important to know to what extent the survey methods might
influence the respondent’s participation rate and/or compliance.
 Respondents’ evaluation of 24-HDR
I Huybrechts et al
S30
The 24-h dietary recall (24-HDR) method is widely used in
national and international epidemiological and dietary
monitoring surveys investigating different target popula-
tions (children, adolescents and adults; Slimani et al., 2000;
Vereecken et al., 2005; De Vriese et al., 2006; Moshfegh et al.,
2008; Elmadfa et al., 2009). The European Food Consump-
tion Method Project advised the use of repeated,
non-consecutive 24-HDR through strictly standardized
procedures for comparable and reliable data collection (Biro
et al., 2002). In addition, the group advised using EPIC-Soft
(the software developed to conduct 24-HDRs in the
European Prospective Investigation into Cancer and Nutri-
tion (EPIC) Study) to standardize the 24-HDRs as the best
way forward for future pan-European dietary monitoring
(Brussaard et al., 2002). This computerized method was
developed by International Agency for Research on Cancer
within the EPIC Study (Slimani et al., 1999, 2000). Although
a number of studies have already been published on the
validity of 24-HDR interviews (Klesges et al., 1987; Horst
et al., 1988; Lytle et al., 1993; Dop et al., 1994; Beer-Borst and
Amado, 1995; Kahn et al., 1995; Takatsuka et al., 1996; Tran
et al., 2000; Conway et al., 2004; Lof and Forsum, 2004;
Vereecken et al., 2005), an evaluation of 24-HDR from the
respondents’ viewpoint is still lacking. To optimize the
comprehensibility of this method and to lower the burden
on the respondent, it is important to know to what extent
the respondents feel capable of answering the different
questions.
Accuracy of recall information can, for instance, be
improved by enhancement of the comprehensibility of
the questions asked during the 24-HDR. In addition,
information about the preferences of the subjects with
respect to the administration method used (by telephone
or face-to-face) during the 24-HDR may improve the
participation rate in future monitoring and other surveys.
Subar et al. (2007) showed that most respondents (72%)
preferred a meal-based approach for the 24-HDR assessment,
and only 17% preferred an unstructured approach. To
inventory the experiences of the respondents, a respondent
evaluation questionnaire was included within the
validation study of the European Food Consumption
Validation (EFCOVAL) Project, which was devoted to the
adaptation and validation of the 2 non-consecutive days of
24-HDR using EPIC-Soft as a future pan-European food
consumption survey instrument.
The aim of this manuscript was to investigate the
respondents’ preferences and perception about the 24-HDR,
administered via face-to-face and telephone interviews
using EPIC-Soft, and to compare these preferences and
perception between population groups (for example, men
versus women or between body mass index categories). In
addition, we considered the respondent’s view on other
components included in the study design (for example,
collection of biological samples, anthropometric measure-
ments, filling in the written questionnaires, answering the
EPIC-Soft 24-HDR or taking the para-aminobenzoic acid
European Journal of Clinical Nutrition
(PABA) tablets) as many nutrition surveys currently include
a large battery of measurements in addition to dietary
intakes.
Subjects and methods
Sampling and study design
Within the EFCOVAL Study, a convenience sample of 600
apparently healthy adults in the age group between 45 and
65 years old were recruited in Belgium, Czech Republic,
France, the Netherlands and Norway. Respondents were
recruited by different sampling methods, for example,
through advertisements in local papers, by email, by flyers
and invitation letters sent to a random selection of
inhabitants, as well as via flyers distributed in the neighbor-
hoods of supermarkets. During recruitment, efforts were
made to include both genders and subjects with lower as well
as with medium and higher educational levels. Exclusion
criteria were as follows: using diuretics, following a pre-
scribed medical diet, being enrolled in another study during
the same timespan, not being able to read or speak the
national language, being pregnant, lactating, having dia-
betes mellitus or kidney disease and donating blood or
plasma during (or less than 4 weeks before) the study.
Subjects taking antibiotics containing sulfonamides were
also not allowed in the study, as well as subjects hypersensi-
tive to sulfonamides or PABA (sulfonamide is a PABA-intake
antagonist and PABA was used to check the completeness of
urine collections).
The study protocol was approved by the local ethics
committees of the centers involved in the EFCOVAL
Study and informed consent was obtained from all study
participants.
The recruitment of subjects and data collection in the
Netherlands were performed 6 months before the other four
countries, from April to July 2007, to test all procedures
of the fieldwork and to be able to instruct the other centers
on the standardized procedures. The other four countries
started the fieldwork in October 2007 following the standar-
dized procedures and study protocols drafted by Wageningen
University in the Netherlands.
Instruments and procedures used
After subjects were found eligible and had signed an
informed consent, they were included in the study sample.
At the start of the data collection, a general questionnaire
about lifestyle, socio-demographical information and nutri-
tional habits (including use of supplements and a food
propensity questionnaire) was completed by all subjects.
In addition, height and weight were measured, and a
non-fasting blood sample was taken in the study center.
Two repeated 24-HDRs and 24-h urine collections (covering
the same reference day) were obtained on 2 days at least one
month apart.

EPIC-Soft was used to conduct the two computer-assisted
24-HDR interviews from each participant. Trained (in
interviewing skills and EPIC-Soft use) nutritionists and
dietitians conducted the interviews. One recall was per-
formed through telephone and one via a face-to-face inter-
view in the study center. The exact purpose of the dietary
interviews (the exact food intake of the previous day)
was not told to the subjects in order to avoid changing their
usual dietary behavior because of the recall interview. The
order of the two recalls with a different mode of adminis-
tration was randomly assigned. During the telephone inter-
view, respondents were allowed to check food packages or
household measures in their home if this could help them
provide more detailed information. They could also ask for
help from another person in the house (for example, the
person responsible for food purchasing and preparations)
during the telephone interview (checking of food packages
and help from another person in the house were not possible
during the face-to-face interview in the study center).
A picture book was available to help in estimating portion
sizes of foods (for example, carrots, cake, French fries and so
on) and mixed dishes or recipes (for example, casseroles,
pizza and so on). Although most of the pictures included in
this picture book were selected from the EPIC-Soft picture
book, some extra (country specific) pictures were added. In
Belgium, Czech Republic and the Netherlands, interviews
were conducted neither on public holidays nor on Sundays;
therefore, intakes of Saturday were recalled 2 days later
(on Monday).
At the end of the data collection, all participants received
an incentive (the incentive differed between countries but
was B50 euros) and an evaluation questionnaire, to assess
the respondents’ viewpoint regarding the different proce-
dures and methods used within the EFCOVAL Study. The
same evaluation questionnaire was used in the five coun-
tries. In the Netherlands, no identification number was put
on the questionnaire to guarantee anonymity. However,
the identification number of the subject was included in the
respondent evaluation questionnaire of the four other
countries to allow linking with the other survey data (for
example, gender, educational level and so on).
The respondent evaluation questionnaire (see Supplemen-
tary Appendix online) included 17 main questions and was
specifically developed to investigate the perception and
preferences of the respondents in the EFCOVAL Study
regarding different aspects of the study. This questionnaire
included open questions and multiple-choice questions
using relative answering scales (for example, to a low degree,
to some degree, to a fairly high degree and to a high degree).
All the questionnaires were coded in the local centers by
using a standard data-entry format and protocol. Answers to
open questions were translated to English and recoded
to the original answer categories wherever possible. An
English version of the evaluation questionnaire used
within the EFCOVAL Project is included in Supplementary
Appendix online.
Respondents’ evaluation of 24-HDR
I Huybrechts et al
S31
More details on the procedures and methods used in this
EFCOVAL Study are given by Crispim et al. (2011a, b).
Data analysis
Most of the questions that used the answering scale with
the four categories were dichotomized in two categories
(for example, to a high degree or not; or to a fairly high and
high degree or not). The w2-test was used to investigate
differences of categorical variables (for example, able to
remember foods consumed on previous day to a high degree
or not) among population groups (for example, genders and
countries), and the Kruskal–Wallis test was used to compare
continuous variables between independent samples (for
example, comparing a score between different groups of
education level). The respondents’ abilities to fulfill the
different aspects in the EPIC-Soft 24-HDR (for example,
memorizing and identifying foods consumed) had four
different answering categories (ranging from 1 ¼ ‘capable to
a low degree’ till 4 ¼ ‘capable to a high degree’) in the
respondent evaluation questionnaire (questions 8.a to 8.d in
Supplementary Appendix online). To reflect the respondent’s
general capability to answer the EPIC-Soft 24-HDR, the
scores for these four different aspects were summed into one
single ‘Total capability score’ (ranging from 4 to 16) to
compare the total capability of respondents for answering all
the different EPIC-Soft questions between population groups
(for example, between genders). All analyses were performed
using the statistical software package SPSS for Windows
version 15 (SPSS, 2003).
Results
A total of 585 respondents (98%) completed the evaluation
questionnaire; however, for the 122 subjects from the
Netherlands, no socio-demographic information could be
linked to the respondent evaluation data (see Subjects and
methods section). Therefore, the subjects of the Netherlands
were included in the analyses including data from the
respondent evaluation questionnaire only. All respondents
were almost equally represented over both genders and by
the different centers (Table 1). Only for the educational level,
there was an imbalance between the lowest and highest
educational level, with an underrepresentation of the lowest
educated level (only 18.8%). On the basis of the body
mass index categories (as calculated from measured height
and weight), 39.5% of the subjects were overweight and
11.7% were obese.
Reasons for participation
In total, 16% of the EFCOVAL subjects participated in the
study because they always participate in scientific studies
whenever they were invited, and 50% participated because
they were interested in the study topic. There were 17% of
European Journal of Clinical Nutrition
 Respondents’ evaluation of 24-HDR
I Huybrechts et al
S32
Table 1 Characteristics and attitudes of the respondents who the subjects who participated because of the incentive (39%
completed the respondent evaluation questionnaire in Czech Republic versus less than 17% in all other study
Respondents centers; Po0.001). In addition, 18% of the respondents
felt motivated by the interviewer and 12% were motivated
na Relative by someone else to take part in the study. It is noteworthy to
percentage
mention that multiple answers were possible for participa-
(%)
tion motivation/justification.
Study center (n ¼ 585) In total, 92% of the participants would participate again in
Belgium 116 (19.8) the study and 72% of them even without incentive. In Czech
Czech Republic 118 (20.2) Republic, only 48% would participate again without incen-
France 111 (19.0)
Norway 118 (20.2)
tive compared with more than 60% in all other countries
The Netherlands 122 (20.9) (Po0.001).
Significantly, more non-obese subjects were motivated to
Gender (n ¼ 463) participate in the study because of their interest in the study
Men 228 (49.2)
Women 235 (50.8)
topic (56 versus 28% in the obese subjects; Po0.001). For the
same reason, more women were motivated to participate
Age, years (n ¼ 463) in the study because of their interest in the study topic
45–54 224 (48.4) compared with men (59 versus 45%; P ¼ 0.003). On the
55–65 239 (51.6)
contrary, significantly more men got motivated to partici-
BMI category, kg/m2 (n ¼ 463) pate in the study through the explanation of the field worker
BMI 20–25 226 (48.8) (25 versus 16% women; P ¼ 0.01).
Overweight (BMI 25–30) 183 (39.5)
Obese (BMI 430) 54 (11.7)

Educational level (n ¼ 463) Evaluation of different aspects of the study

Low 87 (18.8) For all aspects of the study (EPIC-Soft interviews, blood and
Intermediate 126 (27.2) urine collection, PABA intake and general questionnaire), at
High 250 (54.0)
least three-quarters of the population reported that they only
Main reason(s) to participate in the study (multiple answers allowed) experienced problems to a low degree (Table 2).
I always participate in studies or surveys, 96 (16.4) Only 88% of the respondents reported having faced
when I am invited to, no matter what problems, to a low degree, during both the telephone
the subjects or time loads are
The topic really interested me 294 (50.3) and face-to-face EPIC-Soft interviews (Table 2). There were
The interviewers (or other project member) 109 (18.6) no differences between the telephone and face-to-face
motivated me to participate administration, and there were fewer problems for the blood
Someone else motivated me to participate 71 (12.1) collections, questionnaires and PABA tablets than for the
Because of the incentive that was given 101 (17.3)
urine collections.
Would participate again in the study 536 (92.4) When asking the respondents to what degree they
preferred the face-to-face EPIC-Soft interviews to report their
Would participate even without incentive 419 (71.9)
dietary intake, 50 and 41% reported that they preferred the
The subject changed something in his/her diet 26 (4.4) face-to-face method to a high and a fairly high degree,
because of the urine collection respectively. When asking the respondents to what degree
I drank less on the day of the urine collection 6 (1.0)
they preferred the telephone EPIC-Soft interviews to report
I changed something else in my diet because 20 (3.4)
of the urine collection their dietary intake, 41 and 43% reported that they preferred
the telephone method to a high and fairly high degree,
The subject changed something in his/her diet 32 (5.5) respectively (Table 2).
because of the first 24-HDR
Most of the respondents stated that they did not change
I started eating more healthy products, 12 (2.1)
such as fruits and vegetables their dietary habits because of the urine collection (96%) or
I started eating less unhealthy products, 5 (0.9) 24-HDR interview (94%). Only 1% of the respondents drank
such as candies and soft drinks less and 3% changed something in their diet during their
I changed something else in my diet 15 (2.6)
urine collection (Table 1). Moreover, 2% reported that they
because of the first 24-HDR
started eating healthier products and 1% less unhealthy
Abbreviations: BMI, body mass index; EFCOVAL, European Food Consump- products after their first 24-HDR interview (what the
tion and Validation; 24-HDR, 24-h dietary recall. respondent considered as healthy or unhealthy products
a
The total number of respondents who completed the evaluation ques-
was not reported in the questionnaire). There were 3% of the
tionnaire was n ¼ 585 out of the total EFCOVAL Study sample of 600 subjects,
although for the 122 subjects from the Netherlands, no socio-demographic participants who changed something else in their diet after
information was available. their first 24-HDR interview (what exactly was changed was
not specified).

European Journal of Clinical Nutrition

 Respondents’ evaluation of 24-HDR
I Huybrechts et al
S33
Table 2 The perception and preferences of the study participants for different aspects of the study design and methods used (n (%))

Answering categories (scale) To low degree To some degree To fairly high degree To high degree
n (%) n (%) n (%) n (%)

To what degree did you have problems with one of the following sections of the study?
Telephone interview 511 (87.7) 60 (10.3) 11 (1.9) 1 (0.2)
Face-to-face interview 516 (88.4) 62 (10.6) 6 (1.0) 0 (0.0)
Blood collection 543 (93.0) 27 (4.6) 10 (1.7) 4 (0.7)
Urine collection 449 (77.0) 114 (19.6) 15 (2.6) 5 (0.9)
PABA 529 (90.9) 38 (6.5) 13 (2.2) 2 (0.3)
General questionnaire 543 (93.3) 36 (6.2) 2 (0.3) 1 (0.2)

To what degree do you prefer the administration via face-to-face and telephone interview for assessing your diet of the last 24 hours?
Telephone interview 22 (3.8) 66 (11.4) 252 (43.4) 241 (41.5)
Face-to-face interview 6 (1.0) 47 (8.1) 240 (41.2) 289 (49.7)

To what degree did you feel capable to perform the different tasks of the EPIC-Soft interview about the previous day?
Remember foods eaten 5 (0.9) 41 (7.1) 260 (44.8) 274 (47.2)
Describe foods 7 (1.2) 49 (8.5) 278 (48.0) 245 (42.3)
Estimate portion size with photo book 6 (1.0) 55 (9.6) 283 (49.2) 231 (40.2)
Estimate portion size without photo book 72 (12.4) 159 (27.4) 216 (37.2) 133 (22.9)

To what degree did you feel able to perform the different tasks of the EPIC-Soft interview about 2 days in the past? (n ¼ 48)
Remember foods eaten 1 (2.1) 10 (20.8) 23 (47.9) 14 (29.2)
Describe foods 1 (2.1) 9 (18.8) 22 (45.8) 16 (33.3)
Estimate portion size with photo book 0 (0.0) 5 (10.4) 26 (54.2) 17 (35.4)
Estimate portion size without photo book 5 (10.4) 15 (31.3) 21 (43.8) 7 (14.6)

Abbreviations: EPIC-Soft, the software developed to conduct 24-h dietary recalls in the European Prospective Investigation into Cancer and Nutrition Study;
PABA, para-aminobenzoic acid.
The participants could answer the different questions described in the table below by means of a scale using the following categories: to a low degree, to some
degree, to a fairly high degree, or to a high degree (N ¼ 584).

Respondent’s capabilities to answer the EPIC-Soft questions
There were 89% of the respondents who were able to answer
the 24-HDR by themselves during the telephone interview
(96% women versus 87% men; P ¼ 0.006), whereas 8% asked
for help from someone else in the house and 3% needed
help, but could not ask any help during their telephone
interview. Although 85% of the respondents were capable of
answering all the 24-HDR questions during the face-to-face
interview in the study center by themselves, 15% reported
that help from someone else in the household (for example,
the person responsible for meal preparations) would have
been helpful during this face-to-face interview in the study
center (20% men versus 6% women; Po0.001).
As presented in Table 2, 47 and 42% of the subjects felt
capable to remember and, respectively, describe all foods
consumed the day before to a high degree and, respectively,
45 and 48% to a fairly high degree (for these questions, no
differentiation was made between telephone and face-to-face
interviews). In addition, 40 and 49% of the subjects felt
capable of estimating the portion sizes by means of the
picture book to a high and fairly high degree, respectively,
whereas only 23 and 37% without the picture book (Table 2).
Only 29 and 33% of the 48 subjects who completed a
24-HDR from 2 days in the past felt capable to remember
and, respectively, describe all foods consumed 2 days before
to a high degree and, respectively, 48 and 46% to a fairly
high degree (Table 2). In addition, 35 and 54% of the subjects
felt capable of estimating the portion sizes by means of the
picture book to a high and fairly high degree, respectively,
when the interview was completed recalling intakes from
2 days in the past, whereas only 15 and 44% without a
picture book (Table 2).
When summing up the capabilities (ranging from 1 ¼ ‘cap-
able to a low degree’ to 4 ¼ ‘capable to a high degree’) for the
four different aspects investigated into one single total
capability score (ranging from 1 to 16), small but statistically
significant differences were found between the capability score
for the previous day and the capability score calculated for
2 days in the past (median was 13 for the previous day and
12 for 2 days in the past (P ¼ 0.01)). Furthermore, significant
differences were found among countries when comparing the
total capability scores for the previous day (Po0.001), with
Belgium and the Netherlands scoring lowest (see Figure 1).
When comparing this total capability score between popula-
tion groups, no statistically significant differences were found
between genders (13.1 (s.d. 2.4) for men and 13.0 (s.d. 2.3)
for women), nor between educational levels (13.5 (s.d. 2.1) for
the lowest educated, 12.9 (s.d. 2.5) for the intermediate and
12.9 (s.d. 2.3) for the highest educated subjects).
Preferences for administration method and logistics
Two face-to-face interviews were preferred by 28% of the
subjects, whereas 26% preferred two telephone interviews

European Journal of Clinical Nutrition

 Respondents’ evaluation of 24-HDR
I Huybrechts et al
S34
Table 3 Preferences of the respondent for some methodological
16
aspects of the study

14 Number (%) of
respondents who
prefer this method
Capability score

12 n (%)

What way of administration would you prefer for the 24-HDR?

10
Both interviews administered by 149 (25.5)
telephone interview
8 Both interviews administered 165 (28.2)
face-to-face by an interviewer
First interview administered 95 (16.2)
6 face-to-face by an interviewer and
second interview by telephone
First interview by telephone and 30 (5.1)
4 second interview by face-to-face interview
Other way of administration (for example, 105 (17.9)
Belgium Czech France Norway The web-based interview to complete yourself)
Republic Netherlands

Country What location would you prefer for the face-to-face 24-HDR?
At your home 127 (21.7)
Figure 1 Box plot diagram (minimum, 25P, 50P, 75P and In a local center 287 (49.1)
maximum) for between-country comparison of the total capability At your work 51 (8.7)
score, which was calculated from the capability values of the Other location 10 (1.7)
different aspects investigated the previous day (remembering foods,
describing foods, estimating portion sizes with a photo book and What location would you prefer for the telephone 24-HDR?
estimating portion sizes without a photo book). The score can range At your home 371 (63.4)
between 4 and 16, with 16 representing the greatest capability. At your work 64 (10.9)
Other location 13 (2.2)

and 21% a combination of those two administration
methods (for example, face-to-face method first, followed
by a telephone interview, or the other way around; Table 3).
There were 18% of the subjects who preferred another
administration method (for example, self-administered
methods) and the remaining 6% did not have a clear
preference. Statistically significantly more subjects in the
Netherlands and in Norway preferred two telephone inter-
views compared with the other countries (34 and 41%
compared with 25% or less in all other countries; Po0.001).
On the contrary, more subjects preferred two face-to-face
interviews in France, Belgium and Czech Republic compared
with Norway and the Netherlands (41, 34, 30 versus 15 and
25%, respectively; Po0.001). Moreover, a higher percentage
of lower educated subjects preferred two face-to-face admin-
istrations (33%) compared with their higher educated peers
(28%; P ¼ 0.03). On the contrary, a higher percentage
of higher educated subjects would refuse two face-to-face
interviews in future (18% versus 8% in lower educated
subjects (P ¼ 0.03)).
When asking for the respondents’ preferred location for a
future interview, 49% of the subjects would prefer the study
center (versus 22% at home and 10% at work) for the face-to-
face interview, and 63% would prefer to be at home for
the telephone interview (compared with 11% at work). The
remaining respondents did not have a clear preference
(Table 3).
Although all face-to-face interviews in the EFCOVAL
Project were conducted at the study center, in the
Netherlands, significantly more subjects would prefer the
Abbreviation: 24-HDR, 24-h dietary recall.
face-to-face interview at home (39%) compared with all
other countries (less than 25%; Po0.001), in which they
mainly would prefer to have the face-to-face interview in a
study center. No significant differences were identified for
what concerns the location preferences when comparing
higher- and lower educated subjects.
Discussion
In the light of a pan-European food consumption survey, it is
important to know why subjects participate in a dietary
survey and whether the motivation/reason to participate
in such a survey could differ between population groups
(for example, between genders and countries), to understand
and analyze possible selection bias. In addition, it is
important to know the respondents’ preferences and
capabilities to improve participation rate, accuracy and
respondents’ compliance. Therefore, this respondent evalua-
tion questionnaire was used to investigate the respondent’s
viewpoint about the EPIC-Soft 24-HDR, which is currently
used in different national and international surveys.
In this study, most of the respondents reported that they
faced problems only to a low degree for both EPIC-Soft
administration modes. However, 15% of the respondents
(mainly men) preferred help from another person during
the face-to-face interview in the study center. One reason

European Journal of Clinical Nutrition


why mainly men preferred the help of a third/proxy person
may be explained, because in this age group, women are
often more responsible for meal preparations and for
purchasing foods than men. When considering these results,
the availability or accessibility of a third person who is
responsible for food provision and meal preparations could
be recommended in future surveys if this extra information
is important for the accuracy of the dietary intake assessment
(especially when men are included in the target population).
Although 90% of the subjects felt capable of remembering
and describing all foods consumed the day before to a fairly
high or high degree, only 75% felt capable when the recall
was about 2 days in the past (because no interviews could be
conducted on Sundays, the interview regarding intakes on
Saturdays was conducted on Monday). Moreover, the total
capability score was significantly lower for interviews
regarding intakes from 2 days in the past than about the
previous day. Although our results suggest that the respon-
dent himself feels less capable of answering a 24-HDR about
2 days in the past, we do not know whether the results of
those recalls were indeed less accurate. Therefore, it should
be further investigated to what extent the validity of a
24-HDR of 2 days in the past is comparable to the validity
of a 24-HDR interview about the previous day.
In both situations (recall of previous day or 2 days in the
past), more respondents reported to be capable of quantify-
ing portion sizes to a higher degree with, compared with
without, the use of the picture book. Although respondents
preferred the picture book for estimating portion sizes, it is
noteworthy that this does not necessarily mean that the
picture book is the most accurate quantification method
(compared with household measures for instance). The
accuracy of the method can only be evaluated by compar-
ison with weighing of the portions consumed. De Keyzer
et al. (2010) recently concluded from a validation study in
Belgium that the use of food photographs for portion
size estimation of bread and beverages is acceptable for use
in nutrition surveys. However, for photographs of margarine
on bread, further validation using smaller amounts
corresponding to actual consumption is recommended
(De Keyzer et al., 2010).
Statistically significant differences were found among
genders, different educational levels and body mass index
categories for different aspects under study (for example,
more women and non-obese subjects reported that they were
motivated to participate in the study because of their interest
into the study topic compared with men and obese subjects,
respectively). Moreover, significant differences were found in
preferences for the modes of administration used for the
24-HDR interviews among countries. From these analyses, it
could be concluded that no single study method is ideal for
all population groups and that the methods and design to be
used in a survey should be adapted according to the study
population in order to optimize response rate and compli-
ance. For instance, different (interchangeable) administra-
tion methods could be recommended in different countries
Respondents’ evaluation of 24-HDR
I Huybrechts et al
S35
if those different administration modes are comparable in
accuracy (this should be investigated in future research). It is
therefore important to investigate the preferences of a study
population in a pilot or validation study before commencing
large-scale monitoring surveys to optimize participation rate
and accuracy (Brustad et al., 2002). However, it is noteworthy
that the respondent’s preference is often inferior to aspects
influencing the validity/accuracy and feasibility of the
dietary intake assessment (for example, if a self-administered
web-based method is preferred by the respondent, but shown
to be less accurate for estimating usual dietary intakes,
preference should be given to the more accurate methods
such as the face-to-face method). However, practical con-
straints such as housing density in a country could also have
an important role in the decision-making process for the
interview method to be used (for example, in Norway, where
the population density is very low, the telephone method is
often preferred above the face-to-face method because of
feasibility reasons).
Although no other respondent evaluation studies were
found that investigated the motivation, preferences and
self-reported capabilities of the respondents for completing
24-HDR interviews, two recent studies from Subar et al. also
asked for the respondent’s preferences of certain aspects of
the 24-HDR method (portion size estimation with different
types of food photographs (Subar et al., 2010) and the use of
different types of quick lists (Subar et al., 2007)). Subar et al.
(2010) concluded that participants generally preferred aerial
photos for food categories other than spreads, for which
household measures were preferred, and that participants
indicated a striking preference for simultaneous versus
sequential presentation of pictures. In addition, this study
showed that, although photograph size did not affect the
accuracy, participants strongly favored larger over smaller
photographs and four versus eight photographs. From the
other study of Subar et al. (2007), which presents qualitative
information from cognitive testing of two potential
strategies for designing a quick list for an automated
self-administered recall, it could be concluded that most of
the participants preferred the meal-based quick-list approach
over the unstructured quick-list approach. This subjective
preference information from studies of Subar et al. (2010)
helped in decision making during the further development
of their automated self-administered recall when there were
no clear findings regarding accuracy.
The large total sample size (when including all countries),
the geographical spread (across Europe) and the inclusion of
both genders and subjects from different educational level
are important strengths of this respondent evaluation study,
as well as the high response rate for the evaluation
questionnaire. In the studies of Subar et al. (2007, 2010)
discussed above, less than 30 individuals were included in
each study (in the study about the quick-list approaches,
only 18 subjects were included).
Furthermore, the uniqueness of these data is an important
strength of this study, as well as the inclusion of respondents
European Journal of Clinical Nutrition
 Respondents’ evaluation of 24-HDR
I Huybrechts et al
S36
recalling dietary intake data regarding intakes from 2 days in
the past. No other pan-European studies were found
that evaluate the respondent’s viewpoint about the 24-HDR
interview.
A limitation of the study is that small convenience samples
were recruited in each country, which limits the representa-
tiveness of the subjects recruited within a country and
confines generalization of these results to larger study
populations recruited through random sampling procedures,
as usually done in monitoring surveys. Comparisons between
countries should therefore be interpreted cautiously.
Moreover, the underrepresentation of the lower educated
subjects is a limitation of this study. Because the accuracy of
the dietary intake data recalled through a 24-HDR interview
might depend on the level of education of the subject, the
feasibility results reported in this manuscript might be
overestimated because of an overrepresentation of higher
educated subjects. However, from our study results, it could
be concluded that the lower educated subjects did not report
differently on their ability to recall, identify and describe
their food intake in comparison with the higher educated
subjects.
Another limitation of the convenience sample is the fact
that the subjects who participated in our study were likely to
be more motivated than general participants from national
food consumption surveys (the workload in our validation
study with collection of biomarkers in addition to the
24-HDRs was probably higher compared with an ordinary
monitoring study). These highly motivated subjects might
experience less problems than less motivated subjects.
Therefore, one should be careful when generalizing these
results to the general population participating in large-scale
food consumption surveys.
Moreover, the respondent evaluation questionnaire itself
had its limitations, as this questionnaire had not been
evaluated beforehand. For instance, for questions asking
the degree of problems experienced, it could have been
more accurate to include a category ‘no problems’, as now
the lowest category in the scale was ‘to a low degree’.
Another limitation of this study is that anonymous evalua-
tion questionnaires were used in the Netherlands, and no
linking could be made with demographical information for
the subjects recruited in the Netherlands. In addition,
anonymous evaluation questionnaires might differ in truth-
fulness than non-anonymous questionnaires.
Conclusion
Most subjects only experienced problems to a low degree
during the face-to-face and telephone-administered 24-HDR
EPIC-Soft interviews. However, preferences and capabilities
sometimes differed between population subgroups; there-
fore, a respondent evaluation during a pilot study could be
recommended to improve participation and compliance in
future surveys.
European Journal of Clinical Nutrition
Conflict of interest
JR received consulting fees from the Czech Technology
Platform for food and healthy lifestyle. All other authors
declare no conflict of interest.
Acknowledgements
The content of this article reflects only the authors’ views
and the Community is not liable for any use that may be
made of the information contained therein. The Commu-
nity funding under the Sixth Framework Program for the
EFCOVAL Project is acknowledged (FOOD-CT-2006-022895).
References
Beer-Borst S, Amado R (1995). Validation of a self-administered
24-h recall questionnaire used in a large-scale dietary survey.
Z Ernahrungswiss 34, 183–189.
Biro G, Hulshof KF, Ovesen L, Amorim Cruz JA (2002). Selection of
methodology to assess food intake. Eur J Clin Nutr 56 (Suppl 2),
S25–S32.
Brussaard JH, Lowik MR, Steingrimsdottir L, Moller A, Kearney J,
De HS et al. (2002). A European food consumption survey
method–conclusions and recommendations. Eur J Clin Nutr 56
(Suppl 2), S89–S94.
Brustad M, Skeie G, Braaten T, Slimani N, Lund E (2002). Comparison
of telephone versus face-to-face interviews in the assessment of
dietary intake by the 24-h recall EPIC SOFT programme–the
Norwegian calibration study. IARC Sci Publ 156, 17–19, 17–19.
Conway JM, Ingwersen LA, Moshfegh AJ (2004). Accuracy of dietary
recall using the USDA five-step multiple-pass method in men: an
observational validation study. J Am Diet Assoc 104, 595–603.
Crispim SP, de Vries JH, Geelen A, Souverein OW, Hulshof PJ, Lafay L
et al. (2011a). Two non-consecutive 24 h recalls using EPIC-Soft
software are sufficiently valid for comparing protein and
potassium intake between five European centers—results from
the European Food Consumption Validation (EFCOVAL) study.
Br J Nutr 105, 447–458.
Crispim S, Geelen A, Souverein OW, Hulshof PJM, Ruprich J, Dofkova
M et al. (2011b). Biomarker-based evaluation of two 24-h recalls
for comparing usual fish, fruit and vegetable intakes across
European centers in the EFCOVAL Study. Eur J Clin Nutr 65 (Suppl
1), S38–S47.
De Keyzer W, Huybrechts I, De Maeyer M, Ocke M, Slimani N,
Van ‘t veer P et al. (2010). Food photographs in nutritional
surveillance: errors in portion size estimation using drawings of
bread and photographs of margarine and beverages consumption.
Br J Nutr 105, 1073–1083.
De Vriese S, Huybrechts I, Moreau M, Van Oyen H (2006). The
Belgian Food Consumption Survey 1—2004: Report (Enquête de
consommation alimentaire Belge 1—2004: Rapport). D/2006/
2505/16.
Dop MC, Milan C, Milan C, N’Diaye AM (1994). The 24-h recall for
Senegalese weanlings: a validation exercise. Eur J Clin Nutr 48,
643–653.
Elmadfa I, Meyer A, Nowak V, Hasenegger V, Putz P, Verstraeten R
et al. (2009). European Nutrition and Health Report 2009.
Ann Nutr Metab 55 (Suppl 2), 1–40.
Horst CH, Obermann-De Boer GL, Kromhout D (1988). Validity of
the 24-h recall method in infancy: the Leiden Pre-School Children
Study. Int J Epidemiol 17, 217–221.
Kahn HA, Whelton PK, Appel LJ, Kumanyika SK, Meneses JL,
Hebert PR et al. (1995). Validity of 24-h dietary recall interviews

conducted among volunteers in an adult working community.
Ann Epidemiol 5, 484–489.
Klesges RC, Klesges LM, Brown G, Frank GC (1987). Validation of
the 24-h dietary recall in preschool children. J Am Diet Assoc 87,
1383–1385.
Lof M, Forsum E (2004). Validation of energy intake by dietary recall
against different methods to assess energy expenditure. J Hum Nutr
Diet 17, 471–480.
Lytle LA, Nichaman MZ, Obarzanek E, Glovsky E, Montgomery D,
Nicklas T et al. (1993). Validation of 24-h recalls assisted by food
records in third-grade children. The CATCH Collaborative Group.
J Am Diet Assoc 93, 1431–1436.
Moshfegh AJ, Rhodes DG, Baer DJ, Murayi T, Clemens JC,
Rumpler WV et al. (2008). The US Department of Agriculture
Automated Multiple-Pass Method reduces bias in the collection of
energy intakes. Am J Clin Nutr 88, 324–332.
Slimani N, Deharveng G, Charrondiere RU, van Kappel AL, Ocke MC,
Welch A et al. (1999). Structure of the standardized computerized
24-h diet recall interview used as reference method in the
22 centers participating in the EPIC project. European Prospective
Investigation into Cancer and Nutrition. Comput Methods Programs
Biomed 58, 251–266.
Slimani N, Ferrari P, Ocke M, Welch A, Boeing H, Liere M et al. (2000).
Standardization of the 24-h diet recall calibration method used in
Supplementary Information accompanies the paper on European Journal of Clinical Nutrition website (http://www.nature.com/ejcn)
View publication stats
Respondents’ evaluation of 24-HDR
I Huybrechts et al
S37
the European prospective investigation into cancer and nutrition
(EPIC): general concepts and preliminary results. Eur J Clin Nutr
54, 900–917.
SPSS (2003). Statistical Package for the Social Sciences, 12th edn. SPSS:
Chicago.
Subar AF, Crafts J, Zimmerman TP, Wilson M, Mittl B, Islam NG et al.
(2010). Assessment of the accuracy of portion size reports using
computer-based food photographs aids in the development of
an automated self-administered 24-h recall. J Am Diet Assoc 110,
55–64.
Subar AF, Thompson FE, Potischman N, Forsyth BH, Buday R,
Richards D et al. (2007). Formative research of a quick list for an
automated self-administered 24-h dietary recall. J Am Diet Assoc
107, 1002–1007.
Takatsuka N, Kawakami N, Kawai K, Okamoto Y, Ishiwata K, Shimizu
H (1996). Validation of recalled food intake in the past in a
Japanese population. J Epidemiol 6, 9–13.
Tran KM, Johnson RK, Soultanakis RP, Matthews DE (2000).
In-person vs telephone-administered multiple-pass 24-h recalls
in women: validation with doubly labeled water. J Am Diet Assoc
100, 777–783.
Vereecken CA, Covents M, Matthys C, Maes L (2005). Young
adolescents’ nutrition assessment on computer (YANA-C).
Eur J Clin Nutr 59, 658–667.
European Journal of Clinical Nutrition