Appetite 69 (2013) 180–185

Contents lists available at SciVerse ScienceDirect

Appetite
journal homepage: www.elsevier.com/locate/appet

Research report

Agreement between paper and pen visual analogue scales

and a wristwatch-based electronic appetite rating system (PRO-DiaryÓ),
for continuous monitoring of free-living subjective appetite sensations
in 7–10 year old children
P.L.S. Rumbold ⇑, C.J. Dodd-Reynolds, E. Stevenson
Faculty of Health and Life Sciences, Department of Sport, Exercise and Rehabilitation, Northumbria University, Northumberland Building, Newcastle upon Tyne NE1 8ST, UK

a r t i c l e i n f o a b s t r a c t

Article history: Electronic capture of free-living subjective appetite data can provide a more reliable alternative to tradi-
Received 19 February 2013 tional pen and paper visual analogue scales (P&P VAS), whilst reducing researcher workload. Conse-
Received in revised form 1 May 2013 quently, the aim of this study was to explore the agreement between P&P VAS and a wristwatch-
Accepted 5 June 2013
based electronic appetite rating system known as the PRO-DiaryÓ technique, for monitoring free-living
Available online 14 June 2013
appetite sensations in 7–10 year old children. On one occasion, using a within-subject design, the 12 chil-
dren (n = 6 boys; n = 6 girls) recorded their subjective appetite (hunger, prospective food consumption,
Keywords:
and fullness), at two time points before lunch (11:30 and 12:00) and every 60 min thereafter until
Children
Subjective appetite
21:00. The agreement between the P&P VAS and PRO-DiaryÓ technique was explored using 95% limits
Pen and paper visual analogue scales of agreement and 95% confidence intervals (95% CI) calculated using the Bland and Altman (1986) tech-
PRO-DiaryÓ nique. For hunger, prospective food consumption and fullness, the 95% limits of agreement were
1 ± 25 mm (95% CI: lower limit 8 mm; upper limit +6 mm), 0 ± 21 mm (95% CI: lower limit 6 mm;
upper limit +6 mm) and 6 ± 24 mm (95% CI: lower limit 14 mm; upper limit +1 mm), respectively.
Given the advantages associated with electronic data capture (inexpensive; integrated alarm; data easily
downloaded), we conclude that the PRO-DiaryÓ technique is an equivalent method to employ when con-
tinuously monitoring free-living appetite sensations in 7–10 year old children, but should not be used
interchangeably with P&P VAS.
Ó 2013 Elsevier Ltd. All rights reserved.

Introduction
Visual analogue scales (VAS) have been successfully and exten-
sively used to assess subjective appetite sensations (in laboratory
and free-living settings) such as hunger, prospective food con-
sumption and fullness in children (Dodd, Welsman, & Armstrong,
2008; Moore, Dodd, Welsman, & Armstrong, 2004; Rumbold,
Dodd-Reynolds, & Stevenson, in press) and adolescent populations
(Bellissimo, Thomas, Goode, & Anderson, 2007; Bozinovski et al.,
2009; Rumbold & Dodd, 2007; Rumbold, St Clair Gibson, Allsop,
Stevenson, & Dodd-Reynolds, 2011; Rumbold et al., 2013; Thivel,
Isacco, Rousset, et al., 2011; Thivel et al., 2012; Thivel, Isacco, Tail-
lardat, et al., 2011). Thus, VAS provide an invaluable means of mon-
itoring free-living subjective appetite sensations in these
population groups. Typically in paediatric exercise and appetite
regulatory research, VAS take the form of 100–150 mm horizontal
lines, with extreme subjective phrases placed at either end of the
⇑ Corresponding author.
E-mail address: penny.rumbold@northumbria.ac.uk (P.L.S. Rumbold).
line (Hill & Blundell, 1982). This technique requires individuals to
respond to a question by placing a vertical mark between the an-
chor phrases. Visual analogue scales therefore require individuals
to extrapolate a subjective sensory experience into a linear format,
which then enables the subjective sensation to be measured and
quantified (Gift, 1989).
Traditionally VAS are administered using pen and paper (P&P)
and therefore are convenient and quick to complete (Gibbons, Cau-
dwell, Finlayson, King, & Blundell, 2011; Wewers & Lowe, 1990).
Conversely however, the measurement and recording process is
extensive, time consuming and susceptible to human error, given
that the horizontal lines require hand measurement and data entry
(Gibbons et al., 2011). Moreover, P&P VAS often require individuals
to complete such scales unsupervised in their natural environment,
which increases the likelihood of questions being omitted, com-
pleted incorrectly and not at the correct time points stipulated
by the researcher (Stratton et al., 1998).
Various electronic devices which address the limitations inher-
ent when administering P&P VAS in free-living environments have
been developed. These include the Psion hand-held personal digital

0195-6663/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.appet.2013.06.005
 P.L.S. Rumbold et al. / Appetite 69 (2013) 180–185
assistant Electronic Appetite Ratings System (EARS), (ÓUniversity
of Leeds, UK), the Apple Newton Message Pad EARS (ÓR.J. Stubbs
and M. Elia), the EARS II [ÓQueensland University of Technology
(QUT)] and more recently the PRO-Diary (ÓCamNtech Ltd. and
CamNtech Inc.). Such portable devices are relatively inexpensive
and incorporate an integrated alarm to alert participants when to
complete the VAS questions (Gibbons et al., 2011). In addition, re-
searcher error associated with the manual measurement of P&P
VAS is reduced, given that data from electronic devices can be
downloaded directly into appropriate software programmes for
subsequent analysis (Stubbs et al., 2001). These practicalities have
resulted in compliance rates of P90% being reported in the litera-
ture (Hufford & Shields, 2002). The agreement of these devices
compared to traditional P&P VAS have been extensively explored
in adult populations in both laboratory-based (Delargy, Lawton,
Smith, King, & Blundell, 1996; Gibbons et al., 2011; Stubbs et al.,
1997; Stubbs et al., 2001; Whybrow, Stephen, & Stubbs, 2006; Za-
bel, Ash, Bauer, & King, 2009) and free-living settings (Stratton
et al., 1998). Consensual findings in adults indicate that the EARS
provide reproducible results relative to P&P VAS (Stubbs et al.,
2000). However, the cited authors also indicate that individuals
tend to use EARS in a more constrained manner with regards to
the range of responses provided between the extreme anchor
phrases. Therefore in adults, it is advised that P&P VAS and EARS
methodologies should not be used interchangeably (Stubbs et al.,
2000).
With regards to young people however, there are a lack of stud-
ies which have explored the agreement between free-living appe-
tite sensations assessed using P&P VAS and alternative electronic
devices. When working with young people in free-living environ-
ments and considering both the disadvantages associated with
the administration of P&P VAS (time consuming; susceptible to hu-
man error both participants and researchers) and the inherent
advantages of using electronic VAS (inexpensive; integrated alarm;
high compliance rates; data easily downloaded), the use of devices
such as the PRO-DiaryÓ technique to capture appetite sensations
electronically would be extremely useful. More recently, we have
adopted a more representative, free-living and ecologically valid
approach when conceiving paediatric exercise and appetite re-
search study designs (Rumbold, Dodd-Reynolds, et al., 2013; Rum-
bold, St Clair Gibson, Allsop, et al., 2011; Rumbold et al., 2013).
Thus we strongly advocate the use of robust methodological tools
which accurately assess both free-living energy intake and appetite
sensations in paediatric populations. Indeed, we have developed a
combined dietary data collection technique to employ with female
adolescents when quantifying free-living energy intake (Rumbold,
St. Clair Gibson, Stevenson, & Dodd-Reynolds, 2011). At the time of
writing, we are unaware of any studies which have investigated
the agreement between P&P VAS and alternative electronic devices
such as the PRO-DiaryÓ technique, for assessing free-living appe-
tite sensations in young people.
Therefore, the aim of this study was to explore the agreement
between the P&P VAS method and the PRO-DiaryÓ technique for
assessing free-living continuous subjective appetite sensations
(hunger, prospective food consumption and fullness) in 7–10 year
old children.
Methods
Design
On one occasion, using a within-subject design, the agreement
between the P&P VAS method and PRO-DiaryÓ technique for
assessing free-living continuous subjective appetite sensations
(hunger, prospective food consumption and fullness) was explored
181
in young children. One 1-d test session took place in a free-living
environment (primary school and home setting).
The study was approved by the Faculty of Health and Life Sci-
ences Research Ethics Committee at the University of Northumbria.
Written informed consent was obtained from both the children
and parents or guardians prior to data collection. The children were
notified that the study was based on general appetite measures,
but not told that it was concerned with the agreement between
two methods of subjective appetite assessment.
Participants
The sample size of 12 was designed to have more than 90%
power to detect statistical significance at an alpha level of 0.05.
Our calculations are based on previous data which identified that
n = 11, n < 8 and n < 8 participants are required to show detectable
differences of 10 mm in subjective hunger, fullness and prospec-
tive food consumption respectively (Flint, Raben, Blundell, & Ast-
rup, 2000). Therefore, 12 children (n = 6 boys; n = 6 girls) aged
(mean ± SD) 9.0 ± 1.0 years; stature 1.40 ± 0.09 m; mass 30.8 ±
9.7 kg; BMI 16.6 ± 3.1 kg/m2; maturity offset 2.9 ± 1.3 years and
dietary restraint 2.1 ± 0.6 took part in the study. All children were
classified as normal weight (Cole, Bellizzi, Flegal, & Dietz, 2000).
Preliminary measures
During a preliminary testing session at the school, the children’s
stature and seated height were measured to the nearest 0.01 m
using a Harpeden Portable Stadiometer (Holtain Limited, Pembs,
UK) and body mass to the nearest 0.1 kg using Avery Balance scales
(Avery Berkel Ltd., West Midlands, UK). These measures were used
to calculate BMI (kg/m2) and maturity offset (Mirwald, Baxter-
Jones, Bailey, & Beunen, 2002), with the latter providing an indica-
tion of years to/from peak height velocity and thus maturation. In
the presence of the research team, the children completed the
Dutch Eating Behaviour Questionnaire for children (DEBQ-C) (van
Strien & Oosterveld, 2008) to provide an indication of restrained
eating prevalence within the sample.
Chambers and Craig (1998) suggest that young children tend to
respond to appetite rating scales in an extreme manner. Conse-
quently, the children were familiarised with the VAS diaries and
PRO-DiaryÓ (CamNtech Ltd. and CamNtech Inc., Cambridge, UK)
devices and provided with careful instructions on how to use both
methods by trained research assistants (Hanet, Salah, & Lluch,
2010). For the P&P VAS the participants were instructed to mark
a vertical line on the 100 mm VAS to correspond with what they
were subjectively feeling at the time the VAS were administered.
With regards to the PRO-DiaryÓ device, participants were asked
(when alerted by the integrated alarm) to respond using their fin-
ger via the touch sensitive ‘slider’ function at the bottom of the
PRO-DiaryÓ interface. The touch sensitive ‘slider’ enabled the chil-
dren to move a vertical cursor along the horizontal VAS and once in
the desired place, were instructed to press the ‘O’ button to con-
firm their response. Subsequently, the problems encountered with
regards to extreme responses are believed to be eliminated (Price,
McGrath, Rafii, & Buckingham, 1983).
Protocol
The 1-d test session took place on a day that did not involve the
children participating in physical education or any other active
curriculum lessons (dance/drama, etc.), as we have previously
demonstrated the potential for exercise to influence subjective
appetite in children and adolescents (Dodd et al., 2008; Moore
et al., 2004; Rumbold, St Clair Gibson, Allsop, et al., 2011). The
organisation of the 1-d test session followed the typical timings
182 P.L.S. Rumbold et al. / Appetite 69 (2013) 180–185
of a school day to ensure the study day was representative of the
children’s free-living routine. All 12 children attended school as
normal from 08:55 to 15:15 and continued with their typical
free-living routine for the remainder of the day up until 21:00.
The children recorded their subjective appetite (hunger, prospec-
tive food consumption and fullness), at two time points before
lunch (11:30 and 12:00) and every 60 min thereafter until 21:00
(13:00, 14:00, 15:00, 16:00, 17:00, 18:00, 19:00, 20:00 and
21:00). The children completed the P&P VAS and PRO-DiaryÓ
scales immediately after one another, but were instructed to com-
plete these in a randomised order. Between 12:00 and 13:00 the
children consumed school dinner or packed lunch according to
their own preference. No food or drink was consumed between
lunchtime and the end of school (13:00–15:15), however following
this period the children were free to eat as they pleased up until
21:00 (i.e. snacking, evening meals). As per the methods described
in one of our previous studies (Rumbold, Dodd-Reynolds, et al.,
2013) children’s school dinner consumption was covertly weighed
to the nearest gram by the research team, whilst parents/guardians
were asked to weigh and record in a food diary their child’s packed
lunch intake and their child’s food and drink consumption for the
period between the children left school and the end of the study
period (15:15–21:00).
P&P VAS
The P&P VAS were presented to the children individually on
separate pages within small A5 booklets. Subjective appetite re-
sponses were rated on 100 mm horizontal lines, to each of the fol-
lowing questions: ‘How hungry do you feel now?’ anchored by very
hungry (100) and not at all hungry (0); ‘How full do you feel now?’
anchored by very full (100) and not full at all (0); prospective food
consumption and ‘‘How much would you like to eat now?’’ an-
chored by a lot (100) and nothing at all (0). The P&P VAS method in-
volved the children marking a vertical line with a pencil at a point
along the length of the 100 mm horizontal lines, which they felt
represented their subjective feeling at the time with regards to
hunger, prospective food consumption and fullness. All P&P VAS
were measured by hand by one researcher, from the minimum
score of 0 mm to the maximum score of 100 mm and entered into
an Excel spreadsheet.
PRO-DiaryÓ
The PRO-DiaryÓ device took the form of a ‘watch like’ com-
puter, which due to its compact dimensions (51  34  8 mm)
was worn on the children’s wrist. One advantage of the PRO-Dia-
ryÓ over the P&P VAS method is that an alarm (at given time points
set by the researcher) notifies the user to respond to the questions
onscreen. The same subjective appetite questions and anchors
used for the P&P VAS were uploaded onto the PRO-DiaryÓ devices
and appeared as separate VAS on the screen (24  13 mm) one at a
time, in the same order of appearance as the P&P VAS (i.e. hunger,
prospective food consumption, and fullness). The children were
asked to read each question carefully and respond using their fin-
ger via the touch sensitive ‘slider’ (39  5 mm) function at the bot-
tom of the PRO-DiaryÓ interface. The touch sensitive ‘slider’
enabled the children to move a vertical cursor along the horizontal
VAS. Once the vertical cursor was in place, the children were in-
structed to press the ‘O’ button to confirm their response. Using
the micro USB provided with the PRO-DiaryÓ devices the data
was downloaded and subsequently imported into Microsoft Excel.
With regards to the unit of measurement for the PRO-DiaryÓ the
value was given as a decimal between 0.00 and 1.00. Given that
the P&P VAS were rated on 100 mm horizontal lines, the PRO-Dia-
ryÓ values were appropriately converted in accordance with the
P&P VAS ratings. Furthermore, converting the values from decimals
to millimetres provided more ecologically valid results.
Analysis
Microsoft Excel was used for all data analysis. Mean ± SD were
calculated for all data. Visual analogue scale ratings for subjective
appetite (hunger, prospective food consumption and fullness) for
each method (P&P VAS and PRO-DiaryÓ) were calculated as
time-averaged area under the curve (AUC) for the study period
(11:30–21:00; 9.5 h). Data were checked for heteroscedasticity,
to enable the appropriate statistical technique to be employed.
Subsequently, the agreement between the P&P VAS method and
PRO-DiaryÓ technique for assessing free-living continuous subjec-
tive appetite sensations (hunger, prospective food consumption
and fullness) was explored using Pearson product-moment corre-
lation coefficients, 95% limits of agreement and 95% confidence
intervals calculated using the Bland–Altman method (Bland & Alt-
man, 1986). Statistical significance was accepted at p < 0.05 for all
analyses.
Results
Hunger
Mean ± SD for hunger (mm), for the P&P VAS and PRO-Diary
technique over the study period (11:30–21:00; 9.5 h) are pre-
sented in Fig. 1a. Correlation coefficients between the P&P VAS
and PRO-Diary ratings for hunger were significant (r = 0.71;
p = 0.009) (Fig. 1b). Mean time-averaged AUC hunger for the P&P
VAS and PRO-DiaryÓ technique were 45 ± 18 and 44 ± 13 mm,
respectively. The 95% limits of agreement were 1 ± 25 mm
(Fig. 1c). This indicated that 95% of the population will have differ-
ences between their hunger ratings of 26 to +24 mm, when as-
sessed using the P&P VAS and PRO-DiaryÓ techniques. However,
the 95% confidence interval for the mean bias indicated that the
true difference in hunger, assessed using the P&P VAS and PRO-
DiaryÓ technique lies between 8 and +6 mm. Given that the
95% confidence interval contains zero, this indicates that there is
no statistically significant difference in mean time-averaged AUC
hunger ratings when assessed using the P&P VAS and PRO-DiaryÓ
technique.
Prospective food consumption
Mean ± SD for prospective food consumption (mm), for the P&P
VAS and PRO-Diary technique over the study period (11:30–21:00;
9.5 h) are presented in Fig. 2a. Correlation coefficients between the
P&P VAS and PRO-Diary ratings for prospective food consumption
were significant (r = 0.75; p = 0.005) (Fig. 2b). Mean time-averaged
AUC prospective food consumption for the P&P VAS and PRO-Dia-
ryÓ technique were 43 ± 16 and 43 ± 13 mm, respectively. The 95%
limits of agreement were 0 ± 21 mm (Fig. 2c). This indicated that
95% of the population will have differences between their prospec-
tive food consumption ratings of 21 to +21 mm, when assessed
using the P&P VAS and PRO-DiaryÓ techniques. However, the
95% confidence interval for the mean bias indicated that the true
difference in prospective food consumption, assessed using the
P&P VAS and PRO-DiaryÓ technique lies between 6 and +6 mm.
Given that the 95% confidence interval contains zero, this indicates
that there is no statistically significant difference in mean
time-averaged AUC prospective food consumption ratings when
assessed using the P&P VAS and PRO-DiaryÓ technique.
 P.L.S. Rumbold et al. / Appetite 69 (2013) 180–185 183

(a)
100 100
(a)

Prospective food consumption (mm)

90 90
80 80
70 70
Hunger (mm)

60 60
50 50
40 40
30 30
20 20
10 10
0 0
11:30 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 11:30 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00
Time (h) Time (h)
P&P VAS PRO-Diary P&P VAS PRO-Diary

P&P VAS prospective food consumption

70 70
(b) (b)
60 60
P&P VAS hunger (mm)

50 50

40

(mm)
40

30
30
20
20
10
10
0
0 0 10 20 30 40 50 60 70
0 10 20 30 40 50 60 70 80 PRO-Diary prospective food consumption (mm)
PRO-Diary hunger (mm)
Difference in time-averaged AUC prospective food

(c)
consumption (PRO-Diary - P&P VAS; mm)

25
(c)
30 Mean+1.96SD
20
Differecne in time-averaged AUC hunger

Mean+1.96SD

20 15
(PRO-Diary - P&P VAS; mm)

10
10 5
Mean bias
0
Mean bias
0 -5
-10
-10
-15
Mean-1.96SD
-20
-20
Mean-1.96SD -25
0 10 20 30 40 50 60 70
-30 Mean time-averaged AUC prospective food consumption
0 10 20 30 40 50 60 70 (PRO-Diary and P&P VAS; mm)
Mean time-averaged AUC hunger
(PRO-Diary and P&P VAS; mm) Fig. 2. (a) Mean ± SD for prospective food consumption ratings (mm) derived from
the P&P VAS and PRO-Diary technique over the study period (9.5 h). (b) Correlation
Fig. 1. (a) Mean ± SD for hunger ratings (mm) derived from the P&P VAS and PRO- coefficients for prospective food consumption ratings were significant (r = 0.75;
Diary technique over the study period (9.5 h). (b) Correlation coefficients for hunger p = 0.005). (c) Bland and Altman plot for time-averaged AUC for prospective food
ratings were significant (r = 0.71; p = 0.009). (c) Bland and Altman plot for time- consumption ratings. AUC, area under the curve; VAS, visual analogue scales.
averaged AUC for hunger ratings. AUC, area under the curve; VAS, visual analogue
scales.

95% confidence interval contains zero, this indicates that there is

no statistically significant difference in mean time-averaged AUC
Fullness fullness ratings when assessed using the P&P VAS and PRO-DiaryÓ
technique.
Mean ± SD for fullness (mm), for the P&P VAS and PRO-Diary
technique over the study period (11:30–21:00; 9.5 h) are pre-
sented in Fig. 2a. Correlation coefficients between the P&P VAS
and PRO-Diary ratings for fullness were significant (r = 0.65; Discussion
p = 0.023) (Fig. 3b). Mean time-averaged AUC fullness for the P&P
VAS and PRO-DiaryÓ technique were 55 ± 16 and 48 ± 12 mm, This study was the first to investigate the agreement between
respectively. The 95% limits of agreement were 6 ± 24 mm the P&P VAS method and the PRO-DiaryÓ technique, for assessing
(Fig. 3c). This indicated that 95% of the population will have differ- free-living continuous subjective appetite sensations (hunger, pro-
ences between their fullness ratings of 31 to +18 mm, when as- spective food consumption and fullness) in 7–10 year old children.
sessed using the P&P VAS and PRO-DiaryÓ techniques. However, Consequently, there are a very limited number of directly compa-
the 95% confidence interval for the mean bias indicated that the rable studies in young people which can be used to explore these
true difference in fullness, assessed using the P&P VAS and PRO- data, therefore findings from adult studies will be used for the pur-
DiaryÓ technique lies between 14 and +1 mm. Given that the pose of this discussion.
184 P.L.S. Rumbold et al. / Appetite 69 (2013) 180–185

ger = 1.7 mm; prospective food consumption = 0.6 mm) and

(a)
120

100
indeed are more favourable compared to data presented by Zabel
et al. (2009) (hunger = 6.2 mm; prospective food consump-
Fullness (mm)

80 tion = 5 mm). Subjective fullness seemed to be the appetite

60
parameter that disagreed with the current literature, with a mean
difference between techniques of 6 mm in the present study com-
40 pared to 1.6 mm (Stratton et al., 1998) and 2.6 mm (Zabel et al.,
20 2009). Based on the power analysis (as previously described), it
suggests that the sample size of n = 12 in the present study was
0
11:30 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00
sufficient enough to detect any within-subject differences in sub-
Time (h) jective hunger, fullness and prospective food consumption (Flint
P&P VAS PRO-Diary
et al., 2000). The mean difference values for hunger, fullness and
prospective food consumption, between techniques (P&P VAS;
80 PRO-DiaryÓ technique) all lay within the acceptable level of

70
(b) 10 mm (10%). Thus, indicating that there was no meaningful
difference in the free-living monitoring of these subjective appetite
P&P VAS fullness (mm)

60 variables in 7–10 year old children. Moreover, it is imperative to

50 explore the variability of the ratings (standard deviations) and
limits of agreement. In the present study the standard deviations
40
were high, resulting in large limits of agreement (hun-
30 ger = 1 ± 25 mm; prospective food consumption = 0 ± 21 mm;
20 fullness = 6 ± 24 mm). Thus as with the majority of adult litera-
ture, despite low mean differences between techniques (Delargy
10
et al., 1996; Stratton et al., 1998; Stubbs et al., 1997; Stubbs et al.,
0 2001; Whybrow et al., 2006; Zabel et al., 2009), electronic appetite
0 10 20 30 40 50 60 70 80 90 100
PRO-Diary fullness (mm)
monitoring devices and P&P VAS should not be used interchange-
ably, due to the large limits of agreement values. Subsequently,
Mean+1.96SD the findings from the present study confirm that this is also the rec-
Difference in time-averaged AUC fullness

20
ommendation when monitoring free-living, continuous subjective
15
(c)
(PRO-Diary - P&P VAS; mm)

10 appetite sensations (hunger, prospective food consumption and

5 Mean bias
0 regarding the inherent advantages, disadvantages, practicality of
-5
-10
-15
-20
-25
-30
-35
0 10 20 30 40
Mean time-averaged AUC fullness
(PRO-Diary and P&P VAS; mm)
Fig. 3. (a) Mean ± SD for fullness ratings (mm) derived from the P&P VAS and PRO-
Diary technique over the study period (9.5 h). (b) Correlation coefficients for
fullness ratings were significant (r = 0.65; p = 0.023). (c) Bland and Altman plot for
time-averaged AUC for fullness ratings. AUC, area under the curve; VAS, visual
analogue scales.
Despite being administered simultaneously, there was a slight
bias for lower scores to be reported using the PRO-DiaryÓ tech-
nique compared to the P&P VAS for subjective hunger and fullness
(mean differences of 1 mm and 6 mm, respectively). However,
the 95% confidence intervals for hunger ( 8 to +6 mm) and fullness
( 14 to +1 mm) demonstrated a lack of statistical difference in
mean time-averaged AUC hunger and fullness ratings. Similarly,
for prospective food consumption there was no difference (0 mm)
between the PRO-DiaryÓ technique and P&P VAS, which was also
reinforced by the 95% confidence interval ( 6 to +6 mm). Such con-
sistently lower values (although not statistically significant) de-
rived from electronic appetite rating scales compared to P&P VAS
is a consistent finding within the available adult literature (Delargy
et al., 1996; Stratton et al., 1998; Zabel et al., 2009). More specifi-
cally, the findings in the present study with regards to mean differ-
ences between electronic appetite ratings and P&P VAS for hunger
and prospective food consumption ( 1 mm and =0 mm, respec-
tively), agree with data presented by Stratton et al. (1998) (hun-
fullness) in 7–10 year old children. Furthermore, consideration
each technique (PRO-DiaryÓ and P&P VAS), research setting (exper-
imental laboratory or free-living) and research question should be
considered prior to the commencement of appetite and energy reg-
Mean-1.96SD ulation research in young children.
This was the first study to explore the agreement between the
P&P VAS method and the PRO-DiaryÓ technique, for assessing
50 60 70 80 free-living continuous subjective appetite sensations (hunger, pro-
spective food consumption and fullness) in 7–10 year old children.
Subsequently, we believe this study to have a high degree of eco-
logical validity, given that the agreement between the two appetite
monitoring techniques were explored in a free-living, representa-
tive environment. Despite this strength however, we did not ex-
plore the agreement between the PRO-DiaryÓ and P&P VAS
under a more experimental laboratory setting (i.e. fixed intakes
at fixed mealtimes) or indeed the test–retest reproducibility of
the PRO-DiaryÓ under free-living conditions. Thus taken together,
these shortcomings suggest that these findings should only be ap-
plied to the specific sample population in question (7–10 year old
children), whilst appetite and energy regulation research is being
conducted in a free-living, naturalist environment. Consequently,
this work has provided a baseline for further work on two counts.
Firstly, the agreement between the P&P VAS method and the PRO-
DiaryÓ technique, for assessing subjective appetite sensations
(hunger, prospective food consumption and fullness) in children
should be explored (a) in a more experimental laboratory setting
and (b) in various population groups (overweight and/or obese
children and adolescents). Moreover, it must be recognised that
there is no ‘gold standard’ for assessing subjective appetite in en-
ergy regulation studies (in both adults and children), therefore
the aim of the present study was not to validate the use of the
PRO-DiaryÓ technique in young children (i.e. indicate that one
method is better than the other). Indeed in adults, saliva samples
provide a viable and alternative measure for exploring hormonal
appetite responses (ghrelin) (Cetinkaya et al., 2009), however
 P.L.S. Rumbold et al. / Appetite 69 (2013) 180–185
contamination is likely to impinge on hormonal concentrations if
nutritional interventions are implemented in energy regulation
studies. Thus, researchers should seek to explore the level of agree-
ment between simultaneous venous and arterialized finger prick
blood samples for the expression of appetite hormones. An ability
to collect hormonal appetite data via finger prick in children and
adolescent populations would potentially provide an accepted
means for exploring hormonal appetite responses in paediatric en-
ergy regulation studies.
In conclusion, the PRO-DiaryÓ technique is an effective method
to employ when continuously monitoring free-living appetite sen-
sations in 7–10 year old children, but should not be used inter-
changeably with traditional P&P VAS.
References
Bellissimo, N., Thomas, S. G., Goode, R. C., & Anderson, G. H. (2007). Effect of short-
duration physical activity and ventilation threshold on subjective appetite and
short-term energy intake in boys. Appetite, 49(3), 644–651.
Bland, J. M., & Altman, D. G. (1986). Statistical methods for assessing agreement
between two methods of clinical measurement. Lancet, i, 307–310.
Bozinovski, N. C., Bellissimo, N., Thomas, S. G., Pencharz, P. B., Goode, R. C., &
Anderson, G. H. (2009). The effect of duration of exercise at the ventilation
threshold on subjective appetite and short-term food intake in 9–14 year old
boys and girls. International Journal of Behavioral Nutrition and Physical Activity,
6, 66–76.
Cetinkaya, Z., Aydin, S., Cerrahoglu, Y. Z., Ayten, R., Erman, F., & Aygen, E. (2009).
Changes in appetite hormone (ghrelin) levels of saliva and serum in acute
appendicitis cases before and after operation. Appetite, 52(1), 104–107.
Chambers, C. T., & Craig, K. D. (1998). An intrusive impact of anchors in children’s
faces pain scales. Pain, 78, 27–37.
Cole, T. J., Bellizzi, M. C., Flegal, K. M., & Dietz, W. H. (2000). Establishing a standard
definition for child overweight and obesity worldwide. International survey.
British Journal of Nutrition, 320, 1240–1246.
Delargy, H., Lawton, C. L., Smith, F. C., King, N., & Blundell, J. E. (1996). Electronic
appetite rating system (EARS). Validation of continuous automated monitoring
if motivation to eat. International Journal of Obesity, 20(4), 104.
Dodd, C. J., Welsman, J. R., & Armstrong, N. (2008). Short-term energy intake and
appetite responses to exercise in lean and overweight girls. Appetite, 51,
482–488.
Flint, A., Raben, A., Blundell, J. E., & Astrup, A. (2000). Reproducibility, power and
validity of visual analogue scales in assessment of appetite sensations in single
test meal studies. International Journal of Obesity, 24, 38–48.
Gibbons, C., Caudwell, P., Finlayson, G., King, N., & Blundell, J. E. (2011). Validation of
a new hand-held electronic data capture methods for the continuous
monitoring of subjective appetite sensations. International Journal of
Behavioral Nutrition and Physical Activity, 8(57), 1–7.
Gift, A. G. (1989). Visual analogue scales. Measurement of subjective phenomena.
Nursing Research, 286–288.
Hanet, N., Salah, S., & Lluch, A. (2010). A training session to rate appetite feelings
increases the robustness of methodology. Appetite, 55(1), 168.
Hill, A. J., & Blundell, J. E. (1982). Nutrients and behaviour. Research strategies for
the investigation of taste characteristics, food preferences, hunger sensations
and eating patterns in man. Journal of Psychological Research, 17, 203–212.
Hufford, M., & Shields, A. (2002). Electronic subject diaries. An examination of
applications and what in the field. Applied Clinical Trials, 11, 46–56.
Mirwald, R. L., Baxter-Jones, A. D., Bailey, D. A., & Beunen, G. P. (2002). An
assessment of maturity from anthropometric measurements. Medicine and
Science in Sports and Exercise, 34(4), 689–694.
185
Moore, M. S., Dodd, C. J., Welsman, J. R., & Armstrong, N. (2004). Short-term appetite
and energy intake following imposed exercise in 9–10-year-old girls. Appetite,
43, 127–134.
Price, D. D., McGrath, P. A., Rafii, A., & Buckingham, B. (1983). The validation of
visual analogue scales as ratio scale measures for chronic and experimental
pain. Pain, 17, 45–56.
Rumbold, P. L. S., & Dodd, C. J. (2007). Energy intake and appetite responses of
adolescent girls following a netball specific exercise protocol. Proceedings of the
Nutrition Society, 66(38A).
Rumbold, P. L. S., Dodd-Reynolds, C. J., & Stevenson, E. J. (2013). Informing primary
school nutritional policy: Effects of mid-morning snacks on appetite and energy
control. Food and Nutrition Sciences, 4(5), 529–537.
Rumbold, P. L. S., St Clair Gibson, A., Allsop, S., Stevenson, E., & Dodd-Reynolds, C. J.
(2011). Energy intake and appetite following netball exercise over 5 days in
trained 13–15 year old girls. Appetite, 56(3), 621–628.
Rumbold, P. L. S., St Clair Gibson, A., Stevenson, E., & Dodd-Reynolds, C. J. (2011).
Agreement between two methods of dietary data collection in female
adolescent netball players. Appetite, 57, 443–447.
Rumbold, P. L. S., St Clair Gibson, A., Stevenson, E. J., King, J. A., Stensel, D. J., & Dodd-
Reynolds, C. J. (2013). Influence of netball-based exercise on energy intake,
subjective appetite and plasma acylated ghrelin in adolescent girls. Applied
Physiology, Nutrition and Metabolism, 38, 1–8.
Stratton, R. J., Stubbs, R. J., Hughes, D. A., King, N. A., Blundell, J. E., & Elia, M. (1998).
Comparison of the traditional paper visual analogue scale questionnaire with an
Apple Newton electronic appetite rating system (EARS) in free living subjects
feeding ad libitum. European Journal of Clinical Nutrition, 52, 737–741.
Stubbs, R. J., Hughes, D. A., Johnstone, A. M., Rowley, E., Ferris, S., Elia, M., et al.
(2001). Description and evaluation of a Newton-based electronic appetite rating
system for temporal tracking of appetite in human subjects. Physiology and
Behaviour, 72(4), 615–619.
Stubbs, R. J., Hughes, D. A., Johnstone, A. M., Rowley, E., Reid, C. A., Elia, M., et al.
(2000). The use of visual analogue scales to assess motivation to eat in human
subjects. A review of their reliability and validity with an evaluation of new
hand-held computerized systems for temporal tracking of appetite ratings.
British Journal of Nutrition, 84, 405–415.
Stubbs, R. J., Rowley, E., Hughes, D. A., Johnstone, A. M., Reid, C. A., King, N., et al.
(1997). Evaluating a new electronic appetite rating system (EARS). International
Journal of Obesity, 21, 391.
Thivel, D., Isacco, L., Montaurier, C., Boirie, Y., Duché, P., & Morio, B. (2012). The 24-h
energy intake of obese adolescents is spontaneoulsy reduced after intense
exercise. A randomized controlled trial in calometric chambers. PLoS One, 7(1),
1–7.
Thivel, D., Isacco, L., Rousset, S., Boirie, Y., Morio, B., & Duche, P. (2011). Intensive
exercise. A remedy for childhood obesity? Physiology and Behavior, 102,
132–136.
Thivel, D., Isacco, L., Taillardat, M., Rousset, S., Boirie, Y., Morio, B., et al. (2011).
Gender effect on exercise-induced energy intake modification among obese
adolescents. Appetite, 56(3), 658–661.
van Strien, T., & Oosterveld, P. (2008). The children’s DEBQ for assessment of
restrained, emotional, and external eating in 7–12-year-old children.
International Journal of Eating Disorders, 41(1), 72–81.
Wewers, M. E., & Lowe, N. K. (1990). A critical review of visual analogue scales in the
measurement of clinical phenomena. Research in Nursing and Health, 13,
227–236.
Whybrow, S., Stephen, J. R., & Stubbs, R. J. (2006). The evaluation of an electronic
visual analogue scale system for appetite and mood. European Journal of Clinical
Nutrition, 60, 558–560.
Zabel, R., Ash, S., Bauer, J. D., & King, N. A. (2009). Assessment of subjective appetite
sensations in hemodialysis patients. Agreement and feasibility between
traditional paper and pen and novel electronic appetite rating system.
Appetite, 52(2), 525–527.