Introduction

Physical activity wearable devices are a popular and growing market for monitoring
physical activity, energy expenditure, sleep, and other health behaviors. Triaxial accelerometers
have been the gold standard, objective method, for assessing physical activity in field-based
research because they are small, non-invasive, easy-to use, and provide an objective indication
of physical activity behavior (Bassett 2012, Freedson 2012, Welk 2012). While still widely used
in research, companies have now begun to target a larger consumer market. Consumer-based
activity monitors became available in 2008-2009 when FitBit released its first wearable tracker.
FitBit became popular because it engages consumers and provides them with accurate data
related to physical activity at their fingertips through interactive applications on smartphones
(Lyons 2014, Cadmus-Bertram 2015, Almalki 2013). These consumer-based activity monitors
provide consumers and health professionals with the ability to estimate levels of physical activity
by day and track data across time. With technological advances, companies at the forefront of
the market, FitBit and Jawbone, are now focused on enhancing their monitoring technologies to
create more comprehensive and useful self-monitoring tools to help consumers track activity in
free-living.
These technologies include pedometers, global positioning systems (GPS), and heart
rate (HR) monitors. Pedometers and GPS have been calibrated to provide measurements such
as exercise intensity and energy expenditure. HR monitors, originally marketed to athletes, have
been modified and marketed to appeal to recreational consumers interested in health and
weight control and to health professionals interested in objective tracking for their clients during
free-living activities. A company has developed a new accelerometry based medical-grade
device, Metria IH1 (IH1), which has incorporated additional sensors to monitor sleep quality via
heat flux and galvanic skin response (GSR). The IH1 was designed for patients with disease or
patients attempting weight loss. GSR provides a mechanism to determine how much the
individual is sweating through changes in the skins electrical conductivity. Skin temperature is
also reflective of activity level. Heat flux determines how much heat is being produced by your
muscles and radiated into the ambient environment. Collectively, data from the sensors is
claimed to measure calorie burn with less than 10 percent error (McGrath 2013). In addition,
these sensors also measure sleep quantity and quality. Due to the continual technological
advances and release of research-grade activity trackers, data must be obtained related to the
validity under free-living conditions to assist consumers, health professionals, and researchers
in making informed decisions when selecting the appropriate model of an activity tracker. The
consumer based physical activity trackers were previously validated through several various
comparison measures. Validity for steps were compared against manual counts, pedometer
counts, video analysis, and step counts measured by the Actigraph GT3X+. Trackers worn at
the hip generally outperformed trackers worn at the wrist for step counts (Simpson 2015).
Energy expenditure was validated through direct and indirect calorimetry. There were no
substantial differences in energy expenditure correlations between devices being worn on the
hip or wrist (Diaz 2015, Evenson 2015).
The purpose of the study is to compare the leading two commercial physical activity
tracking devices, FitBit Surge (FS) and Jawbone UP3 (JU3), and one medical-grade physical
activity tracking device, Metria IH1, against validated laboratory equipment, EKG Cardiac
Science heart rate monitor and Metabolic System Maxx-II, to analyze the validity of the following
constructs: Heart rate, step count, and energy expenditure. This study will parallel what a typical
individual would do during an independent workout of moderate intensity which was completed
in a laboratory setting. The (IH1) is currently being used for tracking accelerometer based data
in the areas of fitness and lifestyle.

Methods
Participants
Fourteen apparently healthy college-age men (n=12) and women (n=3) were recruited
through word of mouth and class announcements during the fall semester 2015 at a small
private university in the Midwest. All participants met the following inclusion criteria through a
physical activity readiness questionnaire (PAR-Q+): 1) 18 years and older, 2) no
musculoskeletal limitations, 3) no recent surgeries in the past six months, 4) not prescribed any
medications that affect body weight or metabolism, 5) no current tobacco use, 6) participants
need to own an Android or Apple smartphone, and 7) no adhesive, latex, or nickel allergies. All
participants were made aware of the purpose and procedures of the study prior to signing a
release of liability and informed consent.
Instruments
Jawbone UP3 (JU3) The Jawbone UP3 (Jawbone, San Francisco, CA) is a triaxial
accelerometer. Bio-impedance markers include heart rate, respiration, and galvanic skin
response (GSR). Jawbone quantitatively presents the user with information on caloric
expenditure, steps taken, sleep quality, and heart rate.
Fitbit Surge (FS) The FS (Fitbit Inc., San Francisco, CA) is a triaxial accelerometer and
gyroscope that can measures steps taken, floors climbed, distance traveled, caloric
expenditure, sleep quality, and heart rate (HR).
MetriaIH1 Lifestyle Assessment System (IH1) (Avery Dennison Corporation., Pasadena, CA)
includes a triaxial accelerometer along with physiological sensors: skin temperature sensor,
near body temperature sensor and a galvanic skin response (GSR) that estimates the
conductivity of the skin. IH1 collects a plethora of data every minute for 24 hours up to 7 days,
providing an accurate indication of the participants lifestyle including caloric expenditure, activity
level, steps taken, and sleep duration and quality.
EKG Cardiac Science - CareCenter MD (Waukesha, WI) allows the researching team to
monitor the participants heart rate and rhythm during the various exercise modalities. Heart rate
variability will be compared between the modalities. The (make and model) will allow viewing
and analysis, of heart rate and rhythm, between the pre and post-test sessions.
Metabolic System Maxx-II (AEI Technologies, Pittsburgh, PA) will analyze the participants O2
and CO2 levels via breath by breath analysis. Through the metabolic system the participants
energy expenditure will be determined. Data collected through the metabolic system will then be
exported to Excel for further data extrapolation.
Procedures
Participants attended two sessions located in a graduate exercise physiology laboratory
at a small private university in the Midwest. The initial informational was used to record baseline
information, such as the participants height, weight, gender, and date of birth. Weight was
measured to the nearest 0.1 kg with the participants in lightweight shorts and t-shirt, and without
shoes. Height was measured to the nearest 0.1 cm with participants not wearing shoes.
Participants were instructed to download the Fitbit and UP applications (apps) on their personal
smartphone. Participants entered their recorded baseline information into the apps to ensure the
greatest accuracy in energy expenditure values. Participants then watched a video on how to
properly place the FS, on the right wrist, and the JU3, on the left wrist, as well as basic wearing
and upkeep instructions. The researchers synced the devices to the participants smartphones.

A three-inch by three-inch area was then cleansed with an alcohol wipe and dried for two
minutes per the manufacturer's instructions for placement of the IH1 patch. The IH1 patch was
activated and then placed on the medial left triceps. The patch was applied and immediately
held on the participants arms for thirty seconds to ensure adhesion. Participants could carry on
normal daily activities once the tracking devices had been placed. Lastly, the participants
received a handout that regarded their four hour fast prior to data collection the next day with
dietary recommendations. A food log with examples on how to record was given, as well, diet
recommendations for their meal prior to testing will be to consume a light meal consisting of
protein and carbohydrates, limiting fat (400 calories).
After 24 hours passed, participants reported back to the laboratory, dressed in the same
shorts, t-shirt, and shoes worn during the informational session with the FS on right wrist and
JU3 on left wrist. Participants were asked to turn in their dietary food log regarding their pre-test
meal. They then rested for five minutes for collection of resting vitals; heart rate, blood pressure,
and EKG. The following activities were chosen for analyses, after extensive discussions, on the
basis that moderate to vigorous physical activity has established health benefits which are
widely supported in scientific literature and because it is the focus of public health physical
activity guidelines (Diaz 2015). The participants walked and jogged on the treadmill (Woodway
Continuum) at four different intensities for three to four minutes. Duration of time for each
activity was determined by when the participants reached their steady state heart rate plus two
minutes. Intensities are classified as slow (1.9 mph), moderate (3.0 mph), brisk (4.0 mph), and
jogging (5.2 mph). The participants then cycled (Monark Ergomedic 828E) at a moderate
intensity of 50 revolutions per minute (RPM) at a resistance of 60 Watts for three to four minutes
until steady state heart rate was reached. Participants then performed a resistance training
exercise, the bench press at 6-8% body weight (BW) in kilograms. Participants achieved a
steady-state HR of 30% heart rate reserve (HRR) and continued to perform the bench press for
two to three minutes, at a pace of 60 beats/minute set by a metronome. Exercise order was
randomly selected via a random number generator. Participants were randomly selected to
perform two additional activities for re-test. Participants were unaware that they were being retested so results would not be biased. The participants were monitored by a 12 lead EKG,
Cardiac Science - CareCenter MD, to monitor heart rate and heart rhythm during the various
activities in order to classify exercise intensity. Indirect calorimetry was measured through the
use of a metabolic cart, Maxx II - AEI Technologies. Steps were counted manually in order to
compare with step counts from IH1 and FS.
During the next five days, participants were instructed to simultaneously wear the FS
and JU3 trackers for approximately one-24 hour period in order to capture a full 24 hours of
activity data. The wear period was not limited to a particular period of the week, and no
guidelines or restrictions on activity levels or sleep were provided, however, the participants
were highly encouraged to engage in moderate intensity physical activity during that 24 hour
period. This ensured that the study broadly represented free living conditions.
On day six, the participants returned to the exercise physiology laboratory; the IH1 patch
was removed by a researcher, and then the FS and JU3 were turned in. Data collection took
place in November 2015.
Data Analysis
Descriptive analyses of laboratory and demographic data were conducted to examine
associations between the consumer based physical activity trackers and IH1 to standardized
laboratory grade equipment, specifically looking at validity of step count, energy expenditure,

and heart rate. The JU3 physical activity tracker was omitted from individual protocol analysis
because the activity tracker did not report minute-by-minute data.
Step count validity was determined by comparing the consumer level physical activity
trackers with manually counted observed steps for each protocol and for total steps. The validity
of energy expenditure was measured through the Maxx II and compared to the recorded
energy expenditure from the three devices for each protocol and totals. Heart rate validity was
analyzed through maximum heart rate recorded per protocol for the FS in comparison with the
Cardiac Science EKG through mean absolute difference (MAD).
Bland altman plots with corresponding limits of agreement and fitted lines (from
regression analyses between mean and difference) with their corresponding parameters were
presented. A fitted line that provides a slope of 0 and an intercept of 0 signifies perfect
agreement. MAD and Pearsons r were also calculated to provide an indicator of overall
measurement error and linear correlation, respectively. MAD is a measure of statistical
dispersion equal to the average absolute difference of two independent variables, calculated
from a probability distribution. These methods of analysis were chosen because of predicted
outliers and skewed data.
Results
A total of fourteen participants agreed to participate in the study. However, due to subject
illness, non-compliance, data extraction error, and device malfunction only eight participants
data was analyzed. Gender distribution was significantly skewed with eight males (Age 22 1.4)
and two females (Age 21.5 2.1). Additional descriptive statistics for participants are provided in
Table 1.
Table 1. Physical characteristics of male (n=6) and female (n=2) subjects
Male
Range
Female
Age (yrs)
22 1.4
20.0 - 24.0
21.5 2.1

Range
20.0 - 23.0

Height (cm)

177.7 7.9

164.0 - 185.7

161.3 3.6

158.8 - 163.8

Weight (kg)

89.6 15.4

64.9 - 106.6

59.6 1.6

58.5 - 60.8

BMI (kg/m2)

28.3 3.2

24.2-32.6

23 1.7

21.8-24.2

Yrs: years of age

Kg: kilograms

Cm: centimeters
BMI: body mass index

Table 2 shows the correlations and MAD between energy expenditure (kilocalories) and
steps between the consumer based physical activity trackers compared to laboratory grade
equipment. All of the consumer based physical activity trackers measured total steps and
energy expenditure. Energy expenditure measured for FS, JU3, and IH1 had moderate
correlations with the laboratory grade equipment (r=0.51-0.65). Correlations between the
physical activity trackers and observed steps ranged from poor for JU3 (0.08) to very strong for
IH1 (r=0.94)
Table 2. Pearson r and MAD of kcals and steps in laboratory
Kilocalories
Pearson R
MAD
Pearson R
Fitbit 1.9
0.181
7.43
0.94
Fitbit 3.0
0.191
6.10
0.88
Fitbit 4.0
0.513
7.64
0.93
Fitbit 5.2
0.179
12.39
0.58
Fitbit Cycle
0.102
7.85
n/a

Steps
MAD
59.09
57.81
89.25
69.14
2.49

Fitbit BP
Fitbit Total
JU3 Total
Metria 1.9
Metria 3.0
Metria 4.0
Metria 5.2
Metria cycle
Metria BP
Metria total
BP: bench press

0.49
0.56
0.65
0.09
-0.05
-0.11
0.44
0.17
0.45
0.51
JU3: Jawbone UP3

8.69
48.42
61.34
3.09
4.08
8.60
9.17
8.17
6.09
43.42

n/a
0.52
0.08
0.68
0.86
0.96
0.94
n/a
n/a
0.94

108.51
316.92
484.50
51.89
58.37
75.62
50.12
n/a
n/a
228.44

Table 3 depicts the correlation between each protocol for the FS and Cardiac Science
EKG. Recorded heart rate for JU3 was not analyzed in the laboratory or free living due to
smartphone consumer based app only reporting resting and passive heart rate. FS heart rate
was compared to the average of steady state heart rate recorded by Cardiac Science EKG.
Cycling and 4.0 mph had the best correlation (r=0.80-0.89), with the largest MAD, while bench
press reported the worst correlation (r=0.26). The cycling protocol had the largest MAD (20.69).
Table 3. Pearson r and MAD for heart rate in laboratory between FS and Cardiac Science EKG
Pearson R
MAD
1.9 mph
0.47
11.90
3.0 mph
0.61
8.13
4.0 mph
0.89
11.40
5.2 mph
0.56
8.50
cycle
0.80
20.68
BP
0.26
14.92
Table 4 represents an average 24-hour daily routine for each participant. FS for both
steps and total energy expenditure had the strongest correlation when compared to the IH1
(r=0.97). JU3 had a moderate correlation for total energy expenditure (r=0.82) and a strong
correlation for total steps (r=0.92).
Table 4. Pearson r and MAD of FS and JU3 in freeliving
Freeliving Total Steps
Pearson R
FitBit vs Metria Total Steps
0.97
Jawbone vsThe Metria Total Steps
0.92
Freeliving Total Calories
Pearson R
FitBit vs Metria Total Calories
0.97
Jawbone vs Metria Total Calories
0.82

MAD
5403.65
4889.11
MAD
659.96
691.22

Figure 1 shows a scatterplot analyzing the Pearsons r against the MADs for the total
steps and kcals of various physical activity tracking devices within the free-living setting.
Correlations and difference varied between the two physical activity trackers. Kilocalories had
the strongest correlation and smallest differences for the Fitbit, followed by steps.

Table 5. Means (SD), MAD, and Bland Altman output for each device in the laboratory setting on
the constructs of total steps and kcals
FitBit Surge

Jawbone UP3

Metria IH1

Total Steps
Mean (SD)

2766.87 345.68

2253.25 1076.26

2217.12 304.53

0.52

0.08

.94

MAD

316.93

484.50

228.44

Bias

415.87

-97.75

-133.87

CI - Lower

160.11

-1006.20

-221.95

CI - Upper

671.63

810.70

-45.79

Total Calories
Mean (SD)

254.87 66.86

261.28 120.05

158.30 26.71

0.56

0.65

0.51

MAD

48.42

61.34

43.42

Bias

43.23

59.30

-53.33

CI - Lower

-15.82

-26.19

-99.84

CI - Upper

102.29

144.79

-6.83

CI: confidence interval

SD: standard deviation

Table 5 portrays the correlation, MAD, bias, and confidence intervals for total steps and
total calories analyzed by the consumer based physical activity trackers relative to the MAXII
and observed. The IH1 patch had the strongest correlation for steps (r=0.94). Total energy
expenditure for the three devices in the laboratory showed moderate correlations when
compared to Maxx II. JU3 showed the greatest variation in energy expenditure recordings of all
three consumer based physical activity trackers.
Discussions/Limitations/Conclusions
This study aimed to examine the validity of the FS and JU3 physical activity monitors
during a typical moderate intensity workout. In general, the FS and IH1 physical activity tracking
devices were valid in measuring steps. JU3 had weak correlations for steps. However,
measures of energy expenditure for all three devices showed weaker validity when compared to
laboratory grade equipment. This demonstrates, in general, moderate correlation with researchgrade accelerometers. However, large mean absolute deviations were observed with step
counts. In a comparison between data collected from standardized laboratory equipment and
the FS and JU3, the FS generally performed stronger. Steps were generally counted with a high
degree of accuracy by both physical activity trackers. Although, the JU3 had a larger standard
deviation compared to the FS. The findings of the current study concur with similar studies in
that step counts are valid in healthy subjects (Takacs 2013). The current findings from this study

add to the previous research by indicating that these devices are valid tools for health
professionals to utilize in order to prescribe exercise prescriptions and track progress objectively
instead of subjectively.
The FS and JU3 measured total daily energy expenditure (TDEE). In general, analyzing
one-24 hour time period, from 12:00 am to 12:00 pm, showed a moderate to strong TDEE (r =
0.82-0.97). Of the two devices, the FS was better than the JU3 when compared to the Metria
IH1 in free-living. However, the FS, JU3, and IH1 in the laboratory all had moderate TDEE (r =
0.51-0.65) compared to the MaxxII. Opposite of most findings, the FS and JU3 underestimated
energy expenditure in the laboratory setting compared to the MaxxII (Lee 2014). However, in
free-living conditions, both devices had improved validity, another reason health professionals
should use these devices with clients.
Strengths of the current study are that the devices were testing in the laboratory and in
free-living conditions for a 24 hour period. This study analyzed heart rate, total steps, and TDEE
which are typically variables that health professionals care about. With those variables being
validated, health professionals can confidently prescribe exercise prescriptions to clients based
on target heart rate for various exercise intensities. Also, steps and energy expenditure can then
be utilized to track goals and weight loss.
While conducting the study, a number of limitations were identified that should be
addressed. The small sample size (n=8) and testing only college aged, apparently healthy,
young adults does not allow the results to be generalized to a larger population. In addition, the
small number of consumer based physical activity trackers that were analyzed was limited due
to budgetary reasons and concerns around participant burden wearing more than two physical
activity trackers. Furthermore, updated models for FitBit and Jawbone have entered the market
prior to data collection. Incomplete data sets were obtained for some values related to the JU3
(steps, TDEE) due to the inability to flag activities on the JU3, data extraction error, and
participant error. Caution should be taken when interpreting findings for the above mentioned
results.
There was noted inconsistency of heart rate recordings every minute for each protocol.
This could have affected steady state and hence total protocol duration. Also, there were
variations in steady state heart rate between test and re-test due to fatigue. This was
problematic because it was difficult to determine validity of heart rate for repeated measures.
When extracting data from the IH1 patch, it was difficult to sum every minute for each
protocol for steps and energy expenditure because the protocols would not consistently start at
the same minute the IH1 patch records. This lead to under and overestimation in some data
points due to variability in recording time. In addition, IH1 data for energy expenditure needed to
be converted from kilojoules to kilocalories. Total energy expenditure per protocol may not be as
accurate due to rounding.
Further research examining larger sample sizes that reflect the age and demographics
of the various target populations of consumer based physical activity trackers is needed. Also,
studying physical activity tracker size and fit compared to the accuracy of heart rate would be a
good addition to the previous study. Lastly, looking at activities of daily living in the laboratory
would reflect the typical daily routine of the average person.
Conclusions
In conclusion, the present study supports the validity of the FS for steps and energy
expenditure. Results with the FS heart rate must be viewed with caution due to inconsistent
findings compared to laboratory grade equipment. An advantage of wearable physical activity
trackers with bluetooth capabilities is that they offer additional feedback online and are less
obtrusive compared to standard research grade devices. The new physical activity trackers
provide goal setting features, tracking tools, social networking links, and online competitions
with friends.

References
1. Almalki M, Gray K, Sanchez FM. The use of self-quantification systems for personal
health information: big data management activities and prospects. Health Information
Science Systems. 2015;3(Suppl 1 HISA Big Data in Biomedicine and Healthcare 2013
Con):S1. doi: 10.1186/2047-2501-3-S1-S1
2. Bassett DR, Rowlands A, Trost SG. Calibration and validation of wearable monitors.
Med Sci Sports Exerc. 2012;44:S32-8.
3. Cadmus-Bertram LA, Marcus BH, Patterson RE, Parker BA, Morey BL. Randomized trial
of a Fitbit-Based physical activity intervention for women. Am J Prev Med.
2015;49(3):4148.
4. Diaz KM, Krupka DJ, Chang MJ, Peacock J, Ma Y, Goldsmith J, et al. Fitbit: An accurate
and reliable device for wireless physical activity tracking. Intl J Cardiol. 2015;185:13840
5. Evenson KR, Goto MM, Furberg RD. Systematic review of the validity and reliability of
consumer-wearable activity trackers. Intl J of Behavioral Nutrition and Physical Activity.
2015;12:159
6. Freedson P, Bowles HR, Troiano R, Haskell W. Assessment of physical activity using
wearable monitors: recommendations for monitor calibration and use in the field. Med
Sci Sports Exerc. 2012;44:S1-4.
7. Lee JM, Kim Y, Wlk, GJ. Validity of consumer-based physical activity monitors. Med Sci
Sports Exer. 2014; Feb 5
8. Lyons EJ, Lewis ZH, Mayrsohn BG, Rowland JL. Behavior change techniques
implemented in electronic lifestyle activity monitors: A systematic content analysis. J Med
Internet Res. 2014;16(8):e192. doi:10.2196/jmir.3469.
9. McGrath, M. J., & Scanaill, C. N. (2013). Sensor Technologies: Healthcare, Wellness
and Environmental Applications. Apress.
10. Simpson LA, Eng JJ, Klassen TD, Lim SB, Louie DR, Parappilly B, et al. Capturing step
counts at slow walking speeds in older adults: Comparison of ankle and waist placement
of measuring device. J Rehabil Med. 2015;47(9):8305
11. Takacs J, Pollock CL, Guenther JR, Bahar M, Napier C, Hunt MA. Validation of the fitbit
one activity monitor device during treadmill walking. J Sci Med Sport. 2013;17(5):496500
12. Welk JG, Mcclain J, Anisworth BE. Protocols for evaluating equivalency of
accelerometry-based activity monitors. Med Sci Sports Exerc. 2012;44:S39-49