Physiological Measurement

NOTE You may also like

- Validity of an automated algorithm to
Methods for estimating physical activity and identify waking and in-bed wear time in
hip-worn accelerometer data collected with
energy expenditure using raw accelerometry data a 24 h wear protocol in young adults
Joanne A McVeigh, Elisabeth A H Winkler,
Genevieve N Healy et al.
or novel analytical approaches: a repository,
- Parameterizing and validating existing
framework, and reporting guidelines algorithms for identifying out-of-bed time
using hip-worn accelerometer data from
older women
John Bellettiere, Yiliang Zhang, Vincent
To cite this article: Kimberly A Clevenger et al 2022 Physiol. Meas. 43 09NT01 Berardi et al.

- Accessibility and use of novel methods for

predicting physical activity and energy
expenditure using accelerometry: a
View the article online for updates and enhancements. scoping review
Karin A Pfeiffer, Kimberly A Clevenger,
Andrew Kaplan et al.

This content was downloaded from IP address 194.225.195.54 on 15/09/2022 at 09:12

 Physiol. Meas. 43 (2022) 09NT01 https://doi.org/10.1088/1361-6579/ac89c9

NOTE

RECEIVED
Methods for estimating physical activity and energy expenditure
15 November 2021
REVISED
using raw accelerometry data or novel analytical approaches: a
1 August 2022
ACCEPTED FOR PUBLICATION
repository, framework, and reporting guidelines
15 August 2022
PUBLISHED
Kimberly A Clevenger4,∗ , Alexander H K Montoye2, Cailyn A Van Camp1, Scott J Strath3 and
5 September 2022 Karin A Pfeiffer1
1
Michigan State University, Department of Kinesiology, United States of America
2
Alma College, Integrative Physiology and Health Science, United States of America
3
University of Wisconsin Milwaukee, Department of Kinesiology, United States of America
4
Utah State University, Department of Kinesiology and Health Science, United States of America
∗
Authors to whom any correspondence should be addressed.
E-mail: kimberly.clevenger@usu.edu

Keywords: open science, harmonization, machine learning, analysis, device, algorithm

Abstract
The proliferation of approaches for analyzing accelerometer data using raw acceleration or novel
analytic approaches like machine learning (‘novel methods’) outpaces their implementation in
practice. This may be due to lack of accessibility, either because authors do not provide their developed
models or because these models are difficult to find when included as supplementary material.
Additionally, when access to a model is provided, authors may not include example data or
instructions on how to use the model. This further hinders use by other researchers, particularly those
who are not experts in statistics or writing computer code. Objective: We created a repository of novel
methods of analyzing accelerometer data for the estimation of energy expenditure and/or physical
activity intensity and a framework and reporting guidelines to guide future work. Approach: Methods
were identified from a recent scoping review. Available code, models, sample data, and instructions
were compiled or created. Main Results: Sixty-three methods are hosted in the repository, in
preschoolers (n = 6), children/adolescents (n = 20), and adults (n = 42), using hip (n = 45), wrist
(n = 25), thigh (n = 4), chest (n = 4), ankle (n = 6), other (n = 4), or a combination of monitor wear
locations (n = 9). Fifteen models are implemented in R, while 48 are provided as cut-points, equations,
or decision trees. Significance: The developed tools should facilitate the use and development of novel
methods for analyzing accelerometer data, thus improving data harmonization and consistency across
studies. Future advances may involve including models that authors did not link to the original
published article or those which identify activity type.

Introduction

Body-worn accelerometers are widely-used for capturing energy expenditure and physical activity intensity in a
variety of populations and settings (Troiano et al 2014). Coupled with their small size and unobtrusiveness, the
ability to capture frequency, intensity, and duration of activity means that accelerometers are well suited for the
characterization of free-living behaviors (Strath et al 2013). Early approaches for analyzing accelerometer data
involved single linear regression equations to predict energy expenditure and/or development of cut-points for
classifying physical activity intensity (Freedson et al 1998). As early devices were limited by their storage capacity,
this often necessitated the use of pre-processed metrics like proprietary activity counts that are not comparable
across device brands (John & Freedson 2012, Troiano et al 2014, Sasaki et al 2016).
More recently, increased storage capacity and the availability of raw acceleration data in popular research-
grade accelerometers like the ActiGraph (Troiano et al 2014, Sasaki et al 2016) have led to the development of

© 2022 Institute of Physics and Engineering in Medicine

Physiol. Meas. 43 (2022) 09NT01 K A Clevenger et al

different approaches to analyzing these data (de Almeida Mendes et al 2018, Farrahi et al 2019, Álvarez-García
et al 2020). Use of raw acceleration data can facilitate comparisons across devices brands (de Almeida Mendes
et al 2018) while use of machine learning or other advanced modeling techniques can more fully make use of the
vast amount of information collected by a triaxial accelerometer, as numerous time- and/or frequency-domain
characteristics may be used as model inputs (Farrahi et al 2019). Despite these advances, research using
accelerometry is hindered by lack of comparability across studies due to the large variety of methods available to
researchers to analyze accelerometer data (Keadle et al 2019). This is similar to the ‘cut-point conundrum’
(Trost 2007), a term coined to describe the confusion regarding which set of count cut-points to use amidst the
large number available for specific populations, across different monitor wear locations and monitor brands,
and in specific contexts/settings (Migueles et al 2017). Even though the use of proprietary activity counts in
linear regression and/or cut-point based approaches has partially given way to use of raw acceleration data and
more advanced machine learning techniques, researchers find themselves in an all-too-familiar situation in
which it is difficult to identify what approach to use—we have simply replaced the cut-point conundrum with
‘novel method mayhem.’
Pfeiffer et al (2022) report that of 168 ‘novel’ approaches (i.e., not count-based cut-points or linear
regression models) included in their scoping review, only 61 were available for use by other researchers.
Regression equations (including count-based two-regression models) and/or cut-points for raw acceleration
metrics were more commonly made available, but access to more advanced machine learning techniques was
less often provided. This issue is compounded by the fact that use of the more advanced models relies on at least
some knowledge of coding in software programs such as R for implementation. Thus, even when more advanced
models were made available, they still were seldom (if ever) used by researchers other than the research group
who created them, suggesting that they still may be too difficult to use by a large portion of physical activity
researchers. This may be due to confusion over which models are available and should be used (novel method
mayhem) or unclear code and/or lack of instructions for use and/or sample data.
This paper addresses these issues through the description of three tools. First, we developed a repository of
methods for assessing physical activity intensity or energy expenditure using raw acceleration or novel analytic
approaches which will increase visibility of and access to available models. Second, we describe a framework for
classifying model development to encourage further development of existing models and inform future model
validation trials. Third, we provide reporting guidelines to ensure models developed in future research are
accessible and implementable by other researchers. We envision these tools serving both physical activity
measurement and general physical activity researchers by making it easier to identify and use novel methods,
improving accessibility of newly developed models, encouraging cross-validation and further development of
existing methods, and overall, reducing novel method mayhem.

Methods

Scoping review
The methods of the recent review that informed this work are provided in more detail elsewhere (Pfeiffer et al
2022). Briefly, Pfeiffer et al (2022) searched PubMed, Web of Science, and EBSCOhost to identify novel methods
for assessing physical activity or energy expenditure using accelerometry published between January 1, 2000 and
February 4, 2021. Novel methods were those that did not use count cut-points or single linear regression models
with counts as an input; further inclusion/exclusion criteria can be found in the review. For each included study,
the authors identified whether the associated model was accessible (e.g., model was included in the paper or
linked to externally) and whether it had been used in other studies based on a review of the articles that cited the
included article according to Publish or Perish software (Harzing 2007).

Repository
Of the papers included in Pfeiffer et al’s (2022) review, 48 of these papers provided models directly in the paper
(e.g., as equations or cut-points) and 13 need code to be implemented (all in the R programming language). The
models associated with two additional papers were published online separate from the original paper, resulting
in a total of 15 papers including access to their models as code. Available models were added to a repository on
Google Sites, which was selected because it is free to use and can be updated over time as new models are
released. In addition to putting all the available models into a central location, we also aimed to facilitate their use
by providing standardized information with each model. Specifically, each model has information about the
development and validation approach, device type and wear location, and instructions for use. The models that
need to be implemented in R also have sample data and a data dictionary when appropriate. A mechanism was
developed to allow individuals to submit other developed models for inclusion and to provide feedback about

2
Physiol. Meas. 43 (2022) 09NT01 K A Clevenger et al

potential future improvements to the repository. Finally, the ‘home’ page was designed to help users find
appropriate models for their target population, monitor wear location, and monitor brand.

Reporting guidelines
While developing the repository, we noted that, in addition to the fact that most models were not made available
for use, there were disparities in the type and amount of reported information. This made it difficult to
understand the development of the model and to implement the model, even when it was made available. We
created a 22-point checklist of items that should be reported in studies developing new methods of analyzing
accelerometer data (figure 1). In addition to information regarding the study context, model development
approach, data collection, and data processing, we ask for detailed information about the model’s inputs and
outputs and how they were defined and derived. We also suggest that authors provide a reproducible example of
implementation of their model, for example by providing sample data and code.

Framework
To provide a quick indication of how models were developed, and which models may need further development
research, each model in the repository was classified as one of five phases as part of our framework for model
development. This framework is modified but based on a recent framework for device development (Keadle et al
2019). In Keadle et al 2019 framework for device evaluation, Phase 0 is mechanical signal testing, Phase 1 is
laboratory, Phase 2 is semi-structured, Phase 3 is naturalistic, and Phase 4 is adoption of devices in applied
studies. In our framework (figure 2), Phase 0 (pre-model development) includes everything that is done before
an actual model that is meant to be used or further validated is created (e.g., comparison of features, wear
locations, or model types). Phase 1 (laboratory) includes structured activities in controlled conditions. In Phase
2 (semi-structured) studies, participants may be free to choose the order or duration of activities, or the study
may take place in an unstructured location (e.g., a park). Phase 3 (free-living) is more reflective of how the model
would perform in the ‘real world’ as participants may be monitored during daily life or in unstructured settings
(e.g., at home) or may be free to do whatever activities they want and/or go to any location they want. In Phase 4
(independent sample), a developed model is tested in a new, independent sample, which provides the best
indication of how the model will perform when used in new samples/studies. Models do not have to progress
linearly through the phases. While phase designation can be subjective, this provides an indication of what can/
needs to be done to further develop each model.

Results

Of the 168 methods included in the recent scoping review (Pfeiffer et al 2022), 63 are now found in the repository
(https://sites.google.com/view/accelerometerrepository). Most of these approaches (n = 48) were made
available in the paper as a regression formula, cut-points, or decision tree. As can be seen in figure 3, more
models have been made available as code in recent years. A total of 15 papers included access to their models as
code (including two that were published online separate from the original paper).
These models include a variety of populations (6 preschool, 20 child/adolescent, 42 adult), wear locations
(45 hip, 25 wrist, 9 combination, 4 thigh, 4 chest, 6 ankle, 4 other), and use a number of accelerometer brands (3
Actical, 29 ActiGraph, 2 Actiheart, 3 ActivPAL, 14 GENEActiv, 3 Omron, 17 other). While the majority of
models were developed with healthy individuals, two models were available for manual wheelchair users
(Hiremath et al 2012, Kiuchi et al 2014), one for pregnant women (van Hees et al 2011), one for adults with class
2 or 3 obesity (Diniz-Sousa et al 2020), and one for children with cerebral palsy (Trost et al 2016). Methods were
developed in the laboratory (n = 47), a semi-structured setting (n = 16; i.e., a researcher-specified place outside
of the laboratory), or free-living (n = 7). Seven of the included methods involved an independent sample cross-
validation.

Discussion

The cautionary tale of the cut-point conundrum and the ramifications for data harmonization are well known to
physical activity researchers. In describing the evolution of accelerometer methods, Troiano et al (2014) called
on researchers to build and provide models in open-source shareable forms to ‘reduce barriers to adoption and
support replication and cross-validation of new models.’ Yet, while use of raw acceleration data and/or
advanced modeling techniques show promise for the measurement of physical activity or energy expenditure
(de Almeida Mendes et al 2018, Farrahi et al 2019), the lack of accessibility of these methods has limited their use
(Pfeiffer et al 2022). This repository will allow researchers to find and use appropriate techniques for analyzing
these data more easily and to further validate or develop existing models. The framework and reporting

3
Physiol. Meas. 43 (2022) 09NT01 K A Clevenger et al

Figure 1. Reporting guidelines for new methods for analyzing accelerometer data.

Figure 2. Phase framework for model development.

4
Physiol. Meas. 43 (2022) 09NT01 K A Clevenger et al

Figure 3. The number of papers in a recent scoping review that did not make their developed method available, provided the method
within the paper, or made the method available as code.

guidelines will provide a guide for future research to further develop existing methods or ensure that newly
developed methods are accessible for use by other researchers.
Within the repository, 63 methods are made available which can be used in future studies and/or further
developed. We categorized each model using a framework based on the work of Keadle et al (2019) as phase 0
(pre-lab development), phase 1 (laboratory), 2 (semi-structured), 3 (free-living), or 4 (independent sample
cross-validation). Most models in the repository are in phase 1 (n = 47) or 2 (n = 16) of development. Only
seven models in the repository included a free-living component (phase 3) and involved an independent sample
cross-validation (phase 4). Although not all models have to move through every phase of the framework,
progressing existing models through the phases of model development (i.e., from 1 to 4) should be prioritized
over the development of new models unless there is not currently an appropriate model for a given population or
wear location of interest. Models in the repository can also be further improved or refined, such as how Ellingson
et al (2017) did for the Hildebrand et al (2014) model or Crouter et al (2011, 2010) did for earlier versions of the
2-regression models (Crouter et al 2006, Crouter & Bassett 2008).
Of note, the scoping review that we used to identify models for the repository only included articles
describing initial model development. Thus, some of these repository models may have been validated further in
a separate publication. In future updates to the repository, we aim to include this information as independent
sample cross-validation of existing models, particularly in free-living, provides the most realistic indication of
how the model will perform in a new, free-living sample (Lyden et al 2014, Keadle et al 2019). The repository and
framework facilitate this important process by making models that need to be further cross-validated or
developed easier to find and implement.
The repository includes models for accelerometers worn at various wear locations (ankle, chest, hip, thigh,
wrist) using both count and acceleration data in a number of populations (preschool, children/adolescents,
adults), However, the repository also highlights a few gaps in the research that need to be filled. While use of raw
acceleration facilitates transportability of models across device brands (Rowlands et al 2018), it was surprising
that no models using the Axivity monitor were available for inclusion in the repository given Axivity’s use in the
UK Biobank study (Doherty et al 2017). Similarly, although 17 smartphone-based algorithms were reported in
the scoping review, only one was made available (Nolan et al 2014). This is an important area of research moving
forward due to the ubiquitous nature of smartphone ownership and use. Finally, most models were developed in
able-bodied, healthy populations, and more methods for other populations should be made available. We
encourage researchers creating models to provide access to their method so it may be further developed or used
in subsequent studies.
As many researchers do not provide access to their developed models when publishing the findings from a
validation study, the overarching goal is to increase the number of developed models made available to other

5
Physiol. Meas. 43 (2022) 09NT01 K A Clevenger et al

researchers. However, we also note that when these models were provided, lack of consistency in the format and
provided ancillary information would make use of these models difficult for some researchers. As seen in
figure 3, there is a shift towards more code-based models in recent years, making it even more important to
provide enough information for the models to be useable by other researchers who may not have experience
with advanced modeling techniques. The repository and reporting guidelines (figure 1) can serve as an example
of the type of information that is helpful to include when providing access to a model. Detailed instructions on
how to get from raw accelerometer data to the model inputs (epoch, how to calculate features), how to
implement the model, and a description of model outputs (units and epoch) was often lacking. For models that
require code to implement, sample data with the correct headers and data format are useful.
In addition to highlighting models that need to be developed or further validated, the repository may also
serve as an avenue to implementing models in applied studies (Keadle et al 2019). In accordance with the
importance of this step, we will continue our efforts to make repository models easier to use, particularly for
those without coding experience. This may be done through interactive interfaces (e.g., Shiny app), tutorials, or
troubleshooting assistance. As the initial goal was to make these models easier to find, equations are provided in
the format of the original paper, and included code is only modified from the original source for consistency in
format across the models and to ensure that the code works with available sample data. In future updates, we can
make this information more useable. For example, code provided in the source articles does not start from
unprocessed monitor outputs but rather relies on the user to extract relevant features and put the data in a
particular format with specific variable names before using the provided model. A future update to the
repository may make it possible to extract the relevant features within the same code as the model. This will
make it easier, particularly for those with less experience extracting features, and may facilitate consistency
across studies.
The repository is a dynamic document that can expand and change over time. We have provided a feedback
form on the website so users can indicate which features they think will be most helpful in future updates. For
now, the repository will be updated every three months to include newly published models, including from
recent research and/or existing models that were not initially provided as a supplement to their published
articles. However, future adaptations may allow open community control over repository content (e.g., Wiki,
GitHub). Information about subsequent developments or cross-validation of included models will also be
added. While only models developed in R were available at the time the repository was created, models created in
any coding language or software can be included. It may also be of interest to include models for predicting
activity type or other behaviors from accelerometer data, although this was not the original intent of the
repository or review. Further, approaches not deemed novel by the recent review might still be useful for
researchers (e.g., count cut-point, count-based linear regression), and the repository may be a way to make these
methods easier for researchers to find. We may add these models to the repository in the future.
In conclusion, in this paper, we highlight new resources to facilitate the accessibility and use of methods for
predicting energy expenditure or physical activity intensity using raw acceleration data or novel analytic
approaches to analyzing count data. We encourage physical activity researchers to use the models provided in
the repository when appropriate and further cross-validate these models in free-living and/or independent
samples (in accord with the framework). We also encourage model developers to provide access to past and
future developed models to facilitate their use or further development in future research. This access should
include the information outlined in the reporting guidelines.

ORCID iDs

Kimberly A Clevenger https://orcid.org/0000-0003-2993-3587

Karin A Pfeiffer https://orcid.org/0000-0001-6280-9495

References
de Almeida Mendes M, da Silva I C, Ramires V V, Reichert F F, Martins R C and Tomasi E 2018 Calibration of raw accelerometer data to
measure physical activity: a systematic review Gait & Posture 61 98–110
Álvarez-García J A, Cvetković B and Luštrek M 2020 A survey on energy expenditure estimation using wearable devices ACM Computing
Surveys (CSUR) 53 1–35
Crouter S E and Bassett D R 2008 A new 2-regression model for the Actical accelerometer Br. J. Sports Med. 42 217–24
Crouter S E, Clowers K G and Bassett D R Jr 2006 A novel method for using accelerometer data to predict energy expenditure J. Appl. Physiol.
100 1324–31
Crouter S E, DellaValle D M, Horton M, Haas J D, Frongillo E A and Bassett D R 2011 Validity of the actical for estimating free-living physical
activity Eur. J. Appl. Physiol. 111 1381–9
Crouter S E, Kuffel E, Haas J D, Frongillo E A and Bassett D R Jr 2010 A refined 2-regression model for the actigraph accelerometer Medicine
and Science in Sports and Exercise 42 1029

6
Physiol. Meas. 43 (2022) 09NT01 K A Clevenger et al

Diniz-Sousa F, Veras L, Ribeiro J C, Boppre G, Devezas V, Santos-Sousa H, Preto J, Machado L, Vilas-Boas J P and Oliveira J 2020
Accelerometry calibration in people with class II-III obesity: energy expenditure prediction and physical activity intensity
identification Gait & Posture 76 104–9
Doherty A, Jackson D, Hammerla N, Plötz T, Olivier P, Granat M H, White T, Van Hees V T, Trenell M I and Owen C G 2017 Large scale
population assessment of physical activity using wrist worn accelerometers: the UK biobank study PLoS One 12 e0169649
Ellingson L D, Hibbing P R, Kim Y, Frey-Law L A, Saint-Maurice P F and Welk G J 2017 Lab-based validation of different data processing
methods for wrist-worn ActiGraph accelerometers in young adults Physiol. Meas. 38 1045
Farrahi V, Niemelä M, Kangas M, Korpelainen R and Jämsä T 2019 Calibration and validation of accelerometer-based activity monitors: a
systematic review of machine-learning approaches Gait Posture 68 285–99
Freedson P S, Melanson E and Sirard J 1998 Calibration of the computer science and applications, inc. accelerometer Medicine and Science in
Sports and Exercise 30 777–81
Harzing A W 2007 Publish or Perish software. (https://harzing.com/resources/publish-or-perish)
Hildebrand M, VT V H, Hansen B H and Ekelund U 2014 Age group comparability of raw accelerometer output from wrist-and hip-worn
monitors Medicine and Science in Sports and Exercise 46 1816–24
Hiremath S V, Ding D, Farringdon J and Cooper R A 2012 Predicting energy expenditure of manual wheelchair users with spinal cord injury
using a multisensor-based activity monitor Archives of Physical Medicine and Rehabilitation 93 1937–43
John D and Freedson P 2012 Acti graph and actical physical activity monitors: a peek under the hood Medicine and Science in Sports and
Exercise 44 S86
Keadle S K, Lyden K A, Strath S J, Staudenmayer J W and Freedson P S 2019 A framework to evaluate devices that assess physical behavior
Exercise and Sport Sciences Reviews 47 206–14
Kiuchi K, Inayama T, Muraoka Y, Ikemoto S, Uemura O and Mizuno K 2014 Preliminary study for the assessment of physical activity using a
triaxial accelerometer with a gyro sensor on the upper limbs of subjects with paraplegia driving a wheelchair on a treadmill Spinal Cord
52 556–63
Lyden K, Keadle S K, Staudenmayer J and Freedson P S 2014 A method to estimate free-living active and sedentary behavior from an
accelerometer Medicine and Science in Sports and Exercise 46 386
Migueles J H, Cadenas-Sanchez C, Ekelund U, Nyström C D, Mora-Gonzalez J, Löf M, Labayen I, Ruiz J R and Ortega F B 2017
Accelerometer data collection and processing criteria to assess physical activity and other outcomes: a systematic review and practical
considerations Sports Medicine 47 1821–45
Nolan M, Mitchell J R and Doyle-Baker P K 2014 Validity of the apple Iphone®/iPod touch® as an accelerometer-based physical activity
monitor: a proof-of-concept study Journal of Physical Activity and Health 11 759–69
Rowlands A V, Mirkes E M, Yates T, Clemes S, Davies M, Khunti K and Edwardson C L 2018 Accelerometer-assessed physical activity in
epidemiology: are monitors equivalent Medicine and Science in Sports and Exercise 50 257–65
Sasaki J E, da Silva K S, da Costa B G G and John D 2016 Measurement of physical activity using accelerometers Computer-Assisted and Web-
Based Innovations in Psychology, Special Education, and Health (Amsterdam: Elsevier) pp 33–60
Strath S J, Kaminsky L A, Ainsworth B E, Ekelund U, Freedson P S, Gary R A, Richardson C R, Smith D T and Swartz A M 2013 Guide to the
assessment of physical activity: clinical and research applications: a scientific statement from the American Heart Association
Circulation 128 2259–79
Troiano R P, McClain J J, Brychta R J and Chen K Y 2014 Evolution of accelerometer methods for physical activity research Br. J. Sports Med.
48 1019–23
Trost S G 2007 State of the art reviews: measurement of physical activity in children and adolescents American Journal of Lifestyle Medicine 1
299–314
Trost S G, Fragala-Pinkham M, Lennon N and O’Neil M E 2016 Decision trees for detection of activity intensity in youth with cerebral palsy
Medicine and Science in Sports and Exercise 48 958
van Hees V T, Renström F, Wright A, Gradmark A, Catt M, Chen K Y, Löf M, Bluck L, Pomeroy J and Wareham N J 2011 Estimation of daily
energy expenditure in pregnant and non-pregnant women using a wrist-worn tri-axial accelerometer PLoS One 6 e22922