386
2, JANUARY 15, 2017
A Survey on Activity Detection and Classification
Using Wearable Sensors
Maria Cornacchia, Student Member, IEEE, Koray Ozcan, Student Member, IEEE,
Yu Zheng, Student Member, IEEE, and Senem Velipasalar, Senior Member, IEEE
Abstract— Activity detection and classification are very
important for autonomous monitoring of humans for applica-
tions, including assistive living, rehabilitation, and surveillance.
Wearable sensors have found wide-spread use in recent years
due to their ever-decreasing cost, ease of deployment and use,
and ability to provide continuous monitoring as opposed to
sensors installed at fixed locations. Since many smart phones are
now equipped with a variety of sensors, such as accelerometer,
gyroscope, and camera, it has become more feasible to develop
activity monitoring algorithms employing one or more of these
sensors with increased accessibility. We provide a complete and
comprehensive survey on activity classification with wearable
sensors, covering a variety of sensing modalities, including
accelerometer, gyroscope, pressure sensors, and camera- and
depth-based systems. We discuss differences in activity types
tackled by this breadth of sensing modalities. For example,
accelerometer, gyroscope, and magnetometer systems have a
history of addressing whole body motion or global type activities,
whereas camera systems provide the context necessary to classify
local interactions, or interactions of individuals with objects.
We also found that these single sensing modalities laid the
foundation for hybrid works that tackle a mix of global and local
interaction-type activities. In addition to the type of sensors and
type of activities classified, we provide details on each wearable
system that include on-body sensor location, employed learning
approach, and extent of experimental setup. We further discuss
where the processing is performed, i.e., local versus remote
processing, for different systems. This is one of the first surveys
to provide such breadth of coverage across different wearable
sensor systems for activity classification.
Index Terms— Wearable, sensors, survey, activity detection,
activity classification, monitoring.
I. ACTIVITY D ETECTION AND C LASSIFICATION
H UMAN activity detection and classification are impor-
tant for a wide range of applications including mon-
itoring elderly people for assistive living, robotics, human
computer interaction, surveillance, retail, and keeping track of
athletic activities. With developing technology and decreas-
ing cost of devices with various sensors, life-logging has
become more popular and easily accessible. People are logging
everything from the number of steps taken in a day to
the number of times they feed and change their babies. In
terms of activity-level events, many of these applications still
Manuscript received July 2, 2016; revised October 3, 2016; accepted
October 27, 2016. Date of publication November 14, 2016; date of current
version December 20, 2016. This work was supported by the National Science
Foundation CAREER under Grant CNS-1206291 and Grant CNS-1302559.
The associate editor coordinating the review of this paper and approving it
for publication was Dr. Nitaigour P. Mahalik.
The authors are with the Department of Electrical Engineering and Com-
puter Science, Syracuse University, Syracuse, NY 13244 USA (e-mail:
mlscalzo@syr.edu; kozcan@syr.edu; yzheng04@syr.edu; svelipas@syr.edu).
Digital Object Identifier 10.1109/JSEN.2016.2628346
1558-1748 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
IEEE SENSORS JOURNAL, VOL. 17, NO.
require the human to manually record the event(s). However,
this only serves as motivation for more automated human
activity classification systems and approaches. This survey,
therefore, comes at a critical time for forging a clear under-
standing of the current state of the activity monitoring systems,
employing wearable sensors, so that future advances can
be made.
Existing activity monitoring systems can be broadly classi-
fied into two categories based on the manner in which sensors
are employed in an environment: (1) fixed sensor setting,
where information is gathered from static sensors mounted
at fixed locations and (2) mobile sensor setting, where the
sensors are wearable and thus mobile.
Fixed-sensor approaches involve acoustic [1]–[7], vibra-
tional [8]–[11], other ambient-based sensors, or static cameras
installed at fixed locations [12]–[16]. Acoustic, vibrational,
and other ambient sensor-based methods record major events
such as walking or falling based on their characteristic vibra-
tion patterns. This type of activity monitoring is confined to
limited environments, where the sensors are installed, captures
all individuals in the environment, and usually involve the
installation of specialized sensing equipment.
The alternative to fixed-sensor settings for activity moni-
toring is mobile sensing, where the sensors are wearable and
activity is monitored in a more first-person perspective. Before
the last decade and a half, this type of sensing was on par with
fixed-sensor settings in terms of availability and cost. Wearable
technologies were often large and cumbersome devices more
often found in specialized healthcare facilities. What then
changed in the last decade and half? The change was spurred
by a shift of cultural acceptance of mobile devices, the
advancement in hardware size and power, and the marketing of
smartphones. The acceptance of mobile hardware can be seen
through the statistic that in 1998 the U.S. census recorded that
only 31% of households had cellphones compared to the 71%
of households in 2005 [17]. Now, there are smartphones, wrist
worn and sensor-equipped watches, and even glasses equipped
with ubiquitous computing.
Now more than ever, with wearable and mobile technologies
already a part of our daily lives, human activity detection and
classification approaches have become both feasible and more
culturally acceptable. This type of mobile sensing also allows
monitoring of subjects wherever they may travel. With the
availability of affordable hardware for the purposes of mobile
sensing, and the widespread use of mobile devices, the focus of
this survey is therefore on activity classification using wearable
sensors. We cover a variety of different sensors, including
wearable cameras, and do not exclude variants where sensing
CORNACCHIA et al.: SURVEY ON ACTIVITY DETECTION AND CLASSIFICATION USING WEARABLE SENSORS
and computation are not co-located on the same device.
However, where possible, we try to mention systems where
the sensing and computation take place on the same device,
such as in mobile applications. Based on literature, Lara and
Labrador [18] distinguish between seven groups of activities:
(1) ambulation, (2) transportation, (3) phone usage, (4) daily
activities, (5) exercise/fitness, (6) military, and (7) upper body.
The two most pervasive activity types classified are ambulatory
and daily activities. Ambulatory activities are those actions
referred to in health fields that involve the mobility of an
individual, and include walking, running, sitting, lying and
going up and down the stairs. Daily activities include eating,
drinking, brushing teeth, and so on. While we do not disagree
that activities can be sub-divided into these seven more fine-
grained categories, we take a slightly different approach to
grouping activities with two main categories: (1) activities
involving global body motion and (2) activities involving local
interaction. Global motion type activities will include ambula-
tion, transportation, and exercise/fitness activities. We use the
term global motion activities to describe motions that involve
displacement of the entire body, such as walking, running,
and climbing stairs. Local interaction activities are those that
generally involve the interaction with an object and would be
the activities which are categorized in [18] as phone usage and
daily activities. Specifically, local interaction type activities
do not involve motion of the entire body, but rather only
involve motion of the extremities. These local interactions
often require additional context in terms of identification of
an object with which a subject is interacting. For example, the
local interaction activity of brushing teeth involves the use of
a tooth brush. Although we use these two categories to group
papers, not all works classify tasks only from either global
or local activities, works such as [19]–[23] classify activities
from both categories.
We also organize our discussion based on the various sensor
types, sensor placements and combinations used for activity
classification. As the combinations of sensors change, there is
often a shift in the ability to address certain activity types. For
example, for single-sensor approaches using accelerometer,
magnetometer, and/or gyroscope (A/M/G), the majority of
the literature tends to focus on ambulation type activities
[24]–[26]. This is in large part due to the information captured
by these acceleration or orientation-based sensors. Single-
sensor based camera systems on the other hand, provide the
additional information necessary to tackle more types of fine-
grained local interaction type activities, such as distinguishing
between different types of food preparation. Additionally,
hybrid-sensor systems often use a mix of sensors that provide
information to tackle activities that often span both global and
local activities.
Another key point is the placement of the wearable sen-
sors on the human body. Fig.1 depicts these placements
for the works in this survey. As discussed in [27], with
placement of the sensor differing from person to person,
using only the accelerometer data might not be sufficient for
activity classifiers. To address these potential issues, others,
[28]–[34], explore or compensate for the effects of different
orientation-based sensor placements. Saeedi et al. [35] attempt
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
387
Fig. 1. Sensor Locations Employed. (a) Works using one sensor type.
(b) Works using hybrid sensors.
to determine the location of seven accelerometers on the
body to aid activity classification if there is an unknown
sensor configuration. Still others address the problem of single
sensing orientation based information by turning to multiple
combinations of A/M/G based sensors or compensating for
the problems algorithmically such as in [36]. In the case
of A/M/G sensing, the sensor placements can be linked to
the types of activities. For example, A/M/G placements that
handle global body motion are very often placed at a central
body location, such as the waist. A/M/G works that address
local interaction activity types are placed on the extremities.
For camera based systems, however the activity types are in
general placement agnostic, that is in order to prevent blur
or achieve stability, despite the activity type the common
sensor placements include the head or chest, as seen in Fig.1.
Accelerometer-based systems are simple and cost effective.
However, even with multiple accelerometer sensors, these
methods can still create false positive alarms, especially in
environments such as high speed trains and elevators, where
people are exposed to acceleration.
It is for this reason that, even though activity classification
using acceleration-based sensing is more prevalent in the
literature, we provide a review of works that go beyond the
accelerometer-based methods. In addition to accelerometer-
based approaches, this survey also covers modalities such as
wearable vision-based systems [37], as well as more recently
introduced depth-based sensing. We discuss how using visual
or depth images captured by a camera sensor provide an
abundance of data including contextual information about the
surroundings. This is one of the first surveys to provide a
fully comprehensive review of human activity classification
by covering a variety of wearable sensors, that include A/M/G
sensors as well as camera- and depth-based sensing. We pro-
vide an informative summary of each work, classifying papers
based on sensor type, number of sensors used, sensor location,
type of activities detected (global body motion vs. local inter-
action with other objects), processing type (onboard vs remote
computation), extent of experimentation and evaluation, and
the employed learning approach. We additionally provide a
388
TABLE I
PAPERS C LASSIFIED BY A CTIVITY T YPE (G LOBAL B ODY M OTION OR
L OCAL I NTERACTION ) U SING A/M/G S ENSORS
detailed summary and organization of this extensive body
of work in Tables I through VI. Tables I, II, and III focus
on works using A/M/G sensors, camera sensors and hybrid
sensor modalities, respectively, and group references based
on the activity types that they classify. Tables IV and V
provide a comprehensive comparison of all A/M/G, camera,
and hybrid works based on sensor location, number of sensors,
number of subjects, and learning approach. Finally, Table VI
summarizes all A/M/G, camera, and hybrid references based
on higher level activity type, global or local activities, and
processing location. The summarizing information in Table
VI demonstrates the larger distribution of works relating to
global body motion type activities and the still limited set of
works performing onboard processing.
The rest of this survey is organized as follows: Section II
provides a review of the existing work using only accelerom-
eters, magnetometers or gyroscopes. A review of the work
using wearable cameras is presented in Section III. Section
IV provides a summary of the work using other single-type
sensors such as ECG, and Section V focuses on the work
using hybrid sensor modalities. The survey is concluded in
Section VI.
II. R ECENT W ORK U SING O NLY ACCELEROMETERS ,
M AGNETOMETERS OR G YROSCOPES
There is a large body of work using A/M/G sensors for
activity classification. Table I groups these works into two
classes (based on whether they focus on global body motion or
local interaction with objects), and lists different activity types
that these works address. Accelerometer-based systems have
been proposed for fall detection [4], [154]–[158]. There are
systems using single sensor accelerometer-based system [25],
[39], [159], [160] and those fusing several sensors [63], [161]
to detect ambulatory-type activities. Others are performing
posture recognition [162]. A number of approaches have
been proposed for minimizing the false-alarm rate, including
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
IEEE SENSORS JOURNAL, VOL. 17, NO. 2, JANUARY 15, 2017
TABLE II
PAPERS C LASSIFIED BY A CTIVITY T YPE (G LOBAL B ODY M OTION
OR L OCAL I NTERACTION ) U SING C AMERA
watching for no-movement [163], [164] and using statistical
models [165]. Zhang et al. [41] use rule-based reasoning
on smart phone accelerometer data to determine motion
and motionless activities. Many activity classification algo-
rithms have been proposed for Android or other smartphone
platforms using only accelerometer data [157], [166]–[170].
Accelerometer-based systems are used in [170] and [171] to
distinguish between walking, running, cycling, and driving.
Accelerometers have also been used in the activity classifica-
tion of workout [172] or sports activities [27], [61] to capture
and archive various training statistics.
We have organized A/M/G-based activity recognition sys-
tems into two main sections. Sections II-A and II-B will
describe A/M/G-based systems focusing on global body
motion type and local interaction type activities, respectively.
We subdivide Section II-A further to distinguish between sen-
sor placement and quantity. The placement and the number of
sensors used is a distinguishing factor for this sensing modal-
ity. Whereas numerous different features can be extracted
from vision-based systems, there are two main categories of
acceleration-based data (time domain and frequency domain-
based features), and many of the acceleration-based systems
use features from both domains. In other words, while features
vary for camera-based systems, accelerometer-based systems
can be distinguished based on the placement of the sensor
itself, which often dictates the type of activities to be classified.
Table I shows that there are many more accelerometer-based
works for global body motion activity types than for local
interaction types. Table II demonstrates there is almost an even
number of global body motion and local interaction works for
camera-based wearable sensor systems. Examination of Table
I and IV together shows that those works for accelerometer-
based sensors that tackle global body motion activities tend to
be at non-extremity based locations, whereas those for local
CORNACCHIA et al.: SURVEY ON ACTIVITY DETECTION AND CLASSIFICATION USING WEARABLE SENSORS
TABLE III
PAPERS C LASSIFIED B Y A CTIVITY T YPE (G LOBAL B ODY M OTION OR L OCAL I NTERACTION ) U SING A H YBRID S ET OF S ENSORS
interaction type activities tend to place the sensors on the
extremities.
A. Global Body Motion Activity Classification
1) Waist-Mounted: Numerous works place accelerometer-
based sensors at the waist of the subject. In the detection of
global body motion, this central location is in many cases more
stable than placing the sensor on one of the extremities. The
extremities do not necessarily move in conjunction with the
rest of the body. Mounting at the waist also made convenient
belt mounting of the sensor. Initially, works employing this
setup used very limited feature sets. Works such as those by
Mantyjarvi et al. [25], Song et al. [40], He et al. [48], and He
and Jin [49] employed these limited feature sets with a single
selected classification algorithm.
Some other earlier works did not even use a classification
algorithm, but rather employed empirically determined thresh-
olds for classification [38], [39], [173]. Later works expanded
on this threshold-based reasoning, employing a hierarchical
methodology to rule out initial larger classes of activities.
For example, the threshold would be used to determine if
an activity was a motion type activity, such as walking or
running, or a motionless activity, such as standing or sitting.
After determining if an activity was in one of the two larger
classes of motion or motionless, then another method would
often be used to determine the final sub-classification decision
of walking, running, sitting, or standing. A rule-based method
is used in [41] to initially distinguish between motion and
motionless activities and then a final classification decision is
made using an Support Vector Machine (SVM), trained on
3-axis acceleration measurements. Weng et al. [47] use an
SVM to both distinguish between static versus motion activi-
ties and for the final classification within the subcategories.
Ataya et al. [50] also employ a hierarchically structured
approach, first determining static from dynamic activities.
They compare numerous classifiers for making the final clas-
sification decision and determine Random Forests to be an
improvement over SVMs as well as other techniques. Unique
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
389
to [50], Ataya et al. study the temporal coherence of activities
overtime. In other words, a Hidden Markov Model (HMM) is
used to determine whether or not it is plausible for someone
to be lying down after running.
Similar to [50], another major trend in this area, outside of
hierarchical approaches, was comparing numerous classifiers
in the design of their systems. Ravi et al. [51] compared
numerous classification techniques using a small combination
of 12 time and frequency domain features. They did not
only compare classifiers but also attempted to fuse numerous
different combinations of classifiers. One of the novelties
of [51] is effectively performing activity classification with
small feature sets using a combination of low power classifiers.
Lara and Labrador [56] compare numerous techniques and
create a code base for an Android-based operating system.
Kwapisz et al. [42] compared numerous classifiers using a
smartphone-based platform, but placed the emphasis on the
diversity of a 43-element feature set.
With improvements in processing power, more recent works
seemed to follow this trend in improving features as opposed
to classifiers. References [43], [46], and [84] all compare
various classifiers, while leveraging unique feature sets that
range in the number of features from 76 to 110 features.
Feature selection was also a part of some of these works,
in other words the improvements of accuracy seemed to stem
from not a particular classifier, but rather the input provided
to the classifier. Still others focused on novelties outside
of improving activity classification accuracy through feature
sets. Alshurafa et al. [57] focused on distinguishing inten-
sity levels of different activities, whereas [44], [45] attempt
to maintain classification accuracy while improving energy
consumption.
2) Mounted in Other Locations: After 2006, the place-
ment and number of accelerometer-based sensors appeared
to expand for a single system. With increasing number of
sensors at different locations, the average number of activities
classified also rose. Many of the waist-mounted works tackled
activity sets of around five, yet with variation in placement
390
and more sensors, many of the works to be discussed in this
section classified a larger range of activities (up to 20) with the
average number of classified activities being closer to seven
or eight.
However, some works still prescribed to a more central
body location, such as the chest [59], [67], [174] or lower
back [77]. Chung et al. [59] use frequency domain features
from a chest-mounted sensor to distinguish between rest,
walking and running. They use a rough hierarchical threshold
based technique on Signal Magnitude Area (SMA). Curone
et al. [67] likewise use SMA from an upper body trunk
attached accelerometer with a hierarchical threshold based
technique to distinguish moving from non-moving activities.
However, Curone et al. [67] extend the application to tackle a
larger number of classes compared to [59]. Other works, such
as [174], use a chest-mounted sensor with SMA features but
extend the simple threshold-based decision to an HMM.
There are also works placing the sensors on the extremities
of the body. This placement has become more common
recently with arm-band mounted smartphones and wrist-based
hardware such as e-watches. Additionally, these works tend to
be more complex in terms of the application of the learning
algorithm applied to detect certain activities. That is, unlike the
threshold-based techniques applied to the central body loca-
tions, these papers employ supervised learning techniques to
address the classification problem. References [68] and [175]
employ thigh-based deployments of single 3D accelerometers,
and both use multiple hierarchically-structured algorithms to
make their classification decisions. Mannini and Sabatini [68]
apply two cascaded SVMs, whereas Zhu and Sheng [175] use
two Neural Networks in conjunction with an HMM. Another
common extremity placement is on the wrist [69], [70], [73].
Chernbumroong et al. [69] compare numerous different feature
combinations, as well as two learning algorithms, namely C4.5
decision trees and Artificial Neural Networks (ANNs). Andreu
et al. [70] propose a system using an evolving self-learning
fuzzy rule-based classifier and address a more comprehensive
activity set than [69], to include vacuuming, stretching
and bicycling. Margarito et al. [73] likewise classify some
exercise activities, such as cross training, rowing, squatting
and weightlifting in addition to the standard set of walking,
running, and sitting. They apply a template matching technique
on the acceleration signals to make the classification decision.
Lee et al. [74] explore a sensor placement different from thigh
and wrist altogether. They introduce a decision-tree-based
model created from a foot-mounted gyroscope to classify
walking, running, going upstairs and downstairs.
There are other works that did not choose a single location
for an accelerometer, gyroscope or a magnetometer. Dobrucali
and Barshan [85], as well as [58], evaluated different sensor
body placements and selected the location with the best data.
Yet, while [58] and [85] chose the best locations, other systems
were created that consist of using data from all of the multiple
orientation-based sensors. The simplest combination is the use
of pairs of sensor locations. Some of these works applying
pairs of sensor placements, such as [55], [80], and [81], only
tackle the same average activity sets of around five or fewer
activities. In their defense, in [80] and [81], the focus was
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
IEEE SENSORS JOURNAL, VOL. 17, NO. 2, JANUARY 15, 2017
on specific applications such as the ability of the elderly to
ambulate and fall detection, so the goal was not classify-
ing a larger set of activities. On the other hand, numerous
works have been presented discriminating a larger set of
activities by using information from two sensor locations.
Lustrek et al. [71], Xu et al. [75], and Ghasemzadeh and
Jafari [83] were able to discriminate between 11, 14, and 25
activities respectively.
Multi-sensor approaches were not limited to pairs of sen-
sors. Krishnan and Panchanathan [60], Trabelsi et al. [79],
and Mortazavi et al. [86] use data from sensors at three
different locations. Ermes et al. [176], Ugulino et al. [177],
Zhu and Sheng [26], [63] use data from four orientation-based
sensors. Atallah et al. [32], Rednic et al. [78], and Huynh and
Schiele [82] use seven or more orientation based sensors. For
all of these arrangements, containing more than two sensors,
the number of classified activities are also significant, ranging
from 5 to 20, with the average number of classes being
around 10.
The works that are not focused on multi-sensor combina-
tions or placements can be distinguished by their hardware
implementation. Earlier works developed their own specialized
hardware. Wang et al. [62] develop a new hardware instanti-
ation of self organizing maps or neural network algorithms.
Hao et al. [64] propose onboard human activity classification,
using a 3D accelerometer with a GPS module placed anywhere
on the body. Basterretxea et al. [53] present an onboard Field
Programmable Gate Array (FPGA)-based human activity clas-
sification. Numerous later works [47], [52], [54], [65], [72],
[76], [178]–[182] attempt to make use of other commercially
available hardware, such as mobile platforms or smartphones.
Ryder et al. [65] propose a tool for monitoring mobility
patterns over time using a mobile phone. Others tried to
tolerate different placements of these mobile platforms, trying
to prevent constraints on how an individual carries their mobile
platform. Berchtold et al. [76], Siirtola and Roning [72],
and Anjum and Ilyas [52] propose smartphone systems with
various different locations.
B. Local Interaction Activity Classification
In addition to the works classifying ambulatory move-
ments, there are recent works focusing on other types of
activities. This has also been motivated by the introduction
of smart watches and FitBit® like devices. Almost all of
these works place the sensors on a portion of the arm in
order to detect a user’s interactions with the environment.
These works vary based on the types of local interactions
that they are trying to detect and the number of sensors
employed.
Activities of interest to be classified with a single
accelerometer based sensor, mounted at a location on the arm,
typically the wrist, have included eating and drinking [96],
dynamic and static activities [22], work types of activities [97],
different arm movements [99], and long-term motions [98].
Min et al. [88]–[90] were interested in classifying morning
activities. In their earlier work, Min et al. [88] use a single
2D wrist-worn accelerometer and apply a Gaussian Mixture
Model (GMM). In [89], Min et al. also use a GMM, but
CORNACCHIA et al.: SURVEY ON ACTIVITY DETECTION AND CLASSIFICATION USING WEARABLE SENSORS
extend the work by adding a fixed sensor. Min et al. [90]
additionally explore using an audio recorder with a single
2D accelerometer. In this work [90], the addition of another
sensing modality allows the extension from three activities
to the ability to classify six activities including brushing
teeth, washing, shaving, electric brushing, and electric shaving
with the GMM approach. However, placement of a single
sensor was not limited to the arm. Fortune et al. [91]
extract three features from a single accelerometer and gyro-
scope located on the thigh. They classify other types of
daily activities including dressing, walking, sitting, reading,
washing dishes, climbing stairs, writing, dusting, and folding
laundry.
In order to create more comprehensive systems with the
ability to discriminate an increasing number of activities, other
works have explored multi-accelerometer placements [92],
[94], [95]. As with the case of global body motion described
above, with the introduction of multiple accelerometer-based
sensors for local interaction classification, came the necessity
to use more advanced supervised learning methods. In other
words, many of the works, employing a single sensor, used
threshold-based techniques, GMMs, or other types of mod-
els such as HMMs. Works such as, [92], [94], [95], use
more complex learning methods compared to single sen-
sor works. Lin et al. [94] present a wearable module and
neural-network-based activity classification for daily energy
expenditure estimation. They propose to place accelerome-
ters on an individual’s waist, wrists and ankles. They con-
struct energy expenditure regression models from 12 acquired
and nine derived features to classify 14 different activities.
Zhelong et al. [95] deploy five 3D-accelerometers on wrists,
ankles and trunk to classify numerous cleaning, eating, and at
least 10 other daily life activities. Five features are extracted
for each axe per sensor. An overall accuracy of 91.3% is
achieved using a probabilistic neural network (PNN) and
adjustable fuzzy clustering (AFC). The contribution of [95] is
an iterative learning method for adding new training samples
and new activity classes to the already learned AFC and
PNN model. Stickic et al. [93] looked at improving training
by proposing a framework for learning activity from labeled
and unlabeled data. This framework involves using experience
sampling to classify activities such as meal preparation, dish-
washing, grooming, etc. Min and Cho [92] also deploy five
sets of accelerometers and gyroscopes. They place the sensors
on the hands to classify 19 activities that include clapping,
shaking, teeth-brushing, and other hand manipulation tasks.
III. R ECENT W ORK U SING O NLY W EARABLE C AMERAS
Over the years, the flexibility of wearable camera systems
has evolved thanks to decreasing camera and processor sizes
and increasing resolution and processing power. Table II
groups camera-based works into two classes (based on whether
they focus on global body motion or local interaction with
objects), and lists different activity types that these works
address. Initially, many of the efforts involved backpack-based
camera systems such as those in [151]. Camera technologies
have now been developed to the point where image capture
can be accomplished by less intrusive setups. Numerous
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
391
demonstration systems, especially with the introduction of
Google Glass® , have prototyped algorithms for activity clas-
sification based on images captured from an eye-glass worn
camera [21] [103]. Others have adopted head- [106], [114],
[115] or chest-mounted cameras [111], [124]. Bambach [183]
summarizes the advances in computer vision algorithms for
egocentric video.
While these wearable camera-based activity classification
systems can be distinguished from one another in terms of
the placement of the camera on the body, it can be seen from
Table IV that the majority of these camera based setups choose
either a head or torso-mounted configuration. Additionally,
Table IV shows that all works include only a single camera
and often do not specify the number of subjects or have
fewer than 10 subjects. The high-level distinguishing factor
between these camera-based systems is the type of features
extracted from the visual scene. These features may be more
general, high-level scene features that quantify the motion of
the scene or more fine-grained details that try to capture inter-
actions of the subject with objects in the scene. We therefore,
similarly to Section II, group the wearable camera-based
activity classification work into two categories: Global body
motion and local interaction activity classification described
in Sections III-A and III-B, respectively.
A. Global Body Motion Activity Classification
While there are some works in the area of global motion
activity classification based on wearable cameras, one will
notice that this section has comparatively less number of
works than the global body motion activity detection using
A/M/G. Nonetheless, there are some works that have explored
global motion using camera-based techniques. Global body
motion is often inferred from images based on an estimate
of the camera motion. The techniques for estimating this
camera motion often fall into two main groups, (i) tech-
niques that leverage methods based on point-based features
[101], [184]–[186], and (ii) techniques that employ optical
flow like features [103]–[106], [108].
While some works employing point-based features used
standard detection algorithms such as Speeded Up Robust
Features (SURF) [101], others focused on developing novel
point-based features. Zhang et al. [185], [186] use a novel
point feature detection based on defining interest points from
the covariance matrix of intensities at each pixel. They then
calculate motion histograms based on the point features to
capture the global motion distribution in the video.
The optical flow based techniques appear to differ based
on the learning algorithm employed. Kitani et al. [105] inves-
tigate the use of motion-based histograms from optical flow
vectors and unsupervised Dirichlet learning on classification of
11 sports-based ego-actions. Zhan et al. [103] extract optical
flow features for each frame, and pool the optical flow vectors
over numerous frames to provide temporal context. They
compare three classification approaches, namely K-Nearest
Neighbor (KNN), SVM, and LogitBoost. The approach
also includes a comparison with and without an HMM.
Yin et al. [104] also use an optical flow technique, but employ
an SVM.
392
There are still other works that do not exactly fall into
describing global motion based on local point features or
optical flow based features. Ozcan et al. [107] propose a
wearable camera-based fall detection and activity classification
system without applying learning-based methods. They use
separate edge strength and orientation histograms for fall
detection. Song and Chen [100] also use a histogram of ori-
ented gradients approach to detect humans from their camera-
based system. However, in the case of [100], their system
is using a mobile camera on a robot watching the human.
Similarly, Watanabe and Hatanaka [102] also use a different
setup of the camera, wherein a single hip or waist-mounted
camera is employed to describe the gait of individual. Unlike
other works, the camera is facing downward and motion is
relative to the waist position. A walking state at each frame
is composed of the position of the waist, and relative position
of joints, as well as the angular speed of the camera. It is
assumed that the intrinsic parameters of the camera and the
world coordinates of the observed points are known.
B. Local Interaction Activity Classification
The works, described in Section III-A, focusing on higher
level, global body motion activities, while comparable to and
competitive with the accelerometer-based approaches, do not
necessarily take advantage of the visual details captured by
cameras. Thus, other researchers have considered the ability to
classify more fine-grained activities [113], [114], [116], [118],
[119]. While some of these works, like the local interaction
papers of section II-B, look at distinguishing between activities
such as brushing teeth, washing dishes and vacuuming, the
visual information allows for even more fine-grained clas-
sification, such as distinguishing between making coffee or
making tea. To capture this granularity of classification, the
works in this section view activities as involving a subject
and object interaction or the interaction of multiple subjects.
The majority of the works, discussed in this section, will
be on classifying activities that involve object and subject
interactions, however we do not want to ignore works such
as [113] and [121] that classify interactions between multiple
subjects. Aghazadeh et al. [121] propose novelty detection
in egocentric video by measuring appearance and geometric
similarity of individual frames and exploiting the invariant
temporal order of the activity to determine if a subject runs
into a friend, subject gives directions, or the subject goes to
an ice cream shop. Fathi et al. [113] similarly classify the
type of interaction multiple subjects are involved in, such as
a dialogue, monologue, or discussion.
To begin to understand these interactions, there have been
supporting works that contributed to individual aspects of
being able to tackle the larger problem of activity classifi-
cation. These supporting works have addressed problems such
as detecting a hand [120], better recognizing an object from
an egoview [187], methods for gaze/attention focus detection
and 3D estimation of hand-held objects [188], or linking
an object to a location [189]. Another area of work has
been in form of video summarization, which contributes to
activity classification by developing techniques for temporally
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
IEEE SENSORS JOURNAL, VOL. 17, NO. 2, JANUARY 15, 2017
segmenting a video into distinct actions. The ability to properly
segment regions of activity has been furthered by several
works of Grauman et al. [190]–[194] that proposed algo-
rithms for video event summarization for ego-centric videos
including predicting important objects and people [191], [192],
extracting frames that might be a snapshot from ego-centric
video [190], and video summarization of daily activity ego-
centric videos based on detected objects and their relations and
co-occurrences [193]. In addition to temporal segmentation,
Lee and Grauman [194] also explored within frame segmen-
tation using region cues to emphasize high-level saliency.
Attempts at activity classification began with works such
as [118] by Mayol and Murray, who use a shoulder-mounted
camera to recognize manipulations of objects by an individ-
ual’s hands. This work, however, is constrained to a static
workspace view and manipulations are described as actions
carried out by the subject’s hands. In fact, to first determine the
center of focus, they use the center of mass of a detected skin
region, where these skin regions are assumed to be either one
or both hands. More recent works began to use less constrained
experimental setups, where the camera is in fact mobile.
Sundaram and Cuevas [116] use a shoulder-mounted camera
on a moving subject. They, like their predecessors, also attempt
to detect the hands and represent activities as manipulations
by hands. They divide their system into two parts; the first,
where the vision is relied on to recognize the manipulation
motions and the second, where a Dynamic Bayesian Network
(DBN) is used to infer objects and activities from the motions
of the first part. An accuracy of 60.99% is obtained for the
manipulation motions that include making a cup of coffee,
making a cup of tea, washing a dish, and so on.
Fathi et al. [114] also use manipulations by the hand to
classify fine-grained activities and similarly focus on food
preparation tasks such as making a hotdog, a sandwich, coffee,
etc. They propose a framework for describing an activity as
a combination of numerous actions. For example, actions of
the coffee making activity include opening coffee jar, pouring
water in coffee maker, and so on. They attempt to assign action
labels to each segmented image interval and an object, hand, or
background label to each super-pixel in each frame. The action
models are learned using Adaboost [195] and a set of features
that include information such as object frequency, object
optical flow, hand optical flow, and so on. The action verbs
are estimated first, then the object classes are inferred with a
probabilistic model. The final steps involve refining decisions
from previous stages based on the final decisions of activities,
using a conditional random field. This work is evaluated on the
GeorgiaTech egocentric activity dataset, which includes video
from a head-mounted camera. Frame based action recognition
accuracy is 45%, whereas activity classification accuracy is
32.4%. They further demonstrate that object recognition is in
fact improved based on action classification results. In [115],
Fathi et al. demonstrate improved results on the same Geor-
giaTech dataset by modeling actions through the state changes
that are caused on an object or materials. This is in contrast
to their initial work that uses information from all frames. To
detect changed regions the authors sample frames from the
beginning and end of an action sequence. Change is measured
CORNACCHIA et al.: SURVEY ON ACTIVITY DETECTION AND CLASSIFICATION USING WEARABLE SENSORS
in terms of the color difference of matched pixels. An SVM
is trained to detect regions that correspond to specific actions.
Using state change detection, 39.7% accuracy is achieved over
61 classes on the GeorgiaTech egocentric activity dataset. The
activity segmentation results in 42% accuracy.
As accuracies in object detection works began to improve,
others began to use these more specific object detection mod-
els for activity classification. Pirsiavash and Ramanan [110]
use fully supervised learning with dense labels to train
deformable parts-based object detectors. With the additional
label information, this enables more complex understanding
of multiple objects interacting in a single scene. They also
recognize that changes in the objects occur throughout an
action sequence and develop a notion of active versus passive
objects and create separate detectors for objects in each
state. To evaluate their system, they created and annotated a
1 million-frame dataset, and obtain results of around 30%
frame classification accuracy. However, with an assumed ideal
object detector, they demonstrate that this method would
obtain a 60% frame classification accuracy. The created
dataset is from a chest-mounted Go-Pro camera and contains
label information about activities such as brushing teeth,
combing hair, making coffee, and so on. Numerous works
leverage this dataset to evaluate their approaches including
[109], [111], [112]. McCandless and Grauman [109] propose
to learn the spatio-temporal discriminative partitions for ego-
centric videos and apply object detection to obtain a strong
classifier for recognition of 18 activities, achieving an overall
38.7% F score.
Matsuo et al. [111] further extend the work in [110], through
visual attention and saliency, accounting for cases where a
hand may not be involved in the interaction. The groundwork
for using visual attention and saliency can be linked to other
works, including [196], [117], and [197]. Matsuo et al. [111]
propose a method for quantifying visual saliency, not only
through the static saliency of an image, but through the ego-
motions of the first person viewer. The results are based on the
same dataset used in [110]. The results improve the average
recognition accuracy, over all activities, from 36.9% to 43.3%
and decrease variance in accuracy over numerous subjects
from 9.8% to 7.1%.
IV. R ECENT W ORK U SING OTHER S INGLE T YPE S ENSORS
This section will summarize works that only use a sin-
gle type of sensor (other than A/M/G or camera). These
sensor types include Electrocardiogram (ECG), Photoplethys-
mogram (PPG), some acoustic sensors, custom-built sensors
etc. This section does not have as many papers as the other
sections. The reason is that many works use these type of
sensors in conjunction with other sensor types, and we refer
to these as works using hybrid sensor modalities, and will
summarize them in Section V.
As the single sensor type, Pawar et al. [198],
Kher et al. [199] and Keskar et al. [200] use an ECG.
El Achkar et al. [130] use force sensors on the bottom of
an individual’s feet. The total force is estimated and a linear
regression model of the total force was created to classify
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
393
each activity. Accuracy obtained was over 93% for sitting,
standing, and walking. Hardegger et al. [201] proposed the
LocAFusion algorithm for improving activity recognition
while using multiple Inertial Measurement Unit (IMU)
sensors located at different parts of the body such as foot and
hand wrist. Bulling et al. [202] proposed a method for human
activity recognition by using electrooculography system that
records the eye movement. Types of activities include copying
a text, reading a printed paper, taking handwritten notes,
watching a video, and browsing the Web.
Mukhopadhyay [203] provides a recent review of wear-
able sensors for activity monitoring, focusing mostly on
healthcare applications using recent wearable sensors, includ-
ing some works that use custom-designed single sensors.
Cheng et al. [204] proposed capacitive sensors built from
conductive textile patches that can monitor heart and breathing
rate, hand gesture recognition, swallowing monitoring and
gait analysis. Wujcik et al. [205] developed an ion sensor
for quantizing the amount of sodium ions in sweat real-time.
Veltink and Rossi [206] present e-textile and micromechan-
ical sensors for accurate analysis of movement character-
istics during activities of daily living. Shaltis et al. [207]
proposed wearable cuff-less photoplethysmographic monitor
for measuring blood pressure. Corbishley and Rodriguez-
Villegas [208] measured breathing rate, with accuracy more
than 90%, using a wearable acoustic sensor such as a micro-
phone. A heart rate monitoring application is proposed by
Patterson et al. [209], which is based on a flexible and low-
power PPG sensor. Dudde et al. [210] developed wearable
closed-loop drug infusion system that analyzes blood glucose
levels and infuses insulin appropriately. Ahn et al. [211] devel-
oped a low-cost, disposable biochip for biochemical detection
of parameters including glucose and blood gas concentration.
Recently, flexible sensors were presented. Trung et al. [212]
provided a review of flexible and stretchable physical sensors
for applications of activity monitoring and personal healthcare.
Chen et al. [213] proposed body temperature sensor that is
wearable, breathable and stretchable for healthcare monitoring.
They are capable of measuring temperature, pressure and strain
so far. They offer limited capability for recognizing various
daily activities. However, they promise applicability towards
activity detection and recognition if the sensor capabilities are
improved to the level of A/M/G sensors. These custom-built
single sensors allow accurate and robust measurements for
healthcare applications.
V. R ECENT W ORK U SING H YBRID S ENSOR M ODALITIES
With the advances in single sensor type systems, and sensor
technologies becoming smaller, cheaper and more commer-
cialized, there has been many research efforts that now look
at not only a single modality sensor but rather a hybrid of
multiple types of sensors. Table III specifies the activities
addressed by each hybrid sensor work. These approaches are
not as easily separated into the classification of global body
and local interaction type activities, as can be seen in Table III,
and a single work in this area can often address activities
in both categories. Hybrid sensing systems, with multiple
394
sensors, also often employ supervised learning methods as
shown in Tables IV and V. In the next two subsections, we
will discuss some different combinations of different sensor
modalities. Section V.A will discuss works that contain at least
one imaging sensor, while Section V.B will discuss systems,
which do not contain camera sensors or imaging capabilities.
A. Works Using Hybrid Sensor Modalities With Cameras
There are also multiple hybrid systems that include
vision-based sensors in addition to other sensor types
[122], [124], [140]. While Nam et al. [124] simply fuse
information from a camera and accelerometer to increase
the accuracy of classification for ambulatory activities,
Doherty et al. [140] and Wu et al. [122] use the context
provided by cameras to identify the specific class of activity
once an accelerometer has identified the level of activity being
undertaken.
Numerous of these works favor a camera and orientation-
based sensor combination. Spriggs et al. [214] focus on
temporal segmentation of an activity in order to recognize
different temporal parts. The authors explore the usage of
GMMs, HMMs, and K-Nearest Neighbors for segmenting
and classifying various actions involved in the cooking of
different meals. The results demonstrate that using both IMU
and camera data improves results over single modality sensing.
Additionally, the best results were obtained using a K-nearest
neighbor approach, whose success was largely attributed to
the fact that the feature vectors being used had high dimen-
sionality. Li et al. [215] utilize gyroscope to obtain candidate
boundary between different daily activities, and extract visual
features from images for better video segmentation. Ozcan and
Velipasalar [149] proposed wearable camera and accelerom-
eter based fall detection algorithm for portable systems that
can eliminate false positives significantly, compared to camera
only and accelerometer only systems, while preserving correct
detection rate.
More recently, a common proposal, for these camera and
orientation-based sensors has been an eyeglass-mounted sys-
tem [21], [134], [135], [216]. Windau and Itti [21] use the
inertial sensing from a gyroscope and accelerometer to nor-
malize the coordinate system for different head orientations.
For the inertial sensors, energy consumption and movement
intensity, and the mean and variance of the sensor readings
are extracted. As for the camera-based features, a single GIST
vector per frame is calculated. The video data is also used to
classify an indoor versus outdoor environment using the GIST
vectors. The activities classified include lying down, walking,
jogging, biking, washing dishes, brushing teeth, etc. The
results show an 81.5% classification accuracy on 20 different
activities. Zhan et al. [135] also design an eye glasses-like
system with a first person view camera and accelerometer.
This work shows that while accelerometer-only classification
has proven effective for dynamic activities, the camera is
more suitable for static activities. The extracted accelerometer
features include both frequency and time domain information.
The extracted video features include optical flow vectors. The
authors compare classification results for different sensors
using LogitBoost and SVM with numerous different kernels.
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
IEEE SENSORS JOURNAL, VOL. 17, NO. 2, JANUARY 15, 2017
Combining the classifiers through structured prediction using
a Conditional Random Field (CRF) with Tree Re-Weighted
Belief Propagation, they obtain an overall accuracy of 84.45%
on 12 activities. Hernandez et al. [134] propose a real-time
human activity recognition for sitting, standing and supine
with glasses-based system with onboard processing.
Ishimaru et al. [216] additionally make use of an
Infrared (IR) proximity sensor on a glasses-based system.
The IR proximity sensor is used to measure the dis-
tance between the eyes and the eyewear in order to per-
form blink detection. Moreover, the average variance of
3D-accelerometer is calculated to construct a head motion
model. They try to distinguish between the activities of
watching, reading, solving, sawing and talking. An overall
accuracy of 82% is achieved on an 8-person dataset for
these five activities. Other works have used IR, orientation,
and camera systems. Fleury et al. [148] use a webcam,
a 3D-accelerometer, a 3D-magnetometer, an IR sensor, and a
microphone. They apply Principal Component Analysis (PCA)
on extracted features to get ten dominant features that are
fed through a multi-SVM for 35 activities, reaching 86%
accuracy. Punangpakisiri et al. [144] deploy senseCam with
a 3D accelerometer, a light sensor, and a passive IR sensor to
classify ten activities.
With advances in gaming systems, such as the Kinect, Red,
Green, Blue, and Depth (RGB-D) sensors have also surged in
popularity in the last few years. Bahle et al. [145], Damen
et al. [151], and Moghimi et al. [137] explore the use of
RGB-D sensors that provide both a regular imaging capability
and depth information. Bahle et al. [145] reach 92% overall
accuracy by using camera and depth-based information with a
Dynamic Time Warping technique when classifying walking,
writing, using stairs, and using the dishwasher. A helmet-
mounted RGB-D camera is employed in [137]. The features
used include GIST and a skin segmentation algorithm. To
classify the activities, learning based methods such as bag
of SIFT words, Convolutional Neural Networks (CNNs), and
SVMs were explored. They conclude that CNN-based features
provide the best representation for finer-grained activity recog-
nition or tasks that involve manipulations of objects by hands.
These works however are still relatively new and have explored
limited activity sets compared to other hybrid approaches with
around three to six activities classified in each of these works.
B. Works Using Hybrid Sensor Modalities Without Cameras
There are yet other works that rely on a combination
of numerous sensors but do not use an imaging sensor.
To acquire the context otherwise extracted from a camera,
such as object type or an indication of location in an indoor
space, one common sensor suite is one that contains an Radio
Frequency Identification (RFID) sensor in addition to other
modalities. Hong et al. [123], [125] use an accelerometer
combined with an RFID sensor to identify object types and
ultimately provide the context for classification. Shimokawara
et al. [150] use tagging technology to recognize what area of a
house the individual is in and then apply accelerometer-based
information for classification. Torres et al. [126] also use a
CORNACCHIA et al.: SURVEY ON ACTIVITY DETECTION AND CLASSIFICATION USING WEARABLE SENSORS 395

TABLE IV
PAPERS C LASSIFIED BY S ENSOR L OCATION , N UMBER , AND T YPE , N UMBER OF S UBJECTS U SED
FOR E VALUATION , AND VARIANT OF L EARNING A LGORITHM U SED

3D-accelerometer and RFID to classify motions in and out of Another more popular combination for these hybrid systems
bed. Ranasinghe et al. [217] use RFID information to monitor appears to be an ECG sensor used together with other sensor
the motions of patients in a hospital. types [24], [66], [128], [129], [141], [142]. Li et al. [66],

Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
396 IEEE SENSORS JOURNAL, VOL. 17, NO. 2, JANUARY 15, 2017

TABLE V
PAPERS C LASSIFIED BY S ENSOR L OCATION , N UMBER , AND T YPE , AND VARIANT OF L EARNING A LGORITHM U SED C ONTINUED

TABLE VI
PAPERS C LASSIFIED A CCORDING TO A CTIVITY AND S ENSOR T YPE AND P ROCESSING L OCATION

Jia and Liu [129], and Banerjee et al. [24] use an ECG sensor
together with an accelerometer. Li et al. [66] and [129], both
use an SVM approach to classify ambulatory type activities,
differing in the size of feature sets used for the classification.
Reference [129] improve the accuracy over [66] by extracting
262 features, and selecting the six most relevant by performing
Linear Discriminant Analysis (LDA). Others apply ECG sen-
sor combinations for different applications. Tapia et al. [127]
use an accelerometer in conjunction with a heart rate monitor
to classify physical activities and the intensity of the activity.
Lin et al. [94] use an accelerometer and heart rate sensor
to classify not only the intensity level but specific daily life
activities.
These applications for ECG sensor combinations extend
in applying activity classification for the healthcare field to
automatically monitor patients. Yuan and Herbert [218] use
position, ECG, blood pressure, and blood oxygenation sensors
for the general monitoring of individuals. Keskar et al. [200]
propose a wearable on-board processing chip with ECG sen-
sors, using HMM to classify ECG signals to detect abnormal-
ity. Patel et al. [219] use multiple gyrometer and accelerometer
sensors with heart rate and respiration rate data to monitor
the activity of a person with Chronic Obstructive Pulmonary
Disease (COPD). Fletcher et al. [220] use an ankle-mounted
sensor capable of measuring electrodermal activity (EDA), 3-
axis acceleration, temperature, along with a chest-mounted
ECG. These sensors are used in conjunction with an Android-
based software, applying the system to military Post Trau-
matic Stress Syndrome to determine if a user is agitated.
The pressure sensor also appears frequently in the literature
[132], [138], [130], [143], [147], [221]. Edgar et al. [147]
design a system with three pressure sensors on both shoes
and a 3D accelerometer on wrist to classify household and
athletic activities. Sazonov et al. [138] use five force-sensitive
pressure sensors and a 3D-accelerometer on each shoe to
classify sitting, standing, walking, ascending and descending

Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
CORNACCHIA et al.: SURVEY ON ACTIVITY DETECTION AND CLASSIFICATION USING WEARABLE SENSORS
stairs, and cycling. Initially, they extract eight features to
train an SVM and reach an average accuracy of 95.2% [138],
but later achieve a 99.8% overall accuracy using SVM with
rejection and ANN with rejection on the six activity set [132].
Sazonov et al. [133] later extend the activity set to 15 different
activities and apply SVM, Multinomial Logistic Discrimina-
tion (MLD), and Multilayer Perceptrons (MLP).
There are yet other sensor combinations explored in [136],
[139], [146], [152], [153], and [221]. Basak et al. [152]
use a wearable microphone on kids in day-care, and clas-
sify four activities; cough, sneeze, cry and toilet flush.
De et al. [131] and [221] use an accelerometer, a gyroscope, a
temperature/humidity sensor, and atmospheric pressure sensor
to classify a combination of ambulatory and daily life activ-
ities. Pärkkä et al. [136] explore not only combinations of
two different sensors, but employ 22 wearable sensors. They
classify lying, rowing, cycling, sit/stand, running, nordic walk,
or walking by introducing a custom automatically generated
decision tree and an ANN. Riboni and Bettini [139] use a
3D accelerometer, temperature, and light sensor to classify
a mix of global body motion and local interaction activi-
ties through ontological reasoning. Tacconi et al. [146] use
accelerometer and microphone on wrist and hip, classifying
emotion and activity, reaching 47.1% accuracy. These works
show that the hybrid sensing modalities used are dictated
by the application for which the author is attempting to
address. That is, an ECG is often used in the healthcare
field or to measure intensity level of an activity, RFIDs are
used in combination of other sensors to provide locational
or object type context to the classification, and others may
apply a mix of sensors to classify both local and global type
interactions.
VI. C ONCLUSION
A review of wearable sensor approaches for activity detec-
tion and classification has been presented. Different from
existing surveys, which focus on only certain types of sensors,
we have covered a breadth of wearable sensing modalities,
including accelerometer, gyroscope, pressure sensors, depth-
based, and hybrid modality systems, to provide a complete
and comprehensive survey. In addition to the type, number
and placement of sensors, and the type of activities that
are detected, we have classified the large set of research
papers based on the employed learning approach, the extent of
experimentation and evaluation, and whether the processing is
performed onboard or remotely. Another goal is to motivate
additional work to extend wearable camera-based systems.
We have covered a wide range of wearable sensor
approaches by also discussing the progression of how methods
using different sensor modalities approach various different
problems. Initial orientation-based sensing approaches started
with stable sensors at the waist and classified a limited set of
activities. As orientation-based sensors and processing power
advanced, other works explored orientation-based sensors on
the extremities. With this advancement to the extremities, the
number of classes that an approach was able to distinguish
increased. These approaches additionally saw improvements
and can be distinguished from one another in terms of the
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
397
increasing feature sets used. Moving these orientation sensors
to the extremities also enabled the ability to approach new
activity types that involve more localized motions such as
brushing teeth, vacuuming, and so on. However, orientation-
based sensors are limited by the information they provide, and
with improvements in the camera technology, others have been
able to show that cameras can provide details that these other
orientation sensors cannot. That is, images and videos can
provide more detailed information and context for a specific
action sequence, and not only be able to distinguish between
making food, but also making specific types of food. Yet,
despite the additional activities these camera only systems
can tackle, the accuracies of these systems, often below 60%,
are far lower than other works discussed. Hence, more recent
hybrid works have provided the ability to classify both global
and local activities, while maintaining higher accuracies of
above 80%. However, it can be argued that these single-sensor
based approaches have laid the groundwork for more advanced
hybrid approaches. These hybrid approaches have also tended
to be more extensive in their experimentation, often using more
realistic experimentation approaches with more subjects. With
community datasets sparse in all but the camera-based only
works, the experimentation is often specific to each work.
Hence, a potential contribution to activity classification using
wearable sensors would be the creation of a standard dataset.
Table VI also reflects that there is still work to do to develop
methods that perform processing onboard in real-time versus
works where analysis is remotely processed.
R EFERENCES
[1] K. Yatani and K. N. Truong, “BodyScope: A wearable acoustic sensor
for activity recognition,” in Proc. Ubicomp, 2012, pp. 341–350.
[2] O. Amft, M. Stäger, P. Lukowicz, and G. Tröster, “Analysis of chewing
sounds for dietary monitoring,” in Proc. 7th Int. Conf. Ubiquitous
Comput., Berlin, Germany, 2005, pp. 56–72.
[3] G. J. Brown and M. Cooke, “Computational auditory scene analysis,”
Comput. Speech Lang., vol. 8, no. 4, pp. 297–336, 1994.
[4] J. Chen, A. H. Kam, J. Zhang, N. Liu, and L. Shue, Bathroom Activity
Monitoring Based on Sound. Berlin, Germany: Springer-Verlag, 2005,
pp. 47–61.
[5] C. Brian, N. Sawhney, and A. Pentland. 1998. Auditory
Context Awareness Via Wearable Computing. [Online]. Available:
http://citeseer.ist.psu.edu/brian98auditory.html
[6] H. Lu, W. Pan, N. D. Lane, T. Choudhury, and A. T. Campbell,
“Soundsense: Scalable sound sensing for people-centric applications on
mobile phones,” in Proc. 7th Int. Conf. Mobile Syst., Appl., Services,
2009, pp. 165–178.
[7] V. Peltonen, J. Tuomi, A. Klapuri, J. Huopaniemi, and T. Sorsa,
“Computational auditory scene recognition,” in Proc. IEEE Int.
Conf. Acoust., Speech, Signal Process. (ICASSP), May 2002,
pp. II-1941–II-1944.
[8] A. Yazar, A. E. Çetin, and B. U. Töreyin, “Human activity classification
using vibration and PIR sensors,” in Proc. 20th Signal Process.
Commun. Appl. Conf. (SIU), Apr. 2012, pp. 1–4.
[9] D. Gordon, F. Witt, H. Schmidtke, and M. Beigl, “A long-term sensory
logging device for subject monitoring,” in Proc. 4th Int. Conf. Pervasive
Comput. Technol. Healthcare, Mar. 2010, pp. 1–4.
[10] K. Van Laerhoven and H. Gellersen, “Spine versus porcupine: A study
in distributed wearable activity recognition,” in Proc. 8th Int. Symp.
Wearable Comput., Oct. 2004, pp. 142–149.
[11] K. Van Laerhoven, D. Kilian, and B. Schiele, “Using rhythm awareness
in long-term activity recognition,” in Proc. 12th IEEE Int. Symp.
Wearable Comput. (ISWC), Sep. 2008, pp. 63–66.
[12] B. Mirmahboub, S. Samavi, N. Karimi, and S. Shirani, “Automatic
monocular system for human fall detection based on variations in
silhouette area,” IEEE Trans. Biomed. Eng., vol. 60, no. 2, pp. 427–436,
Feb. 2013.
398
[13] H. Nait-Charif and S. McKenna, “Activity summarisation and fall
detection in a supportive home environment,” in Proc. 17th Int. Conf.
Pattern Recognit., vol. 4, Aug. 2004, pp. 323–326.
[14] X. Zou and B. Bhanu, “Anomalous activity classification in the
distributed camera network,” in Proc. 15th IEEE Int. Conf. Image
Process., (ICIP), Oct. 2008, pp. 781–784.
[15] G. Panahandeh, N. Mohammadiha, A. Leijon, and P. Handel, “Continu-
ous hidden Markov model for pedestrian activity classification and gait
analysis,” IEEE Trans. Instrum. Meas., vol. 62, no. 5, pp. 1073–1083,
May 2013.
[16] G. Srivastava, H. Iwaki, J. Park, and A. C. Kak, “Distrib-
uted and lightweight multi-camera human activity classification,” in
Proc. ACM/IEEE Int. Conf. Distrib. Smart Cameras, Aug. 2009,
pp. 1–8.
[17] Census: Homes with Cell Phones Nearly Double in First Half of
Decade. Accessed on Sep. 30, 2010. [Online]. Available: https://www.
census.gov/newsroom/releases/archives/income_wealth/cb09-174.html
[18] O. D. Lara and M. A. Labrador, “A survey on human activity recog-
nition using wearable sensors,” IEEE Commun. Surveys Tuts., vol. 15,
no. 3, pp. 1192–1209, Jul. 2013.
[19] B. Bruno, F. Mastrogiovanni, A. Sgorbissa, T. Vernazza, and
R. Zaccaria, “Analysis of human behavior recognition algorithms
based on acceleration data,” in Proc. IEEE Int. Conf. Robot. Autom.,
May 2013, pp. 1602–1607.
[20] S. Chernbumroong, S. Cang, A. Atkins, and H. Yu, “Elderly activities
recognition and classification for applications in assisted living,” Expert
Syst. Appl., vol. 40, no. 5, pp. 1662–1674, Apr. 2013.
[21] J. Windau and L. Itti, “Situation awareness via sensor-equipped eye-
glasses,” in Proc. IEEE Int. Conf. Intell. Robots Syst., Nov. 2013,
pp. 5674–5679.
[22] J.-Y. Yang, J. S. Wang, and Y. P. Chen, “Using acceleration mea-
surements for activity recognition: An effective learning algorithm for
constructing neural classifiers,” Pattern Recognit. Lett., vol. 29, no. 16,
pp. 2213–2220, 2008.
[23] S. Dernbach, B. Das, N. C. Krishnan, B. L. Thomas, and D. J. Cook,
“Simple and complex activity recognition through smart phones,” in
Proc. 8th Int Conf. Intell. Environ., 2012, pp. 214–221.
[24] D. Banerjee, S. Biswas, C. Daigle, and J. M. Siegford, “Remote
activity classification of hens using wireless body mounted sensors,”
in Proc. Int. Workshop Wearable Implant. Body Sensor Netw., 2012,
pp. 107–112.
[25] J. Mantyjarvi, J. Himberg, and T. Seppanen, “Recognizing human
motion with multiple acceleration sensors,” in Proc. IEEE Int. Conf.
Syst., Man, Cybern., vol. 2, Oct. 2001, pp. 2–7.
[26] C. Zhu and W. Sheng, “Wearable sensor-based hand gesture and
daily activity recognition for robot-assisted living,” IEEE Trans.
Syst., Man, Cybern. A, Syst., Humans, vol. 41, no. 3, pp. 569–573,
May 2011.
[27] K. Taylor, U. A. Abdull, R. J. N. Helmer, J. Lee, and I. Blanchonette,
“Activity classification with smart phones for sports activities,” Proce-
dia Eng., vol. 13, pp. 428–433, Jul. 2011.
[28] D. Coskun, O. D. Incel, and A. Ozgovde, “Phone position/placement
detection using accelerometer: Impact on activity recognition,” in
Proc. IEEE 10th Int. Conf. Intell. Sensors, Sensor Netw. Inf. Process.
(ISSNIP), Apr. 2015, pp. 1–6.
[29] S. J. Preece, J. Y. Goulermas, L. P. J. Kenney, and D. Howard, “A
comparison of feature extraction methods for the classification of
dynamic activities from accelerometer data,” IEEE Trans. Biomed.
Eng., vol. 56, no. 3, pp. 871–879, Mar. 2009.
[30] K. Kunze and P. Lukowicz, “Sensor placement variations in wear-
able activity recognition,” IEEE Pervasive Comput., vol. 13, no. 4,
pp. 32–41, Dec. 2014.
[31] L. Atallah, B. Lo, R. King, and G.-Z. Y. G.-Z. Yang, “Sensor placement
for activity detection using wearable accelerometers,” in Proc. Int.
Conf. Body Sensor Netw. (BSN), Jul. 2010, pp. 24–29.
[32] L. Atallah, B. Lo, R. King, and G.-Z. Yang, “Sensor positioning
for activity recognition using wearable accelerometers,” IEEE Trans.
Biomed. Circuits Syst., vol. 5, no. 4, pp. 320–329, Aug. 2011.
[33] Y. Prathivadi, J. Wu, T. R. Bennett, and R. Jafari, “Robust activity
recognition using wearable IMU sensors,” in Proc. IEEE SENSORS,
Nov. 2014, pp. 486–489.
[34] M. Kreil, B. Sick, and P. Lukowicz, “Dealing with human variability
in motion based, wearable activity recognition,” in Proc. IEEE Int.
Conf. Pervasive Comput. Commun. Workshops (PERCOM Workshops),
Mar. 2014, pp. 36–40.
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
IEEE SENSORS JOURNAL, VOL. 17, NO. 2, JANUARY 15, 2017
[35] R. Saeedi, J. Purath, K. Venkatasubramanian, and H. Ghasemzadeh,
“Toward seamless wearable sensing: Automatic on-body sensor local-
ization for physical activity monitoring,” in Proc. 36th Annu. Int. Conf.
IEEE Eng. Med. Biol. Soc. (EMBC), Aug. 2014, pp. 5385–5388.
[36] A. M. Khan, Y. K. Lee, and S. Y. Lee, “Accelerometer’s position free
human activity recognition using a hierarchical recognition model,” in
Proc. 12th IEEE Int. Conf. e-Health Netw. Appl. Services (Healthcom),
Jul. 2010, pp. 296–301.
[37] A. Betancourt, P. Morerio, C. S. Regazzoni, and M. Rauterberg, “The
evolution of first person vision methods: A survey,” IEEE Trans.
Circuits Syst. Video Technol., vol. 25, no. 5, pp. 744–760, May 2015.
[38] S.-W. Lee and K. Mase, “Activity and location recognition using
wearable sensors,” IEEE Pervasive Comput., vol. 1, no. 3, pp. 24–32,
Jul./Sep. 2002.
[39] D. M. Karantonis, M. R. Narayanan, M. Mathie, N. H. Lovell, and
B. G. Celler, “Implementation of a real-time human movement clas-
sifier using a triaxial accelerometer for ambulatory monitoring,” IEEE
Trans. Inf. Technol. Biomed., vol. 10, no. 1, pp. 156–167, Jan. 2006.
[40] S.-K. Song, J. Jang, and S. Park, “A phone for human activity
recognition using triaxial acceleration sensor,” in Dig. Tech. Papers,
Jan. 2008, pp. 1–2.
[41] S. Zhang, P. McCullagh, C. Nugent, and H. Zheng, “Activity mon-
itoring using a smart phone’s accelerometer with hierarchical clas-
sification,” in Proc. 6th Int. Conf. Intell. Environ. (IE), Jul. 2010,
pp. 158–163.
[42] J. R. Kwapisz, G. M. Weiss, and S. A. Moore, “Activity recogni-
tion using cell phone accelerometers,” ACM SIGKDD Explorations
Newslett., vol. 12, no. 2, pp. 74–82, Dec. 2010.
[43] M. Arif, M. Bilal, A. Kattan, and S. I. Ahamed, “Better physical
activity classification using smartphone acceleration sensor,” J. Med.
Syst., vol. 38, no. 95, pp. 1–10, Jul. 2014.
[44] Z. Yan, V. Subbaraju, D. Chakraborty, A. Misra, and K. Aberer,
“Energy-efficient continuous activity recognition on mobile phones:
An activity-adaptive approach,” in Proc. 16th Int. Symp. Wearable
Comput., Jun. 2012, pp. 17–24.
[45] D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-
Ortiz, “Energy efficient smartphone-based activity recognition using
fixed-point arithmetic,” J. Universal Comput. Sci., vol. 19, no. 9,
pp. 1295–1314, 2013.
[46] M. Zhang and A. A. Sawchuk, “Human daily activity recognition with
sparse representation using wearable sensors,” IEEE J. Biomed. Health
Informat., vol. 17, no. 3, pp. 553–560, May 2013.
[47] S. Weng et al., “A low power and high accuracy MEMS sensor based
activity recognition algorithm,” in Proc. IEEE Int. Conf. Bioinformat.
Biomed. (BIBM), Nov. 2014, pp. 33–38.
[48] Z. He, Z. Liu, L. Jin, L.-X. Zhen, and J.-C. Huang, “Weightless-
ness feature–a novel feature for single tri-axial accelerometer based
activity recognition,” in Proc. 19th Int. Conf. Pattern Recognit., 2008,
pp. 1–4.
[49] Z. He and L. Jin, “Activity recognition from acceleration data based on
discrete consine transform and SVM,” in Proc. IEEE Int. Conf. Syst.,
Man, Cybern., Oct. 2009, pp. 5041–5044.
[50] A. Ataya, P. Jallon, P. Bianchi, and M. Doron, “Improving activity
recognition using temporal coherence,” in Proc. Int. Conf. IEEE Eng.
Med. Biol. Soc., Jul. 2013, pp. 4215–4218.
[51] N. Ravi, N. Dandekar, P. Mysore, and M. L. Littman, “Activity
recognition from accelerometer data,” in Proc. 17th Conf. Innov. Appl.
Artif. Intell., 2005, pp. 1541–1546.
[52] A. Anjum and M. U. Ilyas, “Activity recognition using smartphone
sensors,” in Proc. IEEE 10th Consum. Commun. Netw. Conf., (CCNC),
Jun. 2013, pp. 914–919.
[53] K. Basterretxea, J. Echanobe, and I. del Campo, “A wearable human
activity recognition system on a chip,” in Proc. Conf. Design Archit.
Signal Image Process. (DASIP), Oct. 2014, pp. 1–8.
[54] R. Braojos, I. Beretta, J. Constantin, A. Burg, and D. Atienza, “A
wireless body sensor network for activity monitoring with low trans-
mission overhead,” in Proc. 12th IEEE Int. Conf. Embedded Ubiquitous
Comput., Aug. 2014, pp. 265–272.
[55] F.-C. Chuang, J.-S. Wang, Y.-T. Yang, and T.-P. Kao, “A wearable
activity sensor system and its physical activity classification scheme,”
in Proc. Int. Joint Conf. Neural Netw., Jun. 2012, pp. 10–15.
[56] Ó. D. Lara and M. A. Labrador, “A mobile platform for real-time
human activity recognition,” in Proc. IEEE Consum. Commun. Netw.
Conf. (CCNC), Jun. 2012, pp. 667–671.
[57] N. Alshurafa et al., “Robust human intensity-varying activity recog-
nition using stochastic approximation in wearable sensors,” in Proc.
IEEE Int. Conf. Body Sensor Netw., May 2013, pp. 1–6.
CORNACCHIA et al.: SURVEY ON ACTIVITY DETECTION AND CLASSIFICATION USING WEARABLE SENSORS
[58] U. Maurer, A. Smailagic, D. P. Siewiorek, and M. Deisher, “Activity
recognition and monitoring using multiple sensors on different body
positions,” in Proc. Int. Workshop Wearable Implant. Body Sensor Netw.
(BSN), Apr. 2006, pp. 4–7.
[59] W.-Y. Chung, A. Purwar, and A. Sharma, “Frequency domain
approach for activity classification using accelerometer,” in Proc. 30th
Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBS), Aug. 2008,
pp. 1120–1123.
[60] N. C. Krishnan and S. Panchanathan, “Analysis of low resolution
accelerometer data for continuous human activity recognition,” in Proc.
IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), Mar. 2008,
pp. 3337–3340.
[61] M. Ermes, J. Pärkkä, J. Mäntyjärvi, and I. Korhonen, “Detection of
daily activities and sports with wearable sensors in controlled and
uncontrolled conditions,” IEEE Trans. Inf. Technol. Biomed., vol. 12,
no. 1, pp. 20–26, Jun. 2008.
[62] L. Wang, S. Thiemjarus, B. Lo, and G.-Z. Yang, “Toward A mixed-
signal reconfigurable ASIC for real-time activity recognition,” in
Proc. 5th Int. Workshop Wearable Implant. Body Sensor Netw., 2008,
pp. 227–230.
[63] C. Zhu and W. Sheng, “Human daily activity recognition in robot-
assisted living using multi-sensor fusion,” in Proc. IEEE Int. Conf.
Robot. Autom., May 2009, pp. 2154–2159.
[64] T. Hao, L. Pang, X. Li, and S. Xing, “Wearable activity recognition
for automatic microblog updates,” in Proc. IEEE/ASME Int. Conf. Adv.
Intell. Mechatron., Jul. 2009, pp. 1720–1723.
[65] J. Ryder, B. Longstaff, S. Reddy, and D. Estrin, “Ambulation: A tool
for monitoring mobility patterns over time using mobile phones,” in
Proc. Int. Conf. Comput. Sci. Eng., Aug. 2009, pp. 927–931.
[66] M. Li et al., “Multimodal physical activity recognition by fusing
temporal and cepstral information,” IEEE Trans. Neural Syst. Rehabil.
Eng., vol. 18, no. 4, pp. 369–380, Aug. 2010.
[67] D. Curone, G. M. Bertolotti, A. Cristiani, E. L. Secco, and
G. Magenes, “A real-time and self-calibrating algorithm based on
triaxial accelerometer signals for the detection of human posture
and activity,” IEEE Trans. Inf. Technol. Biomed., vol. 14, no. 4,
pp. 1098–1105, Jul. 2010.
[68] A. Mannini and A. M. Sabatini, “On-line classification of human
activity and estimation of walk-run speed from acceleration data using
support vector machines,” in Proc. Int. Conf. IEEE Eng. Med. Biol.
Soc., Sep. 2011, pp. 3302–3305.
[69] S. Chernbumroong, A. S. Atkins, and H. Yu, “Activity classification
using a single wrist-worn accelerometer,” in Proc. 5th Int. Conf. Softw.,
Knowledge Inf. Ind. Manag. Appl. (SKIMA), 2011, pp. 1–6.
[70] J. Andreu, R. D. Baruah, and P. Angelov, “Real time recognition of
human activities from wearable sensors by evolving classifiers,” in
Proc. IEEE Int. Conf. Fuzzy Syst., Jun. 2011, pp. 2786–2793.
[71] M. Luštrek, B. Cvetković, and S. Kozina, “Energy expenditure estima-
tion with wearable accelerometers,” in Proc. IEEE Int. Symp. Circuits
Syst. (ISCAS), May 2012, pp. 5–8.
[72] P. Siirtola and J. Röning, “Ready-to-use activity recognition for smart-
phones,” in Proc. IEEE Symp. Comput. Intell. Data Mining, Apr. 2013,
pp. 59–64.
[73] J. Margarito, R. Helaoui, A. M. Bianchi, F. Sartor, and A. Bonomi,
“User-independent recognition of sports activities from a single wrist-
worn accelerometer: A template-matching-based approach,” IEEE
Trans. Biomed. Eng., vol. 63, no. 4, pp. 788–796, Apr. 2016.
[74] J. Lee, M. McCarthy, D. Rowlands, and D. James, “Decision-tree-
based human activity classification algorithm using single-channel foot-
mounted gyroscope,” Electron. Lett., vol. 51, no. 9, pp. 675–676,
2015.
[75] W. Xu, M. Zhang, A. A. Sawchuk, and M. Sarrafzadeh, “Co-
recognition of human activity and sensor location via compressed
sensing in wearable body sensor networks,” in Proc. Int. Workshop
Wearable Implant. Body Sensor Netw., 2012, pp. 124–129.
[76] M. Berchtold, M. Budde, D. Gordon, H. Schmidtke, and M. Beigl,
“ActiServ: Activity Recognition Service for mobile phones,” in Proc.
Int. Symp. Wearable Comput. (ISWC), Oct. 2010, pp. 1–8.
[77] D. Naranjo-Hernández, L. M. Roa, J. Reina-Tosina, and
M. Á. Estudillo-Valderrama, “SoM: A smart sensor for human
activity monitoring and assisted healthy ageing,” IEEE Trans. Biomed.
Eng., vol. 59, no. 11, pp. 3177–3184, Nov. 2012.
[78] R. Rednic, E. Gaura, J. Brusey, and J. Kemp, “Wearable posture
recognition systems: Factors affecting performance,” in Proc. IEEE-
EMBS Int. Conf. Biomed. Health Inf. (BHI), Jan. 2012, pp. 200–203.
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
399
[79] D. Trabelsi, S. Mohammed, F. Chamroukhi, L. Oukhellou, and
Y. Amirat, “An unsupervised approach for automatic activity recogni-
tion based on hidden Markov model regression,” IEEE Trans. Autom.
Sci. Eng., vol. 10, no. 3, pp. 829–835, Jul. 2013.
[80] H. Gjoreski, S. Kozina, M. Gams, and M. Lustrek, “RAReFall—Real-
time activity recognition and fall detection system,” in Proc. IEEE Int.
Conf. Pervasive Comput. Commun. Workshops (PERCOM Workshops),
Mar. 2014, pp. 145–147.
[81] T. Hester, D. M. Sherrill, M. Hamel, K. Perreault, P. Boissy, and
P. Bonato, “Using wearable sensors to analyze the quality of use of
mobility assistive devices,” in Proc. Int. Workshop Wearable Implant.
Body Sensor Netw., 2006, pp. 6–9.
[82] T. Huynh and B. Schiele, “Towards less supervision in activity
recognition from wearable sensors,” in Proc. Int. Symp. Wearable
Comput. (ISWC), 2007, pp. 3–10.
[83] H. Ghasemzadeh and R. Jafari, “Physical movement monitoring using
body sensor networks: A phonological approach to construct spatial
decision trees,” IEEE Trans. Ind. Informat., vol. 7, no. 1, pp. 66–77,
Feb. 2011.
[84] N. A. Capela, E. D. Lemaire, and N. Baddour, “Feature selection
for wearable smartphone-based human activity recognition with able
bodied, elderly, and stroke patients,” PLoS One, vol. 10, no. 4,
pp. 1–18, 2015.
[85] O. Dobrucali and B. Barshan, “The analysis of wearable motion sensors
in human activity recognition based on mutual information criterion,”
in Proc. 22nd Signal Process. Commun. Appl. Conf. (SIU), Apr. 2014,
pp. 1938–1941.
[86] B. Mortazavi, S. Nyamathi, S. I. Lee, T. Wilkerson, H. Ghasemzadeh,
and M. Sarrafzadeh, “Near-realistic mobile exergames with wireless
wearable sensors,” IEEE J. Biomed. Health Inform., vol. 18, no. 2,
pp. 449–456, Mar. 2014.
[87] L. Zhang, X. Wu, and D. Luo, “Improving activity recognition with
context information,” in Proc. IEEE Int. Conf. Mechatron. Autom.
(ICMA), Aug. 2015, pp. 1241–1246.
[88] C. H. Min, N. F. Ince, and A. H. Tewfik, “Generalization capability
of a wearable early morning activity detection system,” in Proc. 15th
Eur. Signal Process. Conf., Sep. 2007, pp. 1556–1560.
[89] C.-H. Min, N. F. Ince, and A. H. Tewfik, “Classification of continuously
executed early morning activities using wearable wireless sensors,”
in Proc. 30th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBS),
Aug. 2008, pp. 5192–5195.
[90] C.-H. Min, N. F. Ince, and A. H. Tewfik, “Early morning activity
detection using acoustics and wearable wireless sensors,” in Proc. 16th
Eur. Signal Process. Conf., Aug. 2008, pp. 1–5.
[91] E. Fortune, M. Tierney, C. N. Scanaill, A. Bourke, N. Kennedy, and
J. Nelson, “Activity level classification algorithm using SHIMMER
wearable sensors for individuals with rheumatoid arthritis,” in Proc.
Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., (EMBS), Aug. 2011,
pp. 3059–3062.
[92] J.-K. Min and S.-B. Cho, “Activity recognition based on wearable
sensors using selection/fusion hybrid ensemble,” in Proc. IEEE Int.
Conf. Syst., Man, Cybern. (SMC), Oct. 2011, pp. 1319–1324.
[93] M. Stikic, D. Larlus, S. Ebert, and B. Schiele, “Weakly supervised
recognition of daily life activities with wearable sensors,” IEEE Trans.
Pattern Anal. Mach. Intell., vol. 33, no. 12, pp. 2521–2537, Dec. 2011.
[94] C.-W. Lin, Y.-T. C. Yang, J.-S. Wang, and Y.-C. Yang, “A wear-
able sensor module with a neural-network-based activity classification
algorithm for daily energy expenditure estimation,” IEEE Trans. Inf.
Technol. Biomed., vol. 16, no. 5, pp. 991–998, Sep. 2012.
[95] Z. Wang, M. Jiang, Y. Hu, and H. Li, “An incremental learning method
based on probabilistic neural networks and adjustable fuzzy clustering
for human activity recognition by using wearable sensors,” IEEE Trans.
Inf. Technol. Biomed., vol. 16, no. 4, pp. 691–699, Jul. 2012.
[96] S. Zhang, M. H. Ang, W. Xiao, and C. K. Tham, “Detection of activities
for daily life surveillance: Eating and drinking,” in Proc. 10th IEEE
Int. Conf. e-Health Netw., Appl. Service, (HEALTHCOM), Jul. 2008,
pp. 171–176.
[97] H. Koskimaki, V. Huikari, P. Siirtola, P. Laurinen, and J. Roning,
“Activity recognition using a wrist-worn inertial measurement unit: A
case study for industrial assembly lines,” in Proc. 17th Medit. Conf.
Control Autom., 2009, pp. 401–405.
[98] S.-J. Ryu and J.-H. Kim, “Classification of long-term motions using
a two-layered hidden Markov model in a wearable sensor system,”
in Proc. IEEE Int. Conf. Robot. Biomimetics, (ROBIO), Dec. 2011,
pp. 2975–2980.
400
[99] P. Sarcevic, Z. Kincses, and S. Pletl, “Comparison of different classi-
fiers in movement recognition using WSN-based wrist-mounted sen-
sors,” in Proc. IEEE Sensors Appl. Symp. (SAS), Apr. 2015, pp. 1–6.
[100] K.-T. Song and W.-J. Chen, “Human activity recognition using a mobile
camera,” in Proc. 8th Int. Conf. Ubiquitous Robots Ambient Intel.,
Nov. 2011, pp. 3–8.
[101] L. Li, H. Zhang, W. Jia, Z.-H. Mao, Y. You, and M. Sun, “Indi-
rect activity recognition using a target-mounted camera,” in Proc.
4th Int. Congress Image Signal Process. (CISP), vol. 1, Oct. 2011,
pp. 487–491.
[102] Y. Watanabe, T. Hatanaka, T. Komuro, and M. Ishikawa, “Human gait
estimation using a wearable camera,” in Proc. IEEE Workshop Appl.
Comput. Vis., Jan. 2011, pp. 276–281.
[103] K. Zhan, F. Ramos, and S. Faux, “Activity recognition from a wearable
camera,” in Proc. 12th Int. Conf. Control Autom. Robot. Vis. (ICARCV),
Dec. 2012, pp. 365–370.
[104] B. Yin, W. Qi, Z. Wei, and J. Nie, “Indirect human activity recognition
based on optical flow method,” in Proc. 5th Int. Congress Image Signal
Process. (CISP), Oct. 2012, pp. 99–103.
[105] K. M. Kitani, T. Okabe, Y. Sato, and A. Sugimoto, “Fast unsu-
pervised ego-action learning for first-person sports videos,” in Proc.
IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., Jun. 2011,
pp. 3241–3248.
[106] M. S. Ryoo and L. Matthies, “First-person activity recognition: What
are they doing to me?” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit., Jun. 2013, pp. 2730–2737.
[107] K. Ozcan, A. K. Mahabalagiri, M. Casares, and S. Velipasalar,
“Automatic fall detection and activity classification by a wearable
embedded smart camera,” IEEE J. Emerging Sel. Topics Circuits Syst.,
vol. 3, no. 2, pp. 125–136, Jun. 2013.
[108] D. Sun, S. Roth, and M. J. Black, “Secrets of optical flow estima-
tion and their principles,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit., Jun. 2010, pp. 2432–2439.
[109] T. McCandless and K. Grauman, “Object-centric spatio-temporal pyra-
mids for egocentric activity recognition,” in Proc. Brit. Mach. Vis.
Conf., 2013, pp. 30.1–30.11.
[110] H. Pirsiavash and D. Ramanan, “Detecting activities of daily living in
first-person camera views,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit. (CVPR), Jun. 2012, pp. 2847–2854.
[111] K. Matsuo, K. Yamada, S. Ueno, and S. Naito, “An attention-based
activity recognition for egocentric video,” in Proc. IEEE Conf. Comput.
Vis. Pattern Recognit. Workshops, Jun. 2014, pp. 565–570.
[112] Y. Yan, E. Ricci, G. Liu, and N. Sebe, “Egocentric daily activity
recognition via multitask clustering,” IEEE Trans. Image Process.,
vol. 24, no. 10, pp. 2984–2995, Oct. 2015.
[113] A. Fathi, X. Ren, and J. M. Rehg, “Learning to recognize objects
in egocentric activities,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit., Jun. 2011, pp. 3281–3288.
[114] A. Fathi, A. Farhadi, and J. M. Rehg, “Understanding egocen-
tric activities,” in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011,
pp. 407–414.
[115] A. Fathi and J. M. Rehg, “Modeling actions through state changes,”
in Proc. IEEE Comput. Vis. Pattern Recognit. (CVPR), Jun. 2013,
pp. 2579–2586.
[116] S. Sundaram and W. W. M. Cuevas, “High level activity recognition
using low resolution wearable vision,” in Proc. IEEE Conf. Comput.
Vis. Pattern Recognit. Workshops, Jun. 2009, pp. 25–32.
[117] Y. Li, A. Fathi, and J. M. Rehg, “Learning to predict gaze in egocentric
video,” in Proc. IEEE ICCV, Dec. 2013, pp. 3216–3223.
[118] W. W. Mayol and D. W. Murray, “Wearable hand activity recognition
for event summarization,” in Proc. 9th IEEE Int. Symp. Wearable
Comput., (ISWC), Oct. 2005, pp. 122–129.
[119] A. Fathi, J. K. Hodgins, and J. M. Rehg, “Social interactions: A first-
person perspective,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis.
Pattern Recognit., Jun. 2012, pp. 1226–1233.
[120] T. Starner, J. Weaver, and A. Pentland, “Real-time American sign
language recognition using desk and wearable computer based
video,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 12,
pp. 1371–1375, Dec. 1998.
[121] O. Aghazadeh, J. Sullivan, and S. Carlsson, “Novelty detection from
an ego-centric perspective,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit., Jun. 2011, pp. 3297–3304.
[122] H. H. Wu, E. D. Lemaire, and N. Baddour, “Change-of-state determina-
tion to recognize mobility activities using a blackberry smartphone,” in
Proc. Ann. Int. Conf. IEEE Eng. Med. Biol. Soc., (EMBS), Aug. 2011,
pp. 5252–5255.
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
IEEE SENSORS JOURNAL, VOL. 17, NO. 2, JANUARY 15, 2017
[123] Y.-J. Hong, I.-J. Kim, S. C. Ahn, and H.-G. Kim, “Mobile health
monitoring system based on activity recognition using accelerometer,”
Simul. Model. Pract. Theory, vol. 18, no. 4, pp. 446–455, 2010.
[124] Y. Nam, S. Rho, and C. Lee, “Physical activity recognition using mul-
tiple sensors embedded in a wearable device,” ACM Trans. Embedded
Comput. Syst., vol. 12, no. 2, pp. 1–14, 2013.
[125] Y.-J. Hong, I.-J. Kim, S. C. Ahn, and H.-G. Kim, “Activity recognition
using wearable sensors for elder care,” in Proc. 2nd Int. Conf. Future
Generat. Commun. Netw., vol. 2, Dec. 2008, pp. 302–305.
[126] R. L. S. Torres, D. C. Ranasinghe, Q. Shi, and A. P. Sample, “Sensor
enabled wearable RFID technology for mitigating the risk of falls near
beds,” in Proc. IEEE Intl. Conf. (RFID), Apr. 2013, pp. 191–198.
[127] E. M. Tapia et al., “Real-time recognition of physical activities and
their intensitiies using wireless accelerometers and a heart monitor,” in
Proc. Int. Symp. Wearable Comput., 2007, pp. 37–40.
[128] J. Surana, C. S. Hemalatha, V. Vaidehi, S. A. Palavesam, and
M. J. A. Khan, “Adaptive learning based human activity and fall
detection using fuzzy frequent pattern mining,” in Proc. Int. Conf.
Recent Trends Inf. Technol. (ICRTIT), Jul. 2013, pp. 744–749.
[129] R. Jia and B. Liu, “Human daily activity recognition by fus-
ing accelerometer and multi-lead ECG data,” in Proc. IEEE Int.
Conf., Signal Process., Commun. Comput. (ICSPCC), Aug. 2013,
pp. 1–4.
[130] C. M. El Achkar, F. Masse, A. Arami, and K. Aminian, “Physical
activity recognition via minimal in-shoes force sensor configuration,” in
Proc. 7th Int. Conf. Pervasive Comput. Technol. Healthcare Workshops,
(PervasiveHealth), May 2013, pp. 256–259.
[131] D. De, P. Bharti, S. K. Das, and S. Chellappan, “Multimodal wear-
able sensing for fine-grained activity recognition in healthcare,” IEEE
Internet Comput., vol. 19, no. 5, pp. 26–35, Sep./Oct. 2015.
[132] W. Tang and E. S. Sazonov, “Highly accurate recognition of human
postures and activities through classification with rejection,” IEEE J.
Biomed. Health Inform., vol. 18, no. 1, pp. 309–315, Jan. 2014.
[133] E. Sazonov, N. Hegde, R. C. Browning, E. L. Melanson, and
N. A. Sazonova, “Posture and activity recognition and energy expen-
diture prediction in a wearable platform,” IEEE J. Biomed. Health
Informat., vol. 19, no. 4, pp. 1339–1346, Jul. 2015.
[134] J. Hernandez, Y. Li, J. M. Rehg, and R. W. Picard, “BioGlass: Physi-
ological parameter estimation using a head-mounted wearable device,”
in Proc. EAI 4th Int. Conf. Wireless Mobile Commun. Healthcare
(Mobihealth), Nov. 2014, pp. 55–58.
[135] K. Zhan, S. Faux, and F. Ramos, “Multi-scale conditional random fields
for first-person activity recognition,” in Proc. IEEE Int. Conf. Pervas.
Comput. Commun. (PerCom), Mar. 2014, pp. 51–59.
[136] J. Pärkkä, M. Ermes, P. Korpipää, J. Mäntyjärvi, J. Peltola, and
I. Korhonen, “Activity classification using realistic data from wear-
able sensors,” IEEE Trans. Inf. Technol. Biomed., vol. 10, no. 1,
pp. 119–128, Jan. 2006.
[137] M. Moghimi, P. Azagra, L. Montesano, A. C. Murillo, and S. Belongie,
“Experiments on an RGB-D wearable vision system for egocentric
activity recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog-
nit. Workshops, Jun. 2014, pp. 611–617.
[138] E. S. Sazonov, G. Fulk, J. Hill, Y. Schutz, and R. Browning,
“Monitoring of posture allocations and activities by a shoe-based
wearable sensor,” IEEE Trans. Biomed. Eng., vol. 58, no. 4,
pp. 983–990, Apr. 2011.
[139] D. Riboni and C. Bettini, “COSAR: Hybrid reasoning for context-
Aware activity recognition,” Pers. Ubiquitous Comput., vol. 15, no. 3,
pp. 271–289, 2011.
[140] A. R. Doherty et al., “Using wearable cameras to categorise type
and context of accelerometer-identified episodes of physical activity,”
Int. J. Behavioral Nutrition Phys. Activity, vol. 10, no. 1, p. 22,
Jan. 2013.
[141] T. Fujimoto et al., “Wearable human activity recognition by electrocar-
diograph and accelerometer,” in Proc. IEEE 43rd Int. Symp. Multiple-
Valued Logic (ISMVL), May 2013, pp. 12–17.
[142] K. Ouchi, “Smartphone-based monitoring system for activities of daily
living for elderly people and their relatives etc,” in Proc. UbiComp,
2013, pp. 103–106.
[143] A. M. Khan, A. Tufail, A. M. Khattak, and T. H. Laine, “Activity
recognition on smartphones via sensor-fusion and KDA-based SVMs,”
Int. J. Distrib. Sensor Netw., vol. 10, May 2014, Art. no. 503291.
[144] W. Puangpakisiri, T. Yamasaki, and K. Aizawa, “High level activity
annotation of daily experiences by a combination of a wearable
device and Wi-Fi based positioning system,” in Proc. IEEE Intl. Conf.
Multimedia Expo, (ICME), vol. 9, Jun. 2008, pp. 1421–1424.
CORNACCHIA et al.: SURVEY ON ACTIVITY DETECTION AND CLASSIFICATION USING WEARABLE SENSORS
[145] G. Bahle, P. Lukowicz, K. Kunze, and K. Kise, “I see you: How to
improve wearable activity recognition by leveraging information from
environmental cameras,” in Proc. IEEE Int. Conf. Pervasive Comput.
Commun. Workshops, Mar. 2013, pp. 409–412.
[146] D. Tacconi et al., “Activity and emotion recognition to support early
diagnosis of psychiatric diseases,” in Proc. 2nd Int. Conf. Pervasive
Comput. Technol. Healthcare, 2008, pp. 100–102.
[147] S. R. Edgar, G. D. Fulk, and E. S. Sazonov, “Recognition of household
and athletic activities using smartshoe,” in Proc. Annu. Int. Conf. IEEE
Eng. Med. Biol. Soc., Aug. 2012, pp. 6382–6385.
[148] A. Fleury, M. Vacher, and N. Noury, “SVM-based multimodal clas-
sification of activities of daily living in health smart homes: Sensors,
algorithms, and first experimental results,” IEEE Trans. Inf. Technol.
Biomed., vol. 14, no. 2, pp. 274–283, Mar. 2010.
[149] K. Ozcan and S. Velipasalar, “Wearable camera-and accelerometer-
based fall detection on portable devices,” IEEE Embedded Syst. Lett.,
vol. 8, no. 1, pp. 6–9, Mar. 2016.
[150] E. Shimokawara, T. Kaneko, T. Yamaguchi, M. Mizukawa, and
N. Matsuhira, “Estimation of basic activities of daily living using zig-
bee 3D accelerometer sensor network,” in Proc. Int. Conf. Biometrics
Kansei Eng., pp. 251–256, Jul. 2013.
[151] D. Damen, A. Gee, W. Mayol-Cuevas, and A. Calway, “Ego-
centric real-time workspace monitoring using an RGB-D camera,”
in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Oct. 2012,
pp. 1029–1036.
[152] A. Basak, S. Narasimhan, and S. Bhunia, “KiMS: Kids’ health monitor-
ing system at day-care centers using wearable sensors and vocabulary-
based acoustic signal processing,” in Proc. IEEE 13th Int. Conf.
e-Health Netw., Appl. Services, (HEALTHCOM), Jun. 2011, pp. 1–8.
[153] T. Tagniguchi, K. Hamahata, and N. Iwahashi, “Unsupervised segmen-
tation of human motion data using sticky HDP-HMM and MDL-based
chunking method for imitation learning,” Adv. Robot., vol. 25, no. 17,
pp. 2143–2172, 2011.
[154] N. Noury, A. Galay, J. Pasquier, and M. Ballussaud, “Preliminary
investigation into the use of autonomous fall detectors,” in Proc. Int.
Conf. IEEE Eng. Med. Biol. Soc., Aug. 2008, pp. 2828–2831.
[155] C. C. Wang et al., “Development of a fall detecting system for the
elderly residents,” in Proc. 2nd Int. Conf. Bioinform. Biomed. Eng.,
2008, pp. 1359–1362.
[156] T. Tamura, “Wearable accelerometer in clinical use,” in Proc. Int. Conf.
IEEE Eng. Med. Biol. Soc., vol. 7, Jan. 2005, pp. 7165–7166.
[157] Y. Cao, Y. Yang, and W. H. Liu, “E-FallD: A fall detection system
using Android-based smartphone,” in Proc. 9th Int. Conf. Fuzzy Syst.
Knowl. Discovery, May 2012, pp. 1509–1513.
[158] M. R. Narayanan, S. R. Lord, M. M. Budge, B. G. Celler, and
N. H. Lovell, “Falls management: Detection and prevention, using a
waist-mounted triaxial accelerometer,” in Proc. 29th Annu. Int. Conf.
IEEE Eng. Med. Biol. Soc. (EMBS), Aug. 2007, pp. 4037–4040.
[159] W. Wu, S. Dasgupta, E. E. Ramirez, C. Peterson, and
J. G. Norman, “Classification accuracies of physical activities
using smartphone motion sensors,” J. Med. Internet Res.,
vol. 14, no. 5, p. e130, Oct. 2012. [Online]. Available:
http://www.ncbi.nlm.nih.gov/pubmed/23041431
[160] I. Mandal, S. L. Happy, D. P. Behera, and A. Routray, “A framework for
human activity recognition based on accelerometer data,” in Proc. 5th
Int. Conf. Next Generat. Inf. Technol. Summit (Confluence), Sep. 2014,
pp. 600–603.
[161] L. Gao, A. K. Bourke, and J. Nelson, “Evaluation of accelerometer
based multi-sensor versus single-sensor activity recognition systems,”
Med. Eng. Phys., vol. 36, no. 6, pp. 779–785, Jun. 2014.
[162] M. Yu, A. Rhuma, S. Naqvi, L. Wang, and J. Chambers, “Posture
recognition based fall detection system for monitoring an elderly person
in a smart home environment,” IEEE Trans. Inf. Technol. Biomed.,
vol. 16, no. 6, p. 1, Nov. 2012.
[163] J. Zheng, G. Zhang, and T. Wu, “Design of automatic fall detector
for elderly based on triaxial accelerometer,” in Proc. 3rd Int. Conf.
Bioinformati. Biomed. Eng. (ICBBE), Jun. 2009, pp. 1–4.
[164] T. Degen, H. Jaeckel, M. Rufer, and S. Wyss, “SPEEDY: A fall detector
in a wrist watch,” in Proc. 7th IEEE Int. Wearable Comput. Symp.,
Oct. 2003, pp. 184–187.
[165] J. Yang, S. Wang, N. Chen, X. Chen, and P. Shi, “Wearable accelerom-
eter based extendable activity recognition system,” in Proc. IEEE Int.
Conf. Robot. Autom., May 2010, pp. 3641–3647.
[166] G. A. Koshmak, M. Linden, and A. Loutfi, “Evaluation of the Android-
based fall detection system with physiological data monitoring,”
in Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBS), Jul. 2013,
pp. 1164–1168.
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
401
[167] S. H. Fang, Y. C. Liang, and K. M. Chiu, “Developing a mobile phone-
based fall detection system on Android platform,” in Proc. Comput.,
Commun. Appl. Conf., 2012, pp. 143–146.
[168] J. Dai, X. Bai, Z. Yang, Z. Shen, and D. Xuan, “PerFallD: A pervasive
fall detection system using mobile phones,” in Proc. IEEE Int. Conf.
Pervasive Comput. Commun. Workshops, Mar. 2010, pp. 292–297.
[169] M. Shoaib, S. Bosch, O. D. Incel, H. Scholten, and P. J. M. Havinga, “A
survey of Online activity recognition using mobile phones,” Sensors,
vol. 15, no. 1, pp. 2059–2085, 2015.
[170] P. Siirtola and J. Roning, “Recognizing human activities user-
independently on smartphones based on accelerometer data,”
Int. J. Interact. Multimedia Artif. Intell., vol. 1, no. 5, p. 38, 2012.
[171] X. Long, B. Yin, and R. M. Aarts, “Single-accelerometer-based daily
physical activity classification,” in Proc. Int. Conf. IEEE Eng. Med.
Biol. Soc., Sep. 2009, pp. 6107–6110.
[172] M. Muehlbauer, G. Bahle, and P. Lukowicz, “What can an arm holster
worn smart phone do for activity recognition?” in Proc. Int. Symp.
Wearable Comput. (ISWC), 2011, pp. 79–82.
[173] J. Chen, K. Kwong, D. Chang, J. Luk, and R. Bajcsy, “Wearable sensors
for reliable fall detection,” in Proc. Int. Conf. IEEE Eng. Med. Biol.
Soc., vol. 4, Mar. 2005, pp. 3551–3554.
[174] A. Li, L. Ji, S. Wang, and J. Wu, “Physical activity classification using
a single triaxial accelerometer based on HMM,” in Proc. IET Int. Conf.
Wireless Sensor Netw., Nov. 2010, pp. 155–160.
[175] C. Zhu and W. Sheng, “Motion–and location-based Online human
daily activity recognition,” Pervasive Mobile Comput., vol. 7, no. 2,
pp. 256–269, 2011.
[176] M. Ermes, J. Parkka, and L. Cluitmans, “Advancing from offline to
Online activity recognition with wearable sensors,” in Proc. Int. Conf.
IEEE Eng. Med. Biol. Soc., Aug. 2008, pp. 4451–4454.
[177] W. Ugulino, D. Cardador, K. Vega, E. Velloso, R. Milidiü, and H. Fuks,
“Wearable computing: Accelerometers,” in Proc. Data Classification
Body Postures Movements, 2012, pp. 52–61. [Online]. Available:
http://dx.doi.org/10.1007/978-3-642-34459-6_6
[178] E. Mitchell, D. Monaghan, and N. E. O’Connor, “Classification of
sporting activities using smartphone accelerometers,” Sensors (Basel,
Switzerland), vol. 13, no. 4, pp. 5317–5337, 2013.
[179] C. Torres-Huitzil and M. Nuno-Maganda, “Robust smartphone-based
human activity recognition using a TRI-axial accelerometer,” in Proc.
Circuits Syst., IEEE Latin Amer. Symp., Feb. 2015, pp. 1–4.
[180] C. K. Schindhelm, “Activity recognition and step detection with
smartphones: Towards terminal based indoor positioning system,” in
Proc. IEEE Int. Symp. Pers., Indoor Mobile Radio Commun. (PIMRC),
Sep. 2012, pp. 2454–2459.
[181] R. C. Wagenaar, I. Sapir, Y. Zhang, S. Markovic, L. M. Vaina, and
T. D. C. Little, “Continuous monitoring of functional activities using
wearable, wireless gyroscope and accelerometer technology,” in Proc.
Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBS), Aug. 2011,
pp. 4844–4847.
[182] S. Inoue and Y. Hattori, “Toward high-level activity recognition
from accelerometers on mobile phones,” in Proc. IEEE Int. Conf.
Internet Things Cyber, Phys. Soc. Comput., iThings/CPSCom,
Oct. 2011, pp. 225–231.
[183] S. Bambach. (Jan. 2015). “A survey on recent advances of com-
puter vision algorithms for egocentric video.” [Online]. Available:
https://arxiv.org/abs/1501.02825
[184] L. Li, H. Zhang, W. Jia, J. Nie, W. Zhang, and M. Sun, “Automatic
video analysis and motion estimation for physical activity classifica-
tion,” in Proc. IEEE 36th Northeast Bioeng. Conf., Mar. 2010, pp. 1–2.
[185] H. Zhang, L. Li, W. Jia, J. D. Fernstrom, R. J. Sclabassi, and
M. Sun, “Recognizing physical activity from Ego-motion of a camera,”
in Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBS), Jun. 2010,
pp. 5569–5572.
[186] H. Zhang et al., “Physical activity recognition based on motion in
images acquired by a wearable camera,” Neurocomputing, vol. 74,
pp. 2184–2192, Jun. 2011.
[187] X. Ren and C. Gu, “Figure-ground segmentation improves handled
object recognition in egocentric video,” in Proc. IEEE Comput. Soc.
Conf. Comput. Vis. Pattern Recognit., Jul. 2010, pp. 3137–3144.
[188] K. Sumi, M. Toda, A. Sugimoto, T. Matsuyama, and S. Tsukizawa,
“Active wearable vision sensor: Recognition of human activities and
environments,” in Proc. Int. Conf. Inf. Res. Develop. Knowl. Soc.
Infrastructure, 2004, pp. 15–22.
[189] R. O. Castle, D. J. Gawley, G. Klein, and D. W. Murray, “Towards
simultaneous recognition, localization and mapping for hand-held and
wearable cameras,” in Proc. IEEE Int. Conf. Robot. Autom., Mar. 2007,
pp. 4102–4107.
402
[190] B. Xiong and K. Grauman, “Detecting snap points in egocentric
video with a Web photo prior,” in Proc. Comput. Vis. (ECCV), 2014,
pp. 282–298.
[191] Y. Lee and K. Grauman, “Predicting important objects for egocentric
video summarization,” Int. J. Comput. Vis., vol. 114, no. 1, pp. 38–55,
2015.
[192] Y. J. Lee, J. Ghosh, and K. Grauman, “Discovering important people
and objects for egocentric video summarization,” in Proc. IEEE Conf.
CVPR, Jun. 2012, pp. 1346–1353.
[193] Z. Lu and K. Grauman, “Story-driven summarization for egocentric
video,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR),
Jun. 2013, pp. 2714–2721.
[194] Y. J. Lee and K. Grauman, “Predicting Important Objects
for Egocentric Video Summarization,” Int. J. Comput. Vis.,
vol. 114, no. 1, pp. 38–55, Jan. 2015. [Online]. Available:
http://link.springer.com/10.1007/s11263-014-0794-5
[195] R. E. Schapire, Explaining AdaBoost. Berlin, Heidelberg,
Germany: Springer, 2013, pp. 37–52. [Online]. Available:
http://dx.doi.org/10.1007/978-3-642-41136-6_5
[196] K. Yamada, Y. Sugano, T. Okabe, Y. Sato, A. Sugimoto, and
K. Hiraki, “Attention prediction in egocentric video using motion
and visual saliency,” in Proc. Lecture Notes Bioinformat., 2011,
pp. 277–288.
[197] K. Ogaki, K. M. Kitani, Y. Sugano, and Y. Sato, “Coupling eye-motion
and ego-motion features for first-person activity recognition,” in Proc.
IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Workshops,
Jun. 2012, pp. 1–7.
[198] T. Pawar, N. S. Anantakrishnan, S. Chaudhuri, and S. P. Duttagupta,
“Transition detection in body movement activities for wearable
ECG,” IEEE Trans. Biomed. Eng., vol. 54, no. 6, pp. 1149–1152,
Jun. 2007.
[199] R. Kher, T. Pawar, and V. Thakar, “Combining accelerometer data
with Gabor energy feature vectors for body movements classification in
ambulatory ECG signals,” in Proc. Int. Conf. Biomed. Eng. Informat.,
2013, pp. 413–417.
[200] S. Keskar, R. Banerjee, and R. Reddy, “A dual-PSoC based reconfig-
urable wearable computing framework for ECG monitoring,” in Proc.
Comput. Cardiol. (CinC), Sep. 2012, pp. 85–88.
[201] M. Hardegger, L.-V. Nguyen-Dinh, A. Calatroni, D. Roggen, and
G. Tröster, “Enhancing action recognition through simultaneous seman-
tic mapping from body-worn motion sensors,” in Proc. ACM Int. Symp.
Wearable Comput., 2014, pp. 99–106.
[202] A. Bulling, J. A. Ward, H. Gellersen, and G. Tröster, “Eye move-
ment analysis for activity recognition using electrooculography,” IEEE
Trans. Pattern Anal. Mach. Intell., vol. 33, no. 4, pp. 741–753,
Apr. 2011.
[203] S. C. Mukhopadhyay, “Wearable sensors for human activity moni-
toring: A review,” IEEE Sensors J., vol. 15, no. 3, pp. 1321–1330,
Mar. 2015.
[204] C. Jingyuan, O. Amft, G. Bahle, and P. Lukowicz, “Designing sensitive
wearable capacitive sensors for activity recognition,” IEEE Sensors J.,
vol. 13, no. 10, pp. 3935–3947, Oct. 2013.
[205] E. K. Wujcik, N. J. Blasdel, D. Trowbridge, and C. N. Monty, “Ion
sensor for the quantification of sodium in sweat samples,” IEEE Sensors
J., vol. 13, no. 9, pp. 3430–3436, Sep. 2013.
[206] P. H. Veltink and D. D. Rossi, “Wearable technology for biomechanics:
E-textile or micromechanical sensors?” IEEE Eng. Med. Biol. Mag.,
vol. 29, no. 3, pp. 37–43, May 2010.
[207] P. A. Shaltis, A. Reisner, and H. H. Asada, “Wearable, cuff-less ppg-
based blood pressure monitor with novel height sensor,” in Proc. Int.
Conf. IEEE Eng. Med. Biol. Soc., Oct. 2006, pp. 908–911.
[208] P. Corbishley and E. Rodriguez-Villegas, “Breathing detection:
Towards a miniaturized, wearable, battery-operated monitoring sys-
tem,” IEEE Trans. Biomed. Eng., vol. 55, no. 1, pp. 196–204,
Jan. 2008.
[209] J. A. C. Patterson, D. C. McIlwraith, and G. Z. Yang, “A flexible, low
noise reflective ppg sensor platform for ear-worn heart rate monitoring,”
in Proc. 6th Int. Workshop Wearable Implant. Body Sensor Netw.,
Jun. 2009, pp. 286–291.
[210] R. Dudde, T. Vering, G. Piechotta, and R. Hintsche, “Computer-aided
continuous drug infusion: Setup and test of a mobile closed-loop system
for the continuous automated infusion of insulin,” IEEE Trans. Inf.
Technol. Biomed., vol. 10, no. 2, pp. 395–402, Apr. 2006.
[211] C. H. Ahn et al., “Disposable smart lab on a chip for point-of-
care clinical diagnostics,” Proc. IEEE, vol. 92, no. 1, pp. 154–173,
Jan. 2004.
Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.
IEEE SENSORS JOURNAL, VOL. 17, NO. 2, JANUARY 15, 2017
[212] T. Q. Trung, S. Ramasundaram, B.-U. Hwang, and N.-E. Lee,
“An all-elastomeric transparent and stretchable temperature
sensor for body-attachable wearable electronics,” Adv. Mater.,
vol. 28, no. 3, pp. 502–509, 2016. [Online]. Available:
http://dx.doi.org/10.1002/adma.201504441
[213] Y. Chen, B. Lu, Y. Chen, and X. Feng, “Breathable and stretchable
temperature sensors inspired by skin,” Sci. Rep., vol. 5, p. 11505,
Jul. 2015. [Online]. Available: http://dx.doi.org/10.1038/srep11505
[214] E. H. Spriggs, F. De La Torre, and M. Hebert, “Temporal segmentation
and activity classification from first-person sensing,” in Proc. IEEE
Conf. Comput. Vis. Pattern Recognit., Aug. 2009, pp. 17–24.
[215] Z. Li, Z. Wei, W. Jia, and M. Sun, “Daily life event segmentation for
lifestyle evaluation based on multi-sensor data recorded by a wearable
device,” in Proc. Int. Conf. IEEE Eng. Med. Biol. Soc., Jun. 2013,
pp. 2858–2861.
[216] S. Ishimaru, K. Kunze, and K. Kise, “In the blink of an eye: Combining
head motion and eye blink frequency for activity recognition with
Google Glass,” in Proc. 5th Augmented Human Int. Conf., 2014,
p. 15.
[217] D. C. Ranasinghe, R. L. S. Torres, A. P. Sample, J. R. Smith, K. Hill,
and R. Visvanathan, “Towards falls prevention: A wearable wireless
and battery-less sensing and automatic identification tag for real time
monitoring of human movements,” in Proc. Annu. Int. Conf. IEEE Eng.
Med. Biol. Soc., Aug. 2012, pp. 6402–6405.
[218] B. Yuan and J. Herbert, “Web-based real-time remote monitoring for
pervasive healthcare,” in Proc. Pervasive Comput. Commun. Work-
shops, IEEE Int. Conf., Mar. 2011, pp. 625–629.
[219] S. Patel, C. Mancinelli, P. Bonato, J. Healey, and M. Moy, “Using
wearable sensors to monitor physical activities of patients with COPD:
A comparison of classifier performance,” in Proc. 6th Int. Workshop
Wearable Implant. Body Sensor Netw., 2009, pp. 234–239.
[220] R. R. Fletcher, S. Tam, O. Omojola, R. Redemske, and J. Kwan,
“Wearable sensor platform and mobile application for use in cognitive
behavioral therapy for drug addiction and PTSD,” in Proc. Annu. Int.
Conf. IEEE Eng. Med. Biol. Soc., Jun. 2011, pp. 1802–1805.
[221] D. De, P. Bharti, S. K. Das, and S. Chellappan, “Multimodal wear-
able sensing for fine-grained activity recognition in healthcare,” IEEE
Internet Comput., vol. 19, no. 5, pp. 26–35, Sep. 2015.
[222] C. V. C. Bouten, K. T. M. Koekkoek, M. Verduin, R. Kodde, and
J. D. Janssen, “A triaxial accelerometer and portable data processing
unit for the assessment of daily physical activity,” IEEE Trans. Biomed.
Eng., vol. 44, no. 3, pp. 136–147, Mar. 1997.
[223] M. J. Mathie, A. C. F. Coster, N. H. Lovell, and B. G. Celler,
“Detection of daily physical activities using a triaxial accelerometer,”
Med. Biological Eng. Comput., vol. 41, no. 3, pp. 296–301, 2003.
[224] M. B. D. Rosario et al., “A comparison of activity classification in
younger and older cohorts using a smartphone,” Physiological Meas.,
vol. 35, no. 11, p. 2269, 2014.
[225] B. Xiong and K. Grauman, “Detecting Snap Points in Egocentric
Video with a Web Photo Prior,” Comput. Vis., vol. 869, pp. 282–298,
Sep. 2014.
Maria Cornacchia (S’10) received the M.S. degree
in computer and information science from Syracuse
University in 2009, where she is currently pursuing
the Ph.D. degree in computer information science
and engineering. She is also a Computer Scientist
with the Air Force Research Laboratory, Information
Directorate, Rome, NY, USA. Her research interests
include target tracking, information fusion, data min-
ing, and computer vision.
Koray Ozcan (S’11) received the B.S. (Hons.)
degree in electrical and electronics engineering from
Bilkent University, Ankara, Turkey, in 2011. He is
currently pursuing the Ph.D. degree with the Depart-
ment of Electrical and Computer Engineering, Syra-
cuse University, Syracuse, NY, USA. In 2015, he
was a research intern with the Exploratory Computer
Vision Group, IBM Thomas J. Watson Research
Center, Yorktown Heights, NY, USA. His research
interests include video/image processing, computer
vision, embedded systems, machine learning, video
event detection, and object detection.
CORNACCHIA et al.: SURVEY ON ACTIVITY DETECTION AND CLASSIFICATION USING WEARABLE SENSORS 403

Yu Zheng (S’12) received the M.S. degree in elec- Senem Velipasalar (M’04–SM’14) received the
trical engineering from Syracuse University in 2012, B.S. (Hons.) degree in electrical and electronic engi-
where he is currently pursuing the Ph.D. degree in neering from Bogazici University, Istanbul, Turkey,
electrical and computer engineering. His research in 1999, the M.S. degree in electrical sciences
interests include object detection/tracking, multi- and computer engineering from Brown University,
camera systems, bio-medical image processing, and Providence, RI, USA, in 2001, and the M.A. and
computer vision. Ph.D. degrees in electrical engineering from Prince-
ton University, Princeton, NJ, USA, in 2004 and
2007, respectively. From 2001 to 2005, she was
with the Exploratory Computer Vision Group, IBM
T. J. Watson Research Center, Yorktown Heights,
NY, USA. From 2007 to 2011, she was an Assistant Professor with the Depart-
ment of Electrical Engineering, University of Nebraska–Lincoln, Lincoln, NE,
USA. She joined Syracuse University, Syracuse, NY, USA, in 2011, where
she is currently an Associate Professor with the Department of Electrical
Engineering and Computer Science. The focus of her research has been on
mobile camera applications, wireless embedded smart cameras, multicamera
tracking and surveillance systems, and automatic event detection from videos.
Her current research interests include embedded computer vision, video/image
processing, distributed multi-camera systems, pattern recognition and signal
processing.
Dr. Velipasalar is a member of the Editorial Board of the Journal of
Signal Processing Systems (Springer). She received a Faculty Early Career
Development Award from the National Science Foundation in 2011 and
the Best Student Paper Award at the IEEE International Conference on
Multimedia and Expo in 2006. She was a recipient of the Excellence in
Graduate Education Faculty Recognition Award, the EPSCoR First Award,
two Layman Awards, the IBM Patent Application Award, and Princeton and
Brown University Graduate Fellowships. She is the co-authored the paper,
which received the third place award at the 2011 ACM/IEEE International
Conference on Distributed Smart Cameras.

Authorized licensed use limited to: Tampere University. Downloaded on August 03,2021 at 16:35:44 UTC from IEEE Xplore. Restrictions apply.