Jalpaiguri Government Engineering College

(A Government Autonomous Engineering College under Maulana Abul Kalam

Azad University Of Technology (formerly known as WBUT))

(Established in 1961)

“HUMAN ACTIVITY RECOGINITION”

A PROJECT REPORT SUBMITTED IN PARTIAL FULFILLMENT
OF THE REQUIREMENT FOR THE DEGREE OF

BACHELOR OF TECHNOLOGY
IN
ELECTRONICS AND COMMUNICATION ENGINEERING

Under The Guidance Of:-

Mr. Mirwaiz Rahaman
(Assistant Professor, ECE)

Submitted By:-
1. MANI SAURABH TIWARI 16101105015
2. ATUL KUMAR RAJAK 16101105016
3. ASHISH TANK 16101105017
4. AKASH KUMAR 16101105046
5. ABHISHEK KUMAR JHA 16101105059
6. SHUBHAM SINGH 16101105060

Department Of Electronics And Communication Engineering

Jalpaiguri, West Bengal
30th June 2020

1
 Declaration of Originality and
Compliance of Academic Project
I hereby declare that this thesis contains literature survey and original research work
by the undersigned candidates, as part of their degree of Bachelor of Technology in
Electronics & Communication Engineering.
All information in this document have been obtained and presented in accordance with
academic rules and ethical conduct.
I also declare that, as required by these rules and conduct, I have fully cited and
referenced all materials and results that are not original to this work.

Name:-Mani Saurabh Tiwari Name:-Atul Kumar Rajak

Roll No:-16101105015 Roll No:-16101105016
Registration No:-161010110134 Registration No:-161010110116
Date:-30th June 2020 Date:-30th June 2020

Name:-Ashish Tank Name:-Akash Kumar

Roll No:-16101105017 Roll No:-16101105046
Registration No:-161010110115 Registration
No:-161010110112
Date:-30th June 2020 Date:-30th June 2020

Name:-Abhishek Kumar Jha Name:-Shubham Singh

Roll No:-16101105059 Roll No:-16101105060
Registration No:-161010110110 Registration No:-161010110150
Date:-30th June 2020 Date:-30th June 2020

Thesis Title: - “Human Activity Recognition”

2
 CERTIFICATE OF RECOMMENDATION
This is to certify that the project report entitled “HUMAN ACTIVITY RECOGINITION”,
submitted by Mani Saurabh Tiwari (Reg. No.-161010110134 of 2016-17, Roll No.-
16101105015), Atul Kumar Rajak(Reg. No.-161010110116 of 2016-17, Roll No.-
16101105016), Ashish Tank (Reg. No.-161010110115 of 2016-17, Roll No.-16101105017),
Akash Kumar(Reg. No.-161010110112 of 2016-17, Roll No.-16101105046), Abhishek
Kumar Jha (Reg. No.-161010110110 of 2016-17, Roll No.-16101105059), Shubham Singh
(Reg. No.-161010110150 of 2016-17, Roll No.-16101105060) under my guidance and
supervision in partial fulfillment of the requirement for the degree of “Bachelor Of
Technology" in Electronics & Communication Engineering is a record bona fide work,
carried out by them. The contents of this thesis, in full or part, have not been submitted to any
other university or institution for the award of any degree or diploma.

Mr. Mirwaiz Rahaman

(Assistant Professor), Dept. of Electronics & Communication Engineering
Jalpaiguri Government Engineering College
Maulana Abul Kalam Azad University of Technology

Dr. Jishan Mehedi

(Associate Professor & H.O.D), Dept. of Electronics & Communication Engineering
Jalpaiguri Government Engineering College
Maulana Abul Kalam Azad University of Technology

3
 Acknowledgement

I as a diligent undergraduate student of Electronics and Communications department, I am

highly privileged to have had an opportunity of working under the tutelage of our supervisor
and mentor, Prof. Mirwaiz Rahaman (Assistant Professor, Electronics and Communications
Engineering, Jalpaiguri Government Engineering College). His guidance and key technical
understanding of our project was the key to finding the solution to our project. I am also deeply
thankful for the guidelines from our head of the department, Professor (Dr.) Jishan Mehedi
(Head of the department, Electronics and Communications Engineering, Jalpaiguri
Government Engineering College). His insight on our work and sincere advice helped us
complete our project without any hardships.

4
 Contents
Abstract……………………………………………………………………………………………………... 6

Abbreviations …………………………………………………………………………………………….. 7

List of Figures……………………………………………………………………………………………... 8

1. Introduction ………………………………………………………………………………………… (9 – 11)

1.1Human activity recognition and problem specification………………………………….9
1.2 Challenges in HAR technology………………………………………………………………………10
1.3 Motivation……………………………………………………………………………………………………10
1.4 Literature review………………………………………………………………………………………….11

2. Theoretical background on Human Activity Recognition……………………… (12-16)

2.1 Human Activity Recognition using smartphones……………………………………………12
2.2 Sensors……………………………………………………………………………………………………….13-14
2.3 Learning algorithms…………………………………………………………………………………….14
2.3 Machine learning………………………………………………………………………………………….14
2.5 Support Vector Machine…………………………………...………………………………………….14
2.6 Linear Discriminant Analysis………………………………………………………………………...15
2.7 Summary……………………………………………………………………………………………………...16

3. Methodology…………………………………………………………………………………………... (16-
21)
3.1 Dataset preparation ……………………………………………………………………………………16-19
3.2 Exploratory data analysis……………………………………………………………………………….20
3.3 Calculation of 3 axial liner acceleration……………………………………………………………20
3.4 Signal processing……………………………………………………………………………………………20
3.5 Feature mapping…………………………………………………………………………………………….21

4. Results and Discussions…………………………………………………………………………. (21-

26)

5. Summary and Conclusion………………………………………………………………………(26-27)

6. Bibliography……………………………………………………………………………………………(28-
29)

5
 Abstract

Human activity recognition is becoming very essential in health monitoring and emergency
cases. The activity detection from human motion patterns is challenging because of the
complexity of human life inside a home. Several technologies based on different advanced
sensors and recognition algorithms have been developed to recognize human activity by
detecting their motions. In this work, we have diligently applied our gained expertise in signal
processing techniques to extrapolate raw data from sensors and extract windows of meaningful
information which can be fed into heuristic machine learning models for precise human
activity prediction in a dynamic time series classification problem. Our team has developed a
machine learning algorithm integrating support vector machines (SVM) and linear
discriminant analysis (LDA) on datasets to obtain the high accuracy prediction we have today.
We have implemented a front-end architecture on the model to represent the data in a user
friendly and graphical way for large scale applications.

6
 Abbreviations

1. HAR - Human Activity Recognition

2. AR - Activity Recognition
3. IOT - Internet of things.
4. GSM - Global system of mobile communication.
5. ADL - Activity of Daily Living.
6. SVM - Support Vector Machine
7. LDA - Linear Discriminant Analysis.

7
List of figures Page

1. Inertial signals graph of user number 1 of acceleration (in 1g) vs time 18

in seconds.

2. Gyroscope signals graph for all activities performed by user number 1 18

in experience number 1 of angular velocity measured in radians
per second vs time in seconds.

3. Data splitting architecture diagram. 19

4. Changes in signal graph diagram before and after application of median 21

filter in signal graph of acceleration in 1g vs time in seconds

8
1. Introduction

1.1 Human Activity Recognition and problem specification

Human activity recognition or HAR is dynamic time series classification of human activity
based on sensor data collected from smartphones or other iot (Internet of Things) devices.
Human Activity Recognition (HAR) aims to identify the actions carried out by a person given
a set of observations of him/herself and the surrounding environment. As a result of this
research, many applications, including video surveillance systems, human-computer
interaction, and robotics for human behavior characterization, require a multiple activity
recognition system.

Recognition can be accomplished by exploiting the information retrieved from various

sources such as environmental or body-worn sensors. Smartphones are bringing up new
research opportunities for human-centered applications where the user is a rich source of
context information and the phone is the firsthand sensing tool. Latest devices come with
embedded built-in sensors such as microphones, dual cameras, accelerometers, gyroscopes,
etc. The use of smartphones with inertial sensors is an alternative solution for HAR. Publicly
available datasets such as https://archive.ics.uci.edu/ml/machine-learning-
databases/00240/ provide a freely available source of data across different disciplines and
researchers in the field. For this reason, we present a new dataset that has been created using
inertial data from smartphone accelerometers and gyroscopes, targeting the recognition of six
different human activities

1.2 Challenges in HAR technology

The challenges corresponding to present Human Activity Recognition technology are listed
as follows:
1. Human behavior: Performing multiple tasks at the same time makes the process more
difficult.
2. The definition of physical activities: develop a clear understanding of the definition of the
activities under investigation and their specific characteristics.
3. Intraclass variability: The same activity may be performed differently by different
individuals.
4. Intraclass similarity: Classes that are fundamentally different, but that show very similar
characteristics in the sensor data.
5. Selection of attributes and sensors: The selection of the attributes to be measured and the
sensors that measure it plays an important role in recognition performance.
6. Sensor inaccuracy: The sensor data play an important role in the overall recognition results.
7. Data availability: Collection of training data under realistic conditions.
8. Reusability: Retraining the system for a new set of users.

9
1.3 Motivation

The problems in the field of integration and reasoning are the reason for us developing this
project. The applications of human activity detection can be applied in futuristic smart homes
where various IoT devices might interact with each other and deduce the various activities
performed by the human beings and can be intelligently responsive like automatic temperature
setting and light dimming and brightening. We have partially implemented the HAR based on
progression of static time series classification. In the future we plan to implement dynamic
real time-based human activity recognition.

1.4 Literature Review

Most of the action recognition methods require to manually annotate the relevant portion of
the action of interest in the video. In recent years it has been studied that the relevant portion
of action of interest can be found out automatically and recognizes the action.

In the research of Poppe, Ronald Walter.[1], we found that the Marker less vision-based human
motion analysis has the potential to provide an inexpensive, non-obtrusive solution for the
estimation of body poses. The significant research effort in this domain has been motivated by
the fact that many application areas, including surveillance, Human-Computer Interaction and
automatic annotation, will benefit from a robust solution.

Subsequently in the research paper of Karantonis et al. [2], presents the real-time monitoring
of human movement can provide valuable information regarding an individual's degree of
functional ability and general level of activity. This paper presents the implementation of a
real-time classification system for the types of human movement associated with the data
acquired from a single, waist-mounted triaxial accelerometer unit.

Human activity recognition has been studied and implemented by various learned researchers
and scholars. Each researcher implemented and obtained a unique result and furthering the
progress threshold. There are mainly two types of implementations in the field of human
activity recognition. The two methods are machine learning and threshold learning. With the
development of machine learning and the ease of production of machine learning techniques
the accuracy obtained now-a-days of aforementioned research subject is relatively higher than
that obtained from threshold learning.

To recognize the activity various parameters were considered in the data obtained from the
sensors in smartphones. Multiple users with multiple experiences were asked for data
extraction. The quality of data obtained was needed to be of high priority and was indeed very
reliable as it impacted greatly on the obtained results.

Various real time human activity and activity recognition techniques are available and some
are real time-based activity detection algorithms used by Federal Bureau of Investigation (FBI)
. Our solution is a static time series activity detection algorithm and we hope to further the
improvement scale on this project.

10
 Test and Train splitting of dataset

Image description: -In the above image we demonstrate the splitting technique used in this
project. The above image demonstrates the data being divided in a 2:1 ratio in train and test
sub datasets.

1.5 Construction of the thesis

Chapter 1 of our thesis is the introduction of human activity recognition. In section 1.1 we
have presented human activity recognition as a problem statement which we intend to solve
using machine learning techniques. We have also discussed the challenges faced during
human activity recognition in section 1.2. The chapter includes the literature review in
section 1.4 containing different works implemented in the past on human activity recognition
by various renowned researchers and data scientists. We have also mentioned the motivation
of working towards solving the problem statement in section 1.3 and removing the glitches
found in the works of other scientists. Section 1.5 includes the construction of thesis which
presents the overall summary of how each and every section is divided into and the
information in them at a glance.

Chapter 2 of our thesis focusses on the theoretical background of human activity recognition.
This chapter essentially presents before the viewers a walkthrough of all the theoretical
concepts such as support vector machines and linear discriminant analysis implemented in
our project. In section 2.1 we have discussed the theoretical knowledge about support vector
machines. In section 2.2 we present before the viewers about the linear discriminant analysis.
In section 2.3 we tend to present the basic idea behind the usage of smartphones for human
activity recognition. In section 2.4 we have discussed about the sensors and their
functionality in our project. Section 2.5 demonstrates the learning algorithms we have at our
disposal for successful implementation of human activity recognition. Further in section 2.6
we have demonstrated the technique we have used for our version of human activity
recognition. In section 2.7 we have an overall summary of our theoretical concepts
implemented in this project.

Chapter 3 of our thesis presents the methodology or the fundamental architecture of our
project. Section 3.1 presents the dataset preparation of our project. Section 3.2 we present the
Exploratory data analysis of our thesis. Section 3.3 demonstrates the Calculation of 3-axial
linear acceleration from data extrapolated from our sensors. Section 3.4 demonstrates
11
The signal processing we have done in the project and the next section 3.5 demonstrates the
feature mapping of extrapolated features from the dataset.

Chapter 4 of our thesis is the results and discussion of the overall thesis. We have
demonstrated the accuracy metrics and web application functionalities in our project.

Chapter 5 of the thesis demonstrates the various sources and research papers along with
citations, which we have used to complete this project.

2. Theoretical background on Human Activity Recognition: -

Human Activity Recognition (HAR) is classifying activity of a person using responsive

sensors that are affected from human movement. These facts make HAR more important and
popular. This work focuses on recognition of human activity using smartphone sensors using
different machine learning classification approaches. Data retrieved from smart phones'
accelerometer and gyroscope sensors are classified in order to recognize human activity.
Results of the approaches used are compared in terms of efficiency and precision.

We have implemented a SVM along with LDA approach to get our high accuracy results.

2.1Support Vector Machine: -

SVM are supervised learning models with associated learning algorithms that analyze data
used for classification and regression analysis. The Support Vector Machine (SVM) algorithm
is a popular machine learning tool that offers solutions for both classification and regression
problems. Developed at AT&T Bell Laboratories by Vapnik with colleagues (Boser et al.,
1992, Guyon et al., 1993, Vapnik et al., 1997), it presents one of the most robust prediction
methods, based on the statistical learning framework or VC theory proposed by Vapnik and
Chervonekis (1974) and Vapnik (1982, 1995).

Given a set of training examples, each marked as belonging to one or the other of two
categories, an SVM training algorithm builds a model that assigns new examples to one
category or the other, making it a non-probabilistic binary linear classifier (although methods
such as Platt scaling exist to use SVM in a probabilistic classification setting).

An SVM model is a representation of the examples as points in space, mapped so that the
examples of the separate categories are divided by a clear gap that is as wide as possible. New
examples are then mapped into that same space and predicted to belong to a category based
on the side of the gap on which they fall. In addition to performing linear classification, SVMs
can efficiently perform a non-linear classification using what is called the kernel trick,
implicitly mapping their inputs into high-dimensional feature spaces. A support-vector
machine constructs a hyperplane or set of hyperplanes in a high- or infinite-dimensional space,
which can be used for classification, regression, or other tasks like outlier's detection.
SVMs can be used to solve various real-world problems:
1. Classification of satellite data like SAR data using supervised SVM.
2. Hand-written characters can be recognized using SVM.

12
 3. The SVM algorithm has been widely applied in the biological and other sciences. They
have been used to classify proteins with up to 90% of the compounds classified
correctly. Permutation tests based on SVM weights have been suggested as a
mechanism for interpretation of SVM models. Support-vector machine weights have
also been used to interpret SVM models in the past. Posthoc interpretation of support-
vector machine models in order to identify features used by the model to make
predictions is a relatively new area of research with special significance in the
biological sciences.

2.2 LINEAR DISCRIMINANT ANALYSIS: -

LDA/SVM classifier: The idea of incorporating a combination of SVM/LDA approach gives

rise to a new large margin classifier which helped us get very accurate and precise results for
our large datasets. The SVM/LDA classifier can be also seen as a generalization of linear
discriminant analysis (LDA) by incorporating the idea of (local) margin maximization into
standard LDA formulation.

Linear Discriminant Analysis can be used to perform supervised dimensionality reduction, by

projecting the input data to a linear subspace consisting of the directions which maximize the
separation between classes (in a precise sense discussed in the mathematics section below).
The dimension of the output is necessarily less than the number of classes, so this is in general
a rather strong dimensionality reduction, and only makes sense in a multiclass setting.

2.3Human Activity Recognition using smartphones

Smartphones have been commonly employed to develop HAR solutions because of the
ubiquitous capability which demonstrates the easy availability of sensors and diversity of
sensors embedded in such devices. Smartphones are included in the scope of wearable
computing, and these devices are considered part of mobile computing-based HAR systems.
The advantage of smartphones over other wearable devices is associated with their ability to
(a) capture and process data, (b) transmit and receive data, and (c) connect with other devices
or sensors available in the physical environment. Inertial sensors such as the accelerometer
and gyroscope are most commonly used to capture information related to acceleration and
direction of movement of the human body, respectively. These sensors allow the extraction of
diverse information about the user that can be used to recognize individual physical activities.

Since smartphone is portable device having computational power, communication capability

and variety of embedded sensors, it has been chosen for the work. One of the first historical
milestone was in 2006, when the first HAR solutions appeared that explicitly used
smartphones. At that time, the first studies were performed using data analysis extracted from
the GSM sensors and accelerometer for monitoring users’ mobility. During this period, all
data processing was performed on a computer (offline processing) because smartphones had
limited computational resources.

2.4 Sensors
13
Sensors Types for the Human Activity Recognition: -

Human Activity Recognition has gained much importance in the past decade due to its
numerous applications in human-centric applications as in the field of medical, security and
also military. In recent years, the use of a smartphone has increased, it is small and convenient
to carry around and its computational resource is powerful enough for our project purpose. By
using data from Smartphone sensors, we can classify and identify human activities such as
WALKING, WALKING-UPSTAIRS, WALKING-DOWNSTAIRS, SITTING, STANDING
AND LAYING. We investigate the role of smartphone sensors in activity recognition. We
evaluate their roles in four body positions while recognizing six physical activities.

• INERTIAL SENSORS: -
1. Accelerometer: - It is an electromechanical device which is used to measure static and
dynamic acceleration forces. Accelerometers in mobile phones are used to detect the
orientation or to sense the acceleration event of Smartphones. An accelerometer
measures the linear acceleration of movement. The reading includes three axes whose
directions are predefined. The acceleration is the raw data stream from the
accelerometer. A set of vectors represents the raw data as: Acci = < xi, yi, zi >, (I =
1,2, 3...). An accelerometer will measure the directional movement of a device but will
not be able to resolve its lateral orientation or tilt during that movement accurately.
The three axes readings are combined with a time stamp. To sample the frequency,
most of the existing accelerometers provide a user interface so that the user could
choose a better sampling rate through experiments. Single and multi-axis
accelerometers detect the combined magnitude and direction of linear, rotational and
gravitational acceleration. They can be used to provide limited motion-sensing
functionality. For example, a device with an accelerometer can detect rotation from
vertical to horizontal in a fixed state Location. As a result, accelerometers are primarily
used for simple motion-sensing applications in consumer devices such as changing the
screen of a mobile device from portrait to landscape orientation. Its popularity is
because it directly measures the subject’s physiology motion status. For example, if a
user changes his/her activity from walking to jogging, it will reflect on the signal shape
of the accelerated reading along the vertical axis there will be an abrupt change in the
amplitude. Moreover, the acceleration data could indicate the motion pattern within a
given period, which is helpful in complex activity recognition.

2. Gyroscope: - Gyroscopes measure the angular rate of rotational movement about one
or more axes. Gyroscopes can measure complex motion accurately in multiple
dimensions, tracking the position and rotation of a moving object, unlike
accelerometers, which can only detect the fact that an object has moved or is moving
in a particular direction. Further, unlike accelerometers and compasses, gyroscopes are
not affected by errors related to external environmental factors such as gravity and
magnetic fields. Hence, gyroscopes greatly enhance the motion-sensing capabilities of
14
 devices and are used for advanced motion-sensing applications in consumer devices
such as full gesture and movement detection and simulation in video gaming. A
gyroscope measures the phone’s rotation rate by detecting the roll, pitch, and yaw
motions of the Smartphones along the x, y, and z-axis, respectively. The raw data
stream from a gyroscope sensor is the rate of the rotation in rad/s (radian per second)
around each of the three physical axes: Rotation = (i=1,2,3...). The gyroscope is helpful
in the navigation applications as well as some smartphone games which use the rotation
data. Inactivity recognition research, the gyroscope is used to assist the mobile
orientation detection.

• Accelerometers in mobile phones are used to detect the orientation of the phone. The
gyroscope, or gyro for short, adds dimension to the information supplied by the
accelerometer by tracking rotation or twist.

Figure 2. The sensors used in a smartphone depicted in the figure.

In the figure above we have a pictorial representation of the sensors used in the smartphones.

2.5 Learning Algorithms

Human activity recognition is the problem of classifying sequences of accelerometer data
recorded by specialized harnesses or smart phones into known well-defined movements.
Classical approaches to the problem involve hand crafting features from the time series data
based on fixed-sized windows and training machine learning models, such as ensembles of
decision trees. The difficulty is that this feature engineering requires deep expertise in the
field. We have built our model based on the SVM and LDA classifier which helped us obtain
an accuracy of 98.44%. Recently, deep learning methods such as recurrent neural networks
and one-dimensional convolutional neural networks, or CNNs, have been shown to provide
state-of-the-art results on challenging activity recognition tasks with little or no data feature
engineering, instead using feature learning on raw data.

2.6 Machine learning

15
In past few decades, there has been drastic change in the way, data is stored, perceived and
processed. Tremendous amount of data is generated every second and if this data is used and
analysed efficiently, it can reveal very important insights. Lot of data mining techniques has
evolved in analysing the huge amount of data. One important part of the prediction is the
selection of suitable models. Our SVM+LDA model represents samples as points in space and
separates the points based on outcome categories with a clear dividing gap. The category of
new points is determined based on which side of the gap they fall on.

2.6 Linear Discriminant Analysis:

Linear Discriminant Analysis can be used to perform supervised dimensionality reduction, by

projecting the input data to a linear subspace consisting of the directions which maximize the
separation between classes (in a precise sense discussed in the mathematics section below).
The dimension of the output is necessarily less than the number of classes, so this is in general
a rather strong dimensionality reduction, and only makes sense in a multiclass setting.

LDA/SVM classifier: The idea of incorporating a combination of SVM/LDA approach gives

rise to a new large margin classifier which helped us get very accurate and precise results for
our large datasets. The SVM/LDA classifier can be also seen as a generalization of linear
discriminant analysis (LDA) by incorporating the idea of (local) margin maximization into
standard LDA formulation.

2.8 Summary:

The emerging popularity in the field of human activity recognition has ignited the spark in
millions of brilliant minds and has triggered a surge of modernisation in today’s era. With the
same motivation in our minds we have built a solution to the static time series classification
of human activity in ADL activities. This project reviews the challenges and learning
accolades faced by us and every finding and report is uniquely studied to obtain the results
both desirable and reproducible.

3. Methodology:

For obtaining the dataset a certain group of individuals were selected for this task, aged
between 19 years to 48 years. These individuals were requested to perform a protocol of
activities while wearing a waist-mounted Samsung Galaxy S II smartphone. The six selected
ADL were standing, sitting, laying down, walking, walking downstairs and upstairs. Each
subject performed the protocol twice: on the first trial the smartphone was fixed on the left
side of the belt and on the second it was placed by the user himself as preferred. There is also
a separation of 5 seconds between each task where individuals are told to rest, this facilitated
repeatability (every activity is at least tried twice) and ground trough generation through the
visual interface. we have been demonstrated the use of Smartphone sensors to detect human
activity. We use sensor data of a Smartphone (Samsung Galaxy S2) to record, model and
16
recognize human activity. For validation we use a publicly available activity recognition (AR)
database. This database includes labeled data collected from 30 subjects in age group of 19 –
48 years. Each person performed different activities wearing a smart phone around the waist,
and engaged in six different activities— walking on flat ground and up and down stairs, sitting,
standing, and lying down. A Samsung Galaxy S2 Smartphone was used for data collection,
which contains an accelerometer and a gyroscope for measuring 3-axial linear acceleration
and angular velocity respectively at a constant rate of 50Hz, which is sufficient for capturing
human body motion. The raw data was then pre-processed by applying noise filters and then
sampled with fixed-width sliding windows of 2.56 sec and 50% overlap. From each window,
a vector of 17 features is obtained by calculating variables from the accelerometer signals such
as: tBodyAcc-mean()-X, tBodyAcc-mean()-Y, tBodyAcc-mean()-Z, tBodyAcc-std()-X,
tBodyAcc-std()-Y,tBodyAcc-std()-Z, tBodyAcc-mad()-X, tBodyAcc-mad()-Y, tBodyAcc-
mad()-Z, tBodyAcc-max()-X, tBodyAcc-max()-Y, tBodyAcc-max()-Z, tBodyAcc-min()-X,
tBodyAcc-min()-Y, tBodyAcc-min()-Z, tBodyAcc-sma(),tBodyAcc-energy()-X, tBodyAcc-
energy()-Y, tBodyAcc-energy()-Z, tBodyAcc-iqr()-X in the time and frequency domain (e.g.
mean, standard deviation, signal magnitude area, entropy, signal-pair correlation, etc.). The
other dataset consists of vectors that each contain 561 features and represent 2.56 seconds of
time. Each vector encodes characteristics such as the tri axial average, maximum, and
minimum acceleration and angular velocity over the given interval, as well as more complex
properties such as the Fourier transform and autoregressive coefficients. We used this dataset
for experimental validation of our activity recognition approach. Put this portion in
methodology.
The tech-stack used by us for implementing this project is a minimal system
requirement tech stack, so that it can be deployed over a large scale. We have python as our
programming language for building our machine learning model. We have used a multiclass
classifier using SVM and LDA classifier algorithms in python sklearn. We have also deployed
the model as a back-end input feed to a web application front-end so that the output can be
viewed in a user friendly and graphically pleasing presentation.

The Techstack:1. Python, 2. Django, 3.CSS, 4. Html-5.

3.1 Dataset Preparation: -

The experiments have been carried out with a group of 30 volunteers within an age bracket
of 19-48 years. Each person performed six activities (WALKING, WALKING_UPSTAIRS,
WALKING_DOWNSTAIRS, SITTING, STANDING, LAYING) wearing a smartphone
(Samsung Galaxy S II) on the waist. Using its embedded accelerometer and gyroscope, we
captured 3-axial linear acceleration and 3-axial angular velocity at a constant rate of 50Hz.
The experiments have been video-recorded to label the data manually. The obtained dataset
has been randomly partitioned into two sets, where 70% of the volunteers were selected for
generating the training data and 30% the test data. The sensor signals (accelerometer and
gyroscope) were pre-processed by applying noise filters and then sampled in fixed-width
sliding windows of 2.56 sec and 50% overlap (128 readings/window). The sensor acceleration
signal, which has gravitational and body motion components, was separated using a
Butterworth low-pass filter into body acceleration and gravity. The gravitational force is
assumed to have only low frequency components, therefore a filter with 0.3 Hz cut-off
frequency was used. From each window, a vector of features was obtained by calculating
variables from the time and frequency domain.
17
Dataset Preparation diagrams: -
The following diagram represents the accelerometer and the gyrometer signal fluctuations in
three different axes. The red coloured fluctuations are of the signals in x axis, the green
coloured fluctuations are of the signals in y axis and the blue coloured fluctuations are of the
signals in z axis, when the movement is observed in a particular direction.

Fig 1: Inertial signals graph of user number 1 of acceleration (in 1g) vs time in seconds.
Fig 2: Gyroscope signals graph for all activities performed by user number 1 in experience
number 1 of angular velocity measured in radians per second vs time in seconds.

Data splitting architecture: The data splitting architecture explains the overall flowchart
of the data preparation for the feed input of the machine learning model used in this project.
The architecture starts with splitting of data followed by signal processing performed on the
split data. Windowing of selected activities and feature selection are performed. After
successful windowing implementation the HAR dataset is finally ready for moving forward
with.

18
 Fig 3: Data splitting architecture diagram

3.2 Exploratory data analysis:

The data obtained from publicly available sources such as

https://archive.ics.uci.edu/ml/machine-learning-databases/00240/ was already scaled, so
we did not have to perform any preprocessing. We checked for null values in the dataset and
the distribution of the activities in the dataset was carried out. Mainly there are 'acceleration'
and 'gyroscope' features. A few 'gravity' features are there as well. There are no missing values
in the dataset. Assuming the participants had to walk the same number of stairs upwards as
well as downwards and knowing the smartphones had a constant sampling rate, there should
be the same amount of datapoints for walking upstairs and downstairs. Disregarding the
possibility of flawed data, the participants seem to walk roughly 10% faster downwards.

3.3 Calculation of 3-axial linear acceleration:

The accelerometer in the mobile device provides the XYZ coordinate values, which is used to
measure the position and the acceleration of the device. The XYZ coordinate represents
direction and position of the device at which acceleration occurred. The rotation direction and
position are measured using gyroscope sensors that are found in the Android devices. The
mobile device rest in the Earth includes the acceleration due to gravity (g = 9.81m/s2) and the
acceleration value. The accelerometer values provided by the device normally includes the
gravity as well. Accelerometer along with the linear acceleration and gyroscope will provide
results close to accuracy. Linear acceleration does not include the gravity. The accelerometer
value is passed into the low/high pass filters for refining the result, based on the application
that has been used. The gyroscope helps us get the angular velocity in 3-axial angular velocity
retrieval.

19
3.4 Signal Processing

At a sampling rate of 50 hz the triaxial linear acceleration and angular velocity signals using
the phone accelerometer and gyroscope were collected. These signals were preprocessed for
noise reduction with a median filter and a 3rd order low-pass Butter worth filter with a 20 Hz
cutoff frequency. This rate is sufficient for capturing human body motion since 99% of its
energy is contained below 15Hz.
The acceleration signal, which has gravitational and body motion components, was separated
using another Butterworth low-pass filter into body acceleration and gravity. The gravitational
force is assumed to have only low frequency components, therefore we found from the
experiments that 0.3 Hz was an optimal corner frequency for a constant gravity signal.
Additional time signals were obtained by calculating from the triaxial signals the euclidean
magnitude and time derivatives (jerk da/dt and angular acceleration dw/dt). The time signals
were then sampled in fixed-width sliding windows of 2.56 sec and 50% overlap between them,
since:
• The cadence of an average person walking is within steps/min, i.e. a minimum of 1.5
steps/sec.
• At least a full walking cycle (two steps) is preferred on each window sample.
• People with slower cadence such as elderly and disabled should also benefit from this
method. We supposed a minimum speed equal to 50% of average human cadence.
• Signals are also mapped in the frequency domain through a Fast Fourier Transform (FFT),
optimized for power of two vectors (2.56sec × 50Hz = 128cycles).
Signal Processing methods and observations: -

I. Noise Filtering: The features selected for this database come from the accelerometer
and gyroscope 3-axial raw signals t_Acc-XYZ and t_Gyro-XYZ. These time domain
signals (prefix 't' to denote time) were captured at a constant rate of 50 Hz. Then they
were filtered using a median filter and a 3rd order low pass Butterworth filter with a
corner frequency of 20 Hz to remove noise.

1- Median Filter: was applied to reduce background noise. We can observe that the signal
feed which was having a lot of background noise can be made almost noiseless and only
meaningful signals can be extracted by applying a simple median filter on the
accelerometer signals. The changes in the signals can be observed in the following figure.
In first figure the accelerometer in 1g vs time in seconds signals with noise can be seen,
whereas in the next figure the noise is being reduced by the median filter applied.

20
 Fig 4: Observing changes after applying median filter on features.

2- 3rd order Low pass Butterworth filter with a cut-off, frequency = 20hz was applied to
remove high frequency noise. Resulted Signals are: total_acc_XYZ and Gyro_XYZ

Gravity filtering:
Similarly, the acceleration signal total-acc-XYZ was then separated into body and gravity
acceleration signals (tBodyAcc-XYZ and tGravityAcc-XYZ) using another low pass
Butterworth filter with a corner frequency of 0.3 Hz. Since the gravitational force is assumed
to have only low frequency components.Resulted components are: total_acc-XYZ ==>
tBody_acc-XYZ + tGravity_acc-XYZ

3.5 Feature Mapping:

A total of 561 features were extracted to describe each activity window. The features dataset
was partitioned into training set and testing dataset with 70% of the data used for training and
remaining 30% used for testing. It has been submitted as the Human Activity Recognition
using Smartphones dataset in the UCI Machine Learning Repository.
Standard measures previously used in Human Activity Recognition literature such as the
mean, correlation, signal magnitude area (SMA) and autoregression coefficients were
employed for the feature mapping. A new set of features was also employed in order to
improve the learning performance, including energy of different frequency bands, frequency
skewness, and angle between vectors (e.g. mean body acceleration and y vector).

4.Results and discussion:

Confusion Matrix of the classification results on the test data using the SVM+LDA
classifier.Confusion matrix also termed as contingency table, provides a comprehensive
21
overview by summarizing the classification results. It showcases the individual results for each
of the classes by tabulating the predicted and actual classes.

The confusion matrix of our findings is given as:

WALKING WALKING_ WALKING_ SITTING STANDING LAYING
UPSTAIRS DOWNSTAI
RS

WALKING 496 0 0 0 0 0
WALKING_ 0 471 0 0 0
UPSTAIRS
WALKING_ 0 1 419 0 0 0
DOWNSTAI
RS
SITTING 0 0 0 461 30 0
STANDING 0 0 0 14 518 0
LAYING 0 0 0 0 0 537

The columns are represented in the following order in the confusion matrix:
"WALKING","WALKING_UPSTAIRS","WALKING_DOWNSTAIRS","SITTING"
,"STANDING","LAYING"
The rows are represented in the following order in the confusion matrix:
"WALKING","WALKING_UPSTAIRS","WALKING_DOWNSTAIRS","SITTING"
,"STANDING","LAYING"

The image 2.3 shows a confusion matrix and its components.

Figure 2.3: Confusion Matrix

TP – Indicates the true positives which is the number of positive records correctly
predicted as positive by the model

TN – Indicates the true negatives which is the number of negative records correctly
predicted as negative by the model

FP – Indicates the false positives which is the number of negative records incorrectly
predicted as positive by the model

FN – Indicates the false negatives which is the number of positive records incorrectly
predicted as negative by the model

Each of these measures have a significance of their own depending on the research
question. Multiple standardized metrics can be defined from the confusion matrix –

22
Accuracy – It is the total number of correct predictions proposed by the model
which includes the positive and negative predictions.

Accuracy = number of correct predictions/ total number of predictions

𝑇𝑃+𝑇𝑁
= 𝑇𝑃+𝑇𝑁+𝐹𝑃+𝐹𝑁

The accuracy obtained after implementing the SVM+LDA classifier is found to

be 0.98449401523395 or 98.44% accuracy.

Web application

In addition to the highly accurate machine learning model we have also developed a user
interactive web application for robust and graphical representation of our experiment. The
web application takes in the user number as input to process the learning of the features from
the dataset and accurately predict the activity of the user in a dynamic time series
classification. The output is deployed on a separate webpage for user interactivity ease.

The web application is a django based web application which presents the user with a text-
box in which the user can input his/her assigned number respectively.
The model runs its course to find the person’s action during progressing time series.
The application runs in the following way:

The above figure is the web application asking the user for user number and name.

23
The above figure is the successful prediction of the activity being performed by the user.

Applications of human activity recognition:

Human activity recognition has gained much importance in the past decade due to its
numerous applications in human centric applications as in the field of medical, security and
also military. In recent times, due to the increase of wearable tech devices, the task of human
activity recognition has gained much more gravitas. An important goal of the HAR in the
current scenario is to identify the actions of the user in order to assist them with their tasks
with the help of computing systems. Computer vision research has been contributing a lot in
this aspect of the study. Human activity recognition here mainly refers to physical human
activity and load, as opposite to cognitive, mental activities and workload which are part of a
different, wider research field.

There are a huge number of applications of machine learning and its usages are ever increasing.
Some of the most popular applications are –

• Creation of a good search engine requires the engine to process the search query, identify
pages having the information and sort them according to prescribed algorithms. Machine
learning has increasingly been utilized to automate the process of web page ranking and
determining the best results for a given query.

• Recommendation engines are another area which has seen rapid development as they help
entice the users with newer products or services. Ecommerce companies as Amazon and eBay
utilize this system to analyze past purchases and viewing options to predict and enable future
viewing and purchases. Netflix, which is a video rental store, also utilizes this mechanism.
This application can also be termed as Collaborative filtering.

• Another application that is not quite defined is the text translation problem. This problem is
quite tricky as the machine needs to model the grammar and the language of the document.
There is also huge research in this area of application.

24
• Face recognition is an example of machine learning revolutionizing the current existing
applications. The machine breaks down the face of the user into smaller parts as pixels to
detect and classify the faces. This system is currently being utilized in various security
applications and also social networking websites.

• Other applications that are utilizing machine learning are, speech recognition, handwriting
recognition, named entity recognition, failure and fault detection in several industrial
equipment, gaming consoles etc.

Mobile phones or smart phones are rapidly becoming the central computer and communication
device in people’s lives. Smart phones, equipped with a rich set of sensors, are explored as an
alternative platform for human activity recognition in the ubiquitous computing domain.
Today’s Smartphone not only serves as the key computing and communication mobile device
of choice, but it also comes with a rich set of embedded sensors, such as an accelerometer,
digital compass, gyroscope, GPS, microphone, and camera. Collectively, these sensors are
enabling new applications across a wide variety of domains, such as healthcare, social
networks, safety, environmental monitoring, and transportation, and give rise to a new area of
research called mobile phone sensing. Human activity recognition systems using different
sensing modalities, such as cameras or wearable inertial sensors, have been an active field of
research. Besides the inclusion of sensors, such as accelerometer, compass, gyroscope,
proximity, light, GPS, microphone, camera, the ubiquity, and unobtrusiveness of the phones
and the availability of different wireless interfaces, such as WI-Fi, 3G and Bluetooth, make
them an attractive platform for human activity recognition. The current research in activity
monitoring and reasoning has mainly targeted elderly people, or sportsmen and patients with
chronic conditions. The percentage of elderly people in today’s societies keep on growing. As
a consequence, the problem of supporting older adults in loss of cognitive autonomy who wish
to continue living independently in their home as opposed to being forced to live in a hospital.
Smart environments have been developed in order to provide support to the elderly people or
people with risk factors who wish to continue living independently in their homes, as opposed
to live in an institutional care. In order to be a smart environment, the house should be able to
detect what the occupant is doing in terms of one’s daily activities. It should also be able to
detect possible emergency situations. Furthermore, once such a system is completed and fully
operational, it should be able to detect anomalies or deviations in the occupant’s routine, which
could indicate a decline in his abilities. In order to obtain accurate results, as much information
as possible must be retrieved from the environment, enabling the system to locate and track
the supervised person in each moment, to detect the position of the limbs and the objects the
person interacts or has the intention to interact with. Sometimes, details like gaze direction or
hand gestures can provide important information in the process of analyzing the human
activity. Thus, the supervised person must be located in a smart environment, equipped with
devices such as sensors, multiple view cameras or speakers.

Human Health care Monitoring applications

The healthcare benefits associated with regular physical activity monitoring and recognition
has been considered in several research studies. Solid evidence shows that regular monitoring
and recognition of physical activity can potentially assist to manage and reduce the risk of
many diseases such as obesity, cardiovascular and diabetes. A few studies have been carried
25
out in order to develop effective human activity recognition system using smartphone.
However, understanding the role of each sensor embedded in the smartphone for activity
recognition is essential and need to be investigated. Due to the recent outstanding performance
of artificial neural networks in human activity recognition, this work aims to investigate the
role of gyroscope and accelerometer sensors and its combination for automatic human activity
detection, analysis and recognition using artificial neural networks. The experimental result
on the publicly available dataset indicates that each of the sensors can be used for human
activity recognition separately. However, accelerometer sensor data performed better than
gyroscope sensor data with classification accuracy of 92%. Combining accelerometer and
gyroscope performed better than when used individually with an accuracy of 95%.

Security and Surveillance applications

The analysis of human activities is one of the most interesting and important open issues for
the automated video surveillance community. In order to understand the behaviors of humans,
a higher level of understanding is required, which is generally referred to as activity
recognition. While traditional approaches rely on 2D data like images or videos, the
development of low-cost depth sensors created new opportunities to advance the field. In this
paper, a system to recognize human activities using 3D skeleton joints recovered from 3D
depth data of RGB-D cameras is proposed. A low dimensional descriptor is constructed for
activity recognition based on skeleton joints. The proposed system focuses on recognizing
human activities not human actions. Human activities take place over different time scales and
consist of a sequence of sub-activities (referred to as actions). The proposed system recognizes
learned activities via trained Hidden Markov Models (HMMs). Experimental results on two
human activity recognition benchmarks show that the proposed recognition system
outperforms various state-of-the-art skeleton-based human activity recognition techniques.

Support for aiding visually impared

Obstacle detection and warning can improve the mobility as well as the safety of visually
impaired people specially in unfamiliar environments. Obstacles are detected and localized
and then the information of the obstacles will be sent to the visually impaired people by using
different modalities such as voice, tactile, vibration.This can be done by the analysis of Human
activity recognition.

Sport-Related Human Activity Detection and Recognition

As an active research field, sport-related activity monitoring plays an important role in
people’s lives and health. This is often viewed as a human activity recognition task in which
a fixed-length sliding window is used to segment long-term activity signals. However,
activities with complex motion states and non-periodicity can be better monitored if the
monitoring algorithm is able to accurately detect the duration of meaningful motion states.
However, this ability is lacking in the sliding window approach. In this study, we focused on
two types of activities for sport-related activity monitoring, which we regard as a human
activity detection and recognition task. For non-periodic activities, we propose an interval-
based detection and recognition method. The proposed approach can accurately determine the
duration of each target motion state by generating candidate intervals. For weak periodic
activities, we propose a classification-based periodic matching method that uses periodic
matching to segment the motion sate. Experimental results show that the proposed methods
performed better than the sliding window method.

26
5.Summary and Conclusion:

We have successfully implemented a new approach for Human Activity Recognition using
smartphones and have acknowledged some results using a multiclass SVM+LDA classifier
approach. The multiclass classification model using SVM+LDA showed significantly accurate
results.
However, room for improvement exists as we still have a long way of solving the
independent dynamic time series classification of human activity recognition using
smartphone sensors. We also have room for improvement on filtering the dataset and reducing
the key feature selection in the released dataset.
With our approach we have achieved a highly accurate classification model and the results
show an accuracy of 98.44% on the available dataset.

6. Bibliography:

[1] R. Poppe. Vision-based human motion analysis: An overview. Computer Vision and Image
27
Understanding, 108(1-2):4–18, 2007.

[2] D.M. Karantonis, M.R. Narayanan, M. Mathie, N.H. Lovell, and B.G. Celler.
Implementation of a real-time human movement classifier using a triaxial accelerometer for
ambulatory monitoring. IEEE Transactions on Information Technology in Biomedicine,
10(1):156–167, 2006.

[4] R. Nishkam, D. Nikhil, M. Preetham, and M.L. Littman. Activity recognition from
accelerometer data. In Proceedings of the Seventeenth Conference on Innovative Applications
of Artificial Intelligence, pages 1541–1546, 2005.

[5] L. Bao and S.S. Intille. Activity recognition from user-annotated acceleration data. In T.
Kanade, J. Kittler, J.M. Kleinberg, F. Mattern, J.C. Mitchell, O. Nierstrasz, C. Pandu Rangan,
B. Steffen, D. Terzopoulos, D. Tygar, M.Y. Vardi, and A. Ferscha, editors, Pervasive
Computing, pages 1–17. 2004.

[6] J.R. Kwapisz, G.M. Weiss, and S.A. Moore. Activity recognition using cell phone
accelerometers. SIGKDD Explorations Newsletter, 12(2):74–82, 2011.

[7] T. Brezmes, J.L. Gorricho, and J. Cotrina. Activity recognition from accelerometer data on
a mobile phone. Distributed Computing, Artificial Intelligence, Bioinformatics, Soft
Computing, and Ambient Assisted Living, pages 796–799, 2009.

[8] W. Wu, S. Dasgupta, E.E. Ramirez, C. Peterson, and G.J. Norman. Classification
accuracies of physical activities using smartphone motion sensors. Journal of Medical Internet
Research, 14(5), 2012.

[9] M.F.A. bin Abdullah, A.F.P. Negara, M.S. Sayeed, D.J. Choi, and K.S. Muthu.
Classification algorithms in human activity recognition using smartphones. International
Journal of Computer and Information Engineering, 6:77–84, 2012.

[10] D. Roggen, A. Calatroni, M. Rossi, T. Holleczek, K. Forster, G. Tr ¨ oster, P. Lukowicz,

D. Bannach, ¨ G. Pirkl, and A. Ferscha. Collecting complex activity data sets in highly rich
networked sensor environments. In Proceedings of the 7th International Conference on
Networked Sensing Systems 2010, 2010.

[11] E.M. Tapia, S.S. Intille, L. Lopez, and K. Larson. The design of a portable kit of wireless
sensors for naturalistic data collection. In Proceedings of PERVASIVE 2006, pages 117–134,
2006.
[12] S. Dernbach, B. Das, N.C. Krishnan, B.L. Thomas, and D.J. Cook. Simple and complex
activity recognition through smart phones. In 2012 8th International Conference on Intelligent
Environments, pages 214–221, 2012.

[13] C. Cortes and V. Vapnik. Support-vector networks. Machine learning, 20(3):273–297,

1995.

[14] C. BenAbdelkader, R. Cutler, and L. Davis. Stride and cadence as a biometric in

automatic person identification and verification. In Proceedings of the Fifth IEEE International
Conference on Automatic Face and Gesture Recognition, pages 372–377, 2002.
28
[15] J.Y. Yang, J.S. Wang, and Y.P. Chen. Using acceleration measurements for activity
recognition: Aneffective learning algorithm for constructing neural classifiers. Pattern
recognition letters, 29(16):2213–2220, 2008.

[16] A.M. Khan, Y.-K. Lee, S.Y. Lee, and T.-S. Kim. Human activity recognition via an
accelerometerenabled-smartphone using kernel discriminant analysis. In Proceedings of the
5th International Conference on Future Information Technology, pages 1–6, 2010.

[17] A. Frank and A. Asuncion. UCI machine learning repository, 2010.

[18] Y. Hanai, J. Nishimura, and T. Kuroda. Haar-like filtering for human activity recognition
using 3d accelerometer. In Digital Signal Processing Workshop and 5th IEEE Signal
Processing Education Workshop, 2009. DSP/SPE 2009. IEEE 13th, pages 675 –678, jan. 2009.

[19] D. Anguita, A. Ghio, L. Oneto, X. Parra, and J.L. Reyes-Ortiz. Human activity
recognition on smartphones using a multiclass hardware-friendly support vector machine. In
Proceedings of the International Workshop of Ambient Assited Living, 2012.

[20] Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra and Jorge L. Reyes-Ortiz.
Human Activity Recognition on Smartphones using a Multiclass Hardware-Friendly Support
Vector Machine. International Workshop of Ambient Assisted Living (IWAAL 2012).
Vitoria-Gasteiz, Spain. Dec 2012

[21] Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, Jorge L. Reyes-Ortiz.
Energy Efficient Smartphone-Based Activity Recognition using Fixed-Point Arithmetic.
Journal of Universal Computer Science. Special Issue in Ambient Assisted Living: Home
Care. Volume 19, Issue 9. May 2013

[22] Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra and Jorge L. Reyes-Ortiz.
Human Activity Recognition on Smartphones using a Multiclass Hardware-Friendly Support
Vector Machine. 4th International Workshop of Ambient Assited Living, IWAAL 2012,
Vitoria-Gasteiz, Spain, December 3-5, 2012. Proceedings. Lecture Notes in Computer Science
2012, pp 216-223.

[23] Jorge Luis Reyes-Ortiz, Alessandro Ghio, Xavier Parra-Llanas, Davide Anguita, Joan
Cabestany, Andreu Català. Human Activity and Motion Disorder Recognition: Towards
Smarter Interactive Cognitive Environments. 21st European Symposium on Artificial Neural
Networks, Computational Intelligence and Machine Learning, ESANN 2013. Bruges,
Belgium 24-26 April 2013.

29