Human Activity Recognition

1
Karan Rajgor 2Harsh Kumar Singh UG Scholar,
Birla Vishvakarma
Mahavidyalaya, Vallabh
Vidyanagar, Anand.
1
karanrajgor@gmail.com 2harshkumars818@gmail.com

Abstract— The scenario of The main purpose behind this
project is “Motion Detection and Gesture Automation" and
under definition “HUMAN ACTIVITY RECOGNITION”.
Now a days, we had seen the importance of hygiene and
increased use of contactless commodities due to COVID .
We have to control, switch and modify several things that
currently we does with remote and at workplace it is
vulnerable to several contacts. So it is need of hour to
replace all those devices with automation and control them
using Hand gesture. Even for mechanical motions and
control we can use servo motor for level accordingly.
combination of Arduino and Python.
Instead of using a keyboard, mouse or remote, which often
are thing of common utility, we can use our hand gestures
and figures movement and count to control certain
functions of a computer like play/pause a video, move or
change command, scroll up/down in a web page and many
more, hence we made this project to control several
devises.

Keywords—Machine Learning, Arduino UNO, Python, OpenCV, I. THEORY

Gesture and Activity Recogination, Servo Motor.
A. Circuit Diagram

INTRODUCTION

Now a days , especially in COVID lockdown we are

always surrounded by gadgets and electronic devices. Be
it mobile phones, computers, tablets, TV all day long and
the most important part of using these devices is HMI.

Human Machine Interface i.e HMI is a integration which

consists of hardware and software that helps in
communication and exchange of data and control
between the user (person operating ) and that device. We
normally use LED Indicators, Switches, Remotes, Touch
Screens and LCD Displays as a part of HMI devices.

Simultaneous way to communicate with machines like

Robots or Computers and devices which is controlling is
with the help of Hand Gestures. Instead of using those
taking input from us manually a keyboard, mouse or
remove or control panel, we can use our hand gestures to
control certain functions of a computer like play/pause a
video, move channels , increase or decrease volume in ,
and scrolling in a web page and many more.

Controlling Computer using gestures is generally done by

using a technology known as Leap Motion. The first
laptop to use Leap Motion was Envy 17. Leap Motion SE
erected by HP which used Leap Motion Regulator.

Leap Motion is a veritably precious technology so in this

design we tried to apply  the functionality of controlling a
computer with Hand Gestures using the important
B. Schematic Diagram  DC Current for 3.3V Pin: 50 mA
 Flash Memory: 32 KB of which 0.5 KB
used by bootloader
 SRAM: 2 KB
 EEPROM: 1 KB
 Clock Speed: 16 MHz
 Length: 68.6 mm
 Width: 53.4 mm
 Weight: 25 g

D. Servo Motor
A servomotor is a rotary actuator or linear actuator that
allows for precise control of angular or linear position,
Fig.1 Block Diagram of IoT System velocity and acceleration. It consists of a suitable
Arduino with use of Python and Open CV is designed and motor coupled to a sensor for position feedback. It also
programmed to function with LED and Servo Motor based on requires a relatively sophisticated controller, often a
input receives as gesture from camera i.e, of figures in hand dedicated module designed specifically for use with
displayed..
servomotors.

C. Arduino UNO Servomotors are not a specific class of motor, although

the term servomotor is often used to refer to a motor
suitable for use in a closed-loop control system.
Arduino is an open source computer hardware and
software company, project, and user community that
designs and manufactures single-board microcontrollers
and microcontroller kits for building digital devices and
interactive objects that can sense and control objects in the
physical and digital world.

The Arduino board used for this project is the Arduino Uno.

Fig.4 Servo Motor

Servomotors are used in applications such as robotics,

CNC machinery or automated manufacturing

Fig.2 Arduino in Diagram II. IMPLEMENTATION ON THE SOFTWARE SIDE

Technical specifications A. Python

It is regarded as the data's repository, and it can be accessed by
 Microcontroller: ATmega328P computers with special privileges that specify the data's access
 Operating Voltage: 5v rights. In most cases, the user receives an API key, a
password, and information about the computer for which it is
 Input Voltage: 7-20v registered. The data obtained from the devices could be
 Digital I/O Pins: 14 (of which 6 provide accessed using these API keys and the password that the user
PWM output) has set up by using the user's Wi-Fi hotspot..
 Analog Input Pins: 6
 DC Current per I/O Pin: 20 mA

B. PyAutoGUI
The purpose of PyAutoGUI is to provide a cross-
platform Python module for GUI automation for human
beings
PyAutoGUI can simulate moving the mouse, clicking the
mouse, dragging with the mouse, pressing keys, pressing
and holding keys, and pressing keyboard hotkey
combinations
C. Arduino IDE:
The Arduino IDE was used to programmed the data collected
from the sensors, which was then sent to the server. Since
independently checking the hardware modules, they were
combined aboard. To begin, the Wi-Fi module was configured
by sending commands. We wired the ESP8266 Wi-Fi module
after initialization. Finally, we developed software that reads
sensor data from the DHT11 and BMP180 and displays
temperature, humidity, and pressure readings in real time. We
use ThingSpeak's IoT analytics service to aggregate, visualize,
and interpret live data after sensor data is submitted to the cloud.
The Wi-Fi module uses its allocated IP to transfer data to the
cloud.
III EXECUTION SIDE
Arduino is connected to 5 LEDs and a Servo Motor for its
output source of action. It takes input of gestures of hands and
counts figures from the Web Cam of PC , the code for which
is programmed using Python and Ardunio IDE. in this circuit
led is connected to digital pins and common ground is
provided. And for servo motor the voltage and ground is
provided.
A. ArduinoIDE:
The Arduino IDE is incredibly minimalistic, yet it provides
a near-complete environment for most Arduino-based
projects. The top menu bar has the standard options,
including “File” (new, load save, etc.), “Edit” (font, copy,
paste, etc.), “Sketch” (for compiling and programming),
“Tools” (useful options for testing projects), and “Help”.
The middle section of the IDE is a simple text editor that
where you can enter the program code. The bottom section of
the IDE is dedicated to an output window that is used to see
the status of the compilation, how much memory has been
used, any errors that were found in the program, and various
other useful messages.
B. OpenCV
OpenCV (Open Source Computer Vision Library) is a
library of programming functions mainly aimed at real-
time computer vision. Originally developed by Intel, it
was later supported by Willow Garage then Itseez
(which was later acquired by Intel). The library is cross-
platform and free for use under the open-source Apache
2 License. Starting with 2011, OpenCV features GPU
acceleration for real-time operations.
C. Python IDLE
IDLE is Python’s Integrated Development
and Learning Environment. IDLE has the
following features:
 coded in 100% pure Python, using the tkinter GUI
toolkit.
 cross-platform: works mostly the same on Windows,
Unix, and Mac OSX.
 Python shell window (interactive
interpreter) with colorizing of code input,
output, and error messages.
 multi-window text editor with
multiple undo, Python colorizing, smart
indent, call tips, auto completion, and other
features.
 search within any window, replace
within editor windows, and search through
multiple files(grep).
 debugger with persistent breakpoints,
stepping, and viewing of global and local
namespaces.
 configuration, browsers, and other dialogs. in this we
used the platform Pycharm edition which is basically a
interface the python more easefully.
 PyCharm is an integrated development environment
(IDE) used in computer programming, specifically for
the Python language. It is developed by the Czech
company JetBrains (formerly known as IntelliJ)
D. Working Mechanism

In the circuit there is several connection is made and

at software side more of a work is done so as we can
see the digital pins of arduino is connected to different
led and for grounding purpose we have provided
common ground to circuit of led as we can implement
it on breadboard now as for connecting of servo motor
we have connected digital pin of arduino to its supply
line and afterword there is a voltage is provided by
arduinos on board and for ground line it is connected
to ground pin of arduino. All connection of circuit is
made by jumper wire and implemented it on
breadboard.
This projects circuit situation is we illustrated above
for now working purpose we have get help of software
side. as software side we can see there is two type of
code python code and arduino code now for
controlling led and motor we have to write arduino
code and for detecting hand gesture and recognition of
activity we have to write the python code .

Now as we can see hand detection is done by open cv as we

detect hand after that we can recognition of hand we can draw
a hand using mediapipe library and as we using there
mediapipe it have different points already exit on our hand so
after understanding our hand point we can code for different
situation for different number of fingers to recognize as we
code we display number of finger. after that we have used this
module for our controlling purpose of the project. in main
module there is controlling arduino using the different
situation which is done and as we have control of the servo
and led using arduino so that’s how we done the project
working of human activity recognistation and control
components which are connected with computer.
SCOPE AND APPLICATION
We can implement a this type of prototype to create
different contact less application.
• We can use our kit to control different things like
switch at home , controlling Fan speed .
• We can increase the application to connect
different component to it and control door of home
with the hands..
We can implement a Human Activity
Recognisation for controlling and managing things
with gestures.
• We can use it in remote control of devices.
• We can increase the accuracy up to 3m by
increasing the cost of the GPS receivers or by WiFi
module inbuilt in Arduino.
• We can use that for operating manual (mechanical)
functions using and integrating it with servo
Motor.
• We can implement this technique further by
making use of Internet of Things (IOT).
 CONCLUSION

References
Using Arduino, and Python we were able to control i. https://ww1.microchip.com/downloads/en/
computer and other devices with hand gestures. The DeviceDoc/doc1919.pdf
Camera is used to detect the distance of the hands from the
sensors or in effect computer screen since the sensors are a. •Muhammad Ali Mazidi, Janice Gillispie
mounted on top of the monitor. The transmitter transmits Mazidi “The 8051 Microcontroller and
b. Embedded Systems Using Assembly and C-
ultrasonic waves and the Receiver receives the waves after 2nd-ed”
they are reflected back from an encountered obstacle in
the path of the wave. The obstacles for our purpose are
hands. The Arduino Uno board runs the program used to ii. Vehicle Tracking and Locking System Based on GSM
fetch the input from Camera and based on that calculate and GPS, I.J. Intelligent Systems and Applications,
the number of entity(figures) shown and that is fed to
2013, 09, 86-93 Published Online August 2013 in
Servo Motor as level and LEDs to blink. The Python
program uses these distances to perform the defined MECS
operations on the these are used to establish serial a. (http://www.mecs-press.org/) DOI:
communication with Arduino Uno board. By executing 10.5815/ijisa.2013.09.10
the Python program and playing LEDs and level Servo
Motor, we were able to control the playback using hand iii. Ranjani M, Ramya S, Rajashree BK, Bhagyashree G,
gestures which was the aim of our project. Sowmya CH, Hand Gesture Automation,
international journal advance research in science
and engineering, volume no. 07, April-2018
iv. www.instructables.com
v. www.circuitdigest.com
vi. www.electronicshub.org
vii. www.technicalustad.com
viii. https://pythonhosted.org/pyserial/
ix. www.cs.uky.edu
x. https://en.wikipedia.org
xi. www.python.org
xii. www.cvzone.com