Professional Documents
Culture Documents
REPORT WRITING
Human Gesture
Based Shadow Robot
Semester III
Submitted By Mohit Manglani (Enrolment No.: 230298)
2023-24
CERTIFICATE
This is to certify that the dissertation entitled Human Gesture Based Shadow Robot has
been carried out by Mohit Manglani my guidance in fulfillment of the degree of Bachelor of
Engineering in Computer Science (AI – ML) (3rd Semester), Adani University, Ahmedabad
during the academic year 2023-24.
2
ACKNOWLEDGMENT
Before we get started, I would like to express my gratitude to everyone who has helped with
this project. And I want to express my sincere gratitude to everyone who has helped us with
this initiative, whether directly or indirectly.
First of all, I want to thank Assist for everything. We are grateful to our project guide, Prof.
Darpan Patel, for encouraging and pushing us to work on this project. I would like to express
my gratitude to him for all of his efforts and help during this process. I am grateful for his
continued interest in my work as well as his excellent counsel.
I would like to thank our Head of Department, Professor Alok Kumar Singh, for his constant
attention and wise counsel during the project's development. His depth of knowledge has
profoundly impressed us.
Sincere Thanks.
3
ABSTRACT
Human-robot interaction has advanced significantly with the smooth incorporation of gesture
control technology into robotic systems. Using a small transmitting device worn on the user's
hand, a gesture-controlled robot deviates from traditional button-based controls by responding
to hand gestures in an intuitive manner. An accelerometer is incorporated into this
revolutionary technology, serving as the central component for capturing the subtleties of
human motions.
The transmitting device's main component, the accelerometer, acts as the frontline sensor,
closely monitoring and deciphering the user's hand movements. After that, a complex process
is applied to this real-time data, beginning with the analog-to-digital conversion made possible
by a comparator integrated circuit. This crucial stage makes sure that physical movements are
translated into a digital language, which sets the stage for later smooth communication with
the robotic system.
An encoder integrated circuit (IC) is used to further enhance the encoded digital data, making
it more secure and efficient to transmit. This wireless functionality increases the range of
applications for gesture-controlled robots in addition to providing users with unmatched
mobility.
4
TABLE OF CONTENT
Certificate
Acknowledgement
Abstract
Chapter 1: Introduction
Chapter 2: Problem Statement
Chapter 3: Objective
Chapter 4: Methodology
Chapter 5: Basic AEIOU Canvas
5.1 Activities Observed
5.2 Environment of Domain
5.3 Interaction
5.4 Domain Objects
5.5 Users
5.6 Snapshot of AEIOU Canvas
Chapter 6: Empathy Mapping
6.1 Users and Stakeholders
6.2 Happy Stories
6.3 Sad Stories
6.4 Snapshot of Empathy Mapping Canvas
Chapter 7: Ideation Canvas
7.1 People
7.2 Activities
7.3 Situation / Context / Location
7.4 Props / Possible Solution
7.5 Snapshot of Ideation Canvas
Chapter 8: Future Scope
Chapter 9: References
5
Chapter 1: Introduction To Human Gesture Based
Shadow Robot
In the modern world of technological developments and many approaches, people typically
prioritize comfort and leisure above making changes. With their extremely good living
conditions, nobody makes any concessions. As a result, we use this gadget to increase comfort
and avoid any kind of discomfort.
The principal aim of this investigation was to acquire the comprehensive understanding
required to delineate the fundamental idea of the application, as the subsequent phase towards
transforming a conventional robot into an imitation robot. Consequently, the following are the
study's primary goals. The first goal was to gain a more thorough understanding of the issues
(such as difficult environments, lifting, and unfavorable conditions) and to evaluate the Kinect-
based methods. Determining whether gesture robots are more necessary than regular robots
was the second goal. Examining all research projects related to low-cost technology, with a
focus on its advantages, disadvantages, and technical aspects, was the third goal.
Moreover, from the publications examined here and the prior understanding within our research
team. Depending on the application, the shadow robot can be made to conduct any desired task,
including welding, grabbing, spinning, etc. For instance, robot arms in an assembly line for
cars carry out a range of duties like gripping, rotating, and positioning pieces while they are
being assembled. The other main objectives of this report are:
Domestic intent.
Application in Industry.
6
Chapter 2: Problem Statement
The apparatus and techniques for gesture recognition constitute the subject invention. The
gesture recognition system in the preferred embodiment of the invention is designed for device
control rather than as a means of human-to-human communication. That is, regardless of
whether the motions came from a live or inanimate source, the apparatus preferably
understands commands for the express purpose of controlling a device like a self-service
machine. It is ideal for the system to be able to identify both dynamic gestures and static
symbols, as motion gestures can usually transmit additional information. By using a human
gesture as the transmitter and the module as the receiver, the number of components in this
acknowledged system can be reduced.
Regretfully, there aren't many comprehensive theories or models that offer sound guidance on
how to create human-robot interfaces. A large body of research in the field of human–computer
interaction indicates that an interface is likely to be subpar if it is designed without sound
principles, without consideration for how people think, feel, and behave, and without
assessment. It is useful to have a sensor set up to recognize the gesture of a user carrying the
portable device.
7
Chapter 3: Objectives
Provide a system that enables users to manipulate a shadow robot with simple,
natural hand motions, establishing a fluid and intuitive paradigm for user
interaction.
Real-time reaction between the movements of the shadow robot and detected
gestures can be achieved, reducing latency and enhancing the immersiveness
and dynamic nature of the user experience.
Build a broad and scalable gesture library that will allow users to express a
variety of commands and increase the flexibility of human-robot interaction.
5. Integration of sensors:
Provide functions that let users define and personalize their own set of gestures
to encourage a customized and user-focused method of interacting with robots.
8
7. Feedback Mechanism:
Include feedback systems to let users know that their gestures are recognized by
the system and that the shadow robot is acting accordingly, such as haptic or
visual indications.
Provide algorithms and control systems that allow the robotic arm of the shadow
robot to replicate human hand dexterity and coordination precisely, resulting in
a robotic behaviour that is responsive and lifelike.
9
Chapter 4: Methodology
Assess and choose suitable sensors, like accelerometers and depth cameras, to
record hand movements. Assemble these sensors into the shadow robot's
architecture to guarantee interoperability and best possible data collection.
To precisely understand and categorize user motions, create reliable and flexible
gesture detection algorithms. If needed, you can even apply machine learning
techniques. To improve the system's recognition performance, train it on a
variety of datasets.
To guarantee the least amount of delay between gesture detection and the
shadow robot's response, design and implement a real-time processing
architecture. Optimize algorithms for speed and efficiency while taking the
robotic system's processing limitations into account.
To comprehend the robotic arm's range of motion and capabilities, model and
simulate its kinematics and dynamics. Create control algorithms to precisely
mimic the movements of human hands.
Provide an intuitive user interface that enables users to define and personalize
their own gestures. Install a learning system that gradually adjusts to the
preferences of each unique user.
Test and assess the system comprehensively, taking into account safety features,
robotic arm coordination, real-time responsiveness, and accuracy of gesture
detection. Get input from users to determine what needs to be improved.
The project's objective is to develop a novel human gesture-based shadow robot by utilizing
the described methods, with an emphasis on a user-friendly interface and reliable gesture
identification.
11
Chapter 5: Basic AEIOU Canvas
1. Activities
2. Environment
1. Industrial Setting:
Use coatings and materials on the robotic system that can endure abrasive
industrial environments.
2. Domestic Environment:
To create a robotic system with features that blend in with homes, research the
dynamics of households.
Make sure the robotic arm's motions are tailored to small homes.
12
3. Interaction
Examine how well the system adjusts to different user demographics, taking age
and experience into account.
2. Human-Robot Collaboration:
Examine cooperative activities in which the robotic system and humans work
together in industrial settings.
Provide safety procedures and safeguards to avoid mishaps when people are in
close proximity to robots.
4. Objects
Determine the maximum weight and size that the robotic arm is capable of
lifting in accordance with commercial and residential specifications.
To ensure a smooth integration with other robotic systems, make sure the system
is compatible with industry-standard communication protocols.
13
3. Objects needed:
3-axis accelerometer
Microcontroller
Sensors and transmitters
5. Users:-
To improve the user interface and overall experience, hold iterative user
feedback sessions.
Put user suggestions into practice to make the system more intuitive and
responsive.
To assist with continued user training, provide thorough user manuals and
internet resources.
3. Operating people:-
Industry
Military
Medical sector
14
Summary:-
AEIOU snapshot:-
15
Chapter 6: Empathy Canvas
User
I. Industry
Comprehending the viewpoint of industry participants entails being aware of the
obstacles they encounter in their day-to-day activities, including the requirement
for effective and secure automation, increased efficiency, and the smooth
incorporation of robotic systems into current processes. In this case, empathy
means taking into account the particular wants, worries, and goals of
professionals in the field to make sure that the creation and application of the
gesture-based shadow robot meet their real-world requirements and enhance the
industrial environment.
II. Medical
Comprehending the difficulties and needs of both patients and healthcare
providers is essential to understanding the medical viewpoint. When it comes to
healthcare workers, the gesture-based shadow robot may be most useful for duties
including patient care, rehabilitation, or even surgery. In the medical domain,
empathy means acknowledging that accuracy, dependability, and flexibility are
necessary in clinical settings and making sure that the robotic system is in line
with the objectives of improving patient outcomes and boosting medical
professionals' workflow efficiency.
16
III. Military
Comprehending the military viewpoint necessitates having empathy for the
distinct difficulties, operational demands, and strategic objectives faced by
military personnel. The gesture-based shadow robot's potential utility as a tactical
tool in a range of situations, such as clandestine operations, logistics, and
reconnaissance, may be the main emphasis for military users. In the context of
the military, empathy means understanding the necessity of resilience, flexibility,
and simplicity of integration into current military systems. Stealth, security, and
the capacity to carry out certain activities in line with military goals are also
critical factors to take into account.
IV. Domestic
Inhabitants, and stakeholders in domestic situations who might engage with or be
impacted by the deployment of such robotic technology in the gesture-based
shadow robot project's empathy canvas. Gaining an understanding of the
domestic viewpoint requires having compassion for the needs, goals, and
struggles that people face on a daily basis within their homes. For homeowners,
the emphasis might be on how the gesture-based shadow robot might serve as a
practical and easy-to-use helper for a range of home duties. Understanding the
value of usability, security, and adaptability to various home environments is a
necessary component of empathy in the domestic sphere. Important factors to take
into account include ease of use, the capacity to help with household tasks, and
integration with regular schedules.
17
Stake Holders:-
I. Factory workers
The gesture-based shadow robot may be of particular interest to industry workers
since it can increase productivity, increase safety, and simplify production
procedures. When considering empathy from the perspective of factory workers,
one must acknowledge that the robotic system must be seamlessly integrated into
current workflows in order to minimize disruptions and make sure that the
technology is in line with the realistic requirements of industrial operations. It is
imperative to take into account factors like as worker comfort, training needs, and
the possibility of human-robotic collaboration.
18
Activities
I. Can be used with simple hand gestures
It can be operated by simple hand gestures, so users won't have to understand
complicated concepts that could impede their ability to use it. Additionally, since
most working people can use simple hand gestures, it will be a more
straightforward and dependable solution for most tasks. Finally, it can assist with
tasks that might require more force than is humanly possible, making it an ideal
solution in the given situation.
V. Gesture recognition
Robot operators' hand and body movements will be the basis for gesture
recognition, which will enable the robot to comprehend the algorithm and operate
in a specific manner when a specific gesture is made.
19
Story Boarding:-
HAPPY:-
A series of earthquakes measuring 5.6 on the Richter scale struck New Delhi on April 26
at 12:39. Thanks to the NDRF, many people sought shelter in appropriate locations early
in the morning, but some were unable to escape and became trapped inside the damaged
building. The NDRF used a small surveillance robot that used hand gestures and a night
vision camera to detect the trapped individuals and saw them.
HAPPY:-
With their creativity, computer science and engineering (AI&ML) students took first
place in innovation and technology.
SAD:-
The chief architect cautioned all other workers to exercise caution when handling heavy
gear after one of the workers at the Mumbai building site suffered an arm fracture while
working near big things. Fourteen workers were brought to the hospital.
SAD:-
20
Empathy canvas snapshot:-
21
Chapter 7: Ideation Canvas
People:-
Factory workers :-
In industrial situations, a human gesture-based shadow robot can be very helpful
to workers. Gesture control's user-friendly interface makes it possible to operate
it hands-free, improving worker productivity and security. Because it can lift
objects and operate alongside humans, the robot expedites production
processes, lowering the need for manual labor and lowering the possibility of
accidents at work.
Domestic :-
The human gesture-based shadow robot helps with household chores in home
environments. The robot is accessible and user-friendly since it can be
controlled by anyone with natural hand motions, even those who are elderly or
have restricted mobility. Because of its lifting capabilities, the robotic arm can
perform jobs like reaching high shelves and picking up goods, which helps
people feel more independent in their homes.
Military :-
The human gesture-based shadow robot has multiple applications for the
military. For covert operations, gesture control offers a quiet and unobtrusive
way to operate the robot. The robot's ability to lift objects can be very useful for
logistical and reconnaissance tasks. Because of its modular architecture, it can
be integrated with other military hardware to increase its adaptability in
challenging and unpredictable environments.
22
Disaster response team :-
When crossing difficult terrain, disaster response teams can benefit from the
agility and versatility of a human gesture-based shadow robot. The robot's
lifting powers come in handy for search and rescue missions as well as debris
disposal. Quick and simple commands are made possible by gesture control,
which accelerates reaction times in life-threatening scenarios where every
second counts.
Activities:-
Gesture recognition
The human gesture-based shadow robot interprets and reacts to human hand
movements using sophisticated gesture recognition algorithms, many of which
are based on machine learning. motions are recorded by sensors like depth
sensors or cameras, and the system's algorithms process the data to detect
certain motions and convert them into orders the robot can follow. With the help
of this simple interface, users may effortlessly manage the robot with their
hands.
Domestic use
The human gesture-based shadow robot serves as a helpful assistant for a variety
of chores in a home environment. By using particular hand movements, users
may communicate with the robot and instruct it to carry out tasks like opening
doors, picking up objects, and helping with housework. The robot's design takes
into account the limitations of homes, making sure it can maneuver through
tight places with grace and offer users useful assistance.
23
Able to function as a module for additional devices
The human gesture-based shadow robot's modular architecture enables it to
serve as an adaptable part that may be included into other gadgets or robotic
systems. This entails standardizing communication protocols and interfaces to
facilitate smooth cross-technology cooperation. Its modular design expands its
usefulness beyond its stand-alone capabilities, acting as a foundational element
for a variety of applications.
Situation/Context/Location:-
Industries
Manufacturing and Assembly Lines:
Picking and arranging components on assembly lines is one repetitive, precise
task that the shadow robot may help with in the production process. It can adapt
to a variety of manufacturing needs thanks to its ability to mimic human
motions.
24
Quality Control and Inspection:
In manufacturing facilities, the robot can be used for activities related to
inspection and quality control. Its nimble movements can help ensure parts
fulfill certain quality standards or check items for flaws.
House
Monitoring and Security for the Home:
The robot can function as a home security system by adding sensors and
cameras. It can increase overall home security by patrolling the area, spotting
odd activity, and alerting homeowners.
Domestic Tasks:
To help keep the house tidy and organized, the shadow robot can help with a
variety of household jobs like picking up objects, putting things in their proper
places, and doing light cleaning..
Hospitals
Patient Support and Mobility:
Patients who struggle with mobility can walk around the hospital or carry out
tasks in their rooms with the aid of the shadow robot. Those recovering from
surgery or those with limited mobility can especially benefit from this.
25
Nursing Assistance:
By carrying and delivering supplies, medications, or medical equipment, the
robot can help healthcare personnel in the nursing setting. This lessens the strain
for nursing staff and frees them up to concentrate more on providing direct
patient care.
Props/Possible Solutions:-
3-axis accelerometer
An accelerometer that measures acceleration along three orthogonal axes—X,
Y, and Z—is known as a 3-axis accelerometer. In the context of a 3-axis
accelerometer, acceleration is defined as the rate of change of velocity and as
the acceleration of an object along three mutually perpendicular directions in
three-dimensional space.
26
Processing unit
The core element in charge of overseeing and regulating a shadow robot's whole
operation is referred to as the processing unit. It serves as the system's brain,
interpreting data from a variety of sensors, carrying out control procedures, and
directing the robot's motions. In order to guarantee the robot's responsiveness,
flexibility, and effective task execution, the processing unit is essential.
27
Chapter 8:
Future scope
28
Chapter 9:
References
29