Supplier Bot

Problem Statement 2: Day to day struggles of a covid affected patient

Idea Outline
With hospitals filling up and treatment costs rising, many patients are
instead being looked after at covid care homes or smaller clinics. But
issues like having a small staff count and the risks of entering a covid
patient’s room/ward makes these places a less viable option.

The idea of the supplier bot is to take necessary supplies to the

patients when called upon, allowing minimal interaction, and less
dependency of the patients on the staff.
Implementation
I have thought of using image processing using OpenCV in python to
implement this idea. The bot would follow a path, which would be
marked using a particular colour, and outside each ward, there would
be a patch of a different colour.
Based on which patient ordered the necessary supplies, the bot will
look out for the corresponding colour patch, and when the bot
encounters it, it will enter the ward.
Implementation (Process)
• The patient checks the checkboxes of the supplies or assistance they
need on their phones.
• The corresponding google sheet cells change their values from “no”
to “yes”.
• The python code, using the google sheets API, notes the changes, and
using ‘pyserial’, communicates with the Arduino and the bot starts
following the track till the camera connected to the bot senses the
corresponding colour, after which the bot assists the patient.
• The corresponding google sheet cells values are changed from “yes”
to “no” by the code.
Implementation Method (Colour patch
detection)
To represent the track, I have used my ID-Card’s lanyard, which is blue
in colour. And for the colour patches on the floor, I have used
notebooks of different colours. I found the appropriate HSV limit values
using a trackbar mechanism.
Implementation Method (Path following)
Initially, my implementation was based upon finding just the centroid
of the contour of the path, and comparing that with the centre of the
frame, and deciding whether to move left or right based on that. But to
increase performance, I have included angle calculation, which allows
the path following module to be very accurate and smooth.
Code file and demo video
This is the demo of the software implementation of the project. I randomly change
the values of some cells from “no” to “yes” and upon running the code, we can see
that the bot follows the track and when it encounters the particular colours, it
responds accordingly, and all values of the google sheet cells are switched to “no”
at the end since the bot completed all the tasks.
Code link:
https://drive.google.com/file/d/1fdw8B5KBu7CNCq1f3J4w4rtjjhmEyChn/view?usp
=sharing
JSON File:
https://drive.google.com/file/d/1SZy4Q5oUFpbNK8P09dtX_q_emVWRJx-
n/view?usp=sharing
Video link:
https://drive.google.com/file/d/1RbaoD5T_tVit0d_FkZbC80HCSsEXp9LN/view?usp
=sharing
Pyserial code and Arduino video
During the implementation of the project on hardware, I faced many issues. initially I used
an HC05 Bluetooth module and the bot responded as desired. But due to some failure in
the module I had to shift to connecting the bot using the Arduino UART cable.
Unfortunately, there was a sudden failure in the motor driver as well, although it worked
for some time. The python and the Arduino files will be included, and along with that I
have also shared a 10 second video of the bot when it was working before the issues
popped up.
Python code with pyserial:
https://drive.google.com/file/d/1crwOt6ohI-AKv8fQ-
H4W7eGI02X9FmYW/view?usp=sharing
Arduino code:
https://drive.google.com/file/d/18XYurpnHmYI-P3JYKxOlgXhMCiRbTwd-/view?usp=sharing
Video:
https://drive.google.com/file/d/1gcEGtKuGUMUq5JcUGyL1_n-
WBQGDDhht/view?usp=sharing
Key elements used
• OpenCV module in python for image processing
• Pyserial module in python to communicate with Arduino
• Google sheets API
• Arduino and chassis for movement

For my prototype, I have used an HC-05 Bluetooth module and the

video feed is from my laptop’s webcam, although, for complete
implementation this can be developed using a Raspberry – Pi and a
camera module.
Potential
Although this idea is focused upon helping covid patients get supplies,
this simple skeleton could be used for many applications. For example,
it could act like a waiter bot at a restaurant to deliver food, or in
factories to carry heavy goods.

With further developments, such as using ROS and introducing SLAM,

bots like these could be used extensively for wider range of
applications.
 Thank You

Team name: Alpha

Member(s): Rishab Agrawal