AI ROBOTICS

AND
AUTOMATION
AIDI1012 - 2024
# Final Nano group project
 P R E S E N TAT I O N
T E A M M AT E ’ S :

Student Id Members

T I T L E
200576299 Dhruvkumar Prajapati
200534289 Harkirath Singh Sran
200573391 Vijay chakravarthy 2

200523921 Priya Jogani

M a y
200511247 Hitesh Nirola

2 1 ,
200575060 Sourabh Bhoir

2 0 X X
200541876 Tuan Anh Nguyen
200566953 Bhanu Bhakta Bhattarai
Purviben Rajubhai Patel
200522745
200574156 Divyakumar Patel
 P R E S E N TAT I O N
flash the Jetson Nano with the JetBot image

• A pre-built software package that includes all of the necessary software for the JetBot,
including the JetPack software development kit, the ROS robotics framework, and the
NVIDIA Deep Learning SDK

T I T L E
Setup the swap file(4GB)

3
Git clone: https://github.com/NVIDIA-AI-IOT/jetbot

M a y
Setup the Docker Container to run the jetbot

2 1 ,
2 0 X X
• https://github.com/NVIDIA-AI-IOT/jetbot/tree/master/docker
• Followed the above link to build and install docker
• Run the Jupyter Notebook inside the container
• Jupyter Notebook can be accessed using a web browser on a computer connected to the same
network as the JetBot
 P R E S E N TAT I O N T I T L E M a y 2 1 , 2 0 X X
4
JUPYTER NOTEBOOK :
MobaXterm

P R E S E N TAT I O N
• Terminal emulator
and remote
desktop software

T I T L E
• It comes with an
SSH client that can
securely connect 5
to JetBot over the
network,

M a y
providing a safe
and encrypted

2 1 ,
communication

2 0 X X
channel
 P R E S E N TAT I O N
JETBOT ROAD
FOLLOWING

T I T L E
• JetBot Road Following is a popular application of the JetBot platform, where the JetBot uses
computer vision and machine learning algorithms to follow a road or track.
6
• This application is often used in autonomous vehicle research and development, as well as in
educational settings to teach students about computer vision and machine learning

M a y
• Three steps

2 1 ,
1. Data collection

2 0 X X
2. Training
3. Deployment
P R E S E N TAT I O N T I T L E M a y 2 1 , 2 0 X X
7
D ATA C O L L E C T I O N
P R E S E N TAT I O N T I T L E M a y 2 1 , 2 0 X X
8
Data collection of
the track
 P R E S E N TAT I O N T I T L E M a y 2 1 , 2 0 X X
10
TRAINING JETBOT
We are training a neural network
PyTorch deep learning framework
to take an input image which will
to train ResNet18 neural network
output a set of x, y values
architecture model
corresponding to a target

We need to upload/place the data We train for 50 epochs and save

of the road to be followed best model if the loss is reduced

A model will be generated after

training
P R E S E N TAT I O N T I T L E M a y 2 1 , 2 0 X X
12
P R E S E N TAT I O N T I T L E M a y 2 1 , 2 0 X X
13
Road Following -
Deployment
 P R E S E N TAT I O N
COLLISION

T I T L E
AVOIDANCE
14

• Use computer vision techniques to detect

M a y
obstacles in the robot's path.
• Once an obstacle is detected, the robot can use its

2 1 ,
motor control system to adjust its trajectory and

2 0 X X
avoid the obstacle
• Useful preventing the robot from entering
dangerous situations
P R E S E N TAT I O N T I T L E M a y 2 1 , 2 0 X X
15
Training :
 P R E S E N TAT I O N
Object
Following

T I T L E
• Got error that the Python module
libssd_tensorrt.so is missing even
after adding the pre-trained coco
model inside the folder as per the 17
instructions
• Tried to solve the error by below
methods

M AY
• Checking the file
libssd_tensorrt.so is present in

2 1 ,
the directory

2 0 X X
• Installing pyserial
• Uninstall and then reinstall the
jetbot module or Tensor
 Challenges
• The Jupyter notebook kept crashing while training the neural model
and hence enabled swap file of 4 GB
• Docker related – Fixed the version issue of `protobuf` while building
the docker image on jetson.

• Took a couple of hours to build the docker image on jetson nano

• In the live_demo.ipynb,needed to adjust the speed, steering values
properly so that jetbot start moving as per the training on the track.
• Bot behaved differently on different surfaces
IMPROVING JETBOT TRAINING

P R E S E N TAT I O N
T I T L E
COLLECT MORE EXPERIMENT WITH
DATA DIFFERENT
MODELS
19

M AY
2 1 ,
2 0 X X
HYPERPARAMETER TEST AND
TUNING EVALUATE THE
MODEL ON A
DIVERSE SET OF
DATA
P R E S E N TAT I O N T I T L E M a y 2 1 , 2 0 X X
20
THANK YOU