Detecting a Team Shipping

Element Using TensorFlow

By 13019 Platinum Panthers
Purpose Ultimate Goal:

How many rings

● How to use the FTC Machine Learning are in the stack?
Toolkit with Tensor Flow Lite

● Image Classification in Past Games

○ Number of Rings Detected
○ Duck Location Detection
○ Custom Team Element Detection Freight Frenzy:

● Today, we will customize Where is the duck

a Tensor Flow Lite Model on the bar code?
for detecting a Team Shipping More Points!
Element (TSE) Where is the team
shipping element
on the bar code?
Overview
Model Development (Evan)
• Design Tips for the Team Element
• Taking a Video of the Team Element
• Accessing the First Machine Learning Toolkit
• Training the Model
• Reviewing Model Output for a Good Fit
• Exporting the Model

Block Coding for Team Element Detection (Gargi)

• Importing the Model
• Editing the Code
• Logic for Detection
Design Tips for Team Element
● Contrasting Elements
○ Bright color
○ Different Color than Field
○ Optional to consider stripes/patterns
○ Cone design that is good for stacking
with alliance TSE

● Size Requirements
○ Minimum Size – 3”x3”x4”
○ Maximum Size – 4”x4”x8”
How to Record the Video
For best results:

○ Record with the same camera position

and angle used during autonomous

○ Record TSE in different positions, lighting

conditions, and different backgrounds

○ Try all TSE combinations from:

■ left/right sides, red/blue alliance
sides, bar code spot 1, 2, or 3

○ Include shots without the TSE

○ Record data on an FTC field

Accessing the First Machine Learning Toolkit
● Go to Team’s First Dashboard Account

● Follow the Instructions to Add Users to

the Machine Learning Toolkit
○ Be careful to add only one user so your
machine learning hours are not used
up.

● Go to: https://ftc-ml.firstinspires.org
• If you are already logged in, it will send
you to the main workflow page
• If not logged in, it will send you to the
FIRST Dashboard Login
• Once you are in the main workflow
page, press upload video to upload
your video
Training the Model
Bounding and Labeling the Element
● Open the video

● Draw your bounding box around TSE

● Label must always be exact same name

○ We used “green_TSE” for example

● If you press start tracking, then it will start

making the box for you automatically

● If the TSE isn’t in the frame, then stop the

tracking and get rid of the box
Training the Model
Select Videos for Training
● When all of the frames are ready
○ Go back to the main screen and click
on the box by the videos you want to
use
Settings for Training
● Set percent of training 80%
● Set percent of evaluation at 20%
● Make sure there is only 1 label for a
correct model
● If more than 1 label, be sure to
correct before training starts

Click “Produce Dataset” for Model to

START TRAINING
Training the Model
Finish Training
● Recommend at least 100 epochs/training steps

● The training time is how long you will let the model train

● Each team has a limited number of minutes for the customized

model
○ About 60 minutes = 3,000 training steps.

● The model saves every 100 steps in case it crashes

 Reviewing Model Output for a Good Fit
● When the model is done, you can click on the
model tab.

● Click on your model to see how accurate your

model is.

● What makes a good fit?

○ All loss graphs to be close to zero
○ Eerything else you want to have the graphs close
to 1
○ you will only have one graph for large, medium,
or small depending on the video
Check the Label by TensorFlow vs. User
Exporting the Model
● To export your tensor flow lite model, select your model in the models tab and
then click download to get a .tflite file.
Coding
 Importing Trained Model

Once you have done creating a database, we need to import database into the
block coding system.
● Upload the .tflite file you downloaded into the manage page of the block
coding application.

● Upload in the place where it says update tensor lite model.

Start programming
● First create a new Op-mode.
● We will use a preset specifically made for the tensorflow custom
model.
Editing the code
Step 1 : Initialize Vuforia Library
If you are using a phone camera, then you
need to enable the phone camera block. If
you are using a webcam then enable the
webcam block.

If you want to use webcam then

remember to set the cameraname to
the webcam name in your configuration
file on robot controller or control hub.
Step 2: Initialize TFOD
● We need to initialize and activate TFOD before WaitForStart so that it
will save you some time during autonomous mode
● First Step is to Setting up the labels for detection
Update the block
with your file
name, and label.

.
● The minimum confidence level in the example is set to 70%. This
means that TensorFlow library needs to have a confidence level of
70% or more in order to consider an object as being detected.

● ObjectTracker parameter is set to true for this block , so an object

tracker will be used, in addition to the TensorFlow interpreter, to
keep track of the locations of detected objects
Step 3 : Setting up the camera view
Below the initialization is this block

● . and magnification of the camera,

With this block, you can change the aspect ratio
which is useful for focusing more on your Team shipping element and getting rid of
distraction.

● In order to View all three of the barcode squares we set the magnification to one.
Step 4: Detection
Recognition loop

● If there are 0
recognitions = Telemetry
‘No items detected’.

● Else then each item in

recognition list is
displayed in telemetry.
Display function
● Displays X and Y coordinate
of object with respect to
camera frame using
telemetry block
 Step 5 : Logic For TSE detection
If shipping element detected then set TSEDetected to true, and save the X
coordinate in a variable to detect the shipping hub level.
 Step 6 : The deactivate block
This block at the end deactivates the tensorflow object detection.

It is important to close the library to free up system resources.

Results
Thank You
Website :
http://lmsrobotics.weebly.com/

Contact :
ftc13019platinumpanthers@gmail.com
larsonmsrobotics@gmail.com

YouTube:
First Robotics Platinum Panthers