Institute of Technology of Cambodia

Electrical and Energy Engineering Department

Real-time Minimal Computation Shoes Counter for Production
Monitoring in Manufacturing Plant
Student: Vexo Wetu
ID: E20171023
Advisor: Mr. Chou Koksal
Supervisor: Assoc. Prof. Dr. Chawalit Jeenanunta
Table of
Contents
Page

I Introduction 4

II Research Methodology 5

III Results 10

IV Conclusion 15
 Sirindhorn International Institute of Technology
Thammasat University
SIIT is always working on research and study in order to discover and develop new technologies and innovations.
Including the department of Logistics and Supply Chain Systems Engineering (LogEn) Research Unit, which address
cutting-edge supply-chain problems and advanced research in the area of logistics, and supply chain system with the
emphasis on optimization, production planning, scheduling simulation, inventory management, forecasting, quality
management, lean management, and operations management.

3
I Introduction

Problem Statement
• Company can't track the amount of good shoes
and bad shoes in real-time
• Reduce the amount of time and money on
production quality counting

Solution

Use image processing technique to count good

and bad shoes in real-time.

4
II Research Methodology
System Overview
Data Collection Data Pre-processing Shoes Detection Shoes Tracking Shoes Counting

• Collecting image from • Labeling image • Training data using • Tracking shoes • 2 lines intersection
Lasting QC station • Resize image yolov4-tiny darknet • Detect shoes with method
• Generate new data by • Detect shoes with bounding box with
horizontal flip bounding box tracking ID
• Split data

5
II Research Methodology

Data Collection

Data Collection

SSH

• Connect to the Jetson nano with Secure shell protocol

then get access to the CCTV camera
• Record different video in each day to get different type
of shoes using openCV and python
• Total of 600 images from different recorded video

6
II Research Methodology

Data preprocessing

• Split data into 80% train, 17% validation, 3 % testing

• Resize image to 416 x 416
• Use horizontal flip to generate new data
• Download data in the form of zip file
Figure2.1. Example of labeling and draw box on image

7
II Research Methodology

Shoes Detection

• Cloud base system:

• Free GPU
• File sharing

Yolo is an image processing algorithm which used convolution neural network to detect real time object with high speed and
accuracy. It used 3 techniques type:
• Residual blocks : divide image to grid (SxS)
• Bounding box regression : box outline that specific object detection
• Intersection Over Union (IOU) : describe object detection that overlap, and make perfect object boundary

8
II Research Methodology

Shoes Detection

Model Configuration & Yolo detection process :

- Yolov4-tiny propose to use with small-scale and low computation device
- Yolov4-tiny.conv.29 to train our input picture below.

Bounding Box+Confident

Input Detected Object

Image Output

9
Class Probability Map
II Research Methodology

Shoes Detection

Write
Import data configuration file Training data

Yolo ( You only Look once)

Yolo is an effective real-time object detection based on convolution neural
network. Yolo v4 tiny build up with Bag of freebies (Bof) & Bag of specials (BoS)
and 3 other important component:
• BackBone: CSPDarknet53
• Neck: spatial pyramid pooling layer and Path Aggregation Network
• Head: Yolov3

Figure 2.2. Shoes detection with yolo v4 10

II Research Methodology

Object Tracking
Deepsort

Tensorflow Object Mahalanobis

Model detection Distance
Kalman Hungarian
filter Assignment
Deep
Appearance
Yolov4-tiny
Description
weight

Input
Obeject detection Object Tracking

11
System overview by human monitoring 12

Stamp

Worker put stamp on

shoes

Not Pass QC line

Bad shoes
II Research Methodology

Shoes Counting

2 Lines intersections method

• Start count shoes when the shoes touch the first line then the
second line
• Count the shoes as a none quality shoes when it touch the
bottom line
• Quality shoes = Total Shoes - None Quality Shoes

Counting Line

Not Pass QC Line

13
III Research Results

14
Figure3.1.MaP 92.6% during object training for object detection
III Research Results

Table3.1. Shoes counting system preferment measurement

Video Video Total shoes Total Shoes Not Pass QC Not Pass QC
Duration (Actual) (System) (Actual) (System)

vid 1 5mn 4s 29 18 3 4

vid 2 2mn 15s 14 17 4 2

vid 3 2mn 15s 21 14 2 3

vid 4 3mn 45s 17 20 2 2

vid 5 3mn 2s 22 24 1 3

15
IV Conclusion

This Research use the Yolov4-tiny object detection and deepsort algorithm to detect image. The system uses the
computational power of Corei5 2.6 GHz CPU and 8 GB of RAM. The counting system algorithm didn’t get a good counting
yet due to the limitation of the camera which is installed in the QC station doesn’t gave a good video and because of the fast
movement of the shoes is very difficult for our system to detect the target object in the video.

Future Work
Since the project is still ongoing, there will be more improvements to the system, such as
• Increasing the performant of object tracking and improving the mean average precision of the detector by adding more
datasets to the system.
• Propose a new counting algorithm that uses human motion and the location of the stamp
• Study about how to send the data to the main sever at the factory

16
Thank you
Q&A

16
III Research Methodology

18
Figure3.1.Loss graph during object training for object detection