Professional Documents
Culture Documents
Bachelor of Technology
By
Indraneel kumar
(180103055)
Nitham Madduri
(180103041)
It is certified that the work contained in the project report titled “Machine Learning
based gate opening and closing” by “Indraneel kumar” and “Madduri Nitham” has
been carried out under my supervision and that this work has not been submitted
elsewhere for a degree.
1
Declaration
We declare that this written submission represents my ideas in my own words and
where others' ideas or words have been included, I have adequately cited and
referenced the original sources. I also declare that I have adhered to all principles of
academic honesty and integrity and have not misrepresented or fabricated or falsified
any idea/data/fact/source in my submission. I understand that any violation of the
above will be cause for disciplinary action by the Institute and can also evoke penal
action from the sources which have thus not been properly cited or from whom proper
permission has not been taken when needed.
2
APPROVAL SHEET
This project report entitled "Machine learning based gate opening and closing
” by Indraneel Kumar and Nitham Madduri is approved for the degree of
Bachelor of Technology.
Examiners
Supervisor(s)
Chairman
Date: Place:
3
Abstract
Indraneel kumar
Nitham madduri
IIT Guwahati
April, 2022
4
Contents
List Of Figures 6
List Of Tables 8
Introduction 9
1.1 Automation 9
1.2 Motivation 10
1.3 Aim of project 10
Methodology 11
2.1 Face Mask Recognition 11
2.1.1 Deciding The Algorithm 11
2.1.2 Using Faster R-CNN 12
2.1.3 Problem Statement 14
2.1.4 Data Preparation 14
2.1.5 Training 14
2.2 License Plate Recognition 15
2.2.1 Vehicle Detection 15
2.2.2 License Plate Localization 16
2.2.3 Character Segmentation 17
2.2.4 Character Recognition 18
2.3 System Setup 20
2.3.1 IR Sensor 21
2.3.2 Raspberry Pi Zero 22
2.3.3 Servo Motor 23
Design Of Gate 25
3.1 Linear Actuator 26
3.2 Sliding Gate 28
3.3 Swing Gate 30
3.4 Other Gates To Consider 32
3.4.1 Vertical Lift Gate 32
3.4.2 Bi-Folding Gate 32
3.4.3 Barrier Arm Gate 32
5
3.5 Power Supply 33
3.6 Gears 34
3.7 Material 35
Results and Discussion 36
Conclusion 37
References 38
6
List Of Figures
Fig 2.1 : working of Faster R-CNN 13
Fig 2.2 : YOLO v2 Darknet architecture 16
Fig 2.3 : Vehicle Detection 16
Fig 2.4 : License Plate Localization 17
Fig 2.5 : (a) An example of a detected vehicle license plate,
(b) a binarized plate. 18
Fig 2.6 : Eliminating the upper and lower borders by
horizontal projection. 18
Fig 2.7 : Separating characters by vertical projection 18
Fig 2.8 : Template matching 20
Fig 2.9 : System setup 21
Fig 2.10 : Working of IR sensors 22
Fig 2.11 : IR Transmitter and Receiver 22
Fig 2.12 : Raspberry pi 0 23
Fig 2.13 : Servo motor 24
Fig 2.14 : Overview of Raspberry Pi working 25
Fig 2.15 : Flowchart of overall system 25
Fig 3.1 : Linear Actuator(top view) 27
Fig 3.2 : Linear Actuator(front view) 27
Fig 3.3 : Linear Actuator(cross view) 27
Fig 3.4 : Inner Parts Of Linear Actuator(side view) 28
Fig 3.5 :Inner Parts Of Linear Actuator(cross view) 28
Fig 3.6 : Sliding Gate(Front View) 29
Fig 3.7 : Sliding Gate(Cross View) 30
Fig 3.8 : Sliding Gate(Side View) 30
Fig 3.9 : Sliding Gate(Top Cross View) 30
7
Fig 3.10 : Swing Gate(Top Edge View) 31
8
List Of Tables
9
Chapter 1
Introduction
1.1 Automation
The entryway and the door opener are the two primary parts of a programmed
entryway. Entryway openers are mechanical gadgets that open doors without the
requirement for human communication. When worked with a controller, accessible
door openers use microcontrollers, pinion wheels, and engines to open the entryway.
10
1.2 Motivation
The neural style transfer for security we worked on last semester is still in the
beginning stages of research and it requires the system to be in online mode which
increases complexity of the gate system. So, we worked on automatic plate recognition
which requires less cost and is less complex. We also wanted to use linear actuators to
design a hinge gate system rather than only a sliding gate.
Aim of the project is to design a gate opening and closing system which is capable of
working automatically, Use License plate recognition(LPR) to automatically detect
license plates and compare different frame materials, gears to decide the best possible
system.
11
Chapter 2
Methodology
In the previous semester we worked on face mask recognition using Keras and Faster
R-CNN. The results of its weights and configuration can be used in vehicle detection.
Our method in this article will start with the discussion of Face Mask Recognition
which will be the same for vehicle detection. License Plate Recognition is applied on
these images which decides the opening and closing of gates.
There are a lot of algorithms used for object detection and R-CNN family is one of
them. There are three algorithms in R-CNN family namely R-CNN, FAST R-CNN and
FASTER R-CNN. We chose R-CNN family because it is faster to train.
R-CNN removes a lot of districts from the given picture utilizing specific hunt,
and afterward checks assuming any of these crates contains an article. We first
concentrate these areas, and for every district, CNN is utilized to extricate explicit
highlights. At long last, these elements are then used to distinguish objects. Tragically,
R-CNN turns out to be fairly delayed because of these different advances associated
with the interaction.
12
Conversely, Fast R-CNN sends the full picture to ConvNet, which makes
districts of interest (rather than passing the removed locales from the picture). It
likewise utilizes a solitary model that concentrates highlights from the areas, arranges
them into unmistakable classes, and results the bouncing boxes, instead of three free
models (as displayed in R-CNN).
All of these stages are completed at the same time, making it faster than
R-CNN. When applied to a large dataset, however, R-CNN is not quick enough
because it employs selective search to extract the regions..
By subbing the more slow R-CNN with a Region Proposal Network, the issue
of specific inquiry is tackled (RPN). We use ConvNet to extricate highlight maps from
the information picture, then feed those guides through a RPN to produce object
suggestions. At last, these guides are grouped and the it are anticipated to bound
boxes.
We chose Faster R-CNN because it gives much more accuracy and it is faster.
13
2.1.2 Using Faster R-CNN
Weights -
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2
_detection_zoo.md
Configuration -
https://github.com/tensorflow/models/tree/master/research/object_detection/configs/tf2
● Take an input image and pass it to the ConvNet which returns feature maps for
the image.
● Apply Region Proposal Network (RPN) on these feature maps and get object
proposals.
● Apply ROI pooling layer to bring down all the proposals to the same size.
● Finally, pass these proposals to a fully connected layer in order to classify any
predict the bounding boxes for the image.
14
Fig 2.1 working of Faster R-CNN
Our task is to detect a person wearing a face mask, person not wearing face mask and
person wearing face mask incorrectly in an image and draw bounding boxes around it.
We used kaggle face mask detection dataset. The dataset contains 850 images with
annotations given in .xml format.
The 850 images in the kaggle dataset are divided into 70 percent training set( 594
images) , 20 percent validation set (169 images) and 10 percent test set (85 images).
The training is done using faster R-CNN model provided by tensorflow object
detection API, therefore the dataset must be converted into Tensorflow TF-Record
format. We used roboflow to convert it.
15
The annotations in .xml format are converted to physical bounding boxes by using
roboflow.
2.1.5 Training
We trained the images using Faster R-CNN by tensorflow object detection API in
google colaboratory. We trained for 10,000 steps with an initial learning rate of
0.0002.
Link to code:
https://colab.research.google.com/drive/1n0mestU-1B7t51W6SAnXF965DcHOP-VU
# Training our face mask detection model from pretrained model( Transfer learning)
!python /content/models/research/object_detection/model_main_tf2.py \
--pipeline_config_path={pipeline_config} \
--model_dir={model_dir} \
--checkpoint_every_n=100 \
--alsologtostderr \
16
● character recognition
For our gate system, an IR sensor is used to detect the motion of the car and the
camera will be triggered to capture the vehicles images. Then the above four steps are
implemented.
To observe a tag, we should initially track down the vehicles in the camera's view, and
afterward observe the tag. This technique kills the risk of confusing tags with traffic
signs or advertising. We might utilize similar methodology we utilized in past
semesters to identify facial coverings. We utilized a pre-prepared quicker R-CNN
model to perceive facial coverings. A pre-prepared YOLOv2 model can likewise be
utilized. The Darknet-19 model used by YOLOv2 comprises 19 convolutional layers
and 5 maxpooling layers, as illustrated in the diagram below. Using the 19
convolutional layers and 5 maxpooling layers, YOLOv2 collects features, decreases
dimensions, and compresses an entire image. The original photos have been scaled
down to 7*7 or 13*13 pixels. Then, YOLOv2 conducts direct object recognition.
Compared with the Faster RCNN, YOLOv2 is the most efficient real time object
detection approach.
17
Fig 2.3 : Vehicle Detection
Following the detection of vehicles, we utilize the SVM to recognize tags. For
grouping and relapse examination, SVM is a regulated learning strategy. To prepare a
classifier, we use the SVM OAR (one against rest) engineering and the HOG upsides
of a picture as elements. To accurately recognize tags, we should prepare a classifier
that can recognize tags and non-tags. During the preparation stage, we should get
ready both positive and negative examples. The tags are the positive examples, while
the vehicle's locales without tags are the negative examples. The distinguishing proof
pace of tags has expanded because of the diminished intricacy of the negative
examples.
18
2.2.3 Character Segmentation
We should remove the space outside the characters, sift through clamor, and parcel it
into single characters for later acknowledgment subsequent to recording tags. An
identified tag is displayed in Fig. 2.5a. There are different stages associated with
character division. To eliminate commotion, the gained picture is first changed to
grayscale and afterward binarized. Fig. 2.5b shows the noise-removed binarized plate.
Then, we perform a horizontal projection of the license plate image to determine the
position of the characters arranged on the license plate. As shown in Fig. 2.6, The
upper and lower borders are removed by horizontal projection. Finally, we perform a
vertical projection on the license plate image to determine the position of each
character and then divide it into single characters, as shown in Fig. 2.7.
Fig 2.5 : (a) An example of a detected vehicle license plate, (b) a binarized plate.
Fig 2.6 : Eliminating the upper and lower borders by horizontal projection.
19
Fig 2.7 : Separating characters by vertical projection
This is the last phase in the LPR system. This phase is divided into two stages:
The separated characters are at first refined in the Normalization stage to squeeze them
into a square with no blank areas on each of the four edges. Each character picture is
currently normalized to 40x40 pixels. The window to see port change is utilized for
standardization. Each pixel of the first picture is planned to the relating pixel of the
standardized picture utilizing this planning. The picture got is scale unfeeling.
20
𝑛𝑟𝑜𝑤𝑠 𝑛𝑐𝑜𝑙𝑠
∑ ∑ 𝑚𝑖𝑠𝑚𝑎𝑡𝑐ℎ𝑖,𝑗
𝑖=1 𝑗=1
where
1, if 𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙𝑖,𝑗 ≠ 𝑒𝑥𝑡𝑟𝑎𝑐𝑡𝑒𝑑𝑖,𝑗
Nrows and ncols are number of rows and columns respectively. Because the image is
standardised to 40x40, nrows = ncols = 40 in our example. The mismatches for each
extracted character are found by comparing them to the database's original characters,
and the character with the lowest mismatch value is chosen as the recognised
character.
Character recognition can also be done using artificial neural networks which gives
higher accuracy but it requires more data and takes more time.
21
Table 2.2 Template Matching vs ANN
Accuracy 73 % 88 %
According to the prototype, an IR sensor is used to detect the motion of the car and the
camera will be triggered to capture the vehicle image. The image will be processed
according to the previously explained mechanism and if the vehicle number is
registered within the database then, the gate will open.
22
2.3.1 IR Sensor
Two IR sensors are needed to detect the obstacle (IR transmitter and IR
receiver). The receiver circuit receives the signal coming from the transmitter circuit.
This reception or non-reception of this signal is what determines the opening or
closing of the gate.
23
Fig 2.11 : IR Transmitter and Receiver
The Raspberry PI 0 is a designer board from the PI series. It's like a solitary board PC
that runs the LINUX working framework. The board has a ton of capacities, yet it
likewise has a ton of registering limit, making it reasonable for requesting errands. The
little actual size of the Raspberry Pi Zero opens up an entirely different universe of
conceivable outcomes. Actually, many individuals utilize the pi zero in their last
version of the plan in the wake of probing an alternate regular pi board. This is
because of the way that it consumes definitely less power than the leader Pi sheets,
making it ideal for battery-fueled applications.
The photograph taken by the camera is sent into raspberry pi which carries out tag
acknowledgment and really looks at regardless of whether to open the entryway.
24
Fig 2.12 : Raspberry pi 0
A servo motor is a rotary or linear actuator that provides for exact angular or linear
position, velocity, and acceleration control. It is made comprised of an appropriate
motor and a position feedback sensor. It also necessitates a complex controller, which
is frequently a separate module created exclusively for servomotors.
A servo motor is utilised to open and close the gates in this design. The Futaba S3003
standard servo is developed for remote control buses and boats, but it may be used in a
variety of other applications and is controlled by a PICAXE chip or other
microcontroller system. The servo comes with a bus and a selection of arms to attach
to it. The three wires that operate the servo are ground (usually black/orange), power
(red), and control (brown/other colour).The S03NXF Standard, for example, does not
follow this wiring order. The servo will move in response to pulses sent across the
control line that determine its position. The servo is wired with brown (negative), red
(positive), and orange wires (signal). ine the actuator arm's angle. The servo will move
in response to pulses sent over the control line that determine the angle of the actuator
25
arm. The servo requires a pulse every 20 milliseconds in order to receive precise angle
information.
2.3.4 Flowchart
The camera records an image when a vehicle approaches the gate's entrance, which is
then preprocessed, and the licence plate number is detected using License Plate
Recognition (LPR). If the detected car number matches the database already placed on
the personal computer, the command to open the gate is executed; otherwise, the
system cycles back to the beginning. The diagram below depicts the process flow.
26
Fig 2.15 : Flowchart of overall system
27
Chapter 3
Design Of Gate
The design permits administrators to have exact and precise command over the
creation. The direct actuator's smooth motion implies it requires less upkeep over its
lifetime and is normally energy proficient. They are more affordable, simpler to
introduce, and occupy less room than water powered or pneumatic partners.
28
Fig 3.1 : Linear Actuator(top view)
29
Linear Actuator Without Frame:
30
3.2 Sliding Gate
When sliding gates open and close, they slide side by side. One of the most
significant advantages of sliding automatic gates is that they take up far less area when
they open, making them ideal for short roads. Sliding gates are extremely dependable
and have a long service life.
Sliding gates are also more suited for rocky land because they normally run on
tracks or can be placed with cantilever systems. Many people believe that sliding gates
are more secure since they are difficult to force open.
Sliding gates have a few drawbacks. For example, most sliding gates require a
ground beam to place the wheels. Installation must be done with caution. We'll also
need enough room on either side of the gate for it to slide open sideways.
31
Fig 3.7 : Sliding Gate(Cross View)
32
3.3 Swing Gate
Swinging gates swing open and close, dividing in the middle and swinging apart, or
swinging together in one direction. Because they are so common, swing gates are the
most preferred option. Swing gates can usually be mechanised and come with a range
of control choices.
Swinging gates are easy to build and often affordable. Because they do not
require as much width as sliding gates, they may be better suitable depending on the
shape of the driveway. We can allow openings of up to 180 degrees, but most people
only allow 90 degrees.
In compared to sliding gates, this form of gate will be more affected by wind. In
high-wind scenarios, sliding gates may be an alternative. The swing area must also be
reasonably well maintained in order for the gate to swing freely.
33
Fig 3.11 : Swing Gate(Front View)
Vertical lift entryways drop upward out of control over the door opening. Vehicles
should have the option to pass under the door, so it should be raised sufficiently high.
This style of entryway requires the establishment of transcending vertical help towers
on the two sides of the door entrance. Notwithstanding, lifting the door requires a great
34
deal of force, and extra help is expected to hold the heaviness of the entryway once it
arrives at the top.
Two door boards are pivoted together to frame bi-collapsing entryways. These wall
boards overlap back on themselves while enacted, permitting passage. Bi-collapsing
entryways are regularly utilized two by two, with one sets on one or the other side of
the door opening. At the point when space is restricted, bi-collapsing entryways have a
minuscule impression and are an amazing choice. The opening and shutting paces of
numerous bi-it rush to overlay entryways. Due to the various potential entanglement
spots and the troubles in creating this sort of entryway in contrast with the others, it
isn't suggested that it be used.
An upward boundary arm pivots all through the door opening to make obstruction arm
entryways. Vehicles are constrained by boundary arm doors, not walkers. Boundary
arm doors give basically no assurance since it is somewhat simple to stroll close by,
move over, or under the entryway arm. Boundary arm doors are by and large utilized
to oversee car traffic at staffed security passages or to control access into and out of
leaving offices. These entryways are not difficult to utilize and occupy little room.
The choice of power source plays a major role in choosing different components.
35
As a result of this, the majority of electronic appliances on the market run on
DC.
2. Batteries, solar cells and specific equipment are all examples of DC sources.
Solar cells can be utilised to make the device more cost effective because it will
be used outdoors. Nowadays, all solar panels generate DC power.
AC DC Weightage
Availability of 5 9 0.4
electronic devices
Economical 8 8 0.3
36
3.6 Gears
Because gears can boost the torque generated by the motors, they are required for our
design. The gear ratio can be adjusted to achieve the desired performance based on the
weight of the gate and the time it takes to open it.
Despite the fact that there are a wide range of kinds of entryways, just the cog
wheels that send movement between two equal tomahawks were decided to make the
plan interaction simple. Prod gear, Helical stuff, and Herringbone Gear (Double
Helical stuff) were totally thought about. Prod gear has the uprightness of being both
cheap and productive. It does, in any case, produce clamor. It is minimized and light in
contrast with different types of stuff, permitting the device to be more viable. Helical
pinion wheels are calmer than prod gears, despite the fact that they are more costly.
They do, be that as it may, have a higher burden resilience. Herringbone gears,
otherwise called twofold helical cog wheels, give smooth transmission and high
burden resilience at the punishment of expanded size and cost.
37
3.7 Material
There will be a casing encasing the gadget to shield it from external climate, for
example, downpour, dust and so on.
Three materials that considered are steel, aluminum and carbon fiber. Aluminum is
incredibly light, pliable, tough and impervious to consumption which can turn into a
need as the gadget is available in outside conditions. Steel is nearly weighty and not
extremely impressive and tough. Carbon Fiber is an exorbitant material.
Table 3.3 Comparison of frame material
Cost 8 6 9 0.2
weight 8 9 6 0.4
score 8 8.4 7
38
Chapter 4
We trained the images of face mask detection using Faster R-CNN by tensorflow
object detection API in google colaboratory. We trained for 10,000 steps with an initial
learning rate of 0.0002.
The trained model detects face masks in the image with a confidence score.
As seen in the above image, the accuracy of the model is nearly 95% while training on
850 images for 10,000 training steps. This training on google colab lasted for
approximately 3hrs. This accuracy can be increased by using more powerful GPUs
which trains faster and allows us to train for more steps increasing the accuracy.
39
Chapter 5
Conclusion
We compared different options for frame material, gears and power supply and
provided each option with a score. We also designed some CAD models for Linear
actuator, Sliding gate and Swing gate. Further we can improve by making CAD
models for the whole gate including motors and gears.
References
40
1. M J. Ahmed, M Sarfaz, A. Zidouri, and K G. AI-Khatib, “LICENSE PLATE
RECOGNITION SYSTEM”, 10th IEEE International Conference.
2. Nur Liyana Yaacob, Ammar Ahmed Alkahtani, Fuad M. Noman, Ahmad Wafi
Mahmood Zuhdi, Dhuha Habeeb, “License plate recognition for campus auto-gate
system”, Indonesian Journal of Electrical Engineering and Computer Science,2020.
4. D. Zheng, Y. Zhao, and J. Wang, “An efficient method of license plate location,”
Pattern Recognition. Lett., vol. 26, no.15, pp. 2431-2438, Nov. 2005.
5. Sarbjit Kaur, "An Automatic Number Plate Recognition System under Image
Processing", I.J. Intelligent Systems and Applications, vol. 3, pp. 14-25, 2016
6. Onyinye Ikpeze, Samiat Bola-Matanmi and Kola Michael Kareem, “Design and
Construction of an Automatic Gate”, ABUAD Journal of Engineering Research and
Development (AJERD), 2021
7. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
by Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.
41