Professional Documents
Culture Documents
In
By
M V S Mahesh (191230028)
B. Suchitha (191230018)
K. Kiran (181230021)
(An Autonomous Institute under the aegis of Ministry of Education, Govt. of India)
1
DECLARATION
We hereby declare that the work presented in the report entitled “Number plate
detection using image processing” submitted by us for the fulfilment of the
project, is an authentic record of our work carried out under the guidance of
Dr.Rahul Jaiswal
…………………………
M V S Mahesh
…………………………
B. Suchitha
…………………………
K. Kiran
2
CERTIFICATE
This is to certify that the above statement made by the candidates is correct to the
best of my knowledge.
…………………………….
3
ACKNOWLEDGEMENT
We have taken a lot of effort into this project. However, completing this project
would not have been possible without the support and guidance of a lot of
individuals. We would like to extend our sincere thanks to all of them.
We are highly indebted to Dr. Rahul Jaiswal for his guidance and supervision.
We would like to thank him for providing the necessary information and
resources for this project.
We would like to express our gratitude towards our parents & our friends for
their kind co-operation and encouragement which help us a lot in completing
this project.
Our thanks and appreciations also go to our colleagues in developing the project.
Thank you to all the people who have willingly helped us out with their abilities.
M V S Mahesh
B. Suchitha
K. Kiran
4
CONTENTS
- Filtering
- Affine Transformations
- Image denoising methods with morphology
4. METHODOLOGY 12
5. 15
CODE
5
ABSTRACT
The main objective of this project is to automate the entries of vehicles at main gate.
Recognizing Car License Plate is a very important task for a camera surveillance-based
security system. We can extract the license plate from an image using Image processing
techniques and then we can use Optical Character Recognition to recognize the license
number.
Step 3: Extracting the plate number from the previously extracted number plate.
6
INTRODUCTION
Digital image processing is the use of a digital computer to process digital images
through an algorithm. As a subcategory or field of digital signal processing, digital
image processing has many advantages over analog image processing. It allows a
much wider range of algorithms to be applied to the input data and can avoid
problems such as the build-up of noise and distortion during processing. Since
images are defined over two dimensions (perhaps more) digital image processing
may be modeled in the form of multidimensional systems. The generation and
development of digital image processing are mainly affected by three factors: first,
the development of computers; second, the development of mathematics
(especially the creation and improvement of discrete mathematics theory); third,
the demand for a wide range of applications in environment, agriculture, military,
industry and medical science has increased.
● Finding contours
● Finding vertical edges
● Grey scaling the image
● Binarizing the image
● Applying closing morphological transformation
Contours of an image:
Contours are the line joining all the points along the boundary of an image that are
having the same intensity. It helps in shape analysis.
Gray scale:
In digital images, grayscale means that the value of each pixel represents only the
7
intensity information of the light. Such images typically display only the darkest black to
the brightest white. In other words, the image contains only black, white, and gray colors,
in which gray has multiple levels.
Image Binarization:
Morphing
It is a special effect in motion pictures and animations that changes (or morphs)
one image or shape into another through a seamless transition. Traditionally such a
depiction would be achieved through dissolving techniques on film.
Morphological closing
8
THEORY
Digital image processing is the use of a digital computer to process digital images
through an algorithm. As a subcategory or field of digital signal processing, digital
image processing has many advantages over analog image processing. It allows a
much wider range of algorithms to be applied to the input data and can avoid
problems such as the build-up of noise and distortion during processing. Since
images are defined over two dimensions (perhaps more) digital image processing
may be modeled in the form of multidimensional systems. The generation and
development of digital image processing are mainly affected by three factors: first,
the development of computers; second, the development of mathematics
(especially the creation and improvement of discrete mathematics theory); third,
the demand for a wide range of applications in environment, agriculture, military,
industry and medical science has increased.
1. Filtering
Digital filters are used to blur and sharpen digital images. Filtering can be
performed by:
● convolution with specifically designed kernels (filter array) in the spatial
domain
● masking specific frequency regions in the frequency (Fourier) domain
2. Affine transformations
9
To apply the affine matrix to an image, the image is converted to matrix in
which each entry corresponds to the pixel intensity at that location. Then each
pixel's location can be represented as a vector indicating the coordinates of that
pixel in the image, [x, y], where x and y are the row and column of a pixel in the
image matrix. This allows the coordinate to be multiplied by an
affine-transformation matrix, which gives the position that the pixel value will be
copied to in the output image.
However, to allow transformations that require translation transformations, 3
dimensional homogeneous coordinates are needed. The third dimension is usually
set to a non-zero constant, usually 1, so that the new coordinate is [x, y, 1]. This
allows the coordinate vector to be multiplied by a 3 by 3 matrix, enabling
translation shifts. So the third dimension, which is the constant 1, allows
translation.
Because matrix multiplication is associative, multiple affine transformations
can be combined into a single affine transformation by multiplying the matrix of
each individual transformation in the order that the transformations are done. This
results in a single matrix that, when applied to a point vector, gives the same result
as all the individual transformations performed on the vector [x, y, 1] in sequence.
Thus a sequence of affine transformation matrices can be reduced to a single affine
transformation matrix.
10
Take the Closing method for example.
Dilation first
1. Read the image and convert it into grayscale with Matlab.
1. Get the size of an image. The return value row numbers
and column numbers are the boundaries we are going to
use later.
2. structuring elements depend on your dilation or erosion
function. The minimum of the neighbor of a pixel leads to
an erosion method and the maximum of neighbor leads to
a dilation method.
3. Set the time for dilation, erosion, and closing.
2. Create a zero matrix of the same size as the original image.
3. Dilation first with structuring window.
1. structuring window is 3*3 matrix and convolution
2. For loop extract the minimum with window from row
range [2 ~ image height - 1] with column range [2 ~
image width - 1]
4. Fill the minimum value to the zero matrix and save a new image
1. For the boundary, it can still be improved. Since in the
method, a boundary is ignored. Padding elements can be
applied to deal with boundaries.
Then Erosion (Take the dilation image as input)
1. Create a zero matrix of the same size as the original image.
2. Erosion with structuring window.
1. structuring window is 3*3 matrix and convolution
2. For loop extract the maximum with window from row
range [2 ~ image height - 1] with column range [2 ~
image width - 1]
3. Fill the maximum value to the zero matrix and save a new image
1. For the boundary, it can still be improved. Since in the
method, boundary is ignored. Padding elements can be
applied to deal with boundaries.
4. Results are as above table shown
11
METHODOLOGY
1. To reduce the noise we need to blur the input Image with Gaussian Blur then
convert the it to grayscale.
12
3. To reveal the plate we have to binarize the image. For this apply Otsu’s
Thresholding on the vertical edge image. In other thresholding methods we have
to choose a threshold value to binarize the image but Otsu’s Thresholding
determines the value automatically.
13
5. To detect the plate we need to find contours in the image. It is important to
binarize and morph the image before finding contours so that it can find more
relevant and less number of contours in the image. If you draw all the extracted
contours on original image, it would look like this:
14
CODE
Code for processing the image and extracting the number plate and
subsequently the license number on the plate:
import cv2
import numpy as np
from skimage.filters import threshold_local
import tensorflow as tf
from skimage import measure
import imutils
from datetime import datetime
def sort_cont(character_contours):
"""
To sort contours from left to right
"""
i = 0
boundingBoxes = [cv2.boundingRect(c) for c in character_contours]
(character_contours, boundingBoxes) =
zip(*sorted(zip(character_contours, boundingBoxes),
key=lambda b:
b[1][i], reverse=False))
return character_contours
thresh = cv2.bitwise_not(thresh)
15
# resize the license plate region to a canoncial size
plate_img = imutils.resize(plate_img, width=fixed_width)
thresh = imutils.resize(thresh, width=fixed_width)
bgr_thresh = cv2.cvtColor(thresh, cv2.COLOR_GRAY2BGR)
16
aspectRatio = boxW / float(boxH)
solidity = cv2.contourArea(c) / float(boxW * boxH)
heightRatio = boxH / float(plate_img.shape[0])
17
characters.append(temp)
return characters
else:
return None
class PlateFinder:
def __init__(self):
self.min_area = 4500 # minimum area of the plate
self.max_area = 30000 # maximum area of the plate
self.element_structure =
cv2.getStructuringElement(shape=cv2.MORPH_RECT, ksize=(22, 3))
element = self.element_structure
morph_n_thresholded_img = threshold_img.copy()
cv2.morphologyEx(src=threshold_img, op=cv2.MORPH_CLOSE,
kernel=element, dst=morph_n_thresholded_img)
return morph_n_thresholded_img
method=cv2.CHAIN_APPROX_NONE)
return contours
18
thresh = cv2.adaptiveThreshold(gray, 255,
cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2)
contours, _ = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_NONE)
if contours:
areas = [cv2.contourArea(c) for c in contours]
max_index = np.argmax(areas) # index of the largest contour
in the area array
max_cnt = contours[max_index]
max_cntArea = areas[max_index]
x, y, w, h = cv2.boundingRect(max_cnt)
rect = cv2.minAreaRect(max_cnt)
rotatedPlate = plate
if not self.ratioCheck(max_cntArea, rotatedPlate.shape[1],
rotatedPlate.shape[0]):
return plate, False, None
return rotatedPlate, True, [x, y, w, h]
else:
return plate, False, None
19
return after_check_plate_img, characters_on_plate,
coordinates
return None, None, None
self.after_preprocess = self.preprocess(input_img)
possible_plate_contours =
self.extract_contours(self.after_preprocess)
# PLATE FEATURES
def ratioCheck(self, area, width, height):
min = self.min_area
20
max = self.max_area
ratioMin = 3
ratioMax = 6
if (area < min or area > max) or (ratio < ratioMin or ratio >
ratioMax):
return False
return True
ratioMin = 2.5
ratioMax = 7
if (area < min or area > max) or (ratio < ratioMin or ratio >
ratioMax):
return False
return True
21
if (height == 0 or width == 0):
return False
class NeuralNetwork:
def __init__(self):
self.model_file = "./model/binary_128_0.50_ver3.pb"
self.label_file = "./model/binary_128_0.50_labels_ver2.txt"
self.label = self.load_label(self.label_file)
self.graph = self.load_graph(self.model_file)
self.sess = tf.compat.v1.Session(graph=self.graph)
22
"""
image = cv2.resize(image, dsize=(imageSizeOuput, imageSizeOuput),
interpolation=cv2.INTER_CUBIC)
np_image_data = np.asarray(image)
np_image_data = cv2.normalize(np_image_data.astype('float'), None,
-0.5, .5, cv2.NORM_MINMAX)
np_final = np.expand_dims(np_image_data, axis=0)
return np_final
input_name = "import/input"
output_name = "import/final_result"
input_operation = self.graph.get_operation_by_name(input_name)
output_operation = self.graph.get_operation_by_name(output_name)
results = self.sess.run(output_operation.outputs[0],
{input_operation.outputs[0]: tensor})
results = np.squeeze(results)
labels = self.label
top = results.argsort()[-1:][::-1]
return labels[top[0]]
def main(location):
findPlate = PlateFinder()
23
# Store plates in plates variable
plates = []
cap = cv2.VideoCapture(location)
while (cap.isOpened()):
ret, img = cap.read()
if ret == True:
cv2.imshow('original video', img)
if cv2.waitKey(25) & 0xFF == ord('q'):
break
# cv2.waitKey(0)
possible_plates = findPlate.find_possible_plates(img)
if possible_plates is not None:
for i, p in enumerate(possible_plates):
chars_on_plate = findPlate.char_on_plate[i]
recognized_plate, _ =
model.label_image_list(chars_on_plate, imageSizeOuput=128)
plates.append(recognized_plate)
if (datetime.now() - current).total_seconds > 2:
break
cv2.imshow('plate', p)
if cv2.waitKey(25) & 0xFF == ord('q'):
break
else:
break
cap.release()
cv2.destroyAllWindows()
return list(set(plates))
24
Code for user interface:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width, initial-scale=1.0"
/>
<title>Number Plate Recognization</title>
href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.
css"
rel="stylesheet"
integrity="sha384-1BmE4kWBq78iYhFldvKuhfTAU6auU8tT94WrHftjDbrCEXSU1oBoqyl2
QvZ6jIW3"
crossorigin="anonymous"
/>
<style>
html,
body {
height: 100%;
}
body {
display: flex;
align-items: center;
padding-top: 40px;
padding-bottom: 40px;
background-color: #f5f5f5;
background-image:
url('https://eskipaper.com/images/road-wallpaper-20.jpg');
background-size: cover;
}
25
.output {
font-weight: bold;
}
.form-signin {
width: 100%;
max-width: 330px;
padding: 15px;
margin: auto;
}
.form-signin .checkbox {
font-weight: 400;
}
.form-floating {
margin: 10px 0;
}
.form-signin .form-floating:focus-within {
z-index: 2;
}
.form-signin input[type="email"] {
margin-bottom: -1px;
border-bottom-right-radius: 0;
border-bottom-left-radius: 0;
}
.form-signin input[type="password"] {
margin-bottom: 10px;
border-top-left-radius: 0;
border-top-right-radius: 0;
}
.bd-placeholder-img {
font-size: 1.125rem;
text-anchor: middle;
-webkit-user-select: none;
-moz-user-select: none;
26
user-select: none;
}
27
src="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/js/bootstrap.bundle
.min.js"
integrity="sha384-ka7Sk0Gln4gmtz2MlQnikT1wXgYsOg+OMhuP+IlRH9sENBO0LRn5q+8n
bTov4+1p"
crossorigin="anonymous"
></script>
28
DEPLOYMENT OF USER INTERFACE
29
Fig. 2.3: UI as seen by user showing the number plates found
30
CONCLUSION
● The number plate extraction is done using various techniques of image
processing. The number plate is then segmented and OCR (Optical
Character Recognition) is used to recognize the license number on the plate.
● The UI is connected with the image processing code. The UI takes in the
video from the user and sends the video to the image processing code. This
also displays the output given by the code.
● The same code can be used to extract number plates in real time. This works
to automate the entries of vehicles into campus from any gate reducing the
chance of any error in the record of entries and also reducing the effort and
time required to do so.
REFERENCES
1. https://towardsdatascience.com/build-optical-character-recognition-
ocr-in-python
2. https://docs.opencv.org/4.x/d4/d13/tutorial_py_filtering.html
3. Automatic Number Plate Recognition System (ANPR): A Survey by
Chirag Indravadan Bhai Patel.
4. Documentation of libraries in OpenCV : “https://docs.opencv.org/4.x/”
31