You are on page 1of 6

Paint Coating Quality Inspection in Bridges Using Image Recognition

and Neural Networks

S.A.Dhivhya1 Aishwarya Murali2
II year Computer Science and Engineering
R.M.D Engineering College,Kavaraipettai.
sa.dhivhya@gmail.com1 aishwaryamuralimv@gmail.com2

Abstract conventional computer programming techniques that

could be used to automate the observation and scoring
The most established technique of Visual Inspection of process suffered from a lack of flexibility, and were not
painted body surfaces of bridges is a very time particularly robust.
consuming and labor intensive process. Inaccurate or
subjective assessment techniques have been identified as By using a Back propagation type of neural network to
a critical obstacle to effective infrastructure or perform the quality-scoring operation, a system can be
constructed facilities management. This paper illustrates constructed that captures the expertise of human
a more objective and reliable assessment method to inspectors, and is relatively easy to maintain and update.
improve the conditions of the infrastructures or the To improve the performance of the system, algorithmic
quality of constructed facilities. The proposed system will techniques can be coupled with the above approach to
automate the coating assessment process by using simplify the problem.
computers to analyze digital images of the areas to be
assessed based on the Back Propagation Network (BPN) The main thrust of that research was the non-destructive
System in Neural Networks. The paint-Quality can be evaluation of bridge coatings using several techniques,
inspected in four simple steps. namely: Color Visible Imaging, Electrochemical
Impedance Spectroscopy (EIS), and Infrared Imaging.
1. Introduction. The results were not reliable and inconsistent. The
research result presented a generic framework, but was
not developed to the implementation stage.
Engineers need accurate, consistent, and reliable
information regarding paint system performance for the
effective management and rehabilitation of bridge.Such 3.Base Concepts applied in our Coating
information includes quantitative paint system ratings and Quality Assessment Model
accurate estimates of the useful life of the coating system.
The extent of coating failure is most important in 3.1. Artificial neural networks
determining when a coating is to be repaired. Currently, Conventional computers concentrate on emulating human
the means available in the evaluation of coating thought processes, rather than actually how they are
performance is limited. Visual rating of the coating achieved by the human brain. Neural computers, however,
condition, which represents the most established take an alternative approach in that they directly model
technique, is plagued by a number of significant the biological structure of the human brain and the way it
problems. processes information (although at a much simpler level).
This necessitates a new kind of architecture, which, like
2.Earlier Paint Quality Assessment the human brain, consists of a large number of heavily
Techniques interconnected processing elements operating in parallel
manner. Such an architecture is now both technically and
Conventionally ,a laser beam was made to fall on the commercially feasible to be deployed on a standard
painted panel and on to a projection screen. Since the computer (from the laptop and desktop to the mainframe)
light source is a coherent beam (has constant phase and is certain to increase in general usage.
relation), the amount of scatter observed in the reflected
image of the laser provides an indication of the quality of Neural networks are mathematical models, originally
paint in bridges. In the past, inspection of scatter-pattern inspired by biological processes in the human brain. They
was performed primarily by humans, because are constructed from a number of simple processing
elements interconnected by weighted pathways to form performing nontrivial mapping functions is the Back
networks. Each element computes its output as a non- Propagation Network.
linear function of its weighted input. When combined into
networks, these processing elements can implement 3.2. Back propagation (BPN)
arbitrarily complex non-linear functions which can be
used to solve classification, prediction or optimization
problems .An artificial neural network or simply a neural A BPN network is designed to operate as a multi-layer,
network is a system based on the operation of biological feed forward network, using supervised mode of learning.
neural networks, in other words, is an emulation of The basic back propagation model is a three-layered
biological neural system. forward architecture. The first layer is the Input Layer, the
second layer the Hidden Layer, and the third layer the
Output Layer. Each layer contains a group of nodes that
A Neural Network will have 3 basic parts which are are linked together with nodes from other layers by
connections among the nodes. Layers are connected only
1)Input Layer 2)Hidden layer and 3)Output layer. to the adjacent layers. The network is a feed-forward
network, which means a Unit’s output can only originate
Although computing these days is truly advanced, there from a lower level, and a unit's output can only be passed
are certain tasks that a program made for a common to a higher level.
microprocessor is unable to perform; even so a software
implementation of a neural network can be made. An The Input Layer receives the features of the data that are
Artificial Neural Network is an adaptive system that entered into the neural network. If n feature values are to
learns to perform a function (an input/output map) from be entered into the Input Layer, then there must be n
data. Adaptive means that the system parameters are nodes; where n is the number of features supplied to the
changed during operation, normally called the training net. The values are passed to the Hidden Layer through
phase. connections from the Input Layer. The nodes in the first
layer distribute the individual inputs to all of the nodes in
the Hidden Layer. The main function of the hidden layer
is to process the input data during the network training to
connect to the output, i.e. to do the regression process.

The connections between the layers are weighted to

emphasize or de-emphasize the relative value of the input.
Each node sums the n weighted inputs to the Hidden
Layer. The value of the summation can be different for
each node, due to differently weighted connections
between the first and second layers. The values that are
summed in the hidden nodes are then passed to the nodes
in the Output Layer via another set of weighted

Fig 1:Neural Network for Back Propagation Algorithm

After the training phase the Artificial Neural Network RESULT
parameters are fixed and the system is deployed to solve
the problem at hand (the testing phase). The Artificial
Neural Network is built with a systematic step-by-step BACK
procedure to optimize a performance criterion or to follow NETWORK-
some implicit internal constraint, which is commonly PROCESSING
referred to as the learning rule. An Artificial Neural STAGE
System (ANS) that is found to be useful in addressing
problems requiring recognition of complex patterns and
Fig 2: BPN System for Paint Coating Quality Assessment
Fig 3:Steps involved in Paint Quality Inspection
Each hidden node is connected with each output node. The following steps are framed to assess paint coating
The Output Layer generates the output of the neural Quality:
network. In the same way the Hidden Layer functions, the
Output Layer's input values are summed and the Step 1: DATA ACQUISITION
summation becomes the output value. The weighted
inputs are summed and used as the node output value. To automate the paint inspection process, a digital video
Several Output Layers can exist for a back propagation system was easily substituted for the human visual
network. Because the Back propagation algorithm is an system. The input video image was run through a digital
iterative algorithm, it can be trained by adjusting the video frame grabber to record a snapshot of the reflected
connection weights among the nodes. The weights are laser image. To keep the size of the network needed to
recalculated after every complete cycle until the weights solve the problem manageable, we elected to take 10
converge and the mean square error falls within the sample images from the snapshot centered on a region of
specified acceptable range. the image with brightest intensity
After acquisition the images are transferred to the
5.Model of Paint Quality Inspection System computer on site or in a remote office via any
communication protocol.
By using a BPN to perform the quality-scoring
application, we have constructed a system that captures Step 2: PREPROCESSING
the expertise of the human inspectors, and is relatively
easy to maintain and update. To improve the performance Here image analysis techniques are used to analyze and
of the system, we have coupled algorithmic techniques to enhance the image by applying algorithms such as
simplify the problem. Now, let us see the system that is filtering and edge detection. Moreover, image pre-
developed to address this application. processing is used to obtain the numerical parameters of
the image such as the gray level and brightness in a
numerical format. At this stage a statistical pattern
recognition algorithm is utilized to identify defects
according to the numerical representation of images. The
DATA ACQUISITION result of this classification is then passed to the third step.

The algorithm that is used to accomplish this pre-

processing stage is the K-Means Algorithm. This
algorithm self-organizes data to create clusters. This
algorithm fits the coating inspection application well
because of the number of clusters need to be
predetermined, which is the case in the inspection
process, are: defects and non-defects. It is used for
classification because it best fits this specific application
in that the number of clusters to classify must be


ANALYSIS After being trained, neural networks are used to identify

defects in the images by assigning binary variable of 0 or
1 for each pixel in the image. During the network training,
the neural network is fed with different images and their
parameters such as the pixels' gray levels. The network is
also fed mapped values of 0 or 1 for each pixel value. The
network will learn to assign the binary variable 0 or 1 for
different scenarios according to image parameters.
Table 1: Image Gray Levels

Pattern recognition is done and the numerical values are

Step 4: ASSESSMENT RESULTS classified into two clusters: rust or no rust. K-Means
algorithm is used for classification because it best fits this
The Assessment Results stage, where quantitative specific application in which the number of clusters to
measures of defects are obtained from the output of the classify must be predetermined.Given the data set
previous stage. From mapped output of the neural obtained from the image numerical presentation ,the K-
network, the whole image is represented as 0's or 1's. The means Algorithm is applied.
1 values represent the defect; hence, defects can be
identified and quantitatively measured. The system can be V= {245,21,90,260,18,45,54,56,57}
trained to identify different types of defects according to
the specific application. Any two values are selected randomly as centre.
Let us assume that the initial centers selected are R1=245
Once, the network was constructed (and trained), 10 and R2=56
sample images were taken from the snapshot using 2 Next, we assign each sample to the class with the closest
different sampling techniques. In the first test, the samples center, i.e. to class 1 if it is closer to R1=245 and to class
were selected randomly from the image (in the sense that 2 if it is closer to R2=56
their position on the beam image was random); in the .The following table shows the difference between each
second test, 10 sequential samples were taken, so as to pixel value and the initial two centers and hence to which
ensure that the entire beam was examined. In both cases, class it is assigned. The difference is calculated based on
the input sample was propagated through the trained the gray level, not based on the location of the pixel.
BPN, and the score produced as output by the network
was averaged across the 10 trials. The average score, as Pixel Difference Difference Class 1 Class 2
well as the range of scores produced, were then provided Value from R1 from R2
to the user for comparison and interpretation. 245 0 189 X
21 224 35 X
90 155 34 X
6. Examining Rust on a Steel Bridge- 260 15 204 X
An Example 18 227 38 X
45 200 11 X
The following example illustrates how a simple 3 pixel x 54 191 2 X
3 pixel mage could be used to illustrate the methodology. 56 189 0 X
57 188 1 X
STEP 1: Data Acquisition:
Table 2: First Iteration of K-Means Algorithm
A digital image of a steel bridge coating area is obtained
and transferred to the computer. C1={245,260}
STEP 2:Preprocessing
Image analysis software is used to enhance the image and
to represent the image in numerical format. Consider the
figure to be an image of a part of a rusted steel bridge 3
beam with a coating on. The darker areas represent defect 245 21 90
object (rust) and the lighter areas represent background.

260 18 45

1 2 3 1

54 56 57
1 245 21 90
1 2 3 X
2 260 18 45 Fig:4: Graph Representing Initial Clustering
Initial Clusters with centers R1=245 and R2=56
3 54 56 57
Next, each cluster's samples are averaged to obtain new
optimal centers.
1 2 3
New center R1 = ∑ Gray levels values in cluster1
Number of points in cluster1 1 1 0 0

=(260+245)/2 2 1 0 0

=252.5 3 0 0 0

New center R2 = ∑ Gray levels values in cluster1 Table 4:Rust Identification in the Image
Number of points in cluster1
STEP 3:Analysis
Data from the image shown will be used to train the
=42.28 neural network. The purpose of using the neural network
is to accommodate for errors that may occur when
The same procedure is applied again to find the distance depending only on statistical pattern recognition.
between each sample Adjustments are made using experts’ knowledge and then
the neural network is trained in order to automate the
Pixel Difference Difference Class 1 Class 2 process in the future.
Value from R1 from R2
245 7.5 202.7 X The complete neural network will consist of three layers:
21 231.5 21.28 X the input layer, the hidden layer, and the output layer. The
90 162.5 47.72 X input layer consists of the image data and has four
260 7.5 217.72 X elements: The x-axis location of the pixel, then the y-axis
18 234.5 24.28 X location of the pixel, the value of the gray level of the
45 207.5 2.12 X pixel, and finally the difference between the gray level of
54 198.5 11.72 X the pixel and a predetermined threshold value. The
56 196.5 13.72 X predetermined threshold value will be determined by
57 195.5 14.72 X experts’ knowledge of coating inspection in coordination
Table 3: Second Iteration of K-Means Algorithm with results from image analysis

C1={245,260} STEP 4:Assessment Results

Quantitative measures of defects are obtained from the
output of the previous stage. From mapped output of the
neural network, the whole image is represented as 0's or
1's. The 1 values represent the defect; hence, defects can
be identified and quantitatively measured. The system can
be trained to identify different types of defects according
to the specific application.
245 21 90
2 7.Advantages
260 18
Neural network based paint quality inspection systems
have the ability to learn from experience, to improve their
54 56 57 performance and to adapt their behavior to new and
changing environment. Hence commonly occurring
1 2 3
X imperfections in paint can be detected easily utilizing the
Fig:5: Graph Representing Initial Clustering gained experience.
Secondary Clusters with centers R1=252.5and R2=42.28
Finally, each sample is assigned a binary variable of 1 if it Unlike conventional rule-based systems, neural networks
belongs to cluster 1 and 0 if it belongs to cluster 2. are not programmed to perform a particular task using
rules. Instead, they are trained on historical data, using a
learning algorithm. The learning algorithm changes the
functionality of the network to suit the problem by
modifying the values of the connection weights between
processing elements.

The time needed to develop a neural application is often [1] Neural Networks and their Applications in Industry
less than that in a conventional approach, since the Dhruba J Sarma and Susmita C Sarma-DESlDOC
interaction between the analyst and the expert is Bulletin of Information Technology, January & March
minimized-there are no algorithms or rules to define. 2000

[2]Using Images Pattern and Neural Networks for

The scope and accuracy of the finished application is Coating Quality Assessment-
improved since the neural computer can be exposed to L.M. Chang and Y.A. Abdel Razig
many more examples than can be assimilated by a single
human. This system will make the assessment process [3] Freeman, James A. & David, M. Neural networks:
more objective, quantitative, and reliable. Algorithms, applications, and programming techniques.
Skapura Addison-Wesley Longman, Inc.
[4] De La Blanca, N. (1992). Pattern recognition and
The basic concept of this paper is to automate the Image Analysis. World Scientific.
assessment process by taking digital images of the rust
areas and analyzing the images to identify and measure [5] Wikipedia Online Encyclopedia
coating defects. Neural Networks do not require explicit
programming by an expert and are robust to noisy,
imprecise or incomplete data.

Furthermore, knowledge is encapsulated in a compact,

efficient way that can easily be adapted to changes in
business environment. It saves time and reduces human
burden. Hence, the application of Neural Network and
BPN technique is very crucial and of extremely
significant value. . The system will make the assessment
process more objective, quantitative, and reliable. This
vital technology can never be overlooked