CIMCA 2008, IAWTIC 2008, and ISE 2008

Detection and Classification of Surface Defects of Cold Rolling Mill Steel

Using Morphology and Neural Network

1
Mohammad Reza Yazdchi, 2Arash Golibagh Mahyari, 3Ali Nazeri
1
Department of Biomedical Engineering, Faculty of Engineering, Isfahan University, Iran
2
Department of Electronic and Electrical Engineering, Faculty of Engineering, Shiraz
University, Iran
3
Mobarakeh Steel Company, Isfahan, Iran
1
yazdchi@eng.ui.ac.ir, 2mahyari.arash@gmail.com, 3nza@mobarakeh-steel.ir

Abstract This paper is organized as follows: In section 2, the

As manufacturing speed increases in the steel
industry, fast and exact product inspection becomes
more important. This paper deals with defect detection
and classification algorithm for high-speed steel bar in
coil. We enhance an acquired image by use of a special
subtractive method and find the position of defect using
local entropy and morphology. The extracted
statistical features are then presented to a classifier.
We use neural network and fuzzy inference system as a
classifier and compare their results. The best
accuracy, % 97.19, is obtained by the neural network.
1. Introduction
In recent years, there is an increased need for
quality control in the manufacturing sector. The
aerospace and automotive industries reject any
materials with defects in manufacturing processes
because a minor defect in a manufactured part might
result in a disaster at a later stage. Without automatic
machine vision technology, steel rolling operations is
not able to perform real-time inline surface defect
inspection. The failure to identify some defects can
lead to factory down time and significant economic
losses.
Cold rolling mill steel could include different
defects. The surface defects are more common. They
appear as a variety of shapes. In this paper, we detect
the region of occurrence of five famous defects and
classify them. These five defects are: Sticking,
Emulsion Rust, Under Pickled, Dirty Surface and
Anneal Stain Defect. The defects considered in similar
papers [1,2] are different and much simpler; hence the
results obtained are not sufficient.
utilized hardware and database are introduced. Section
3 describes the enhancement method. In section 4, we
explain how the position of defect is identified by
morphology methods. In section 5, appropriate features
are extracted. Section 6 describes the classifier that has
been used. Section 7 shows the results of the
implementation.
2. Hardware and Database
Because of the high speed movement of steel sheet,
we use a line-scan camera that has a resolution of
1×4096 pixels. So, the camera can acquire details with
size more than 0.37 mm. Also, its sampling frequency
is 10.8 kLine sec , which leads to the same accuracy of
width in length.
The database we used contains 2300 images which
have been collected during eight months. These images
have been labeled by experts of cold rolling mill.
3. Image Enhancing
Because of the fast movement of the steel sheet and
its vibrating surface, the environment light is reflected
by the surface of the sheet. This appears as black and
white vertical belts. A sample image of sticking defect
that contains these belts is shown in Figure 1.
In order to eliminate these belts, several images of
a sheet were randomly selected to compose a new
image which contains complete information gathered
from the set of selected images. This image is called
the background image. We select the median of
corresponding pixels in these images and substitute it
into the same position of the background image. The
background image formed for the sheet displayed in

978-0-7695-3514-2/08 $25.00 © 2008 IEEE 1071

DOI 10.1109/CIMCA.2008.130
Figure 1 is shown in Figure 2. Since the black and
white belts are common in all images, an enhanced
quality image is obtained by subtracting the
background image from the main image. The image
displayed in Figure 1 is enhanced and the result is
shown in Figure 3.
Figure 1. A sample image of Sticking defect before
enhancement
Figure 2. The background image of Figure 1
1072
Figure 3. Image of Figure 1 after enhancement
4. Defect Detection
To find the position of defect in an image, the local
entropy of the pixels of an image is calculated. The
local entropy of a pixel is defined as the entropy of the
pixels in a 9×9 matrix around it. The local entropy
matrix for the sample image in Figure 4 is calculated
and shown in Figure 5. Considering an appropriate
threshold, the local entropy matrix is converted to a
binary image. This threshold is computed adaptively
using Otsu’s method [3]. The binary image is
presented in Figure 6. In order to eliminate nonrelated
regions, we use morphological approaches [4,5].
Subsequently the binary image is eroded by a disk
structure element which has a radius of 5 pixels. So,
areas of all white regions decrease uniformly. In the
next stage all regions with an area below 2000 pixels
are deleted. We then apply dilation operation to joint
discrete areas. Since some defects lie in a vertical and
some others in a horizontal direction, we chose a
15×15 square structure element. The application of
these operations on the image in Figure 6 is shown in
Figure 7. The white regions in Figure 7 indicate
positions of defect in Figure 4. The defected region is
easily identified and separated with the use of this
image. The final result is shown in Figure 8.
The values of the structure elements scale and
threshold are selected by experience.
Figure 4. A sample image of Sticking defect Figure 6. Result of converting Figure 5 to binary
image

Figure 5. Local entropy image of Figure 4

Figure 7. Result of applying morphological operations
on Figure 6

1073
 Figure 9. A sample image of Emulsion Rust Defect

Figure 8. Separated region of defect

5. Feature Extraction
Sample images of five defects are shown in Figure
4 and Figure 9-Figure 12. These defects have different
shapes and can be distinguished by their edges. So, we
should extract features that describe the difference
between neighborhood pixels. Appropriate features can
increase the accuracy of classification and bad features
decrease it. We chose nine features that describe these
defects. These features are as follows: entropy, Figure 10. A sample image of Under Pickled Defect
variance of the variances of the columns of the images
matrix, variance of variances of the rows of the images
matrix, median of image, average of variances of the
columns of the images matrix, average of variances of
the rows of the images matrix, average of local entropy
matrix, variance of averages of the columns of the
images matrix, power of correlation matrix of image.
All of these features are calculated over the detected
region of defect.

1074

Figure 11. A sample image of Dirty Surface Defect
Figure 12. A sample image of Anneal Stain Defect
6. Classification
After the appropriate features are extracted, they
are normalized and the vector of features is presented
to the classifier. A well designed classifier can increase
the accuracy of classification.
A three layer neural network can implement nearly
all nonlinear functions. In this paper, we use a
feedforward neural network with one hidden layer. The
transfer functions of its hidden and output layer are
tansig and the transfer functions of the input layer are
linear. We train this network by Error Back
Propagation method. There is no specific criterion for
1075
selecting a suitable number of neurons of the hidden
layer, yet this number is important to determine the
accuracy of classification. If it is chosen low, the
network does not train well and its accuracy decreases.
If it is very high, the network adjusts itself with the
learning samples, hence the accuracy of classification
for the learning samples will increase but the accuracy
of classification for the test samples will decrease
[6,7].
In many cases, data does not strictly belong to one
class and because some classes are in close proximity
to each other, the data has a partial degree of
membership to each class. In this situation, Fuzzy
Inference System (FIS) is used to classify data. There
are two famous FIS’s methods, mamdani and sugeno.
Sugeno’s method uses fewer rules to describe data,
therefore it is faster than mamdani’s method for on-line
classification. We use this method in the simulation.
To generate FIS rules, there are several approaches;
one of them is Fuzzy C-Mean (FCM) approach [8,9].
This algorithm classifies N feature
vectors X = {x1 , x2 ,..., x N } to C cluster by minimizing
Equation (1) which is called the cost function. This
minimization is done iteratively.
C n
¦¦ uijm vi − x j
2
J (U , v1 , v2 ,..vc ) = 1
i =1 j =1
Where m is the Weighting Exponent on each fuzzy
membership and determines the amount of fuzziness of
the resulting classification. m=2 would be widely used
for it is accepted to be a good choice of fuzzification
parameter [10]. v j is the cluster center.
A fuzzy c-partition of the given data set is the fuzzy
partition matrix U = [uij ] ; i = 1,2,3,..., C ; j = 1,2,..., N
and uij ∈ [0,1] is the membership of feature vector
x j to cluster vi and must satisfy Equation (2).
c
¦ uij = 1 ; j = 1,2,.., N 2
i =1
The algorithm of FCM is as follows:
Step 1) Initialize the membership matrix U and
thresholds į and Ȝ.
Step 2) Compute the cluster center using
Equation(3)
N
¦ uijm x j
vi =
j =1
3
n
¦ uijm
j =1
 Step 3) Update the memberships uij in the was enhanced by special subtractive method and the
position of defect was determined using local entropy
membership matrix with Equation (4).
−1
and morphology. Finally, neural network classifies the
§ 2 · statistical feature vector extracted from the region of
¨ C § x j − vi · m −1 ¸
¦
uij = ¨ ¨ ¨ ¸
¨ l =1 ¨ x j − vl ¸¸
¸
¸
4
defect.
All of the used methods have acceptable accuracy
¨ © ¹ ¸ and time efficiency. By slight modification they can be
© ¹
Step 4) If the difference between the last two extended to cover more defects types. Also, because of
membership matrices is lower than į, or if the value of their low computation times, they can be employed in
the cost function is lower than Ȝ, terminate the online applications.
algorithm; otherwise, return to step 2.
9. Acknowledgement
7. Experiments and Comparison This work has been supported by Mobarakeh Steel
Company under grant No. 48222675.
We used 2300 labeled images from these five
defects to simulate the methods. 75 percent of the data 10. References
were chosen randomly to train the classifier and the
remaining 25% were used as test samples. The [1] Jong Pil Yun, YoungSu Park, Boyeul Seo,Sang Woo
accuracy of classification is defined as the ratio of the Kim, Se Ho Choi, Chang Hyun Park, Ho Mun Bae, Hwa
Won Hwang,” Development of Real-time Defect Detection
number of correct classification over the number of all
Algorithm for High speed Steel Bar in Coil(BIC)”, SICE-
data. ICASE International Joint Conference 2006, Oct. 18-2 1,
In our experiments, we used a 9×13×5 neural 2006 in Bexco, Busan, Korea.
network. The above experiment was repeated 1000 [2] Hongbin Jia, Yi Lu Murphey,Jianjun Shi, Tzyy-Shuh
times with random initialization values. In each Chang, “An Intelligent Real-time Vision System for Surface
simulation, the accuracy of classification was Defect Detection” , Proceedings of the 17th International
computed for the test and training data separately. The Conference on Pattern Recognition (ICPR’04).
average and variance were calculated and the results [3] N. Otsu, "A Threshold Selection Method from GrayLevel
are mentioned in Table 1. Histogram", IEEE Trans. Systems, Man and Cybernetics, Vo
l. 9, pp. 62-69, 1979.
[4] Rafael C.Gonzalez, Richard E.Woods, “Digital Image
Table 1. Accuracies of NN and FIS classification Processing”, Prentice Hall, New Jersey, 2008.
Accuracy of the Accuracy of the [5] J.Serra, “Image Analysis and Mathematical
training data test data Morphology”, Academic Press, New York, 1982.
Ave Var Ave Var [6] Martin T.Hagan, Howard B.Demuth, Mark Beale,
Feedforward “Neural Network Design”, Thomson Learning, Singapore,
97.89% 1.8×10-5 97.19% 5.1×10-5
NN 1996.
FIS- FCM [7] Richard O.Duda, Peter E.Hart, David G.Stork, “Pattern
82.47% 1.7×10-5 82.46% 1.3×10-5
method Classification”, John Wiley & Sons,China,2001.
[8]James C.Bezdek, “Pattern Recognition with Fuzzy
Considering the results of the above table, in which Objective Function Algorithms”, Plenum Press, New York,
the average of NN is greater than the average of FIS, 1981.
we chose F.F. NN as the classifier. Also, the small [9]James C.Bezdek, “Fuzzy Mathematics in Pattern
Classification”, Ph.D. thesis, Cornell University, Applied
value of variances shows that the network is valid. Mathematics Center, Ithaca, NY,1973.
[10] J.C. Bezdek, J. Keller, R. Krisnapuram, N.R. Pal, Fuzzy
8. Conclusion models and Algorithm for Pattern Recognition and Image
Processing, Kluwer Academic Publisher, 1999.
In this paper, we considered five famous surface
defects of cold rolling mill steel. The acquired image

1076