INDUSTRIAL EXPERIENCES ON APPLICATION OF MULTIVARIATE IMAGE ANALYSIS IN

STEELMAKING PROCESSES

Yale Zhang

Process Automation, Dofasco Inc.

Box 2460, Hamilton, Ontario, Canada, L8N 3J5

Abstract: Multivariate Image Analysis (MIA) has been developed and applied in process
industry primarily for process monitoring and control purposes. This paper discussed
some industrial experiences on application of MIA in the steelmaking industry that are
illustrated using two case studies: (1) hot mill rolled-in scale defect classification; and (2)
real-time monitoring of vacuum degasser ladle eye area. The preliminary results showed
that many benefits could be realized through these high-value added applications.
Copyright © 2007 IFAC

Keywords: multivariate image analysis, rolled-in scale, surface defect, ladle eye, vacuum
degassing.

1. INTRODUCTION by experienced operators. Continuous efforts have

Multivariate Image Analysis (MIA) technology has
been developed and applied in recent years in
process industry, primarily for process monitoring
and control purposes. In this technology, an image is
treated as a multi-dimensional matrix that includes
both spatial and spectral information of each pixel.
Multivariate statistical analysis methods, such as
Principal Component Analysis (PCA) or Projection
to Latent Spaces (PLS), are then applied in order to
extract features of interest. More details about MIA
technology and recent development can be found in
Yu, etc. (2003).
been taken to develop an off-line imaging analysis
tool to aid the operators in automatic rating of the
rolled-in scale (Chupil, 1996; Bharati and
MacGregor, 2000). The latest development focused
on using the multivariate image analysis (MIA)
technique to identify the total number of rolled-in
scale pixels in the images of clean untreated steel
surface (Bharati, 2003). Unfortunately, some
mismatches exist between the number of rolled-in
scale pixels and the rolled-in scale rating given by
the operators. This implies the total number of scale
pixels is not a dominant factor that can be used to
characterize the rolled-in scale features.

Over the past few years, Dofasco has been utilizing
this technology to develop high-value added process
automation applications. In this paper, two case
studies of applying MIA in steelmaking processes
are described in the following sections.
2. HOT MILL ROLLED-IN SCALE
CLASSIFICATION
Rolled-in scale (specifically, the fleck and wave
scale) is the most frequent and serious surface defect
encountered in the hot mill operation. At Dofasco,
the degree and the pattern of rolled-in scale are
evaluated using a surface code rating system. The
surface inspection and rating is conducted manually
The method proposed in this case study combines the
MIA technique and some morphological operations
in the conventional image analysis area. It consists of
four steps: (1) image acquisition and preprocessing;
(2) automatically detecting rolled-in scale pixels
using MIA; (3) extracting rolled-in scale features by
two newly proposed indices: Area Index and
Connectivity Index; and (4) determining the rolled-in
scale rating based on a decision matrix defined in the
Area-Connectivity index space. These steps will be
descried in detail as follows.
(1) Image Acquisition and Preprocessing. A total of
40 color images of clean untreated steel samples
were selected for this feasibility study. These images
were acquired by a flatbed scanner. Each original
image was cropped to a rectangle (in various size) to
eliminate the sample label and the black edges. The
obtained images were used in the following steps.
(2) MIA-based Rolled-in Scale Detection. Fig. 1
shows an example of cropped color image, where the
steel is shown in blue and white color, the rolled-in
scale is shown in pink color, and the black color
represents the burn marks due to the sample being
cut from the steel coil or strip. An MIA model was
developed to distinguish the pixels of rolled-in scale
from that of the steel, as well as the burn marks. As
shown in Fig. 2, MIA transforms all pixels into a
new PCA t1-t2 score space, in which the pixels
falling into a predetermined polygon mask (shown in
magenta color) are part of the rolled-in scale. These
pixels were rearranged based on their original spatial
position and coded by the white color in a black
background. The resulting binary image is shown in
Fig. 3, which highlights all identified rolled-in scale.
The features of rolled-in scale will be extracted from
this binary image.
Fig. 1. Cropped color image of untreated steel
surface with rolled-in scale (shown in pink color)
Fig. 2. T1-T2 score space and the polygon mask
determined by MIA model.
(3) Feature Extraction. Fig. 4 shows a closer look of
the binary image containing identified rolled-in
scale, where some pixels are joined together to create
a large object, and some pixels are standing alone to
form many one-pixel objects. In this case study, each
object area (i.e., number of pixels) was calculated
and its histogram was plotted. The histograms of 40
steel samples were color-coded based on their rating
given by the operators. It can be observed that certain
correlation exists between the degree of scale (i.e., 1:
no scale, 2: light, 3: medium and 4: heavy), and the
distribution of object areas. In order to characterize
this relationship, an index, named Area Index, was
defined as follows:
Area _ Index =
(1)
100
the number of objects containing i pixels
i×
i=2 the total number of objects
Fig. 3. Identified rolled-in scale pixels
Note that the i in the above definition starts from 2,
which implies that the single-pixel objects are treated
as noise in this study and therefore are filtered out.
Fig. 4. A closer look of identified rolled-in scales
In order to characterize the scale pattern, i.e., open
space fleck (OF), closed space fleck (CF) and wave
pattern (W), another index, named Connectivity
Index, was proposed based on the concept of pixel
connectivity:
4-connected: pixels are connected if their edges
touch. This implies that a pixel is part of one object
only if this pixel is connected to the object along the
horizontal or vertical direction.
8-connected: pixels are connected if their edges or
corners touch. This implies that if a pixel is
connected to one object, regardless of whether they
are connected along the horizontal, vertical or
diagonal direction, this pixel belongs to the object.
The Connectivity Index is then defined as:
Connectivi ty _ Index =
number of 8 - connected objects
1− × 100%
number of 4 - connected objects
(2)
(4) Rolled-in Scale Rating and Evaluation. Based on
the defined Area Index (AI) and Connectivity Index
(CI), the rolled-in scale can be automatically rated by
the following decision matrix (Table 1):
Table 1. Rolled-in scale rating decision matrix
RATING AI<=3 3<AI<=4 4<AI<=5 AI>5
CI<=5 1 2OF 3OF 4OF
5<CI<=16 1 2CF 3CF 4CF
CI>16 1 2W 3W 4W
A simple way to evaluate the rating performance is
to compare the automatic rating directly with the
rating determined by operators, and any difference
will be counted for an error. However, this method
may devalue the automatic rating performance if the
large variance of operator’s rating is considered.
Currently, the operators determine the rolled-in scale
rating mainly depending on human judgments. The
accuracy and consistency of this rating process
cannot be guaranteed, even though the operators are
experienced and well trained. It was reported that
one standard deviation of operator’s rating can be up
to 0.66 for the rating of scale degree (1,2,3,4) and
0.78 for the rating of scale pattern (OF=1, CF=2,
W=3).
Taking into account the variance of operator’s rating,
this study tried to evaluate the automatic rating
performance based on a given tolerance:
Zero Tolerance: the automatic rating is correct if it
is exactly same as the operator’s rating.
One Tolerance: the automatic rating is believed to
be correct if the difference between the automatic
rating and the operator’s rating is less than or equal
to 1. For example, for certain steel sample with the
operator’s rating 4W, the automatic rating 3W or
4CF are both correct according to the “One
Tolerance” rule.
The automatic rating performance based on the
defined tolerance level was summarized in Table 2.
Simulations showed very promising results: for the
selected 40 steel samples, the average accuracy rate
for the proposed method is 77.5% for the scale
degree rating (1, 2, 3 and 4) and 90% for the scale
pattern rating (OF, CF and W).
Table 2. Accuracy of automatic rating
ACCURACY DEGREE PATTERN
RATE RATING RATING
(1,2,3,4) (OF, CF, W)
Zero Tolerance 72.5% 85%
One Tolerance 82.5% 95%
3. REAL-TIME MONITORING OF LADLE EYE
IN VACUUM DEGAS PROCESS
Dofasco produces Ultra Low-Carbon (ULC) steel
through the vacuum tank degasser, in which argon
stirring from the bottom of metal bath is used to
facilitate carbon transfer and reduce the bath carbon
to ultra-low level. Ladle eye in vacuum degasser
refers to a metal-air reaction region on the surface of
metal bath, which is uncovered by slag. The area of
ladle eye is an important operation parameter to
indicate the intensity of argon stirring as well as
decarburization reaction (Kuhl, etc., 2004) At
Dofasco, two video cameras are installed at the
vacuum tank degasser roof. These cameras provide
operators with the real-time ladle eye images so that
they are able to respond appropriately to the
changing operating conditions, for example,
manually increasing the argon flow rate to promote
argon stirring. The argon flow rate is currently set by
a pre-defined time-series profile. Since real-time
quantitative measurement of ladle eye is not
available, it is difficult to achieve a desired, stable
stirring regime inside the vacuum degasser by
implementing a feedback control strategy to
accelerate the chemical reaction and improve steel
quality.
3.1 Model Development
In this case study, a process monitoring scheme was
developed based on the MIA technology to provide
online measurement of the ladle eye area. Similar
work was also reported in Brooks and Subagyo
(2003). Fig. 5 shows some sample images,
demonstrating the variations of ladle eyes at the
different time along the vacuum degassing process.
For example, the image at the top-left corner shows
multiple open eyes combined together, whereas the
image at the bottom-right corner shows no eye is
open at that moment.
 Fig. 6. Identified ladle eye area (highlighted in green)
based on the polygon mask in the MIA score plot
(showed in bottom-right corner)

3.2 Preliminary Results

Fig. 7 shows the ladle eye area profile during the

deep vacuum process based on the developed MIA
model. Some valuable observations can be obtained
from this profile: the most intensive decarburization
occurs in the middle of process, lasting a few
minutes. After that, the intensity of Carbon-Oxygen
reaction decays exponentially and the
decarburization becomes very slow at the end of
Fig. 5. Sample images of vacuum degasser ladle eye process. These observations have also been
for model development confirmed by vacuum degasser off-gas analysis (Liao
and Du, 2006).
Similar to the modeling method discussed 12
x 10
4

previously, an MIA model was developed based on

many sample images. The ladle eye area was
10
extracted from the image by defining a polygon mask
in the PCA t1-t2 score plot, as shown in Fig. 6.
Therefore, the ladle eye area can be easily computed Ladle Eye Area 8

as:
6

Ladle _ eye _ area =

number of pixels falling into the ladle eye mask 4
× 100%
total pixels of the image
(3) 2

0
0 0.5 1 1.5 2 2.5 3 3.5 4
4
Time x 10

Fig. 7. Ladle eye area profile during deep vacuum

process

Based on these observations, the pre-defined argon

flow rate has been optimized to promote Carbon-
Oxygen reaction during the second half of deep
vacuum process. This has led to over 15% reduction
of degas operation time, which implies a significant
productivity gain.

Fig. 8. Real-time monitoring of ladle eye area during deep vacuum process
An MIA-based online monitoring system has been
implemented at Dofasco’s vacuum degasser to
compute the ladle eye area every 5 seconds. Fig. 8
shows an example of three vacuum degassing
sequences, where the dash-line indicates the vacuum
degassing process state, and the solid-line indicates
the ladle eye area. It can be found that the ladle eye
area profile behaves differently from Fig. 7 after the
argon flow is optimized. This system is able to
provide operators real-time quantitative
measurements of ladle eye area. Future work will be
focused on developing a feedback control strategy to
further reduce the variation of ladle eye area and to
achieve a desired, stable stirring and reaction regime.
4. SUMMARY AND CONCLUSIONS
This paper has provided a high-level overview of two
case studies in steelmaking industry, where MIA
technology has been applied for surface defect
classification and process monitoring. The
preliminary results demonstrated the capability of
MIA technology and showed that many benefits
could be realized through these high-value added
applications. The ladle eye real-time monitoring
application also reveals a great potential for utilizing
MIA technology to improve ladle control in
steelmaking industry.
REFERENCES
Bharati, M. and J.F. MacGregor (2000), Texture
analysis of images using principal component
analysis, Proceedings of the SPIE - Process
Imaging for Automatic Control, Boston, MA,
November 2000.
Bharati, M. (2003), Monitoring rolled-in scale using
MIA, Dofasco internal report, October, 2003.
Brooks, G.A. and Subagyo (2003), Ladle eye
detection using multivariate image analysis,
ISSTech Conference proceedings, pp.843-852
Chupil V. (1996), Hot mill surface analysis project,
completion of phase 2, Dofasco internal report,
April 10, 1996.
Kuhl, T., S. Sun and M.K. Trinh (2004), Equipment
and practice enhancements at Dofasco’s vacuum
degas tank for ULC steel, Iron & Steel
Technology, 4, pp.21-27
Liao D. and K. Du, personal communication, 2006
Yu, H., J. F. MacGregor, G. Haarsma and W. Bourg
(2003), Digital imaging for on-line monitoring
and control of industrial snack food processes,
Industrial Engineering and Chemistry Research,
42(13), pp.3036-3044