IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO.
6, NOVEMBER 2013
Merchant Vessel Classification Based on
Scattering Component Analysis for
COSMO-SkyMed SAR Images
Hong Zhang, Member, IEEE, Xiaojuan Tian, Chao Wang, Member, IEEE,
Fan Wu, Member, IEEE, and Bo Zhang, Member, IEEE
Abstract—Ship classification in high-resolution synthetic aper-
ture radar (SAR) satellite images is a hotspot and a continuing
problem in SAR applications. The scattering components of ships
are the strong scatter of objects in SAR images, and these can rep-
resent the superstructure of different ship types. Based on analyses
of different scattering components of bulk carriers, oil tankers,
and container ships, we propose a new classification method for
these three ship types in COSMO-SkyMed SAR images. First,
morphological preprocessing is applied to suppress sidelobes.
Second, based on Hough transform (HT), the orientation of the
principal axis is extracted, and the modified minimum enclosing
rectangle (MER) of the ship is obtained and rotated along the
principal axis. Finally, the ship type is decided according to the
width ratio of MER between the HT line, the ratio of ship and
nonship points on the principal axis, and the scattering density.
The results show that this method has good performance in ship
classification, with an overall accuracy of over 80%.
Index Terms—COSMO-SkyMed, scattering component analy-
sis, ship classification.
I. I NTRODUCTION
M ICROWAVE remote sensing is the most effective way
of marine exploration. Satellite-based synthetic aper-
ture radar (SAR) sensors can provide a powerful surveillance
capability, allowing observation without the cooperation of
ships during day and night, and are independent of weather
conditions.
SAR images were used for ship detection in early SAR
application. There are presently two common ways of detecting
ships, i.e., one based on ship wake and another based on ship
features, and many mature ship detection systems are now being
applied [1]. Ship classification is also an important research
area in the field of marine surveillance. Most of the previous
literature in this area used simulated images initially due to
limitation of data and resolution. These features applied to ship
classification are based on position and shape, length, principal
axis, etc., of the scatterers [2], [3]. The Lincoln Laboratory of
Massachusetts Institute of Technology has developed a neural
Manuscript received August 29, 2012; revised December 5, 2012; accepted
December 16, 2012. Date of publication February 6, 2013; date of current
version October 10, 2013. This work was supported in part by the National
Natural Science Foundation of China under Project 40871191 and in part by
the Italian Space Agency COSMO-SkyMed Project (ID 2247).
The authors are with the Center for Earth Observation and Digi-
tal Earth, Chinese Academy of Sciences, Beijing 100094, China (e-mail:
hzhang@ceode.ac.cn; later-early@163.com; cwang@ceode.ac.cn).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/LGRS.2012.2237377
1545-598X © 2013 IEEE
1275
network based on the automatic ship classification system for
inverse synthetic aperture radar images [4]. In addition to
initial attempts at dealing with macroscale features and huge
databases of possible ship radar signatures [5], [6], recent works
have focused on exploiting the properties of polarimetric SAR.
In such cases, particular procedures based on coherent target
decomposition theorems are designed to express the complex
polarimetric scattering behavior of ships in terms of specific
combinations of elemental mechanisms. Normally, such simple
mechanisms have an associated physical meaning from which
the main geometrical properties of the observed target could
be delineated [7]. In [8] and [9], a new method based on
the symmetric scattering characterization method of coherent
target decomposition is defined, with the aim of exploiting the
advantages of coherent and incoherent theorem decompositions
[10]. The method for vessel classification based on single-pass
polarimetric SAR interferometer shows that the polarimetric
response of many types of vessels can be described by trihedral-
and dihedral-like mechanisms [11]. In 2007, the COSMO-
SkyMed and TerraSAR-X satellites were launched successfully.
The resolution of the two satellites under standard mode is
above 3 m, which makes the extraction and analysis of the fea-
tures of ships in SAR imagery and the classification of the types
in further research possible. The DeMarine Security project is
part of the Global Monitoring for Environment and Security
program, which is a joint initiative of the European Commission
and the European Space Agency (ESA). The Detection of
Artificial Objects project is integrated in DeMarine Security
and focuses on the detection and classification of ships and
offshore artificial objects, relying on TerraSAR-X as well as on
optical images. Texture features, invariant moments, or gradient
features are used for the classification of ships and nonships.
However, ship-type classification is not mentioned [12]. At
present, studies based on COSMO-SkyMed and TerraSAR-X
SAR data focus mainly on ship detection. Few research works
are related to classification work. Ship-type classification with
other SAR images is not satisfactory [13].
In general, merchant ships can be divided into five groups,
including oil tankers, bulk carriers, container ships, general
cargo ships, and other types of ships (such as ferries and
passenger ships, chemical tankers, miscellaneous tankers, etc.).
Oil tankers, bulk carriers, and container ships are three types of
mainstream ships that account for about 70%–80% of the inter-
national shipping market. In this letter, following an analysis of
the characteristics of ships in high-resolution SAR images, we
propose a new classification method based on scattering com-
ponent analysis for three common merchant ships, namely, bulk
1276 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 6, NOVEMBER 2013
TABLE I
O PTICAL I MAGES OF S HIPS AND T HEIR A SSOCIATED SAR I MAGES
carriers, oil tankers, and container ships. COSMO-SkyMed
SAR data with 3-m resolution are used. In situ experiment and
automatic identification system (AIS) data are applied to verify
the effectiveness of the algorithm.
II. S CATTERING C OMPONENT A NALYSIS IN
COSMO-S KY M ED SAR I MAGES
In SAR images, ships are always strong scatterers toward
dark clutters. The backscattering depends on several properties,
such as the structure of the ship, orientation of the ship relative
to the radar, material, motion, and SAR parameters. Even
bounce and odd bounce are the dominant scattering mecha-
nisms that are caused by surfaces, corners, edges, cables, and
the ship–sea configuration, among others. The flat and slightly
curved plates can give a strong specular reflection, reflect the
energy away from the radar, and give very little backscattering,
e.g., of the deck and hull. Thus, these are always dark areas
in SAR images. The superstructure (e.g., funnel, engine room,
and arm of the crane) can form dihedral and trihedral reflections
that account for the dominant scattering region in the image
[14]. The engine room, which is always built at stern, mostly
has sidelobes in the range and azimuth directions because of
high reflection. A considerable portion of the backscatter comes
from the intersection between the waterline and the side of ship
facing the radar, due to a large dihedral reflector that is made
up of the water surface and the side of the ship. In the case
of the other side of the hull, it gives relatively low returns.
The location of strong scatterers and low backscattering in the
SAR images can represent the type and geometric features of
the ship. If the ship is in motion, this can give distortions and
displacement in the azimuth direction.
Different ship types have different superstructures and ma-
terials, so the strong scatterers in SAR images are different as
well. In this experiment, COSMO-SkyMed X-band images with
3-m resolution are used to analyze the detailed characteristics
of bulk carriers, oil tankers, and container ships. Table I shows
typical images of all three kinds of ships. The images show that
the scattering components can clearly represent the physical
meaning of the ships’ structure.
A. Bulk Carriers
The deck and the edges of the closed hatch will form a
dihedral reflection and give strong returns. Bulk carriers always
have cargo handling gears beside the hatch. The ships are
typically fitted with derricks or deck cranes to load or discharge
cargo without assistance. The cargo handling gears are metallic,
and their shapes present approximately cylindrical form; they
can also form a dihedral reflection with the deck. Thus, both the
single bounce and the double bounce are strong in SAR images.
In SAR images, the hatches and the cargo handling gears will
form evenly bright lines, and the distances among the lines are
approximately the same and equal to the hatch width.
The intersection between the waterline and the hull that
faces the radar will form a bright line because of the dihedral
reflection, while the other side of the hull will give relatively
low returns.
The engine room forms strong scatter and always has high
sidelobes in the azimuth or range direction.
B. Oil Tankers
An oil tanker has an oil pipeline in the middle of the deck
that generally spans the total length of the ship. In COSMO-
SkyMed images, the oil pipeline forms a strong bright line from
bow to stern, and its ancillary facilities are discrete points in the
dark deck. Because the deck has few buildings on it, except the
engine room, most peaks are located at stern (e.g., the 10% most
intense pixels in the image).
The engine rooms are built at stern and always form strong
scatter with sidelobes, as we had analyzed earlier.
C. Container Ships
The backscatter signature of a container ship depends on its
laden situation. The iron containers laden on the ships have
strong reflections because of the corners among them, and the
intersection between the waterline and hull facing the radar
forms a bright line.
Container ships in SAR images can be divided into two cate-
gories: ships without cargo and fully laden ships. The openings
of the hatch of a container ship stretch the entire breadth of the
cargo holds and are surrounded by hatch coaming. On top of the
hatch coamings are the hatch covers. Cell guides are also used
in ships. If the container ship is not loaded, the hatch and cell
guide will form evenly bright lines, while the distances among
lines are smaller than in the bulk ship. If the ship is fully laden,
the intersection among the containers will show a scattering
combination of dihedral scattering and trihedral surface scat-
tering, and most parts of the hull will become strong scatter.
III. DATA S OURCES
During July 12–19, 2010, ship classification campaign was
performed in East China Sea. The fishery law enforcement
ship (No. 206) cruised in the experiment area for collection
of ship names, ship types, in situ photographs, GPS positions,
sea states, etc. COSMO-SkyMed SAR images were acquired
during the campaign. The in situ information and AIS data were
acquired and later used for identification of ships in COSMO-
SkyMed SAR images and ship classification validation. The
classification algorithm was tested with two COSMO-SkyMed
ZHANG et al.: MERCHANT VESSEL CLASSIFICATION BASED ON SCATTERING COMPONENT ANALYSIS
TABLE II
I MAGE PARAMETERS OF THE T EST I MAGES
Fig. 1. Diagram of ship classification technique.
Fig. 2. Effect of preprocessing.
SAR images acquired on July 12, 2010. Table II gives the image
parameters.
IV. M ETHODOLOGY
We divide the classification algorithm into several stages, as
shown in Fig. 1.
In this letter, the sea clutter surrounding the ship is homoge-
neous, which makes ship detection and isolation easier. Radio-
metric calibration is done first by the Next ESA SAR Toolbox.
Then, the order statistic constant false-alarm rate algorithm and
discriminator stage are responsible for ship detection and the
separation of ships from “not relevant” objects.
After the ships’ regions of interest are extracted, the clas-
sification algorithm based on scattering components is per-
formed. The classification algorithm is outlined as follows:
1) preprocessing; 2) principal axis direction extraction based
on Hough transform (HT); 3) modified minimum enclosing
rectangle (MER) extraction; and 4) ship classification.
A. Preprocessing
The sidelobes of strong backscattering in the azimuth and
range directions will disturb the extraction of the principal
axis direction and MER. We apply ⎡ a morphological
⎤ opening
1 0 1
with the structuring elements ⎣ 0 1 0 ⎦ to the segmented
1 0 1
images. Because the sidelobes are both in the azimuth and
range directions, the matrix angles at 45◦ . Then, small thin areas
are erased. The preprocessing process is prepared for the next
stages. Fig. 2 shows the difference of a tanker image before and
after the preprocessing.
1277
B. Principal Axis Direction Extraction Using HT
HT [15] is a classical approach of finding the lines in binary
image; this approach is introduced to detect parameterized
shapes in binary images and map each image point to all points
in the parameter space. Thus, each image point votes for the
shape parameters that could have produced it. The points in the
parameter space garnering the greatest number of votes, which
appear as peaks, are the most likely to have produced the shapes
in the true image. Thus, HT reduces the problem of detecting
spatially spread patterns in the image space, to find localized
peaks in a dual-parameter space.
When applied to the problem of line detection in 2-D images,
the parameter space for HT is 2-D. Originally, the slope and the
intercept were used to parameterize this space of linear features.
However, because this parameterization has a singularity for
lines with large slopes (i.e., m → ∞), straight lines may most
usefully be parameterized by their distance from the origin and
orientation of their normal vector. In this formulation, each
image point (x, y) is mapped to all values of (ρ, θ) that satisfy
ρ = x cos θ + y sin θ. (1)
This parameterization maps an image point onto a sinusoidal
curve in the 2-D parameter space. The line parameters are
estimated by determining points of intersections of these curves
(i.e., peaks) in the parameter space. Thus, according to the
detection of the local maximum in parameter space, we can get
the longest line in the image.
After sidelobe suppression, the longest line named L inside
the ship can be obtained by HT.
C. Modified MER Extraction
MER is a simple method of obtaining the object size. The
conventional approach to achieve MER is by rotating the object
equally within 90◦ and then calculating the rectangle area
and finding the minimum rectangle. The preprocessing cannot
suppress the sidelobes totally, so the initial MER extracted by
the conventional approach is always larger than the ship area. To
get the length and width of the ship, MER needs to be modified.
We take a bulk carrier for example and show the detailed steps
in Fig. 3.
1) First, detect the longest line L in the ship using HT after
preprocessing.
2) Second, find the initial MER of the ship, which is always
large enough to contain the whole ship area.
3) Third, rotate the MER to the direction of the line L. L
will divide the MER into two parts along its width, and
we name the width of two parts S1 and S2 . Then, count
the sum of bright pixels of images and divide it by the
width of MER. The average pixel numbers in each line
of the L direction can then be obtained. Here, we name it
ave_number.
4) Fourth, calculate the number of bright pixels in the line L
direction from the narrow side of MER. If the number is
less than 0.6 times ave_number, the MER is smaller until
the number is larger than 0.6 times ave_number.
5) Fifth, calculate the number of bright pixels in the line L
direction from the wide side of MER. If the number is
less than 0.5 times ave_number, the MER is smaller until
the number is larger than 0.5 times ave_number.
1278 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 6, NOVEMBER 2013
Fig. 3. Modified MER extraction. (a) Example of MER extraction of a bulk
carrier. (b) MER extraction algorithm.
6) Sixth, if necessary, repeat the process in direction of
length, and the final result of MER can be extracted.
Then, the ship area can be cropped down from the original
image.
The selection of the thresholds is empirical. Different tests
on other ship categories in COSMO-SkyMed SAR images have
confirmed that this method provides good performance on MER
extraction.
D. Ship Classification
According to the analysis of scattering components, the three
criteria used for ship classification in this letter are described as
follows.
1) Width Ratio of Each Side of MER: The longest line L
has its corresponding physical meaning in SAR images. Bulk
carriers and not loaded container ships have at least a clear side,
which is the longest line in the hull in SAR images. Thus, the
line L is the side of the ship facing the radar. For oil tankers and
fully laden container ships, it is the principal axis. L will divide
MER into two parts along its width. The rate R is defined as the
ratio of S1 and S2
max{S1 , S2 }
R= . (2)
min{S1 , S2 }
2) Ratio of Ship and Nonship Points: Some bright lines
are approximately perpendicular to the principal axis inside
the ships. In bulk carriers, the distances between the lines are
wider, which are equal to the hatch width. In container ships,
the distances are narrower and are equal to the containers’
width. MER is divided into two parts by the line L. After target
segmentation, we count the ratio of ship pixels and nonship
pixels on the principal axis of the wider parts.
3) Scattering Density: Most hulls of container ships have
backscattering, while only the oil pipeline, its ancillary facil-
ities, and the engine room have backscattering inside the hull
of an oil tanker. Thus, there are more holes inside an oil tanker
than in a container ship that is placed irregularly. The scattering
Fig. 4. Ship classification rules.
density that can represent the points’ intensity is defined as
follows:
√
2 Aπ
Scattering density = (3)
P
where A is the total pixels of the ship and P is the perimeter
of the ship. Obviously, the scattering density of the oil tanker is
smaller than that of the container ship.
Fig. 4 shows the ship classification algorithm based on
scattering components. If R > 1.5, it illustrates that L is not
located in the middle of the ship but on the side, so the ship
should be a bulk carrier or an unloaded container ship. Then,
according to the ratio of ship and nonship points, the bulk
carrier and the unloaded container ship can be classified. If
R ≤ 1.5, it means that L is located approximately in the middle
of the ship. According to the earlier analysis, the ship should
be an oil tanker or a fully laden container ship. If the scattering
density is higher than 1.6, the ship is a fully laden container
ship; otherwise, it is an oil tanker. Here, through the analyses
of slices of three types of merchant ships, the threshold of the
scattering density is set to be 1.6.
V. E XPERIMENTAL R ESULTS
Finally, 50 merchant vessels, consisting of 30 bulk carriers,
ten oil tankers, and ten container ships, were verified in two
Cosmo-SkyMed images.
Fig. 5 shows the longest line L and the MER extraction
results after sidelobe suppression. From the results, we can see
that, for bulk carriers and container ships, which are placed
smoothly, line L represents the strong backscattering caused by
ZHANG et al.: MERCHANT VESSEL CLASSIFICATION BASED ON SCATTERING COMPONENT ANALYSIS
Fig. 5. Green lines are the longest lines in the extraction results, the blue
rectangles represent the original MERs, and the red rectangles are the modified
MERs.
TABLE III
C ONFUSION M ATRIX FOR G IVEN R ESPONSES
Fig. 6. Misclassified ships: (a) and (b) are bulk carriers, (c) is an oil tanker,
and (d) and (e) are container ships.
the dihedral reflections made up of the waterline and the side
of the ship facing the radar. For oil tankers and container ships
placed irregularly, the line is the principal axis.
Most original MERs are larger than the ship areas, because
of the sidelobes in the azimuth direction. After modification,
however, the MERs contain proper ship areas in both range
and azimuth directions and can be used to calculate further
parameters (length, width, etc.).
Table III shows the classification results. From the results of
scattering component extraction and ship classification, we can
see that this new ship classification algorithm can preferably
acquire the ship orientation, MER, and other criteria, and the
overall accuracy is above 85%.
Five ships are misclassified, as shown in Fig. 6. When the
ship is fully laden, it will have a shallow draft, which will
reduce the strength of the dihedral reflector, made up of the wa-
terline and the side of ship, and increase the difficulty in finding
the side. Thus, those in Fig. 6(a) and (b) are misclassified. The
oil tanker [Fig. 6(c)] is oriented in the azimuth direction, and
the superstructures of the deck are mapped by SAR layover
to the right side of the ship’s body, which makes it difficult
to extract the scattering components. When the container ship
is not loaded or is partly loaded, the structure may vary a
lot and cannot be summarized in certain rules, as shown in
Fig. 6(d) and (e).
VI. C ONCLUSION
In this letter, we have proposed a new method to classify
bulk carriers, oil tankers, and container ships in COSMO-
SkyMed SAR images, considering their scattering components
1279
that can represent the structure, materials, orientation, and other
information of ships. The results prove that the approach is able
to classify ship types correctly for COSMO-SkyMed SAR data
with an overall accuracy of about 90%.
The experiments also have some limitations. First, there is a
need to verify the result with some more examples in further
work. Second, more features have to be included to reduce the
robustness of the algorithm. Finally, detailed physical reasoning
corresponding to the SAR mapping should be researched in
future works.
ACKNOWLEDGMENT
COSMO-SkyMed data are provided by the Italian Space
Agency in the COSMO-SkyMed announcement of opportunity
project (ID 2247). The Fishery Law Enforcement Bureau of
East China Sea Region and Shanghai Maritime Bureau are
grateful for in situ campaign and provision of automatic identi-
fication system data.
The authors would like to thank the anonymous reviewers
for their constructive comments, which significantly improved
this letter.
R EFERENCES
[1] W. Li, “Detection of ship in optical remote sensing image of median-low
resolution,” Ph.D. dissertation, Nat. Univ. Defense Technol., Changsha,
China, 2008.
[2] S. Musman, D. Kerr, and C. Bachmann, “Automatic recognition of ISAR
images,” IEEE Trans. Aerosp. Electron. Syst., vol. 32, no. 4, pp. 1392–
1404, Oct. 1996.
[3] D. Pastina and C. Spina, “Multi-feature based automatic recognition of
ship targets in ISAR images,” in Proc. IEEE Radar Conf., Rome, Italy,
2008, pp. 1–6.
[4] M. M. Menon, E. R. Boudreau, and P. J. Kolodzy, “An automatic ship
classification system for ISAR imagery,” Lincoln Lab. J., vol. 6, no. 2,
pp. 289–308, 1993.
[5] L. Gagnon and R. Klepko, “Hierarchical classifier design for airborne
SAR images of ships,” in Proc. SPIE, 1998, vol. 3371, pp. 38–49.
[6] S. R. Cloude and E. Pottier, “A review of target decompositions theorems
in radar polarimetry,” IEEE Trans. Geosci. Remote Sens., vol. 34, no. 2,
pp. 498–518, Mar. 1996.
[7] R. Touzi and F. Charbonneau, “Characterization of target symmetric
scattering using polarimetric SARs,” IEEE Trans. Geosci. Remote Sens.,
vol. 40, no. 11, pp. 2507–2516, Nov. 2002.
[8] R. Touzi, R. K. Raney, and F. Charbonneau, “On the use of permanent
symmetric scatterers for ship characterization,” IEEE Trans. Geosci. Re-
mote Sens., vol. 42, no. 10, pp. 2039–2045, Oct. 2004.
[9] G. Margarit, J. J. Mallorqui, J. M. Rius, and J. Sanz-Marcos, “On the
usage of GRECOSAR, an orbital polarimetric SAR simulator of complex
targets, for vessel classification studies,” IEEE Trans. Geosci. Remote
Sens., vol. 44, no. 12, pp. 3517–3526, Dec. 2006.
[10] J. M. Rius, M. Ferrando, and L. Jofre, “High frequency RCS of complex
radar targets in real time,” IEEE Trans. Antennas Propag., vol. 41, no. 9,
pp. 1308–1319, Sep. 1993.
[11] G. Margarit, J. J. Mallorqui, and X. Fabregas, “Single-pass polarimetric
SAR interferometry for vessel classification,” IEEE Trans. Geosci. Re-
mote Sens., vol. 45, no. 11, pp. 3494–3502, Nov. 2007.
[12] G. Saur and M. Teutsch, “SAR signature analysis for TerraSAR-X-based
ship monitoring,” in Proc. SPIE, Image and Signal Processing for Remote
Sensing, Toulouse, France, 2010, vol. 7830, pp. 78301O-1–78301O-8.
[13] Z. Xi, Z. Jie, and J. Yong-gang, “The capability analysis of ship classifica-
tion by structure feature using SAR images,” Acta Oceanol. Sin., vol. 32,
no. 1, pp. 146–152, 2010.
[14] A. O. Knapskog, S. Brovoll, and B. Torvik, “Characteristics of ships
in harbour investigated in simultaneous images from TerraSAR-X and
PicoSAR,” in Proc. Radar Conf., Washington, DC, 2010, pp. 422–427.
[15] J. Cha, R. H. Cofer, and S. P. Kozaitis, “Extended Hough transform for
linear feature detection,” Pattern Recognit., vol. 39, no. 6, pp. 1034–1043,
Jun. 2006.