See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/321533094

Sonar detection and tracking of seafloor pipelines

Conference Paper · July 2011

CITATIONS READS

3 1,350

3 authors:

Øivind Midtgaard Thomas Krogstad

Forsvarets forskningsinstitutt Forsvarets Forskningsinstitutt
51 PUBLICATIONS 327 CITATIONS 34 PUBLICATIONS 538 CITATIONS

SEE PROFILE SEE PROFILE

Per Espen Hagen

Kongsberg Discovery AS
78 PUBLICATIONS 686 CITATIONS

SEE PROFILE

All content following this page was uploaded by Øivind Midtgaard on 05 March 2018.

The user has requested enhancement of the downloaded file.

 SONAR DETECTION AND TRACKING OF SEAFLOOR PIPELINES

Øivind Midtgaarda, Thomas R. Krogstada and Per Espen Hagenb

a
Norwegian Defence Research Establishment (FFI), Kjeller, Norway
b
Kongsberg Maritime, Horten, Norway

Contact: Øivind Midtgaard, Address: FFI, P. O. Box 25, NO-2027 Kjeller, Norway,
Tel: +47 63807653, Fax: +47 63807509, Email: Oivind.Midtgaard@ffi.no

Abstract: External inspection of seafloor pipelines is presently carried out with towed or
remotely operated vehicles (ROV) operated from advanced offshore vessels, making it a
time-consuming and complex activity. In co-operation with Kongsberg Maritime, FFI is
developing a concept based on autonomous underwater vehicles (AUV) that has potential
as a cost-effective augmentation of the ROV-based inspection. The concept was
successfully demonstrated with a HUGIN 1000 AUV along a 30 km long section of an oil
& gas pipeline on the western coast of Norway in February 2011.

The AUV surveys the pipeline unaccompanied by the surface ship, which is then relieved
to perform parallel inspection with ROV. The AUV must follow the pipeline within a
cross-track range interval given by the sensor swaths. Preprogrammed mission paths may
then be inadequate, in case of large prior uncertainties in pipe route and drift in vehicle
position estimates. To ensure optimal pipe following, the pipeline should be automatically
recognized in the sensor data and the vehicle path adjusted accordingly. This paper
addresses the detection and tracking tasks when the AUV is travelling 50-100 meters to
the side of the pipeline, imaging the pipeline and its surrounding seafloor with long-range,
high resolution synthetic aperture sonar (SAS).

Keywords: Sonar, Detection and Tracking, Pipeline Inspection, Autonomous Underwater

Vehicle
1 INTRODUCTION

Seafloor pipelines are critical infrastructure for transportation of oil and gas, and timely
inspection is required to verify their integrity and determine the need for maintenance.
Failures in these pipes may not only interrupt oil and gas distribution, but also cause
severe harm to the environment. External inspection is today carried out with towed or
remotely operated vehicles (ROV); the latter are typically rolled on top of the pipeline by
specially designed wheels. These operations are conducted with large and advanced
offshore vessels, making it a time-consuming and complex activity. The objective is to
detect burial, exposure, free spans and buckling of the pipeline, as well as indications of
damages due to third party activities, such as trawling, anchoring and debris near the
pipeline.
The use of autonomous underwater vehicles (AUV) holds potential as a cost-effective
augmentation of the ROV-based pipeline inspection. AUVs are more stable platforms that
can travel faster (typically 3-5 knots, compared to 1-2 knots for inspection ROVs), and
they can perform their task without being closely followed by a large surface vessel. The
AUV can be used to acquire pipeline status information in order to identify the sections
that do not require more detailed inspection or intervention. The ROV efforts can then be
focused on the specific locations where potentially hazardous conditions have been found.
The two inspection systems can be operated in parallel from the same surface vessel, as
the AUV only depends on the vessel for launching and retrieval. Alternatively, the AUV
can be operated alone using a much smaller mother vessel.
Traditional sensors for pipeline inspection are side-scan sonar (SSS), multi-beam echo
sounder (MBES) and video camera. Pre-programmed mission paths may be inadequate for
autonomous operations, as these sensors typically have a swath (or at least a “sweet spot”)
that is narrower than the combined uncertainties of the vehicle navigation and the pipe-lay
position. To ensure optimal pipe following, the pipeline must thus be detected and tracked
in real time onboard the AUV, using data from the inspection sensor. The control system
must then adjust the vehicle path accordingly. The tracking software must handle buried
pipe sections, scenarios with multiple pipelines and a wide variety of seafloor
environments; from soft, flat mud to hard, rocky seabed.
This detection and tracking problem has previously been studied by others. In [1], pipe-
like shadow pixels were segmented in the SSS image. Straight line sections were then
fitted to these pixels and statistically evaluated using a Bayesian approach. The method
was later refined by applying a particle filter to track the shadow detections over time [2].
In [3], phase congruency and the Hough transform were used to detect shadows of
seafloor cables in synthetic aperture sonar (SAS) imagery.
Together with Kongsberg Maritime, FFI is developing a new concept for AUV-based
pipeline inspection showing that sensor data quality and area coverage can be significantly
improved by equipping the AUV with a high-resolution, interferometric SAS and a still
image camera with a multiple light emitting diodes (LED) strobe [4]. The resulting system
was first demonstrated on a 30 km long pipeline section in February 2011 in the Hjelte
fjord outside Bergen, Norway.
The proposed inspection concept is briefly described in chapter 2. The image products
and detection and tracking algorithms for side-looking sonar are presented in chapters 3
and 4, respectively. Chapter 5 presents the sea demonstration and the conclusions are
given in chapter 6.
2 INSPECTION CONCEPT

The inspection system is a HUGIN 1000 AUV equipped with HISAS 1030 SAS,
EM3002 MBES and TileCam camera. The AUV is launched from a support vessel and
programmed to follow a route 50-100 m to the side of the as-laid position of the pipeline.
HISAS provides a constant-resolution SAS swath to each side of 150-200 metres, thus
imaging the pipeline and its surrounding seafloor. Because of the wide sonar swath and
the high navigation accuracy of HUGIN, prior knowledge about the pipeline position does
not have to be very precise. The pipeline is detected and tracked in real time based on
HISAS data together with prior position data. The results are used to update pipeline
positions and adjust the vehicle path, if needed.
The AUV can survey 20-30 nautical miles of the pipeline in this way, while the support
ship performs other tasks (e.g., ROV based intervention on previously surveyed sections).
HUGIN then turns 180 degrees and returns to the launch position. On the way back, it may
travel directly above the pipeline, guided by the pipeline position data generated during
the first half of the mission. The AUV reduces its altitude to 5-10 meters above the pipe
and turns on its camera and LED strobe, to record high resolution optical imagery of the
pipeline. In addition, the MBES will be used to generate bathymetry of the pipeline and its
immediate surroundings. The data will be used to again track the pipeline in real time and
adjust the AUV path as needed, to ensure optimal imaging geometry.
Pipeline burial will be detected and mapped on the first half of the dive, so the AUV
does not have to perform time consuming searches when passing over buried pipeline
sections during the optical imaging phase.

3 SONAR DATA PRODUCTS

This paper focuses on pipeline detection and tracking in HISAS data, which can be
processed into different image products, as illustrated in Fig. 1. The high-resolution SAS
images are valuable for inspection, but not ideal for time-critical vehicle control, due to
the inherent processing delay for broad-side SAS. To minimize tracking delay, it is
beneficial to process the sonar data from each ping as soon as it has been recorded. We
thus base the detection and tracking algorithms on the lower resolution dynamically
focused SSS data (pixel spacing 52 cm x 6 cm), which is available in real time.

Fig.1: Three different image products obtainable from processing of multi-element

side-looking sonar data (e.g. HISAS 1030 data).
 Fig. 2: Sector scan imagery of two pipelines at different aspect angles near maximum
range (sonar range =185 m). 65 beams are generated within a 20-degree angular sector
from single ping data. Standard side-scan sonar would give one beam along centre of
sector as illustrated to the right of each sector scan image.
Survey data from HISAS shows that pipelines can give very strong returns (due to
specular reflection) when the AUV travels parallel to the pipeline, while the echo may be
barely discernible at other aspects angles. The shadow still indicates the pipeline, but
typically the response then must be tracked over longer time to provide a high-confidence
track. HISAS ensonifies over a 20-30 degree range of azimuth angles, however, and
beams can be formed in any direction within this swath. From just a single ping, it is thus
possible to generate a sector scan image revealing the specular highlight of the pipeline, as
long as it is within that ±10-15 degree sector from the AUV’s heading. Additionally, the
pipeline and its shadow can robustly be detected from just a single ping, as its straight line
response across the sector-scan image distinguishes it from e.g. pipe diameter sized rocks
that cause spurious detections in single ping SSS data.

4 SSS DETECTION AND TRACKING

Our detection and tracking processing chain is outlined in Fig. 3. The single ping SSS
backscatter is first filtered to reduce noise and then convolved with a deformable match
filter mask to enhance pipe-like responses. The filter is adapted from automatic detection
of seafloor mines in sonar imagery [5]. Both mines and pipes are protruding objects that
can typically be distinguished from the surrounding seafloor in sonar images as a transient
highlight closely followed by a dark shadow region. The highlight and shadow contrasts
depend, however, on factors such as the object surface, seafloor reverberation, sensing
geometry and level of multi-path signal and additive noise. The one-dimensional match
filter mask consists of four adjacent regions of different lengths: pre-highlight, highlight,
shadow and post-shadow. The average pixel values within the four regions (ypre, yhigh, yshad
and ypost) are combined to an output pixel value, ymf, as:

y high y post
y mf = min( ,1.0) (1)
y shad y pre
 Fig. 3: Pipeline detection and tracking processing steps.

The filter outputs large ymf values when the pixels in the highlight region correspond to
a sonar highlight and the pixels in the shadow region correspond to a sonar shadow. The
contribution from excessive sized highlights and shadows is subdued by multiplying with
the ratio of the post-shadow and pre-highlight values. The lengths of the four regions
should reflect a typical pipeline response, and thus depend on sonar range and input pipe
parameters, e.g. pipe diameter. Fig. 4 shows SSS images before and after enhancement.
The enhanced data is then thresholded to produce detections. Pipelines can be
distinguished from spurious false alarms (e.g. rocks) as consistent detections along an
approximately straight line over many successive pings. This can be modelled as noisy
observations of a point target moving with almost constant velocity. We can then estimate
the state vector at ping k, xk, by the discrete Kalman filter with linear dynamic model [6]:

x k = Fk x k −1 + Bk u k + wk (2)
z k = H k x k −1 + v k

Here Fk is the state transition matrix, Bk is control input matrix, uk is the control vector,
zk is the measurement vector and Hk is the measurement matrix. wk and vk denote mutually
independent, zero-mean, Gaussian process and measurement noise. Our system is defined:

r  1 dTk  (3)

x k =  , Fk =  , z k = r , H k = (1 0)
 r  0 1 

Here r is the track’s slant range and dTk is the time interval between pings k-1 and k.
For each ping, new detection positions are compared with predicted positions of existing
tracks. When a detection is associated a track, the track position is updated as a weighted
sum of the predicted and measured positions. Each track has a log-likelihood score, LLk,
which is the logarithm of the probability ratio between hypotheses H1 (track is a pipeline)
and H0 (non-pipeline) given the observations up to ping k. The track’s log-likelihood score
is updated for each ping depending on whether a new detection is associated to it [7]:

∆LLSk + ∆LLK k , track detection in ping k

LL k = LL k -1 +  (4)
 log(1 - Pd ), no track detection in ping k
 Fig. 4: Two 350m x 150m real time SSS images showing pipeline crossing and burial
(top) and pipeline in rough terrain with free spans and burials (bottom). Original image
(left), after enhancement (centre) and reported tracks imposed in green on original (right).

Here Pd is the probability of detecting the pipeline in ping k (if it lies within swath) and
ΔLLS and ΔLLK are two log-likelihood ratios calculated from the detection’s enhanced
signal amplitude and kinematics (how well its position fits the track), respectively. When
LL has reached a given threshold value, the full track is reported to the vehicle control
system as a candidate pipeline. A track is deleted if its LL drops a given threshold below
its obtained maximum value. This occurs if the track misses detections over multiple pings
or the detected positions are inconsistent with straight line movement.
5 SEA DEMONSTRATION

The proposed inspection concept was demonstrated with a HUGIN 1000 AUV on
February 9th and 10th 2011 in the Hjelte fjord outside Bergen, Norway. A 30 km long
section of a subsea pipeline going to the Mongstad oil refinery was inspected in an 8-hour,
two-pass mission at a constant survey speed of 4 knots. The procedure was repeated the
second day on a second, partly parallel pipeline of slightly smaller outer diameter.
The mission plans were created using approximate pipeline positions from standard
nautical charts (Fig. 5). The SSS detection and tracking software was implemented as a
plug-in module in the standard HUGIN payload system. It was used during the first pass
each day to provide updated pipeline positions in real time to the AUV guidance module,
which then positioned the vehicle at optimal range for HISAS imaging. The whole process
ran fully automated inside the AUV, and status data was transferred over acoustical links
to the mother ship for monitoring (Fig. 5). Returning on the second pass, HUGIN followed
on top of the pipeline tracks identified in the first pass at low altitude (4-7 m), recording
high-resolution data with its optical camera and MBES.
The inspection area proved to be very useful for system verification, as it offered a
range of different environments and pipeline scenarios (Fig. 4). The tracker had to handle
two adjacent and crossing pipelines, as well as multiple sections where the pipe was
buried beneath piles of dumped pebbles. The seafloor conditions varied from smooth to
cluttered and rough bathymetry introduced several free pipe spans in addition to large
fluctuations in vehicle pitch angle. Water depth was 180-560 m.

Fig. 5: Left: Mission plan for February 9th 2011. Top right: HUGIN 1000 AUV. Bottom
right: In-mission status display on mother vehicle, showing HUGIN path (orange line),
pipeline detections on active track (red crosses) and prior pipeline positions (dotted line).
 The operation was overall successful, as the AUV managed to keep the correct pipeline
within the HISAS swath along the full pipeline length and record high quality inspection
data. Pipeline track was quickly regained after fall-out due to burial, free spans or heavy
vehicle manoeuvring. Improved robustness to vehicle manoeuvring is currently being
developed. It was observed that detection performance is sensitive to sensor aspect angle
and the use of real time sector scan imagery may thus improve robustness. Also, using the
depth beneath the vehicle to estimate global positions of pipe tracks proved inaccurate on
sloping seafloors. Real time swath bathymetry from e.g. HISAS should preferably be used.

6 CONCLUSIONS AND FURTHER WORK

Automated detection and tracking of seafloor pipelines based on real time side-scan
sonar data has been developed and demonstrated with a HUGIN 1000 AUV on a 30 km
long section of an oil & gas pipeline. The system performed well for varied seafloor
environments and pipeline scenarios. The sea trials have provided valuable sensor data for
further improvement of system robustness and for the development of a corresponding
detection and tracking module for multi-beam echo sounder data.

ACKNOWLEDGEMENTS

FFI’s work in this project has been funded by the Research Council of Norway through
their FORNY programme. The authors thank the Mine Warfare Service in the Royal
Norwegian Navy for their assistance during the sea trials, and Even Børhaug at Kongsberg
Maritime who has developed the pipeline inspection vehicle guidance module.

REFERENCES

[1] Petillot Y. R., Reed S. R. and Bell J. M., Real time pipeline detection and tracking
using side scan sonar and multi-beam echo sounder, In MTS/IEEE OCEANS ’02,
Biloxi, MS, USA, 2002.
[2] Evans J., Patron P., Privat B., Johnson N. and Capus C., Autotracker: Autonomous
Inspection – Capabilities and Lessons Learned in Offshore Operations, In MTS/IEEE
OCEANS ‘09, Biloxi, MS, USA, 2009.
[3] Isaacs J. C. and Goroshin R., Automated Cable Tracking in Sonar Imagery, In
MTS/IEEE OCEANS ’10, Seattle, WA, USA, 2010.
[4] Hagen P. E., Børhaug E. and Midtgaard Ø., Pipeline Inspection with
Interferometric SAS, Sea Technology, volume 51 (6), pp 37-40, 2010.
[5] Dobeck G. J., Fusing Sonar Images for Mine Detection and Classification, In SPIE
Conference on Detection and Remediation Technologies for Mines and Minelike
Targets IV, volume 3710, pp 602-614, Orlando, FL, USA, 1999.
[6] Gelb A., Applied Optimal Estimation, The Massachusetts Institute of Technology
Press, 374 pages, 1974.
[7] Blackman S. and Popoli R., Design and Analysis of Modern Tracking Systems,
Artech House, 1230 pages, 1999.

View publication stats