Maritime Avoidance Navigation, Totally Integrated System (MANTIS)

Image, Speech & Intelligent Systems Group Department of Electronics and Computer Science Southampton University SO17 1BJ Email: tt97r@ecs.soton.ac.uk

T Tran , C J Harris, P A Wilson

1

Abstract
A collision avoidance system is proposed to improve the e ciency and safety of marine transport, namely Maritime Avoidance Navigation, Totally Integrated System (MANTIS). The principle behind its operation is to remove the di culties and uncertainties involved in marine navigation through a system structure which makes marine transport deterministic - reminiscent of Air Tra c Control. The key features of MANTIS involve; localisation of vessel states and its environment (LVSE), Automatic Collision Avoidance Advisory Service (ACAAS), an Integrated Display System (IDS), Path Planning and Scheduling Service (PPSS), and Automated Ship Guidance and Control (ASGC).

Keywords: marine navigation, fusion, adaptive, modelling, control, fuzzy, expert.

1 Introduction
Ship collisions have occurred from when the rst ships were set a oat. The problem has escalated due to increases in tra c, speed and size of present day vessels. Unlike road tra c, there are generally no boundaries constraining what path a ship may take moving between any two points. As a result there are situations where navigation schedules of two or more ships overlap - giving potential for collision. It is important to understand the process and demands required of the ship operator during navigation to establish the problem areas 5]. These areas need to be targeted and improved upon for safety and e ciency of ship operation. Navigators must collect information that is required for navigation from sensory and data sources. The number of independent sources of information means it is di cult for operators to sustain continuous monitoring. This leads to slow response times and mistakes. There is a need to integrate all information which is delivered independently.
1 2

Information collection

Supported by grants from RACAL Research and EPSRC School of Engineering Science, Southampton University.

Information analysis

Most information is presented to the navigator in its raw form. Due to limitations in humans analysing ability it is impossible to analysis and digest all the available data. Consequently, navigators are more concerned with their immediate situation (i.e. the most dangerous ship) and pay insu cient attention to the global surroundings or future potential predicament. There is a need to deliver the e ective information in an easily understood way for rapid situation assessment to ease decision-making.

Decision making

Predictive analysis of the situation is very important, and is traditionally based on visual observation which can often be di cult to extrapolate (e.g. in fog). Test results show that sea mariners response for any given situation are subject to a number of physical and psychological factors. Their inconsistency causes uncoordinated actions between mariners because neither can be certain of the other's intent. There is a need to automate or aid the decision making process deterministically and to display to the mariner the most appropriate collision avoidance action. The collision avoidance action is a very complex one and causes a high work load for the navigator. He has to decide on the timing and operational qualitities of the actuators and consider external environmental forces and the maneuverability of own ship. Throughout he has to pay attention to the behaviour of other ships while deciding the timing to release the actuators. There is a need to automate or aid the collision avoidance action by controlling or advising the movement of actuators.

Execution of Collision Avoidance Action

1.1 The MANTIS solution

The underlining cause of the majority of marine collisions can be put down to human error, and it has been shown that human error is directly related to work load 1]. Thus by minimising the human work load the room for error is reduced. Unfortunately, for economic reasons there is a continual reduction in the number of human operators, many of which are poorly trained 2]. To counter this adverse e ect, the only viable solution is to increase the level of automation in all areas of ship operation. From the above analysis, a system to improve marine safety can be identi ed and needs to consist of the following parts:
Localisation of Vessel States and its Environment (LVSE). Provide accurate and robust navigational information (position, velocity) of all ships, and information on sea depth, current and wind states. Con dence intervals also needs to be given for each data value. Path Planning and Scheduling Service (PPS). Safe and e cient navigational routes are generated by considering other ship paths and environmental conditions before the journey starts; thus minimising journey time, and more importantly, the event of close encounter situations. Automatic Collision Avoidance Advisory Service (ACAAS). For unforeseen or dynamic events, potential risk situations are resolved using a knowledge base system which comply with Collision Regulations (COLREGs) 3]. The algorithm needs to be capable of dealing with complex multiship encounter situations in an intuitive and predictable manner. Integrated Display System (IDS). These provide decision support and visualisation of collision avoidance advise. By superimposing danger zones 4]and/or encounter on scheduled course line 5] on-top of an Electronic Chart Display Information System (ECDIS). Automated Ship Guidance and Control (ASGC). Given the general collision avoidance advice from ACAAS, this subsystem calculates the precise trajectory via way-points for the ship to navigate,

Satellite

Ship A

Vessel Traffic Centre

Ship B

Figure 1: Diagram showing the communication between the Vessel Tra c Centre and all operational ships. within the constraints of ship dynamics and environmental conditions. Automation and control of ship rudder and engine revolution can be made allowing the ship to smoothly interpolate between way-points. At present some of these areas are only partially satis ed via Vessel Tra c Services (VTS) and electronic navigational aids. The contribution of VTS to navigational safety is in its ability to coordinate tra c ow to minimise tra c density in speci c areas 6]. Navigation aids such as Automatic Radar Plotting Aid (ARPA) and ECDIS at their present state allow e cient navigational support with regard to speed and accuracy of calculation and gives an e ective graphical display of own-ship immediate disposition in relation to target vessels, obstacles and land 4]. MANTIS is reminiscent of an Air Tra c Control system. The structure and deterministic approach to navigation provided by MANTIS minimises the uncertainties which causes uncoordinated vessel actions. Even potential risk situations which are unforeseen are made deterministic via collision avoidance advice. And if need be, automatic control of ships can be made. The system architecture is a fundamental part of MANTIS. A range of distributed sensors are used to provide a rich data pool. The data is integrated in two stages - locally on-board the vessels giving local consensus features and global fusion at the Vessel Tra c Centre (VTC) giving global consensus features. Adaptive sensor and ship models are used for estimation and prediction. The combination of these methods ensures that accurate and robust consensus information can be provided for picture compilation and collision avoidance computation. PPSS, ACAAS and the way-point guidance aspect of ASGC are functions of the VTC, IDS and ship control are handled by the on-board ship computer. Communication is made via satellite using ship-toshore data exchange topology, gure 1. Data transmitted to the VTC consist of locally fused navigational data sent by each vessel or external marine sensor. The VTC transmits global consensus information to all ships and any way-point modi cations to individual vessels. With reference to gure 2, consider any arbitrary ship j . On-board sensors on the ship gather data about the ship and its environment y1 ; y2 ; ; ys ]T . Sensor models transform these measurements into a set of common features x1 ; x2 ; ; xs ]T and compensates for noise components, this is combined with ^ estimates from the ship model using the extended Kalman lter to form local consensus features xj . The common feature set consist of ship states, wind, sea and current states. 3

1.2 MANTIS architecture

Input into the VTC consist of local consensus features from all vessels and external marine sensors At the VTC, chart data is used to compliment depth and land features integration. ^ The output from the global fusion process forms the global consensus feature set x. This information is fed back to all vessels to update their local feature states. The VTC also uses this information to assess whether any collision risk exists between the ships and if necessary the collision avoidance action or decision d is generated prompting an alteration of vessel course via a subset of modi ed way-points P = xi ; yi; Ui]n=+m, where xi ; yi are the way-point absolute position and Ui is the traveling speed i n advised moving from the previous way-point to way-point i, nt is the initial way-point of the avoidance manoeuvre and m are the number of way-points necessary to execute the avoidance manoeuvre. The guidance and control subsystem determines the course and velocity change required for the vessel to reach a designated way-point p = x; y; U ], the outputs u = c; nc ]T are rudder angle and shaft revolution commands to the ship actuators. Prior to the voyage, or when a complete route reassessment is needed, given the vessel's present position, the nal destination point and journey time, J = x0 ; y0 ; xd ; yd ; t] the navigator formulates his navigation plan as a set of way-points for the whole journey, P = xi ; yi ; Ui ]N . To aid this process, the i=0 path planning and scheduling service which contains update information on the tra c situation and sea states can help advise navigators on this task. Data of all vessel routes (way-points) are actively stored in the Global way-point database.

^ ^ ^ x1; x2 ; ; xn ]T .

2 On-board vessel processing

On-board vessel processing involves modelling of the ship and sensor dynamics, estimating the ship and environment states and guidance and control of the ship. All these tasks are interlinked as shown in gure 3. Adaptive networks (indicated by blocks with feedback) clearly plays a key part in this local framework. Neurofuzzy and neural networks have been shown to be capable of modelling any system within an arbitrary accuracy 7]. However, both types of networks encounter problems when the system to be modelled is highly complex, consisting of multiple inputs and outputs. The computational cost increases dramatically by the order of mnk , where n is the number of inputs, m is the number of outputs and k is the number of rules. Training these networks (parameter adaption) in real-time has so far been limited to small super cial problems. Given this di culty, structure adaptation, where the rule base is automatically grown and pruned have been given little attention. Ship and sensor modelling and control are dynamic and complex tasks. There characteristics change with time and environmental conditions. In this research, a transparent realtime adaptive network is presented, capable of modelling highly non-linear problems in realtime with stucture and parameter adaptation. This has made it possible for the implementation of the control structure shown in gure 3. The network consists of B-spline membership functions and linear output functions (with respect to the inputs), see gure 4. The outputs share the same antecedents of fuzzy rules which allow correlations to be made between the outputs. In addition, the number of adjustable parameters are drastically less than the case, if multiple Adaptive Neural Fuzzy Inference System (ANFIS) networks were employed for the same number of outputs. Linear order B-splines are used as membership functions for the following reasons: When these are combined with linear rule outputs, smooth quadratic network outputs are generated. B-splines have compact support, which means that only a prede ned number of splines are activated at any one time. This is particularly important for robustness during on-line learning and 4

2.1 Adaptive modelling

Real world Ship j Environment

Sensor 1

Sensor 2

Sensor 3

Sensor s

y1
Sensor model 1

y2
Sensor model 2

y3
Sensor model 3

ys
Sensor model s

x1

x2

x3

xs update

Ship model

Local fusion On-board vessel processing

Ship guidance and control

^ xj
Display assessment parameters

ECDIS

p
Local way-point database

^ x1

^ x2

^ xn

Chart data ^ x

Global fusion

Vessel Traffic Centre

J
Path planning and scheduling

Global way-point database

Collision avoidance

dj

Way-point modification

Pj

Pj

Figure 2: MANTIS architecture 5

Control diagram for local on-board processing

Disturbances from wind and waves Environment and sensor noise Measurements Ship Controller Rudder and engine revolution Ship True states Sensors error Sensor model Model outputs andacobians J Ship model Desired ship heading and speed error Estimated ship states Guidance Update Extended Kalman Filter

Error back propagation

Vessel Traffic Centre

Figure 3: On-board data fusion control architecture.

x 01 x 02 f 41 x 31 f 42 x 32
f11

x 01 Inputs x 02

x 11
f12

f 21 x 21 f 22 x 22 f 23 x 23 f 24 x 24 Weights

f 43 x 33 f 44 x 34 f 41 x 35 f 42 x 36 f 43 x 37 f 44 x 38 Linear rule output functions f 51 x 41 Outputs f 52 Summation x 42

x 12
f13

x 13
f 14

x 14 B- membership spline functions

Figure 4: A two input, two output adaptive network structure. 6

mn2 . Thus is dependent of the size of the rule base k.

dramatically improves the computational e ciency of the network. Its e ciency is in the order of

The number of adjustable parameters is much less than any other type of membership function, and its evaluation and derivative are simple to compute. B-splines have inherent desirable properties, they form a partition of unity. This has meant that the normalisation layer usual associated with neurofuzzy networks is not required. The input space is -complete; meaning that there is always a spline activated with value greater or equal to . Moderate fuzziness; the overlap between splines is not excess, thus one spline always dominates. This ensures that the representation remains meaningful. For a general multi-input, multi-output system, (1) where y is the network outputs, i is its inputs and are the network parameter set. The network is trained using input i and output y data pairs, Df = i(k); y(k)] are required. The hybrid learning rule which combines least squares estimation and error back propagation is used to update the linear and nonlinear network parameters, respectively. This is achieved through minimisation of an error function E , typically, Additional B-splines are inserted using Activity-based Structure-level Adaption paradigm (Lee 1991). It is observed that if a network is not exible enough (having too few adaptable parameters) to learn a particular problem, then the membership function outputs (weights) will continually uctuate. A measure of weight wi uctuation per unit time is given by the Walking Distance (WD) de ned as, where is a memory term, and i is the parameter vector of weight wi . A new spline should be inserted if, where is some threshold value. And the error derivative with respect to the walking distance is proportional to,

y(k) = f (i(k); )

E (k) = jjd(k) ? f (k)jj2

WDi (k) = WDi (k ? 1) + (1 ? )jj i (k) ? i (k ? 1)jj @E @WDi WDi >

@E / @wi @E @WDi @ xi @wi

Sensor modelling

Sensor modelling is needed for estimation of the ship states via the extended Kalman lter where both its output and Jacobian is required. The superiority of this network over ANFIS is fully exploited in this application where a single network can be used to model all the sensors on-board the ship. From equ. 1 the sensor model can be written as, (2) Where h is some function containing the network parameter vector , the inputs x into the network are the ship states and the outputs z are the estimated sensor measurements. In the simplest case this can 7

z(k) = h(x(k); )

be viewed as a coordinate transformation from ship features to sensor coordinates, e.g. in the case of a radar system, from Cartesian to polar coordinates. However, should the sensor characteristics change during its operation (e.g. due to temperature e ect, atmospheric e ects), the network will adapt on-line to compensate for these changes and can therefore remove bias e ects. To train the sensor model network, data pairs Dh (k) = x(k); z(k)] are needed. Where z = zradar zGPS zINS : : :]T are measurements from the real sensors.

Ship modelling

The ship model is required for state estimation via the extended Kalman lter and its Jacobian is also needed to update the controller parameters. The inputs into the model, i = x; u]T , consist of the ship states x and actuator control inputs u, and the output y are the updated ship states x(k +1). The model f is thus represented by, The ships states, x = u; v; r; x; y; ]T consist of velocities in body xed coordinates, its position in Cartesian coordinates, and heading, respectively. The inputs, u = c ; nc; d]T , are commanded rudder

x(k + 1) = f (x(k); u(k); )

(3)

angle, engine revolution, and sea depth. The ship characteristics can change depending on its load, and changes in the sea state, thus the bene ts of on-line adaptive networks are again of great asset in this application. Training data consist of current and past ship states and control commands, Df (k) = x(k ? 1); u(k ? 1); x(k)].

Ship controller

The objective of the controller is to determine the control action u(k) that would minimise the di erence between the desired ship states xd (k) and the actual ship states x(k). The ship controller is trained using specialised learning which is a direct method of minimising the system error by back propagating error signals through the ship model. Inverse learning can also be used, which has the advantage that the Jacobian of the ship dynamics are not required, however minimisation of the network error (control action) does not guarantee that the system error will also be minimised. The controller network g is de ned as follows where is a parameter vector to be updated. Substituting into equ.3 gives, The criteria or error measure given below also penalises the amount of control action used,

u(k) = g(xd(k); x(k); )

(4) (5)

x(k + 1) = f x(k); g(xd (k); x(k); ); ]

E(k) = e(k)T Qe(k) + u(k ? 1)T Ru(k ? 1) (6) where e(k) = (xd (k) ? x(k)). Two diagonal matrices Q and R are used to weight the ship states and

control action. Back-propagation is used to update the controller parameters to minimise the error measure. Disturbances a ecting ship motion come from; wind, wave and sea current 10]. All are dependent on the local wind conditions. Wind and wave disturbances result in external forces acting on the ship. For slowly varying forces the ship actuators can compensate for these rst order e ects. The sea current can 8

2.2 Disturbances

be treated as an additive term on the velocity of the ship. It remains to be seen whether on-line adaptive networks can compensate for these e ects or whether additional input terms such as wind speed Uw and direction w are needed as inputs into the network. The extended Kalman lter is used for on-line state estimation of the ship's non-linear dynamics. The ship and sensor model outputs are combined with sensor measurements to predict future states. From equ. 3 and 2 the system f and sensor models h are,

2.3 Extended Kalman lter

where w(k) N (0; Q(k)) is the system noise and ship modelling error, and v(k) N (0; R(k)) is sensor noise and sensor modelling error. The noise covariance matrices Q and R can be obtained from their respective network error residues E. For robustness a pre lter should be used to remove surplus measurements from sensor readings and to detect sensor failure.

x(k + 1) = f (x(k); u(k)) + w(k) y(k) = h(x(k)) + v(k)

(7)

3 Vessel Tra c Centre

The fusion of local features xj from ships and external sensors to form global consensus features is ^ achieved using the standard Kalman lter. The process combines similar features j optimally taking into account their error covariance Qj . Furthermore, Qj is adjusted to take into account delays between feature extraction and the nal fusion process. A simple linear system model is used for propagation of the states. Imaging sensors giving data on land and xed objects are integrated with electronic chart data. Sea states such as current and wind velocities are measured, estimated and predicted for use by the Path Planning and Scheduling Service. In situations where there is potential for collision the VTC noti es the navigator and advises him of the avoidance procedure. The advise can be derived either from a human operator and/or expert system. The expert knowledge-base is constructed from collision avoidance regulations (COLREGs). The following highlights the importance of COLREG for collision avoidance 8]: There is worldwide acceptance and understanding of its general procedures for avoiding collision. The Regulations are acknowledged (in its formulation) to contain a distillation of historical navigational experience. With continual improvements and speci c guidance to re ect current state of development, thus the Regulations can be assumed to re ect the present optimum practice in the inexact art of marine navigation. The Regulations can be easily interpreted as a series of production rules (IF-THEN statements). A necessary requirement of a collision avoidance system is its predictability. Devising an avoidance route which optimises a mathematical function may produce time and spatially e cient paths but these paths may be non-intuitive and thus hard to foresee by other ships in the vicinity - causing uncoordinated 9

3.1 Global fusion

3.2 Automatic collision avoidance advisory service

ship manoeuvres. Here a number of heuristical stages are used making up the expert. The transparency (interpretability) of the knowledge-base allows the avoidance advise given by the expert to be validated.
Target ship classi cation 5]. Each target ship is classi ed with respect to own ship as being either; clear - no threat whatever alteration of course own ship makes, restricting - prevents own ship from performing speci c manoeuvres, threat - collision potential if both ships maintain their current speed and course. Restriction on own ship movement. For restricting ships determine the constraint they impose on own ship movements. For example the restricting ship may prevent own ship from turning to starboard or port, and/or, changes in own ship speed may cause problems astern or ahead. Encounter type. For restricting and threatening ships classify the encounter type relative to own ship. e.g. own ship overtaking, target crossing starboard to port, head-on, etc. Risk stage 9]. For threatening ships determine their current level of risk against own ship, i.e. developing, manoeuvring, critical. These categories determines which actions are permitted in accordance with COLREGs. Collision avoidance advice. Given the risk stage, encounter type and constraint imposed on own ship movements, the expert determines the most appropriate action to proceed, i.e. starboard, port alterations and/or speed alterations. The nal avoidance advice is purposefully simple.

Given the expert collision avoidance advice, the next task is to generate a subset of way-points in the general direction permitted. Constraints on ship manoeuvrability and environmental conditions are considered. Furthermore, rule 8 of COLREG states, any alteration of course and/or speed be large enough to be readily apparent to another vessel... (and) a succession of small alterations of course or speed should be avoided. A major part of the VTC is to supply the navigator with information such as tra c density and weather conditions allowing them to best plan their journey. If the journey is planned correctly then potential hazardous situations are avoided and journey time and fuel will be minimised. The advisory service may also suggest a route if required, or on re ection, object to the navigator's planned route for safety reasons. Given the set of way-points xd (k); yd(k)]N=1 , Line of Sight (LOS) guidance can be used to direct the ship k in the desired direction of travel 10]:
d = tan?1

3.3 Way-point modi cation

3.4 Path planning and scheduling service

3.5 Guidance law

Once the ship lies within a circle of acceptance with radius 0 around the way-point xd (k); yd (k)] the next way-point can be selected xd (k + 1); yd(k + 1)].

yd(k) ? y(t) xd (k) ? x(t)

(8)

10

Dangerous area

Collision avoidance route

Scheduled course line Own ship Target ship

Figure 5: Danger zone situation assessment display

4 Integrated display system

An appropriate display of the current and predicted future situation is essential to help the navigator in the decision making process. Information should be delivered to the human operators with the aim of improving navigation safety, i.e. the display is easy to understand and interpret and is expressed in a manner consistent with the method used to navigate the ship. ECDIS have been shown to be an e ect tool for understanding the ship current predicament. Evaluation and visualisation of future predicaments are possible using situation assessment displays, and by overlaying these displays on top of ECDIS gives an integrated display system. Two types of situation displays for integration in ECDIS are considered here; danger zones 4] and encounter situation on the scheduled course line 5]. The modi ed course as the result of the collision avoidance advise can be visualised and validated by either one of these display types which helps to reassure the navigator of the advice given by the system. To reduce clutter of the display a de nable number of target ships purposing the greatest threat can be set.

4.1 Danger zones

Basically the task of collision avoidance is to keep a de ned zone around own ship free. Traditionally a circle around own ship is used with radius equivalent to the permissible closest point of approach CA . The safety circle moving along with own ship gives no useful information for collision avoidance advise in ECDIS. A more initiative approach is to de ne boundaries in the marine environment where own ship should not encroach, known as 'danger zones' .

4.2 Encounter situation on scheduled course line

The scheduled course line of own and target ship are drawn on the display. The target ship position at the distance of closest point of approach (DCPA) of the encounter situation is shown and the ship symbols are red when own ship crosses the bow of target ship, yellow when own ship crosses the stern of target ship, and white during passing or overtaking encounters.

5 Summary
The problems with the present situation in marine navigation have been discussed giving provocation for this research. In this paper a system has been proposed to improve the e ciency and safety of marine 11

Target ship

Scheduled course line

DCPA

Target ship

Own ship Own ship

Crossing situation

Passing situation

Figure 6: Encounter situation on scheduled course line assessment display. transport by alleviating these identi ed problem areas. An overview of the architecture and components of MANTIS have been given.

References
1] Shinya Nakamura & Kuniji Kose. The strategic maneuvering to avoid collision. Yusen Marine Science Inc. Hiroshima University, Japan. Maritime Collision and Prevention, Vol 2. Edited by J.S. Zhao, P.A. Wilson, Z. Hu & F. Wang. September 1996. p172-181. 2] P.Barber. The need for improved curriculum development in marine simulation training. MARSIM '96. Marine Simulation and Ship Manoeuvrability. Edited by M.S. Chislett. p77-85. 3] A.N. Crockcroft and J.N.F Lameijer. A guide to the Collision Avoidance Rules. BH Newnes. Fifth Edition, 1996. 4] J. Froese & S. Mathes. Computer-assisted collision avoidance using ARPA and ECDIS. Maritime Collision and Prevention, Vol 2. Edited by J.S. Zhao, P.A. Wilson, Z. Hu & F. Wang. September 1996. p200-213. 5] Y. Ishioka, K. Kose, H. Kobayashi, C. Yang, H. Yamda & S. Nakamura. A study on a support of decision-making for collision avoidance in INS. MARSIM '96. Marine Simulation and Ship Manoeuvrability. Edited by M.S. Chislett. p49-58. 6] T. Degree. The management of marine tra c by VTS and VTMIS. The International Conference of Preventing Collision at Sea, pages 1-6, 1996. 7] J.-S.R. Jang, C.-T. Sun, E.Mizutani. Neurofuzzy and Soft Computing. Prentice Hall, 1997. 8] G.P. Smeaton and F.P. Coenen. Developing an intelligent marine navigation system. Computing & Control Engineering Journal, March 1990. p95-103. 9] H. Hilgert & M. Baldauf. A common risk model for the assessment of encounter situations on board ships. Maritime Collision and Prevention, Vol 2. Edited by J.S. Zhao, P.A. Wilson, Z. Hu & F. Wang. September 1996. p136-148. 12

10] T.I. Fossen. Guidance and Control of Ocean Vechicles. Wiley, 1994.

13