biocybernetics and biomedical engineering 38 (2018) 71–89

Available online at www.sciencedirect.com

ScienceDirect

journal homepage: www.elsevier.com/locate/bbe

Review Article

Medical image registration in image guided surgery:

Issues, challenges and research opportunities

Fakhre Alam a,*, Sami Ur Rahman a, Sehat Ullah a, Kamal Gulati b

a
Department of Computer Science & IT, University of Malakand, Dir (L), Khyber Pakhtunkhwa, Pakistan
b
School of Computer Science and Information Technology, Stratford University, VA, USA

article info abstract

Article history: Multimodal images of a patient obtained at different time, pre-surgical planning, intra-
Received 5 July 2017 procedural guidance and visualization, and post-procedural assessment are the core com-
Received in revised form ponents of image-guided surgery (IGS). In IGS, the goal of registration is to integrate
18 September 2017 corresponding information in different images of the same organ into a common coordinate
Accepted 6 October 2017 system. Registration is a fundamental task in IGS and its main purpose is to provide better
Available online 18 October 2017 visualization and navigation to the surgeons. In this paper, we describe the most popular
types of medical image registration and evaluate their prominent state-of-the art issues and
Keywords: challenges in image-guided surgery. We have also presented the factors which affect the
Medical image registration accuracy, reliability and efficiency of medical image registration methods. It is not possible
Image registration methods to achieve highly successful IGS until all the issues and challenges in registration process are
Image-guided surgery identified and subsequently solved.
© 2017 Nalecz Institute of Biocybernetics and Biomedical Engineering of the Polish
Academy of Sciences. Published by Elsevier B.V. All rights reserved.

diagnosis and accurately resect the tumor based on image

1. Introduction
Over the last few decades, image-guided surgery (IGS) and
radiotherapy has brought several changes in the way patients
are treated and operations are performed. IGS is now an
alternative to conventional surgery because it provides several
advantages such as minimum risk of tissue damage, improved
localization and targeting, shorter procedures through in-
creased visualization of surgical field and improved hand-eye
coordination. The availability of these techniques provides
several benefits to the patients such as reduced surgical
trauma, fast recovery and reduced hospital stay and cost [1,2].
Nowadays, IGS techniques assist clinicians in qualitative
information. The availability of precise image information
during IGS is due to the integration pre-operative images
(images of a patient obtained before surgery) and intra-
operative images (images of a patient obtained during surgery)
with navigation technology.
Real time visualization of interested anatomical regions in
medical images during surgical process is an essential
requirement for IGS system [3]. The required images of the
interested anatomical regions are obtained through high
resolution 3D scans such as magnetic resonance imaging
(MRI), ultrasound (US), computed tomography (CT) and
positron emission tomography (PET). Similarly, multiple
images are obtained in different time-frames or from different

* Corresponding author at: Department of Computer Science & IT, University of Malakand, Dir (L), Khyber Pakhtunkhwa, Pakistan.
E-mail addresses: fakhrealam@uom.edu.pk (F. Alam), srahman@uom.edu.pk (S.U. Rahman), sehatullah@uom.edu.pk (S. Ullah),
kgulati@stradford.edu (K. Gulati).
https://doi.org/10.1016/j.bbe.2017.10.001
0208-5216/© 2017 Nalecz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences. Published by Elsevier
B.V. All rights reserved.
72 biocybernetics and biomedical engineering 38 (2018) 71–89
angles of the same subjects. Generally, information in the
individual images are not enough for accurate diagnoses and
need to be aligned to reveal high information. IGS relies on
preoperative images to provide visualization and facilitate
surgical navigation within the human organ. During surgery,
preoperative images are matched against their correspon-
dences in the intraoperative patient volumetric images using
the process of registration. More specifically, alignment of
corresponding information in the same underlying tissue or
patient's organ for analysis, visualization and navigational
guidance is performed through the process of registration in
image-guided surgery [4]. IGS involves the creation of precise,
patient-specific models of relevant anatomy for the surgical
process, and registering the models and corresponding image
data to the patient [5]. In IGS system, the registration process is
maintained over the course of therapy.
Successful IGS greatly depends on the registration of
preoperative image data with intraoperative physical anato-
my. In image-guided surgical system, preoperative image based
techniques are integrated with intraoperative guidance system.
In registration process, the same coordinate frames and
anatomical structures in both preoperative image (prior to
surgery) and intraoperative image (during surgery) are mapped
to each other. Successful IGS also needs multiple time
conduction of procedure i.e. obtaining images of interested
regions at multiple-time frames or with multiple scanners.
Therefore, it is not a good practice in IGS to rely on a single image
obtained either at the beginning of procedure/in the same time-
frames or with a single modality. The quality of obtained images
is further improved with alignment and registration. These high
quality and more informative images help surgeons to accu-
rately locate region of interest while the surgery is in progress.
In spite of numerous research development and clinical
use, image registration procedures undertaken in IGS are still
need further improvement. Moreover, a continuous research
interest is seen in this area due to strong dependency of IGS on
image registration methods. It is therefore necessary to
develop more advanced methods in the area of registration
having capability to accurately and efficiently align medical
images. In this regard, researchers and clinical practitioners
need to come forward and work on some of the prominent
issues and challenges in the area of medical image registra-
tion, which are being presented in this survey paper.
Therefore, the objective of the paper can be summarized as:
(a) to give an overview of medical image registration types,
putting emphasis on their role in image-guided surgery. (b)
Additional emphasis has been given to the issues and
challenges in the field and their possible solutions and
research guidelines are presented. (c) In order to study medical
image registration in depth, most recent state-of-the-art
knowledge is presented in a systematic fashion. The contri-
bution of this survey paper is to provide comprehensive
knowledge on medical image registration in a systematic
manner, focusing on the issues and challenges. Rest of the
paper is organized as follows: Section 2 presents general
concepts and methods used for the registration of medical
image in IGS. Section 3 describes issues and challenges in the
available registration methods for IGS, while Section 4
presents some solutions and techniques to cope with these
issues. Concluding remarks are presented in Section 5.
2. General concepts and methods
Image-guided surgery (IGS) involves anatomical images of a
patient (obtained preoperatively) and patient physical space
(intraoperative images). Preoperative images spatially localize
pathology which are further aligned with corresponding
features of interest in intraoperative images of a patient in
operating room. The alignment preoperative and intraopera-
tive images greatly help surgeon in proper guidance during
surgery [6]. Moreover, registration methods also quantitatively
compare medical images taken at different times, from which
information about evolution over time can be inferred [7].
These information also help surgeons in the monitoring of
tumor growth over time.
The combined information obtained from multiple input
images (preoperative and intraoperative images) provide a
larger field of view and better image quality than that available
in one type of image (e.g. intraoperative images). On the
completion of registration process in IGS, all the interested
information in the preoperative images is mapped to the
intraoperative patient anatomy. As a result, surgeons get
combined information intraoperatively in operating room
which helps them in surgical guidance and treatment. Fig. 1
shows surgical view of preoperative diagnostic images and
intraoperative navigation in prostate surgery [8]. In the figure,
image (a) is operating view of the surgeon while image (b) is
preoperative T2-weighted MRI images overlaid onto stereo-
endoscopic intraoperative camera view. The preoperative T2-
weighted MRI images contain prostate and the tumor location
which were in use clinically for preoperative assessment by
the surgeons.
Every modality has its own features to extract different
types of information from human organs i.e. magnetic
resonance image (MRI) and computer tomography (CT) extract
anatomical structures while positron emission tomography
(PET) and single-photon emission computed tomography
(SPECT) access functional information [9,10]. The integration
of both functional and anatomical information is always
required in IGS and greatly helps surgeons in diagnosis and
treatment planning.
The main aim of registration is to find the geometrical
transformation and mapping between two or more images in
order to obtain maximum information [11–13]. Registration is
an iterative process, source image is transformed to target
image, similarity measures between them is computed and
the resultant image is generated if the similarity measures are
fulfilled. The process is repeated for the optimization of
transformation parameters if the measure of similarity
between source and target image is not perfectly aligned.
The mapping of 2D and 3D source and target images can be
expressed as
tðx0 ; y0 Þ ¼ gðTðsðx; yÞÞÞ (1)
tðx0 ; y0 ; z0 Þ ¼ gðTðsðx; y; zÞÞÞ (2)
where t is fixed target image, s is moving source image, g is
intensity mapping function, T is special transformation, x0 , y0 ,
z0 are coordinates of source image and x, y, z are coordinates of
 biocybernetics and biomedical engineering 38 (2018) 71–89 73

Fig. 1 – Surgical view of intraoperative video data and preoperative diagnostic images in prostate surgery. In the figure, image
(a) is operating view of the surgeon while image (b) is preoperative T2-weighted MRI images overlaid onto stereo-endoscopic
intraoperative camera view. The T2-weighted MRI images contain prostate and the tumor location which were in use
clinically for preoperative assessment by the surgeons.

target image. For the transformation of 2D images, the
coordinates are x0 , y0 and x, y while for 3D images, the coor-
dinates are x0 , y0 , z0 and x, y, z for source and target images
respectively.
Image registration algorithm involves four main steps: (1)
features detection, (2) feature matching, (3) transform model
estimation and (4) resampling and transformation [14]. The
result of registration depends on these four steps. Fig. 2 shows
the process of registration from input images to the resultant
registered image.
In medical image processing, feature detection is per-
formed in a task specific manner, usually as product of a
segmentation preprocessing step. Important and desirable
features in source and target images are detected in the first
step of registration during surgical procedure. These features
consist of diagnostic information about anatomy and pathol-
ogies from different types of organs. In image-guided surgery,
the accurate detection of desirable features i.e. tumors ensure
successful removal of abnormal tissues. Advancement in
imaging modalities and registration methods lead to the
precise detection of abnormal tissue (tumor) in its early stages.
Moreover, these techniques also improve the diagnostic
procedures and facilitate better staging and preoperative
planning. Accurate Information in feature detection step is
also important for the surgeon because he/she further uses the
obtained information for object recognition and matching.
Therefore, proper detection of features during registration
process is essential for successful IGS.
The second step in image registration is feature matching,
in which the corresponding detected features in source and
target images are mapped to each other. In feature matching
step, the correspondence among similar features between set
of input images is established [15]. The correspondence is
made through intensity distribution in a neighborhood of
every pixel in both source and target images [16]. In IGS
applications, the features are matched based on similarity
measures among corresponding anatomical and pathological
information in preoperative and intraoperative image data
sets. In these applications, precise transfer of corresponding
information between preoperative image and intraoperative
patient image is crucial but a challenging task. Poor matching
can result alignment errors and corresponding information
can be transferred to the wrong position, with potentially
serious clinical consequences for the patient.
Transform model estimation is another important step of
medical image registration in image-guided surgery. In IGS,

Fig. 2 – The process of image registration.

74 biocybernetics and biomedical engineering 38 (2018) 71–89
the process of patient registration involves calculating a
transformation that maps corresponding points between
preoperative data sets to the patient's physical anatomy.
The transformed parameters are estimated before surgery
which minimize the distance between two images. The
transform model estimation is performed by first locating
features/point landmarks on the patient's intraoperative
images, then matched them to patient's preoperative images
and the transformation relating the two point sets is
calculated [17].
Transformation depends on the situation, it can range from
a simple rigid transformation (translation and rotation) to a
non-rigid transformation (affine, elastic and fluid). Rigid
transformation was a perfect choice for the estimation of
minimum deformation in intra-subject registration. However,
in case of complex deformation in preoperative and intraop-
erative imaging, high dimensional transformation methods
are required. Some of the popular transformation methods for
registration of preoperative and intraoperative images include
Splines, finite element models (FEM) and optical flow based
methods [18]. The selection of appropriate transformation
method is crucial because the deformation and appearance
change in the resultant image after the registration greatly
depends on it [19]. For accurate and reliable estimation of
deformation in images, a priori known information about the
image distortions, degradation and errors is essential for the
chosen transformation model [20]. In case of un-availability of
the required information, the model should be flexible and
general enough to cope with all types of possible degradation
problems.
Finally, the process of image resampling is performed
which geometrically transform the coordinates of source
image into the target image using mapping function [21,22]. In
IGS, several types of images of a patient are obtained i.e.
preoperatively and intraoperatively with different resolutions
and coordinate systems. The process of resampling brings all
of them into a common coordinate frame. In registration,
resampling uses transformation function, which extract and
interpolate the coordinates of pixel locations in source image
(e.g. preoperative) to the geometry of the target image (e.g.
intraoperative). The transformation is performed on the basis
of computed pixel values in each image region. These pixel
values are computed with proper interpolation techniques and
the transformation of each pixel values is performed from source
image to the target image according to the estimated mapping
function. Thus, information in both images is properly mapped
and a more informative registered image is generated.
2.1. Classification of medical image registration
The existing medical image registration methods are classified
into feature-based, intensity-based, segmentation-based and
fluoroscopy-based, as shown in Fig. 3. The detail is given in the
sub-sections below.
A. Features/landmark based registration
Image features such as landmark points, lines, edges and
curve are the abstract representation of an image and show its
behavior. Image features are extracted from raw pixel values in
Fig. 3 – Classification of medical image registration.
the image because they are more robust and easily processed
than raw pixel values [23]. Different types of cues such as color,
shapes and texture are used to represent image features.
Images contain local and global features in which the earlier
covers a large portion of the image while the later one focuses
on the specific portion. Local and global features are extracted
with different types of techniques for the analysis and
registration of medical images.
Feature-based registration consists of correlating fiducials
and/or anatomical landmarks present in both preoperative
image and intraoperatively on the patient's organ. It is the
method of choice in current commercial neurosurgery
navigation systems. The process of registration is performed
by the transformation of corresponding features (e.g. blood
vessel outline) between preoperative image and intraoperative
patient image [24]. The transformation is performed on the
bases of best similarity measures between preoperative image
and intraoperative image data i.e. 2D–3D and 3D–2D images.
Feature-based registration approaches are computationally
efficient because transformation is based on the analytical
values of geometric points landmarks. Furthermore, these
approaches also show high robustness to illumination
changes and are better suited for large displacements.
However, in IGS and real time applications, the available
feature-based registration methods could not properly extract
specific features such as blood vessel outline from clinical
images i.e. 3D–2D and 2D–3D registration. The extraction and
matching of corresponding features in the preprocessing step
and the manual and semi automatic specification of land-
marks made these approaches less accurate. Images with large
homogenous areas and appearance variations are mostly
registered with feature/landmarks-based methods. Fig. 4
shows the landmarks-based registration and fusion of retinal
images with large homogenous areas and appearance varia-
tions. As can be seen from Fig. 4, the landmarks in the source
image are mapped with their corresponding landmarks on the
target image. After landmarks matching and transformation,
the registered and fused image is shown on the right side of
Fig. 4.
The three most popular methods for the implementation of
feature-based registration are surface-based, point-based and
curve-based registration [4]. In surface-based registration, the
 biocybernetics and biomedical engineering 38 (2018) 71–89 75

Fig. 4 – Landmarks based registration and fusion of retinal images.

surfaces (structural information) of one image (source image)
are matched with the corresponding surfaces of another image
(target image) and are transformed accordingly. In this type of
registration, a dense set of corresponding points between two
surfaces is determined. The proper finding of such corre-
sponding points is usually challenging because the surface
may undergo large deformations, and sometimes there might
be missing data, such as unexpected holes and different
boundary locations, in the surface [25]. Fig. 5 shows the
deformable surface-based registration and fusion of endo-
scopic 3D movie clip of pharyngeal surface (a) and a 3D CT
image (b) of the same region of a patient having head and neck
cancer. The surface-based registration of (a) and (b) will permit
fusion of the endoscopically available information about the
size of tumor on the pharyngeal surface with the same
information detected in the CT image. In Fig. 5(a) and (b), a
large deformation between two surfaces can be seen which is
caused by the swallowing process and posture change of a
patient. Due to the limitation of endoscopic procedure, a part
of the pharyngeal anatomy is visually inaccessible by the
camera. Images in Fig. 5(c) and (d) are obtained by the color-
coded alignment of a CT surface and a real reconstruction. The
reconstructed surfaces in registered images are only partial
surfaces with respect to CT surface and pharyngeal surface.
Moreover, many holes are visible in the surfaces of registered
images, which show that surface-based registration is not
suitable for images between modalities with large deformations.
Rigid and deformable methods are available for the proper
implementation of surface-based medical image registration
[4]. Rigid methods for surface-based registration extract the
same anatomical structure surfaces from source and target
images. These image surfaces are used as input for the
registration process. In the registration process, the same
image surfaces in both images are detected and iteratively
transformed. This transformation is continuously performed
until the closest fit between the two equivalent surfaces is
found. Rigid surface-based registration methods are widely
used in medical diagnostics but are mostly prone to errors for
convoluted surfaces. Deformable methods for surface-based
medical image registration extract surfaces from one image i.e.
source image and elastically deform them to best fit in another
image i.e. target image. The deformable surfaces are also
called curves and these curves are implemented as snakes or
active contours. Deformable methods for surface based
medical image registration are successfully applied to inter-
subject and atlas registration. However, the main drawback is
the requirement of good initial pre-registration for proper
convergence.
Several types of registration algorithms are available which
extract the corresponding structure surfaces by various
segmentation techniques. These corresponding structure
surfaces are used as unique features in the registration
process. Among the available algorithms, head and hat is
one of the most popular methods for the registration of rigid
body multimodal images [26]. Head and hat algorithm
determine the two surfaces of same region in source and
target images. The first surface is obtained from higher
resolution image while the second surface is extracted from
lower resolution image. The first surface is called as ‘‘head’’
and is represented as a stack of disk. The second surface is

Fig. 5 – Deformable surface based registration and fusion of the pharynx images of a patient with head and neck cancer: (a) is a
segmented CT image; (b) An endoscopic video reconstruction; images (c, d) are color-coded correspondences between a CT
surface and a real reconstruction. Registration of these images will permit fusion of endoscopically available information
about the tumor extent on the pharyngeal surface with the tumor information seen in the CT, thereby improving the
radiation plan [25].
76 biocybernetics and biomedical engineering 38 (2018) 71–89
called as ‘‘hat’’ and is represented as a list of unconnected 3D
points. In this algorithm, iterative transformation is performed
on head surface with respect to the hat surface. This process is
repeated until the closest fit of the hat onto the head is found.
The computational cost of head and hat algorithm is low and
its segmentation task is easy, however, it is prone to error for
convoluted surfaces. The iterative closest point (ICP) algorithm
is another popular method for rigid body surface-based image
registration [27]. This method is used with several types of
geometrical primitives such as point sets, line sets, curve sets
and surfaces. The transformation and distance between a
point set and a surface is calculated iteratively until the
convergence of metric to minimum value.
Point-based registration is another popular method for the
registration of corresponding anatomical point landmarks in
the pre-operative data sets and patient's physical anatomy.
Point-based registration involves determining the coordinates
of corresponding points in preoperative images and intraop-
erative anatomy and computing the geometrical transforma-
tion that best matches these points [28].
Anatomical point landmarks are fiducial markers, either
intrinsic or extrinsic attached to the patient's body. Extrinsic
markers are used as a reference points for registration and
intrinsic markers are the anatomical landmarks or pixel values
in the image. A point-based registration method is broadly
adopted in commercial navigation system and is mostly used
in existing IGS system. This method properly aligns preopera-
tive image data (MRI, CT images) with intraoperative physical
anatomy [29]. An optimal match is often achieved between
interested features (point landmarks) of preoperative image
data and intraoperative physical anatomy. The popularity and
common use of point-based method is due to the well
established intraoperative workflow. Moreover, in this type of
registration method, registration error properties such as
fiducial localization error (FLE), fiducial registration error (FRE)
and target registration error (TRE) are extensively analyzed [30].
Despite its success, physical contact with multiple point
landmarks over the patient physical anatomy using a
navigated instrument is an issue. As a result, surgeon often
needs to create a bigger exposure than required for minimally
invasive surgery. Moreover, the extreme care in the handling
of tracker reference body and single time conduction of
Fig. 6 – Preoperative image to intraoperative patient anatomy registration using point land marks (fiducial markers) methods
in a typical IGS system. Image (a) is preoperative CT image obtained before intervention while (b) show physical anatomy of
the patient.
procedure at the beginning of image-guided surgery are some
other issues in point-based registration method.
Fig. 6 [31] shows preoperative image to intraoperative
patient anatomy registration using point landmarks (fiducial
markers) methods in a typical IGS system. The registration
process begin by the determination of interested points
(fiducial markers) in the preoperative image and intraopera-
tive physical anatomy of a patient with tracked pointer once
the patient is immobilized on the operating table, as shown in
the figure. In the next step, the preoperative image is
converted into the coordinates of physical anatomy of a
patient with spatial transformation mechanism. In the last
step, surgeon point on the physical anatomy of a patient using
a tracked pointer and see the corresponding location in the
images on the computer screen.
Feature-based registration methods also involve the
extraction, matching and transformation of curves in both
source and target images. Curves are also called snakes or
active contours and are geometrical features, which provide
important information of the image structure. Curve-based
methods are also called elastic methods and are mostly used
for the registration of deformable tissues [9]. Curve-based
registration methods perform feature matching and geometric
transformation independently in several regions of input
images. Curve-based registration computes similarity mea-
sures and mutual information both locally (on the sub image)
and globally (on the whole image). Curve-based registration
methods are more flexible and apply translation, rotation,
scaling and shearing parameters to objects during transfor-
mation of source and target images. Furthermore, these
methods deform slice into volume and transform multi-modal
3D images with high accuracy. Curve-based methods are also
suited for inter-subject and atlas registration [4]. However,
proper initial pre-registration and precise extraction of start
point and end point are essential to achieve successful
registration. Fig. 7 shows feature-based registration of brain
MRI images using points, curves and counters [32]. Registra-
tion is performed on the basis of seven point-pairs (red dots
inside (a) and (b)), one counter-pair (outside the region of (a)
and (b)) and four curves-pairs (blue curves inside (a) and (b)). In
the pre-processing step, the difference image between target
and source images is obtained, as shown in Fig. 7(c). Fig. 7(d) is
 biocybernetics and biomedical engineering 38 (2018) 71–89 77

Fig. 7 – Feature based registration of brainMRI images using points, curves and counters: (a) target MRI image; (b) source MRI
image; (c) difference between target and source images; (d) resultant registered image; (e) difference between (a) and (d); (f)
the mesh of transformation function.

the registered image obtained from the proper mapping of
points, contours and curves in source and target images. Fig. 7
(e) is difference of target image and resultant registered image
while (f) show a grid of transformation function. This grid is
obtained as a result of applying the registration method to a
regular grid and shows the form of deformation. Feature-based
registration in Fig. 7 uses non-uniform cubic B-splines inter-
polation to model the extracted curve-pairs. Interpolation
makes the parameter space almost the same in the extracted
curve-pair. After interpolation, the extracted curve pairs are sub-
divided into a set of points (according to curvature and length of
curves) to achieve best similarity measure between curves.
B. Intensity-based image registration
Intensity-based registration is currently the most widely
used method in which image intensity i.e. scalar values in the
image pixels or voxels is considered for registration. Registra-
tion methods based on image intensity directly operate on
image pixel or voxel values (image gray values) without
considering sparse feature landmarks [33]. Within certain
space of transformation, these methods search for maximum
similarity measures between source and target images.
Intensity-based image registration methods use different
matching parameters for establishing the correspondence of
similar intensity values between source and target images.
These matching parameters includes the sum of squared or
absolute differences (SSD and SAD, respectively), cross
correlation (CCor), correlation coefficient (CCoef), mutual
information (MI), Normalized Mutual Information (NMI),
Normalized Correlation (NC) and Mean Squared Difference
(MSD). All the mentioned matching parameters play an
important role in intensity-based image registration by
maximizing intensity similarity measures and by reducing
cost function. Fig. 8 shows registration of MRI (a) and CT (b)
images using their intensities [34]. The intensity values of hard
tissues (bone) are higher in CT image and lower in MRI.
Therefore, the bones are more visible in CT image and less
78 biocybernetics and biomedical engineering 38 (2018) 71–89
Fig. 8 – Intensity-based registration of MRI and CT images. Target CT image (b) which shows brighter bone structure is
registered with source MR image (a) with darker bone structure. Intensity based registration transform midlevel gray values
into high-level values, and high-level gray values (bone) into low values. Image (c) is remapped CT with an approximation of
the MR intensity distribution.
visible in MRI image. Intensity based registration map midlevel
gray values into high-level values, and high-level gray values
(bone) into low values. The resultant registered image is
remapped CT with an approximation of the MR intensity
distribution as shown in Fig. 8(c).
Intensity-based registration methods operate on image
intensity values and the transformation is performed itera-
tively. At each iteration, the similarity measures between
voxel intensities of source and target images are optimized.
This iterative transformation of image intensity values involve
interpolation between sample points and map both the
position and related intensity value at that particular position
[35]. In feature-based methods, the delineation of feature
landmarks is important for accurate registration which some
time affect the accuracy of registration. On the other hand,
intensity-based registration methods provide high accuracy by
taking into account more image information. Performing
retrospective registration with intensity-based methods need
minimum amount of preprocessing or user interaction [36]. As
a result, the automation of these methods are easy when
compared to point-based or surface-based registration algo-
rithms. However, the lack of human supervision in intensity-
based methods may also produce inaccurate registration
results. Registration of single and multi-modal images,
registration of same or different dimensional images (2D–
2D, 2D–3D, 3D–3D) and registration of rigid and deformable
models are the widely used application areas where intensity-
based registration methods are successfully applied.
Reductive registration and full image content-based regis-
tration are the two types of intensity based medical image
registration. Movements and principle orientations based
method is a popular example of reductive registration which
directly operate on image gray values [37]. Moments and
principle orientations method is also called reduction to
scalars/vectors registration because it directly reduce image
gray levels to representative scalars/vectors. In image-guided
surgery, reductive registration is used to register medical
images of different modalities by mapping their corresponding
volumes or points or surfaces. Speed, easy implementation
and automatic behavior are the main features of reductive
based registration. On the other hand, the use of reductive
registration methods create sensitivity for surgeons in several
types of image-guided surgeries such as marginal hypo-
metabolic swelling in PET, poor handling of dissimilarities in
scanned volume and image to image registration. Full image-
contents based method uses similarity measures such as
cross-correlation, image uniformity, square intensity differ-
ences and intensity variance for transformation and establish-
ing correspondence between images. In image-guided surgical
procedure, this method provides improved visualization in
the registration of inter-subject and atlas images. However,
the computation cost is high in some clinical application such
as the registration of 3D–3D data [38]. Moreover, full image
content-based method has not been introduced yet in the
registration of time-constrained applications such as intra-
operative 2D–3D.
C. Segmentation-based registration
Medical image registration based on segmentation extract
corresponding features and organs surfaces from source and
target images and aligned them properly. The alignment of
corresponding structure between two images is performed
with either rigid models or deformable models. Segmentation-
based registration using rigid models are simple and are widely
used methods in clinical applications. Deformable models, on
the other hand, are complex but are successfully used for the
organs with large deformation. Segmentation is performed
prior to registration, which identify either boundaries of the
structure or categorize every voxel on the basis of its intensity
properties. In medical image analysis, segmentation is used for
the localization of pathology, quantification of tissue volumes,
study of anatomical structure, computer-aided diagnosis and
treatment planning, and image-guided surgery [39].
In image-guided surgery, segmentation is used for the
extraction of 3D anatomical data, which is necessary for
 biocybernetics and biomedical engineering 38 (2018) 71–89
Fig. 9 – Segmentation-based registration: a labeled image (the atlas) is registered with an input image. The label from the atlas
is then overlaid on the deformed input image so the segmented structures of the atlas are also available in the resultant
image.
planning and guiding interventions [1]. Registration methods
based on image segmentation are widely used in IGS and
minimally invasive intervention. In segmentation based
registration, the anatomical structures and other regions of
interest in preoperative and intraoperative images are
segmented before actual transformation [14]. Segmentation
is performed on the basis of corresponding landmarks in the
images. In the second step, transformation function is
repeatedly applied on the images till the alignment of
corresponding landmarks. Fig. 9 shows atlas-based segmen-
tation and registration [40]. In the figure, an atlas source image
is registered with another target input image. Registration is
performed on the bases of voxel representation of both
images. In the first step, a segmented atlas image is aligned
with original input image and a new deformed input image is
generated. The deformed input image contains all the
segmented structures labeled in atlas image. In the next step,
labels from the atlas are overlaid on the deformed input image
so the segmented structures of an atlas are also available in the
resultant image.
Segmentation-based registration methods are more suc-
cessful than feature-based and intensity-based registration
methods when the images contain low or missing information
about human anatomy. These registration methods are
computationally efficient and support multi-modal registra-
tion. The accuracy is also high but depends on segmentation
accuracy because continuous splitting in input images
sometimes compromise accuracy. Furthermore, the available
method for segmentation-based registration is not fully
automatic because segmentation step is mostly performed
semi-automatically.
79
D. Fluoroscopy-based registration
C-arm-based fluoroscopy is an imaging technique that uses
X-rays to create images of the internal organs in real time [41].
It is the most commonly used interventional system, available
in all electrophysiology laboratories. During fluoroscopic
examination computer creates continuous images of structure
on the screen so that the body part and its motion can be seen
in detail. Contrast material such as barium or iodine is used to
enhance the quality of images [42]. Fluoroscopic techniques
have replaced invasive open surgical procedures with mini-
mally and non-invasive image-guided procedures. With each
incremental advancement in the technology, smaller vessels
and more subtle contrast differences can be visualized in real
time, often with low radiation dose [43].
Fluoroscope allows physician and surgeon to see the
internal structure and function of different organs such as
heart, lung, kidneys, bones, muscles and joints. For example,
fluoroscope is used to watch the pumping action of the heart or
the motion of swallowing. Similarly, in cardiac catheterization,
fluoroscopy allows the physicians to view the flow of blood
through the coronary arteries in order to evaluate the presence
of arterial blockages. Fluoroscopy is useful for both diagnosis
and therapy in angiography, general radiology, interventional
radiology and image-guided interventions (IGI). In fluoroscopy,
2D real time high resolution X-rays images of human anatomy
are obtained through an imaging scanner intensifier called C-
arm. 2D C-arm allows the physician to monitor progress and
immediately make any corrections. However, due to lack of 3D
spatial information, it is not sufficient for the proper visualiza-
tion of complex structures and their spatial relationship in
80 biocybernetics and biomedical engineering 38 (2018) 71–89
human anatomy. In other words, accurate path planning on the
C-arm AP-view image is difficult. Therefore, registration of
fluoroscopic images with improved preoperative imaging (high
quality 3D CT/MR images) of relevant anatomy would provide
accurate and efficient guidance in complex structures and their
spatial relationship [44].
2D–3D registration is an important and commonly used
technique of medical image registration. 2D–3D medical image
registration is performed by the alignment of preoperative 3D
data sets (e.g. CT/MR volume) with intraoperative 2D fluoro-
scopic X-rays images (e.g. CBCT volumes). In 2D–3D registra-
tion, C-arm is accurately posed on patient anatomy and the
associated surgical plan for successful image-guided naviga-
tion [45]. Fast and accurate registration of intraoperative 2D
fluoroscopic images and preoperative 3D data sets is useful for
intraoperative guidance and can greatly help the interven-
tionists during surgical procedure. The available 2D–3D
registration approaches provide useful decision support
system for quick localization of the target surgical site,
minimum radiation exposure, reduced contrast dose and
low risk of wrong-site surgery [46]. Integration of 3D CT/MR
image data sets with fluoroscopic images (2D–3D registration)
has potentially overcome the limitations of 2D angiography
due to proper visualization of complex vascular structures [47].
The alignment of preoperative 3D images and intraoper-
ative 2D fluoroscopic live images is however a challenging task
because finding intermodal correspondence is a nontrivial
problem. Furthermore, nonlinear aspect of the underlying
optimization and estimation of 3D CT/MR volume poses of an
object from its 2D X-rays projections are also challenging
problems in 2D–3D registration. 2D–3D rigid registration works
precisely in the initial alignment and in rigid structures e.g.
bones, but might not be enough for the precise alignment of
non-rigid structures e.g. heart due to breathing and natural
movement [48].
Modern fluoroscopy consists of new and innovative
imaging techniques with advanced real time 3D capabilities.
Real time 3D fluoroscopy (3D–3D) has addressed some of the
shortcomings of 2 dimensional fluoroscopy and increased the
accuracy of image-guided surgical procedures. This technique
is used to obtain intraoperative real-time 3D images and
perform automatic registration of real time 3D CT/MR data
with 3D live fluoroscopic images. Co-registering both 3D CT/
Fig. 10 – 3D–3D registration in axial (a) and coronal (b) planes. The cone-beam computed tomography (CT; gray) is
superimposed on the CT angiography reconstruction (red). Note the matching of bony structures and arterial calcifications
[51].
MR image with the fluoroscopic image space and placing them
in the same co-ordinate system helps to obtain real-time 2D
and 3D pictures of human organ. Generally, CT/MR C-arm
system (Ziehm or Siemens Healthcare) is used to register 3D
CT/MR data with 3D live fluoroscopic images. Following
successful image fusion, the anatomic landmarks marked in
preoperative images were overlaid on live fluoroscopy. The
accuracy of image registration was determined by measuring
the distance between overlay markers and a reference point in
the image [49].
Registration of 3D MRI or CT data with 3D fluoroscopic
images provides high spatial and temporal resolution along
with promising accuracy and feasibility [50]. Co-registration of
3D CT/MR images and real time 3D fluoroscopy (3D–3D
registration) is a reliable, efficient and accurate technique in
real-time image-guided interventions. Real time 3D fluorosco-
py (using C-arm) could offer significant advantages in the
catheterization laboratory by potentially reducing procedure
time, contrast and radiation dose. This technique is success-
fully used for non-rigid registration and soft tissue deforma-
tion e.g. structural heart disease and coronary artery
interventions [49]. One the other hand, the success of this
technique is relatively low in the treatment of patients with
congenital heart disease.
Real-time 3D fluoroscopy guidance with cone beam CT is a
new, promising, and feasible technique for needle interven-
tions. Because of the integration of the information of cone
beam CT and live fluoroscopy, real-time 3D fluoroscopy can be
used for real-time needle intervention, with a high degree of
accuracy. Cone-beam CT (3D–3D registration) makes use of
intraoperative rotational scan. The intraoperative scan is done
with the patient on the table in the position that is needed for
the procedure. In the registration process, the operator track
the position and movement of the C-arm which precisely
identify the position of region of interest in the patient.
Furthermore, pre-operative CT/MR volume is merged with the
intraoperative cone-beam CT based on anatomical landmarks
visible in both 3D scans. Real-time 3D fluoroscopy guidance
with cone beam CT is shown in Fig. 10 [51]. In the figure, the
cone-beam computed tomography (CT; gray) is superimposed
on the CT angiography reconstruction (red). The matching of
bony structures and arterial calcifications are clearly visible in
the figure.
 biocybernetics and biomedical engineering 38 (2018) 71–89 81

Real time 3D fluoroscopy is currently a key technology in

image-guided radiation therapy, radiosurgery, minimally
invasive surgery, endoscopy, and interventional radiology.
In endoscopy, 3D virtual images of human organs and vessels
are generated from preoperative 3D CT/MR images and
registered to real-time live endoscopic images. This type of
registration provides an augmented reality environment,
which show anatomical structures that are hidden from the
direct view by currently exposed tissues. Similarly, in
interventional radiology, real time 3D fluoroscopic registration
allows 3D visualization of tools, like catheters and needles
which can greatly improve guidance. In image-guided radia-
tion therapy, real time 3D fluoroscopy allows registration of
preoperative CT data and intra-interventional data images.
This allows precise patient positioning, which is of utmost
importance for exact dose delivery to the target and for
avoiding irradiation of healthy critical tissue [52]. The
complexities and challenges of 3D fluoroscopy includes:
accurate navigation in supra-aortic vessels and visceral
branches, miss registration of preoperative and intraoprative
images, respiration-related and cardiac cycle-related vessel
displacement, vessel elongation, and displacement by stiff
devices and patient movement. Fig. 11 – Issues and challenges in medical image
registration.
3. Issues and challenges

The development of highly sophisticated data scanning

devices and advancement in imaging techniques raises more
challenges in the area of medical image registration. The main
challenge is the development of more accurate and efficient
registration methods in clinically acceptable time-frames [53].
Moreover, real time registration of preoperative and intraop-
erative images is difficult to achieve due to the difference of
image modality and dimensionality. This section investigates
issues and challenges in medical image registration and
presents different solutions provided by researchers. However,
in order to be an effective instrument for the clinical practice,
registration algorithms must be computationally efficient,
accurate and most importantly robust to the multiple biases
affecting medical images. We have investigated several types
of issues and challenges in medical image registration (see
Fig. 11). These issues and challenges are further discussed in
the subsections below.
3.1. Efficiency, accuracy and robustness
An efficient, accurate and robust registration between pre-
operative images and intraoperative patient anatomy is
critical for successful and effective image-guided surgery.
Efficient, accurate and robust registration of corresponding
information from different types of images provides a basis for
diagnostic and medical decision-making, treatment monitor-
ing, and healthcare support [4]. Registration method cannot be
accepted as a clinical tool for patient healthcare and
management until it has been proved efficient, accurate and
robust. However, computational efficiency (performance),
accuracy in the alignments of images and robustness against
multiple biases affecting medical images are the three main
issues in non-rigid registration [54]. The natural behavior of
medical image is not consistent due to the effect of noise, blur
and organ movement. Therefore, highly robust and consistent
registration is required to manage small amount of variations
in the source and target images during IGS. Similarly, without
high accuracy in medical image registration method, it is not
possible to obtain successful results [14]. Accuracy is always
affected by the introduction of errors (either actual or timely)
in the medical images during the registration process.
Similarly, robustness is greatly affected by the variation of
intensity and missing of required data in the input images [54].
The performance, robustness and accuracy in medical
image registration methods depends on several parameters
including modality, effects on image contents, similarity
measures, transformation, optimization and implementation
mechanism [55]. These complex parameters are interdepen-
dent and it is difficult to assess the effect of each one on the
registration method. However, the initial assessments up-to
some level about the influences of these parameters is
important prior to registration. Registration methods based
on image features are efficient than intensity-based methods
but have low accuracy and robustness.
3.2. Similarity measures
Similarity measure is an important criterion for the evaluation
of medical images during registration process. In other words,
it is the criteria to evaluate how much two or more images of
the same organ are similar. In registration process, similarity
measures statistically evaluate and relate source/preoperative
and target/intraoperative images during IGS [56]. Various
similarity measures such as sum of squared differences,
mutual information, correlation coefficient and joint entropy
82 biocybernetics and biomedical engineering 38 (2018) 71–89
are used to evaluate voxel intensity differences between
source and target images. Among the various similarity
measures, sum of square differences (SSD) [57] is the simplest
one which can evaluate the differences between transformed
source image and fixed target image as
X not suitable to register 3D images because a large number of
SSD ¼ xk 2 f T ðf S ðgðxk ÞÞf T ðxk ÞÞ2 (3)
where fS is transformed source image, fT is target image and xk
denotes the total number of pixels that belong to the overlap of
source image and transformed target image. Transformation
function g( ) is applied to the source image and transformed
image and is denoted by fS(g(xk)). Similarity measures uses
measurements of absolute scale and registration positions
during the process. For absolute scale measurements, correla-
tion coefficient finds the values range between +1 and 1.
These values indicate how good the two images (source and
target) are related. The similarity between two images is also
evaluated on the bases of registration position by a maximum
or minimum value at the registration location. This is done by
correlation function and the sum of the absolute value of the
difference between the two images.
3.2.1. Mutual information
Mutual information (MI) is an intensity based similarity
measure, which automatically estimates the similarity in
multi-modal images [58]. In general, mutual information show
statistical relationship between two variables and it measures
the amount information that one variable contains about
other. In image-guided surgical intervention, MI is widely used
similarity measure which estimates the corresponding rela-
tionship between preoperative image and intraoperative
patient anatomy. During registration process, MI is maximized
for the achievement of effective matching of intensity values
between two images. In the process, when MI is reach to a
maximum level, it finds the most complex corresponding
regions between images [59]. MI as similarity measure is
successfully used in image-guided surgery because it is robust
to outliers and efficiently calculated.
In high volume multi-modal images the corresponding
points greatly varies which results the differences in intensi-
ties. These differences require the estimation of joint histo-
gram, which alternatively increases the computation time of
registration. Moreover, in the registration of multimodal
images, local intensity variations also degrade the perfor-
mance of mutual information because the joint histogram
computation is adversely affected. Another issue related to
estimation of similarity measure in multimodal image
registration is the exclusion of spatial and geometrical
information about the voxel. Like that of intensity informa-
tion, the estimation of spatial and geometrical information are
also important because they may provide additional cues
about the optimal registration.
There exists a lot of work on the integration of spatial
information while estimating mutual information (MI) as a
similarity measure. Pluim et al. [60] described an approach
which combines spatial information by multiplying the MI
with an external local gradient. Pluim et al. further incorpo-
rated both gradient magnitude and orientation for the
calculation of MI. Rueckert et al. [61] proposed a framework
which considered six nearest neighbors of each voxel to
calculate MI for the improvement of robustness and accuracy
in non-rigid registration. Rueckert et al. framework was based
on second-order MI to the problem of 2D registration by using a
4D joint histogram. Computationally the proposed approach is
samples are required in it to compute a high dimensional
histogram.
Russakoff et al. [59] extended mutual information as a
similarity measure to incorporate spatial information by
considering regional MI on both corresponding pixels and
their neighbors. Although the method proposed by Russakoff
et al. efficiently takes region of corresponding pixels into
account but may not be reliable when the number of samples
is small. The main reason was the use of high-dimensional
histogram to perform registration. Yi et al. [62] incorporated
spatial relationships into the registration by the addition of
global normalized mutual information and local matching
statistic. Loeckx et al. [63] proposed conditional mutual
information (cMI) as a similarity measure for multi-modal
non-rigid image registration. The proposed method integrates
both intensity and spatial dimensions to express the location
of the joint intensity pair. Loeckx et al. approach overcomes
several problems inherent to the use of global MI, using
artificial and clinical images, but its main drawback was
increased computation time as compared to traditional MI
measures. Another approach presented by Zhuang et al. [64]
uses spatially encoded mutual information in the computa-
tion of a spatial variable and its associated entropy measures.
In this method, spatial variable information are combined into
the computation of their joint histogram and a hierarchical
weighting scheme is used to regularize the locality of the
weighting function. Similarly, Myronenko et al. [65] also
developed a new similarity measure for image registration,
which accounts for complex spatially-varying intensity dis-
tortions and non-stationeries of the images. The method
proposed by Myronenko et al. is although simple and fast but
only applicable to mono-modal registration.
The recent work proposed by Woo et al. [66] addresses the
problems of local intensity variations and missing of spatial
and geometric information about the voxel. In order to achieve
better MI, J Woo incorporated spatial and geometric informa-
tion via a 3D Harris operator which decompose the image into
three disjoint and geometrically distinct regions. Computa-
tionally the proposed approach may not suitable when applied
to large population studies and routine clinical practice. The
reason is its implementation on CPU instead of implementing
it on GPU or with the use of parallel computing.
3.2.2. Correlation coefficient
The correlation coefficient is another similarity measure for
medical image registration. It symmetrically measures the
linear dependence between image intensities of correspond-
ing voxels in both images. In order to determine how strong
the pixels of two images are related, correlation coefficient
uses a value that can range between 1 and +1. If the
correlation coefficient value is in the negative range, then the
relationship between the pixels is negatively correlated, or as
one value increases, the other decreases. On the other hand, if
the correlation coefficient value falls in the positive range, the
 biocybernetics and biomedical engineering 38 (2018) 71–89
relationship between the pixels is positively correlated, or both
values increase or decrease together. This means that, greater
the absolute value of a correlation coefficient, the stronger the
linear relationship between two images. Pearson originally
developed the mathematical formula for correlation coeffi-
cient which estimates the degree of relationship between two
quantities [67]. For monochrome images, the formula is define
as
P unrelated, then the joint entropy will be the sum of the
i ðxi xm Þðyi ym Þ
r ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
P ffiqffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
P ffi
2 2
i ðxi xm Þ i ðyi ym Þ
where xi and yi are intensity values of ith pixel in source and
target image respectively while xm and ym are mean intensity
values of the said images. The value of r shows the level of
similarity between two images i.e. in case of absolute similari-
ty the value will be 1, if they are completely unrelated the value
will be 0.
Correlation coefficient accurately and efficiently evaluates
the accuracy of mono-modal medical image registration. For
the registration of multi-modal images, however, correlation
coefficient is not a favorable similarity measure because of
poor statistical and computational efficiency [68]. In image-
guided surgery and radiotherapy, the available images are
mostly belongs to the same type of modality. Therefore,
correlation coefficient as similarity measure in these applica-
tions is a useful choice for clinicians. Registration of medical
images with correlation coefficient as similarity metric
provides several advantages including easy implementation,
no need to estimate probability densities at every iteration,
insensitivity to geometric distortion, intensity inhomogeneity
and data missing. On the other hand, correlation coefficient is
greatly affected by the outliers, which consequently degrade
registration performance. Moreover, local extrema and large
errors in registration also affect the performance of correlation
coefficient. In order to avoid such problems in registration,
appropriate techniques are required for sampling and visual
inspection.
3.2.3. Joint entropy
Combining images with misaligned structure results an image
with duplicated information. The basic purpose of registration
is to reduce the duplicated information and make it more
simple and informative. Registration uses several types of
matrices for such type of information measure in multiple
images. Joint entropy is commonly used information measure
in digital image processing [5]. The measurement of uncer-
tainty in both joint distribution and conditional distribution of
a pair of random variables is performed with joint entropy. The
relative transformation of source image to target image is
always occurred when the joint entropy is minimum [69].
During transformation, the volume of overlap between source
image and target image also changes as they are transformed
relative to one another. The relative transformation and
volume overlap greatly affects the reliability of alignment and
registration. However, the solution for this problem is already
done by Collignon et al. [70] and Wells et al. [71] using mutual
information (MI) as registration metric.
In order to align source image s with target image t in the
registration process, the two symbols at each voxel location
83
are used for the estimation of transformation T. The alignment
of source image s with target image t results a combined image.
Joint entropy estimates the amount of information in the
combined images. If the two images s and t are dissimilar (i.e.,
no relation between them), then sum of the entropies of
individual images is called joint entropy. In case of high
similarity in the images, the joint entropy is low compared to
the sum of the individual entropies. If s and t are totally
(4) entropies of the individual images. The more similar (i.e. less
independent) the images are, the lower the joint entropy
compared to the sum of the individual entropies. Joint
histogram calculated from image s and t is used for the
visualization of joint entropy as
Hðs; tÞHðsÞ þ HðtÞ (5)
In entropy-based image registration, only pixel intensity
values are used for alignment and image histograms are used
for computation [72]. The use of pixel intensity values only as
alignment measure neglect the spatial information in the
images which may affect alignment accuracy. Similarly,
entropy-based measures are more complicated than simpler
measures and are therefore computationally expensive than
simpler one.
3.3. Registration of multimodal images
Each modality exhibit different characteristics i.e. CT, ultra-
sound (US), rotational C-arm angiography and MRI are used for
anatomical imaging while PET, SPECT and functional magnetic
resonance imaging (fMRI) are used for functional imaging. The
appropriate matching of diverse features (functional and
anatomical contents) in multimodal images is important for
successful registration. However, proper matching in multi-
modal image registration is more challenging and harder
task. The appropriate matching of corresponding features is
still an issue because images obtained from multiple
modalities differ in spatial resolution. Some approaches,
such as information theoretic approaches and reduction to
mono-modal registration have been proposed to solve the
problem of appropriate feature matching in multimodal
image registration. Information theoretic approaches use
mutual information (MI) as similarity metric to solve the
problem of mismatching in multimodal registration. How-
ever, MI is a general similarity metric and it neither overlap
invariant nor assumes any relationship between the image
intensities. Similarly, reduction to mono-modal registration
approach was developed which aims to simplify the problem
and find its solution i.e. simulating one modality from
another or mapping multi-modal images into a common
domain.
In multi-modal image registration, the association between
intensity values of related pixels is also complex and
unknown. The missing of features in one image and presence
in another image, mapping of single intensity value in one
image to multiple values in another image are the challenging
issues in multi-modal image registration [73]. These issues
greatly affect the proper computation of similarity measures
based on their intensity values in medical image registration
[74].
84 biocybernetics and biomedical engineering 38 (2018) 71–89
3.4. Detection of reliable landmarks
The reliable identification of anatomical landmarks in mutli-
modal (CT, MR, PET, etc.) 3D images are essential and one of
the important first step in medical image registration. Land-
marks are detected either with manual method or with
automatic method [75]. The manual method for the identifi-
cation of landmarks requires medical expertise and takes
more time. The available automatic methods for landmarks
identification are fast and could reliably detect landmarks in
medical images. The automatic methods for landmarks
selection are mostly depend on machine learning approaches.
Therefore, quality of training data sets plays an important role
in the reliable identification of anatomical landmarks. In
computer vision, it is easy to obtain large training data sets but
in medical field creating a large database of images are
challenging and requires a lot of efforts and time.
3.5. Outliers rejection
The basic aim of medical image registration is to find the
optimal transformation between two images by maximizing
similarity measures such as mutual information (MI), entropy
and correlation-coefficient. However, mutual information is
always affected by the presence of outliers (objects in one
image but not in another) in source and target images. In
medical image registration, the presence of unpredictable
outliers in preoperative and intraoperative images greatly
affects mutual information [76]. Therefore, several approaches
have been used for the rejection of outliers in medical image
registration. The most prominent among them include
consistency test [77], intensity transformation [78], gradient-
based asymmetric multifeature MI [79], graph-based multi-
feature MI [80], joint saliency map (JSM) [76] and normalized
gradients [81]. The rejection of outliers is a challenging task in
medical image registration because a large number of outliers
are present in the image-guided surgery applications. There-
fore, more efforts are required to improve the robustness of
available similarity measures toward outliers.
3.6. Convergence of optimization methods to local maxima
In the process of registration, optimization is the optimal
transformation of intensity values that best align source and
target images. Several types of optimization techniques are
available which solve various problems in medical image
registration. These techniques include Powell's method,
steepest gradient/gradient decent, Conjugate gradient,
Powell's conjugate Quasi-Newton, Gauss-Newton and Sto-
chastic gradient decent. Each method uses its own mechanism
to properly extract and match the corresponding features in
medical image registration. The selection of optimization
technique plays an important role in performance of registra-
tion process [82]. Powell's method is one of the common
choices for solving optimization problems in image registra-
tion due to its proven performance and simple implementa-
tion. This method allows to simplify multidimensional
problem to one dimensional optimization task.
Sometime registration accuracy is compromised during
optimization in the presence of local maxima. Similarly, in
elastic transformation, the inaccurate extracted landmarks
also produce registration error in the presence of local maxima
[83]. The available optimization methods avoid the problem of
local maxima and improve similarity measures. However,
further investigation is needed to develop advanced optimi-
zation methods for medical image registration.
3.7. Guidance to clinicians
In image-guided surgery and radiotherapy, clinicians faces
several problems while taking pre-operative and intra-opera-
tive measures. The main problem is the accurate mapping of
contrast information in multi-modal images i.e. organ
scanned multiple times with different scanners. In such type
of scenario, it is difficult for the clinicians to know exactly the
location and orientation of patient with respect to different
imaging systems. Image registration and fusion in treatment
room provides more guidance and help to the clinicians while
operating on patient data. With image to patient registration,
data is associated precisely and the treatment is given to the
patient according to pre-operative plan [35].
In surgical guidance system, registration methods process-
es information obtained from physical devices. Information is
processed with an algorithmic procedure for optimal transfor-
mation of images. Most of the transformation is optimal due to
the advancement in medical image registration methods but it
is not ideal. As a result, the chances of error called target
registration error (TRE) is high. Similarly, the role of image
registration is also highly important when the surgical
guidance is based on preoperative images. Here, accurate
registration is required for the surgical guidance system.
Inaccuracy in the surgical guidance system will be useless and
dangerous to the patient life. In typical image-guided surgery
(IGS), the anatomy of a patient captured in the preoperative
image remains rigid from image acquisition to surgical
procedure. Non-rigid registration, which is successfully used
for image-to-image registration need further research and
improvement in image-to-patient registration [84]. For accurate
registration and transformation of corresponding points from
image-to-patient, further improvement is required in surgical
guidance system especially in case of non-rigid registration.
3.8. Relating contrasting information
Relating contrasting information in different types of medical
images is a challenging task in multimodal image registration.
In IGS, the patient's organ is scanned multiple times with
different types of imaging modalities which create difficulties
for the identification/fixation of patient location and orienta-
tion with respect to different imaging systems. Therefore, it is
necessary to developed more advanced registration methods
which can easily remove the differences in patient positioning
and relate information from different types of images.
3.9. Parameters determination and their correspondence
Parameters such as points, landmarks and curves are the
components of an image and their proper determination and
mapping are essential for accurate registration. Image registra-
tion algorithms determine the corresponding parameters in
 biocybernetics and biomedical engineering 38 (2018) 71–89
both source image and target images and aligned them properly
[85]. The correspondence between two images is either
functional or structural. The former relates the equivalent
anatomical structures in the two images while the later line up
the same functional regions. Image registration algorithms,
which determine high number of corresponding parameters are
more flexible. However, the efficiency of such algorithms is slow
and requires more computation time. Rigid and affine
registration algorithms are computationally efficient because
they take less parameters for correspondence. On the other
hand, non-rigid registration algorithms are mostly slow
because they determine a large number of parameters by
matching voxel intensities in images. Moreover, the transfor-
mation in non-rigid registration algorithms is asymmetric and
there is no guarantee of mapping each landmark/point in the
source image to its corresponding position in the target image.
3.10. Automatic image registration
Automatic registration in medical images aligns the common
detected features in preoperative and intraoperative images
without user interaction. Automatic registration methods are
widely used in medical image processing and several types of
image-guided surgeries are successfully performed with
automatic medical image registration methods. The perfor-
mance of automatic image registration is high because it
requires less time and minimum efforts from the user while
aligning the subject images. Moreover, the point/landmarks in
automatic registration method transform globally and with
high efficiency [14]. The accuracy of automatic image registra-
tion methods is also high but greatly depends on the precision
and optimization of algorithms. Automatic image registration is
still an open problem in medical imaging and some of the
challenges include the proper selection of 3D landmarks,
extraction of same features in multi-modal images, variable/
limited anatomical coverage and low contrast to noise.
3.11. Other issues and challenges
Medical image registration also faces several other important
issues and challenges other than above which are highlighted
below.
 'Feature-based' methods required the difficult extraction of
specific features (e.g. blood vessel outline) from the 3D and
2D images before registration. This is very difficult to achieve
accurately and robustly using clinical images and has
limited the clinical adoption of such methods. An alterna-
tive, 'intensity-based' approach matches images on raw
image intensity values.
 During surgical procedures, the registration between the
preoperative images and the tracking system can become
inaccurate, due to movement of the patient, deformation of
tissue, shifting of tracking equipment, and the like. Correcting
an inaccurate registration traditionally requires a time-
consuming interruption to the surgical procedure. The
interruption may be omitted, but doing so introduces
inaccuracies in image registrations, and may result in medical
personnel being provided with incorrect information as to the
location of surgical instruments such as imaging probes.
85
 Soft tissue deformation that occurs during image-guided
surgeries causes errors in registering preoperative images to
the current position of a patient during the surgery.
Therefore, it is desirable to estimate tissue deformation
during surgery and compensate for it. One of the solutions to
this problem is to use stereo images. Preoperative MRI or CT
images should be registered with stereo images taken from
the surface of the soft tissue during surgery.
 The alignment of functional images of low quality and the
determination of functional abnormality is often a difficult
task in medical image registration. The proper identification
of functional abnormality in organs is performed by the
registration of functional images with anatomical images of
the same organ [86].
 In computer assisted surgery and radiotherapy the efficiency
is always required while aligning multiple images. However,
the processing time of current registration algorithms is
slow due to the complex imaging modalities and anatomical
structures in human body. Moreover, most of the current
registration algorithms are also sensitive to initial position-
ing of images.
 For accurate registration, the proper understanding of the
healthy and pathological states of organs before image
mapping at different scale is necessary.
 Online image to physical space registration and interfacing
of computational devices for the guidance of surgical
operation need further research work.
 The lack of accurate correspondence of parameters in source
and target images due to asymmetric nature is also a big
issue in medical image registration.
 Registration method, which is robust to geometric distor-
tions i.e. non-linear and local, is more accurately align two
images. However, registration of medical images with highly
complex nonlinear and local distortions often produces in
accurate result. Moreover, the absence of knowledge of
correspondences between images also affects the accuracy
of registration.
 In feature-based registration, point pattern matching cre-
ates difficulties when medical images contain noise and
missing data. Similarly, feature based image registration
methods are less robust in feature extraction and less
accurate in feature matching.
 Image segmentation is often required in feature-based
methods before registration. In case of manual segmenta-
tion procedures, skilled human interaction is required which
create difficulties toward the automatic process of registra-
tion. In case of automatic segmentation, errors may occur in
the registration process.
 Registration methods based on extrinsic landmarks require
high skill from surgeon and often produce less accurate
results due to skin movement during surgery. Similarly,
registration methods based on anatomic landmarks requires
user interaction for the identification of landmarks.
4. Research opportunities in alleviating issues
and challenges
In order to make IGS a practical reality, the issues and
challenges in medical image registration must be addressed.
86 biocybernetics and biomedical engineering 38 (2018) 71–89
To the best of our knowledge, this paper describes the first
attempt to highlight the issues and challenges in medical
image registration in a comprehensive manner. We have also
made an effort to provide a comprehensive survey on the
image registration concepts and methods and its applications
in image-guided surgery (IGS). In Section 2 we have described
general concepts and methods for medical image registration
in the field of IGS. We have presented the main issues and
challenges and their possible solutions in medical image
registration in general and in image-to-patient registration in
particular in Section 3. After comprehensive analysis of
medical image registration methods and the available issues
and challenges, this section describe some guidelines that will
would be helpful for the development of new advance
registration methods. The adaptability of these guidelines in
clinics would greatly help surgeon in preoperative planning
and intraoperative surgical navigation. These guidelines include:
1. The accurate transfer of corresponding features in
preoperative image to intraoperative patient anatomy is
a challenging task in image-guided surgery (IGS). Al-
though, the location of virtual resection line is visible and
easy to estimate in IGS but it is difficult to directly locate
tumor and relevant vessels because they are hidden
underneath the organ. Advancement in preoperative
imaging techniques and intraoperative navigation sys-
tems will support surgeon to precisely and directly
visualize spatial relationship of surgical instruments to
anatomical structures.
2. The occlusion of tissue or small objects in IGS creates a
challenge for registration. In the occluded scenario,
interested regions are either not present or not clearly
visible in all preoperative and intraoperative images.
Although, registration of images with minor occlusions
produce reasonable results but images with major occlu-
sions is a challenging task and it is important to address
this issue.
3. One of the most important questions concerning medical
image registration is its use in real clinical settings. Clinical
data is always affected by intensity consistencies such as
noise, motion and intensity in-homogeneity. The current-
ly available registration algorithms provide limited capa-
bility to efficiently and accurately cope with these issues in
real clinical setting. In order to increase the use of
registration in clinical practice and make it an effective
instrument for the above issues, accurate, robust and
computationally efficient algorithms are desired.
4. In medical images, landmarks provide anatomy specific
constraints and guide the deformation process in regions
with uneven information. However, the detection and
extraction of significant landmarks to perform an accurate
registration remains a very challenging task.
5. In multimodal registration, features in the same images
obtained from different scanning device are aligned. Due
to different scanning devices, images of the same subject
show different feature characteristic i.e. functional and
anatomical. Therefore, accurate correspondence of fea-
tures between source and target images in multimodal
registration remains a challenge in image-guided surgery.
Although, several types of image registration methods
based on mutual information are available which create
statistical relationship among the features in source and
target images. Although, mutual information is a standard
similarity measure for multimodal image registration but
its performance degrades when the images contains local
intensity variations. Moreover, mutual information only
considers intensity information in the images and ignores
spatial information. Therefore, the development of ad-
vance techniques in which mutual information can easily
cope with local intensity variations and fully consider
spatial information along with intensity information will
bring a great change in multimodal medical image
registration.
6. Correlation coefficient is one of the important statistical
similarity measures for the registration of medical image
in image-guided surgery. Correlation coefficient accurately
and efficiently evaluates the similarity between two
images in mono-modal medical image registration. How-
ever, in multi-modal registration, the performance of
correlation coefficient is greatly affected by the outliers,
local extrema and large errors. An enhancement in the
available correlation measures and development of ad-
vanced techniques will solve such problems in multi-
modal medical image registration.
7. Joint entropy is another similarity measure which esti-
mates the amount of information in the two or more
combined images. However, image registration based on
joint entropy is complicated, computationally expensive
and consider only pixel intensity values in the images.
Considering only pixel intensity values and neglecting
spatial information in images greatly affect the alignment
accuracy. Therefore, more research is needed in the area of
image registration with joint entropy to overcome the
above issues.
8. The alignment of functional images of low quality and the
determination of functional abnormality is often a difficult
task in medical image registration. Therefore, resolution of
functional images and the accuracy of functional analysis
techniques need further improvement.
9. In medical image registration, the identification of reliable
landmarks is performed with either manual or automatic
method. The former requires medical expertise and takes
more time while the later is fast but depends on machine
learning approaches. In other words, automatic methods
depends on the quality of training data sets which are
easily obtained in computer vision but requires more
efforts and time in medical field. Therefore, the develop-
ment and availability of large databases of images in the
medical field will easily solve reliable landmarks identifi-
cation problem in automatic methods.
10. In medical image registration, optimal transformation is
performed by maximizing mutual information in the
source and target images. In case of pre-operative and
intra-operative images, mutual information is greatly
affected by the presence unpredictable outliers. Although,
several types of approaches have been developed for the
rejection of outliers but it is still a challenging issue in
image-guided surgery. The development of new techni-
ques for the minimization of large number of outliers in
IGS will reduce their effect on MI.
 biocybernetics and biomedical engineering 38 (2018) 71–89
11. Most of the optimization methods in medical image
registration converge to local maxima which is not
desired. Further research is needed for the development
of advanced optimization methods for medical image
registration, which can easily avoid local maxima.
12. In surgical guidance system, transformation of corre-
sponding points is usually optimal but not ideal. However,
ideal transformation (i.e. maps every point in the image
space onto its correct counterpart in physical space and
vice versa) is only possible if the target registration error
(TRE) is very low. Therefore, further research is required to
minimize the chances of TRE in image to physical space
registration.
13. In image-guided surgery, it is difficult to relate contrasting
information in multi-modal images due to differences in
images and patient positioning. The main reason is the
identification/fixation of patient location and orientation
with respect to different imaging systems. Therefore, it is
necessary to developed more advanced registration
methods, which can easily remove the differences in
patient positioning and relate information from different
types of images.
14. Parameters determination and their correspondence in
non-rigid registration is not computationally efficient as
compared to rigid registration. The efficiency in non-rigid
registration is affected by the identification of large
number of parameters and asymmetric transformation.
The computationally efficiency in non-rigid registration
can be improved by using symmetric algorithms for
transformation and the introduction of techniques which
uses minimum number of parameters for correspondence.
15. Despite the wide spread use, accuracy and performance,
automatic image registration methods are still an open
problem. Some of the challenges in automatic image
registration include the proper selection of 3D landmarks,
extraction of same features in multi-modal images,
variable/limited anatomical coverage and low contrast
to noise. Moreover, the accuracy of automatic image
registration methods also depends on the precision and
optimization of algorithms. The investigation of new
optimization algorithms for automatic image registration,
as well as the development of advanced schemes for 3D
landmark selection, feature extraction, anatomical cover-
age and contrast to noise will improve the performance of
automatic image registration.
5. Conclusion
The registration problem is one of the great challenges that
must be addressed in order to make image-guided surgery
(IGS) a practical reality. In this paper, medical image registra-
tion, its use and importance in IGS and prominent issues and
challenges has been discussed. We have discussed what has
been done to alleviate these issues, and what needs to be done
in the form of different research opportunities and guidelines.
One of the most significant findings of this study is that the
researchers from the medical imaging community have done a
lot of work to cope with the issues and challenges in medical
87
image registration. They have also developed solutions that
have mostly resolved the issues. However, there is substantial
room for improvement in image registration in clinical
applications because current researches are difficult to achieve
fast and accurate registration to satisfy clinical demand.
Therefore, further research and developments are needed for
the advancement of image registration methods and their
proper implementation in clinical applications.
references
[1] Cleary K, Peters TM. Image-guided interventions:
technology review and clinical applications. Annu Rev
Biomed Eng 2010;12:119–42.
[2] Lindseth F, Langø T, Selbekk T, Hansen R, Reinertsen I,
Askeland C, et al. Ultrasound-based guidance and therapy.
In: Gunarathne GPP, editor. Advancements and
breakthroughs in ultrasound imaging. 2013.
[3] Castro Pareja CR. Real-time 3D elastic image registration.
The Ohio State University; 2004.
[4] Bali RK. Clinical knowledge management: opportunities
and challenges. Idea Group Pub.; 2005.
[5] Bankman I. Handbook of medical imaging: processing and
analysis management. Elsevier Science; 2000.
[6] Miga MI, Clements LW, Galloway RL, Miga MI, Clements LW,
Galloway RL. Apparatus and methods of compensating for
organ deformation, registration of internal structures to
images, and applications of same. Google Patents; 2008.
[7] Olver PJ, Tannenbaum A. Mathematical methods in
computer vision. Springer; 2003.
[8] Thompson S, Penney G, Billia M, Challacombe B, Hawkes D,
Dasgupta P. Design and evaluation of an image-guidance
system for robot-assisted radical prostatectomy. BJU Int
2013;111:1081–90.
[9] Alam F, Rahman SU, Khusro S, Ullah S, Khalil A. Evaluatin
of medical image registration techniques based on nature
and domain of the transformation. J Med Imaging Radiat Sci
2016;47:178–93.
[10] Dogra A, Patterh MS. CT and MRI brain images registration
for clinical applications. Cancer Sci Ther 2014;6:18–26.
[11] Alam F, Rahman SU, Khalil A, Khusro S, Sajjad M.
Deformable registration methods for medical images: a
review based on performance comparison. Proc Pak Acad
Sci A: Phys Comput Sci 2016;53:111–30.
[12] Alam F, Rahman SU, Khalil A, Ullah S, Khusro S.
Quantitative evaluation of intrinsic registration methods
for medical images. Sindh Univ Res J – SURJ (Sci Ser)
2017;491:43–8.
[13] Alam F, Rahman SU, Ullah S, Khalil A, Uddin A. A review on
extrinsic registration methods for medical images. Tech J
Univ Eng Technol Taxila 2016;21:110–9.
[14] Alam F, Rahman SU. Intrinsic registration techniques for
medical images: a state-of-the-art review. J Postgrad Med
Inst (Peshawar-Pakistan) 2016;30.
[15] Zitova B, Flusser J. Image registration methods: a survey.
Image Vis Comput 2003;21:977–1000.
[16] Gonzalez RC. Digital image processing. Pearson Education;
2009.
[17] DeLorenzo C, Papademetris X, Staib LH, Vives KP, Spencer
DD, Duncan JS. Image-guided intraoperative cortical
deformation recovery using game theory: application to
neocortical epilepsy surgery. IEEE Trans Med Imaging
2010;29:322–38.
[18] Risholm P, Golby AJ, Wells WM. Multi-modal image
registration for pre-operative planning and image guided
88 biocybernetics and biomedical engineering 38 (2018) 71–89
neurosurgical procedures. Neurosurg Clin N Am
2011;22:197–206.
[19] Csapo I, Davis B, Shi Y, Sanchez M, Styner M, Niethammer
M. Longitudinal image registration with non-uniform
appearance change. Medical Image Computing and
Computer-Assisted Intervention – MICCAI 2012. Springer;
2012. p. 280–8.
[20] Patel PM, Shah VM. Image registration techniques: a
comprehensive survey. Int J Innov Res Dev 2014;3:68–78.
[21] El-Samie FEA, Hadhoud MM, El-Khamy SE. Image super-
resolution and applications. CRC Press; 2012.
[22] Goshtasby AA. 2-D and 3-D image registration: for medical,
remote sensing, and industrial applications. Wiley; 2005.
[23] Deng H, University OS. Image feature detection and
matching for biological object recognition. Oregon State
University; 2007.
[24] Navab N, Jannin P.Information Processing in Computer-
Assisted Interventions: First International Conference,
IPCAI 2010. Proceedings. Springer; 2010.
[25] Zhao Q, Pizer S, Niethammer M, Rosenman J. Geometric-
feature-based spectral graph matching in pharyngeal
surface registration. Medical Image Computing and
Computer-Assisted Intervention: MICCAI. International
Conference on Medical Image Computing and Computer-
Assisted Intervention, vol. 17. 2014. pp. 259–66.
[26] Feng DD. Biomedical information technology. Elsevier
Science; 2011.
[27] Yankeelov TE, Pickens DR, Price RR. Quantitative MRI in
cancer. CRC Press; 2011.
[28] Maurer C, Maciunas RJ, Fitzpatrick JM. Registration of head
CT images to physical space using a weighted combination
of points and surfaces [image-guided surgery]. IEEE Trans
Med Imaging 1998;17:753–61.
[29] Shamir RR, Joskowicz L, Shoshan Y. Fiducial
optimization for minimal target registration error in
image-guided neurosurgery. IEEE Trans Med Imaging
2012;31:725–37.
[30] Otake Y, Armand M, Sadowsky O, Armiger RS, Kazanzides P,
Taylor RH. An iterative framework for improving the
accuracy of intraoperative intensity-based 2D/3D
registration for image-guided orthopedic surgery. In: Navab
N, Jannin P, editors. Information Processing in Computer-
Assisted Interventions: First International Conference,
IPCAI 2010. Proceedings. Berlin, Heidelberg: Springer Berlin
Heidelberg; 2010. p. 23–33.
[31] Lindseth F, Langø T, Selbekk T, Hansen R, Reinertsen I,
Askeland C, et al. Ultrasound-based guidance and therapy.
Advancements and Breakthroughs in Ultrasound Imaging.
2013. pp. 28–82.
[32] Wen P. Medical image registration based-on points, contour
and curves. 2008 International Conference on BioMedical
Engineering and Informatics. 2008. pp. 132–6.
[33] Chou Y-Y. Transitive and symmetric nonrigid image
registration. Citeseer; 2004.
[34] Meyer J. Histogram transformation for inter-modality
image registration. 2007 IEEE 7th International Symposium
on BioInformatics and BioEngineering. 2007. pp. 1118–23.
[35] Hajnal JV, Hill DLG. Medical image registration. CRC Press;
2001.
[36] Beutel J. Handbook of medical imaging: medical image
processing and analysis. Society of Photo Optical; 2000.
[37] Wilson D, Laxminarayan S. Handbook of biomedical image
analysis: vol. 3: Registration models. Kluwer Academic/
Plenum Publishers; 2007.
[38] Lee M-E, Kim S-H, Seo I-H. Intensity-based registration of
medical images. 2009 International Conference on Test and
Measurement. 2009. pp. 239–42.
[39] Guo Y. Medical image registration and application to atlas-
based segmentation. Kent State University; 2007.
[40] Erdt M, Steger S, Sakas G. Regmentation: a new view of
image segmentation and registration. J Radiat Oncol Inform
2012;4:1–23.
[41] Rivera T, Uruchurtu E. Radiation monitoring in
interventional cardiology: a requirement. J Phys: Conf Ser
2017;012098.
[42] Lisle DA. Imaging for students. 4th ed. CRC Press; 2012.
[43] Gingold E. Modern fluoroscopy imaging systems. Image
Wisely; 2014.
[44] Sra J. Cardiac image registration. J Atrial Fibrillation 2008;1
(September–November):25.
[45] El Hakimi W. Accurate 3D-reconstruction and-navigation
for high-precision minimal-invasive interventions.
Technische Universität; 2016.
[46] Otake Y, Schafer S, Stayman J, Zbijewski W, Kleinszig G,
Graumann R, et al. Automatic localization of vertebral
levels in X-ray fluoroscopy using 3D–2D registration: a tool
to reduce wrong-site surgery. Phys Med Biol 2012;57:5485.
[47] Fagan T, Truong U, Jone P, Bracken J, Quaife R, Hazeem A,
et al. Multimodality 3-dimensional image integration for
congenital cardiac catheterization. Methodist Debakey
Cardiovasc J 2014;10:68–76. This study gives an overview on
how fusion imaging can be used for planning and
performing interventions in patients with congenital heart
disease..
[48] Rivest-Henault D, Sundar H, Cheriet M. Nonrigid 2D/3D
registration of coronary artery models with live fluoroscopy
for guidance of cardiac interventions. IEEE Trans Med
Imaging 2012;31:1557–72.
[49] Suntharos P, Setser RM, Bradley-Skelton S, Prieto LR. Real-
time three dimensional CT and MRI to guide interventions
for congenital heart disease and acquired pulmonary vein
stenosis. Int J Cardiovasc Imaging 2017;1–8.
[50] Narayan SA, Qureshi S. Multimodality medical image
fusion: applications in congenital cardiology. Future
Medicine; 2017.
[51] van den Berg JC. Update on new tools for three-dimensional
navigation in endovascular procedures. AORTA J 2014;2:279.
[52] Markelj P, Tomaževič D, Likar B, Pernuš F. A review of 3D/2D
registration methods for image-guided interventions. Med
Image Anal 2012;16:642–61.
[53] Alam F, Rahman SU, Hassan M, Khalil A. An investigation
towards issues and challenges in medical image
registration. J Postgrad Med Inst (Peshawer Pakistan)
2017;31:224–33.
[54] Liu Y. On the real-time performance, robustness and
accuracy of medical image non-rigid registration. College of
William & Mary; 2011.
[55] Neri E, Baert AL, Caramella D, Bartolozzi C. Image
processing in radiology: current applications. Springer;
2007.
[56] Svedlow M, McGillem C, Anuta PE. Experimental
examination of similarity measures and preprocessing
methods used for image registration. LARS Symposia. 1976.
p. 150.
[57] Suehling M, Huber M, Soza G. Method and system for
semantics driven image registration. Google Patents; 2012.
[58] Keyvanpour M-R, Alehojat S. Analytical comparison of
learning based methods to increase the accuracy and
robustness of registration algorithms in medical imaging.
Int J Adv Sci Technol 2012;41.
[59] Russakoff DB, Tomasi C, Rohlfing T, Maurer Jr CR. Image
similarity using mutual information of regions. European
Conference on Computer Vision. 2004. pp. 596–607.
[60] Pluim JPW, Maintz JBA, Viergever MA. Image registration by
maximization of combined mutual information and
gradient information. In: Delp SL, DiGoia AM, Jaramaz B,
editors. Medical Image Computing and Computer-Assisted
Intervention – MICCAI 2000: Third International
 biocybernetics and biomedical engineering 38 (2018) 71–89
Conference. Proceedings. Berlin, Heidelberg: Springer Berlin
Heidelberg; 2000. p. 452–61.
[61] Rueckert D, Clarkson MJ, Hill DLG, Hawkes DJ. Non-rigid
registration using higher-order mutual information;
2000;438–47.
[62] Yi Z, Soatto S. Nonrigid registration combining global and
local statistics. IEEE Conference on Computer Vision and
Pattern Recognition, 2009. CVPR 2009. 2009. pp. 2200–7.
[63] Loeckx D, Slagmolen P, Maes F, Vandermeulen D, Suetens P.
Nonrigid image registration using conditional mutual
information. IEEE Trans Med Imaging 2010;29:19–29.
[64] Zhuang X, Arridge S, Hawkes DJ, Ourselin S. A nonrigid
registration framework using spatially encoded mutual
information and free-form deformations. IEEE Trans Med
Imaging 2011;30:1819–28.
[65] Myronenko A, Song X.Image registration by minimization
of residual complexityIEEE Conference on Computer Vision
and Pattern Recognition, 2009. CVPR 2009. 2009. pp. 49–56.
[66] Woo J, Stone M, Prince JL. Multimodal registration via
mutual information incorporating geometric and spatial
context. IEEE Trans Image Process 2015;24:757–69.
[67] Kaur A, Kaur L, Gupta S. Image recognition using coefficient
of correlation and structural similarity index in
uncontrolled environment. Int J Comput Appl 2012;59.
[68] Kim J. Intensity based image registration using robust
similarity measure and constrained optimization:
applications for radiation therapy. Citeseer; 2004.
[69] Bailey DL, Townsend DW, Valk PE, Maisey MN. Positron
emission tomography: basic sciences. London: Springer;
2006.
[70] Collignon A, Maes F, Delaere D, Vandermeulen D, Suetens P,
Marchal G. Automated multi-modality image registration
based on information theory. Information Processing in
Medical Imaging. 1995. pp. 263–74.
[71] Wells WM, Viola P, Atsumi H, Nakajima S, Kikinis R. Multi-
modal volume registration by maximization of mutual
information. Med Image Anal 1996;1:35–51.
[72] Sabuncu MR. Entropy-based image registration. Citeseer;
2004.
[73] Kim YS, Lee JH, Ra JB. Multi-sensor image registration based
on intensity and edge orientation information. Pattern
Recognit 2008;41:3356–65.
89
[74] Liu X, Lei Z, Yu Q, Zhang X, Shang Y, Hou W. Multi-modal
image matching based on local frequency information.
EURASIP J Adv Signal Process 2013;2013:1–11.
[75] Riegler G, Urschler M, M, R., ther, Bischof H, Stern D.
Anatomical landmark detection in medical applications
driven by synthetic data. 2015 IEEE International Conference
on Computer Vision Workshop (ICCVW). 2015. pp. 85–9.
[76] Qin B, Gu Z, Sun X, Lv Y. Registration of images with
outliers using joint saliency map. IEEE Signal Process Lett
2010;17:91–4.
[77] Auer M, Regitnig P, Holzapfel GA. An automatic nonrigid
registration for stained histological sections. IEEE Trans
Image Process 2005;14:475–86.
[78] Likar B, Pernuš F. A hierarchical approach to elastic
registration based on mutual information. Image Vis
Comput 2001;19:33–44.
[79] Tomazevic D, Likar B, Pernus F. 3-D/2-D registration by
integrating 2-D information in 3-D. IEEE Trans Med Imaging
2006;25:17–27.
[80] Staring M, Van Der Heide UA, Klein S, Viergever MA, Pluim
JP. Registration of cervical MRI using multifeature mutual
information. IEEE Trans Med Imaging 2009;28:1412–21.
[81] Pszczolkowski S, Zafeiriou S, Ledig C, Rueckert D. A robust
similarity measure for nonrigid image registration with
outliers. 2014 IEEE 11th International Symposium on
Biomedical Imaging (ISBI). 2014. pp. 568–71.
[82] Kosiński W, Michalak P, Gut P. Robust image registration
based on mutual information measure. J Signal Inf Process
2012;3:175.
[83] Xuan Y, Jihong P. Elastic image registration using attractive
and repulsive particle swarm optimization. In: Wang T-D,
Li X, Chen S-H, Wang X, Abbass H, Iba H, et al., editors.
Simulated Evolution and Learning: 6th International
Conference, SEAL 2006. Proceedings. Berlin, Heidelberg:
Springer Berlin Heidelberg; 2006. p. 782–9.
[84] Fitzpatrick JM. The role of registration in accurate surgical
guidance. Proc Inst Mech Eng Part H: J Eng Med
2010;224:607–22.
[85] Crum WR, Hartkens T, Hill D. Non-rigid image registration:
theory and practice. Br J Radiol 2014;77(s2):S140–253.
[86] Lüders H, Comair YG. Epilepsy surgery. Lippincott Williams
& Wilkins; 2001.