This article has been accepted for publication in a future issue of this journal, but has not been
fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 <
Shape Sensing Techniques for Continuum Robots
in Minimally Invasive Surgery: A Survey
Chaoyang Shi, Xiongbiao Luo, Peng Qi, Tianliang Li, Shuang Song, Zoran Najdovski, Toshio
Fukuda, Fellow, IEEE, Hongliang Ren

Abstract—Continuum robots provide inherent structural
compliance with high dexterity to access the surgical target sites
along tortuous anatomical paths under constrained
environments, and enable to perform complex and delicate
operations through small incisions in minimally invasive surgery.
These advantages enable their broad applications with minimal
trauma, and make challenging clinical procedures possible with
miniaturized instrumentation and high curvilinear access
capabilities. However, their inherent deformable designs make it
difficult to realize three-dimensional (3D) intraoperative real-
time shape sensing to accurately model their shape. Solutions to
this limitation can lead themselves to further develop closely
associated techniques of closed-loop control, path planning,
human–robot interaction and surgical manipulation safety
concerns in minimally invasive surgery. Although extensive
model-based research that relies on kinematics and mechanics
has been performed, accurate shape sensing of continuum robots
remains challenging, particularly in cases of unknown and
dynamic payloads. This survey investigates the recent advances
in alternative emerging techniques for 3D shape sensing in this
field, and focuses on the following categories: fiber optic sensors
based, electromagnetic tracking based and intraoperative
imaging modalities based shape reconstruction methods. The
limitations of existing technologies and prospects of new
technologies are also discussed.
Index Terms—shape sensing, shape reconstruction, fiber
Bragg grating, continuum robot, electromagnetic tracking,
intraoperative imaging modalities.
Manuscript received May 30, 2016. This work is supported by the
Singapore Academic Research Fund under Grant R-397-000-227-112, NMRC
Bedside & Bench under grant R-397-000-245-511 and Singapore Millennium
Foundation under Grant R-397-000-201-592. C. Shi and X. Luo contributed
equally to this work. Corresponding authors: Xiongbiao Luo and Hongliang
Ren.
C. Shi is with Department of Mechanical and Industrial Engineering,
University of Toronto, Toronto, ON M5S 3G8, Canada (e-mail:
chaoyanghit@gmail.com).
X. Luo is with Department of Computer Science, Xiamen University,
Fujian 361005, China, (e-mail: xiongbiao.luo@gmail.com).
P. Qi is with Department of Control Science & Engineering, College of
Electronics and Information Engineering, Tongji University, Shanghai, China.
T. Li and H. Ren are with the Department of Biomedical Engineering,
National University of Singapore, Singapore, (e-mail: ren@nus.edu.sg ).
S. Song is with School of Mechanical Engineering and Automation, Harbin
Institute of Technology, Shenzhen, China.
Z. Najdovski is with the Institute for Intelligent Systems Research and
Innovation (IISRI), Deakin University, Victoria, 3216, Australia.
T. Fukuda is with Department of Micro-nano Systems Engineering,
Nagoya University, Nagoya, 464-8603, JAPAN.
0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
1
I. INTRODUCTION
I n recent years, continuum robots have offered significant
advantages and demonstrated great promise in terms of
technological advances and extensive clinical applications in
minimally invasive surgery (MIS) [1-4]. MIS procedures
involve performing delicate operations on anatomical
structures through small incisions or natural orifices along
tortuous paths inside the human body, resulting in both access
and operational constraints, and furthermore technical
challenges [2, 5-7]. Continuum robots provide not only
curvilinear and flexible accessibility through these small
incisions or orifices, but are also capable of generating large
forces at the distal ends to support various operations [1, 4, 8, 9].
They are defined as actuatable structures whose constitutive
materials form curves with continuous tangent vectors [1],
including concentric tube robots, active cable/tendon-driven
catheters and needles, single-backbone and multi-backbone
continuum robots, and pneumatically and hydraulically driven
continuum manipulators. Their compliant structures with high
flexibility and precision allow reaching targeted treatment sites
with complex morphologies and tortuous path access under
constrained spaces, and enable to complete complex and
delicate operations inside the patients’ body through small
incisions. Such advantages are beneficial for patients with
reduced blood loss, minimal trauma, fewer postoperative
complications and shorter recovery time, and improve the
current clinical procedures and make new workflow possible [1,
4, 7]. Consequently, they have been increasingly and
extensively introduced into various MIS procedures, including
otolaryngology, ophthalmic surgery, neurosurgery, abdominal
surgery and intravascular interventions, particularly for cardiac
surgery and stenting surgery [1, 7, 10-14].
However, to achieve precise and reliable motion control of
continuum robots used in these surgical procedures requires
accurate and real-time shape sensing. Due to their inherent
deformable design and inevitable collisions with the anatomy
during surgical procedures, accurately modeling their shape
remains a challenge [15, 16]. Intensive model-based shape
reconstruction methods that rely on kinematics and mechanics,
have been developed for closed-loop control, path planning and
collision detection [1, 16-25]. Their fundamental approach
typically endeavors to balance the tradeoffs among
mathematical model complexity, accuracy, and computational
expense [4, 15, 24, 26, 27]. The accuracy of these modeling
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 <
techniques is model dependent and relies on the identified
modeling parameters that suffer from inaccuracies and change
over time. This is especially encountered in dynamic scenarios,
experiencing dynamic friction, backlash and internal
deformation [26, 28-30]. Moreover, these models are sensitive
to unknown external payloads that might cause a large-scale
structural instability and have their kinematics altered [17, 20].
The majority of them do not deal with torsion conditions, which
can significantly reduce robot stiffness relative to out-of-plane
loads. Additionally, some of these methods are unable to meet
the requirements for real-time applications [17].
Despite the extensive model-based research centered on
kinematics and mechanics reported in the literature to date,
accurate intraoperative shape sensing of continuum robots
remains challenging. This is particularly emphasized in cases
of unknown and dynamic payloads exerted on the end
effector. 3D real-time shape estimation of continuum robots
remains an ongoing and active research field in the robotics
community, with concentrated efforts towards further
developing the closely associated techniques of closed-loop
control, path planning, human–robot interaction and
manipulation safety in MIS. Nevertheless, a general solution
remains
Besides
to be
thedeveloped.
model-based approaches, other emerging and
alternative techniques for shape reconstruction together with
tip localization have been recently proposed. Strain and
curvature sensing that rely on fiber Bragg gratings (FBG)
sensors have received an increase attention in the robotics
field. FBG-enabled sensing techniques are capable of
providing both real-time force measurement and shape
estimation without requiring the kinematics-based modeling.
These sensors also support easy integration with continuum
robots, and provide high biocompatibility for most of surgical
environments, [31, 32]. Electromagnetic (EM) tracking
techniques have been widely applied to track and localize
continuum robots within the human body, due to their
miniature size and freedom from line-of-sight constraints.
Such tracking techniques have not only been extended to
realize both tip tracking and shape reconstruction with
multiple miniature EM sensors attached along the continuum
robots, but are also used in conjunction with simplified robotic
modeling to achieve better accuracy and reduce computational
expense [33]. In addition, intraoperative imaging modalities
such as fluoroscopy, endoscopy and ultrasound can support
the most direct visualization or straightforward observation for
surgical continuum manipulators and human anatomy [34].
Various intraoperative imaging approaches have demonstrated
great potentials to estimate or reconstruct the shape of
continuum robots and consequently improve the current
clinical
Shapeworkflows
sensing [34,
techniques
35]. have been emerging and
beneficially employed in generalized continuum robots used in
MIS, including the extended concept of pseudo-continuum
robots that employ discrete-link or discrete-joint structures to
closely resemble continuum robots [1]. Existing review papers
on continuum robots have focused on their structural designs [1,
8], kinematic and mechanic modeling [15, 17, 20], and force
sensing based on fiber optic sensors (FOS) [31, 32]. In contrast,
this survey discusses the technological advances and recent
0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
2
progress enabled by shape reconstruction techniques on the
generalized continuum robots through the utilization of FBG
sensing, EM tracking and intraoperative imaging, in addition to
state-of-the-art applications.
II. FBG SENSING BASED SHAPE RECONSTRUCTION
FOS-based sensing techniques have been widely employed
for strain, force, torque, displacement and tissue temperature
measurements and pressure monitoring in various medical
applications. Their use encompasses magnetic resonance
imaging (MRI)-guided hyperthemic treatments, thoracic and
abdominal movement recording, respiratory and heart activity
monitoring [31, 32, 36-39]. They have the advantage of small
size, high elasticity and flexibility, and support easy
integration and miniaturized design of continuum robots with
minimal effects on their stiffness [40]. This favorably has little
effect on their modeling and control. Their inherent properties
of biocompatibility, non-toxicity, immunity from EM
interference and the absence of electrical connection allow
FOS to be suitable and intrinsically safe for most medical
applications, including the high EM environments of MRI
[32]. These FOS-enabled sensing techniques can be classified
into three categories: light intensity modulation (LIM)-based
FOS, interferometer-based FOS and FBG element-based FOS
[31, 32].
A. Comparison among FOS-enabled Sensing
LIM-based FOS sensing techniques measure the light
intensity reflection via an optical mirror, light coupling
between fibers or a macro-bending fiber configuration [32, 41].
These techniques have mainly been applied to the measurement
of tool-tissue interaction force, torque and pressure sensing.
They are temperature independent, and do not require complex
and expensive components, resulting in cost-effectiveness [31].
However, such LIM-based techniques suffer from undesired
drift that is generated by input light intensity changes and
unwanted fiber bending losses [32, 39]. Interferometer-based
FOS techniques typically involve the use of multiple reflective
mirrors to generate fringes, and are capable of measuring the
parameters that change the distance between the mirrors
induced by force, torque or pressure [31, 42]. However, both
techniques cannot support straightforward multipoint strain
measurements in the axial direction along the continuum robots
for direct shape estimation [32]. Approaches that have applied
LIM-based FOS with three straight or macrobend fibers in a
triangular configuration have been implemented to realize
indirect curvature or pose measurements on a short segment of
continuum robots [43, 44]. These implementations provide a
shape sensing method with low-cost and are shown to be
temperature independent. They typically related the output light
intensity changes caused by bending with the segment length
variation to estimate the curvature or pose information,
however, they did not consider the intensity loss due to twist
[43]. Therefore, their applications require a great deal of
pre-calibration between the bending angles and variation in
length. Alternatively, these approaches can be applicable for
measurements on short continuum robots due to the bending
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 <
loss. In addition, the complex intensity loss under multiple
bending modes poses a challenge for calibration, and has also
limited their main applications to the single bending mode. The
twisting angle measurement has not been involved, and needs
further investigation [43].
In contrast, FBG sensors are not affected by the change of
the input light intensity, and allow achieving a better strain
sensitivity and resolution to improve the procedural outcomes
[32]. Another advantage of FBG sensing is to support
distributed strain measurements along the continuum robots
using multiple FBG sensors inside a fiber, and enable both
force and shape sensing of the continuum robots [39, 45, 46].
However, FBG sensing-enabled shape estimation techniques
require a high-cost investment for the optical spectrum
interrogator with multiple channels, especially when multiple
fibers are employed for temperature compensation and twist
angle determination.
B. Shape Reconstruction Based on FBG Sensing
An FBG sensor is normally written onto a short segment of
an optical fiber, and it is able to reflect only a narrow range of
particular wavelengths of the input light with the full spectrum
while transmitting all the other ranges [40, 47]. The optical
fibers that are embedded with multiple FBG sensors need to
be connected to an interrogator, which detects the wavelength
of the reflected light through each FBG sensor. The reflected
wavelength is affected by the mechanical strain and
temperature change on the FBG sensor [39, 47], and this
principle is commonly used for temperature, force and
curvature sensing. When there is an external force perceived
by a shape sensor consisting of FBG elements, the induced
deformation can produce mechanical strain, resulting in the
central wavelength shift for the reflected light by the FBG
sensors.
FBG sensing-based shape reconstruction techniques mainly
rely on curvature estimation, which is associated with the
strain calculated from the wavelength shift. These techniques
have been increasingly applied for continuum robots under
small deformation, such as steerable interventional needles for
biopsy and ablation. They have inherent properties of high
stiffness to support the nearly perfect strain transfer to the
attached FBG sensors.
Park et al. first introduced FBG sensing into a small-gauge
MRI-compatible biopsy needle to realize the real-time shape
estimation of the deflection profile [48]. This triplet shape
sensor consisted of three optical fibers embedded with two
FBG sensor array nodes, and was placed along the three
grooves made on the needle surface. Each node has a triangular
configuration that consists of 3 FBG sensors distributed at an
equal interval of 120 degrees, and these nodes were distributed
at different positions along the needle [48]. The beam theory
reconstructed the deflected needle shape using the integral
transform of the curvature calculation from the FBG sensors.
The experiments have been performed inside a water bath
within an MRI environment. After image processing for MRI,
the reconstruction results achieved root mean square (RMS)
values of tip deflection errors of 0.38 mm and 0.28 mm for
0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
3
different planar cases, with the tip deflection in the range of ±15
mm. Such a biopsy needle equipped with radiofrequency (RF)
tracking coils was used for real-time autonomous interventional
scan plane control under MR imaging, and yielded a large
average RMS error of 4.2 mm [49]. The same configuration
with two sets of FBG sensors was employed and attached along
a trocar needle for liver tumor ablation by Henken, but utilized
different sensor placement positions [50]. Frenet-Serret
formulas with Euler integration converted curvature
information into shape profile [51, 52], and yielded an average
tip accuracy of 1.1 mm with tip deflections of up to 12.5 mm in
free space. However, these two groups only considered needle
deflection for 2D planar cases in either water bath or free space.
To achieve the real 3D out-of-plane needle shape
reconstruction, Roesthuis et al. proposed the use of three fibers,
each embedded with four sets of FBG sensors [53, 54]. This
design supports the measurement of curvature information at
more locations along the needle shaft. The deflection
experiments have been performed in both free space and a
soft-tissue simulator (gelatine phantom). The maximum tip
tracking error for experiments completed within the soft-tissue
simulant was 0.74 mm, compared to respective errors of 3.77
mm and 2.20 mm for the kinematics- and mechanics-based
models. This shape reconstruction outcome enabled to realize
real-time closed-loop control of a steerable Nitinol needle and a
tendon-driven needle within 3D workspace to access the target
sites in soft-tissue phantoms [54, 55].
These shape estimation techniques have also been extended
for other common surgical continuum robots, such as
cable-driven continuum robots, single and multi-backbone
continuum robots, and pre-curved continuum robots. Roesthuis
et al. also applied the triplet configuration in [53] on a
multi-segment tendon-driven continuum robot with a hollow
backbone for general MIS uses [56]. The triplet optical fibers,
each with four sets of FBG sensors, were attached into three
grooves made along a nitinol wire that was placed inside the
hollow backbone. The shape estimation was achieved with a
mean tip tracking error of 0.67 mm for the circular trajectory
[56]. This approach was further improved by combining
kinematic rigid-link modeling with FBG-based shape sensing.
The improvement realized closed-loop control to steer this
continuum robot and yielded lower tip trajectory tracking errors
of 0.24 mm and 0.09 mm for the two cases with moving object,
respectively [57]. Dupont et al. used FBG sensors with the
triangular configuration on a continuum robot with micro
forceps, and embedded the sensors within the compliant
polymer tubes. These tubes have a lower stiffness than that of
commonly used optical fibers, but allow for deformations with
large curvatures [58]. A strain transfer model was proposed to
cope with the associated low strain transfer ratio, and has been
validated by shape reconstruction experiments with an average
tip tracking error of 0.84 mm. Liu et al. placed 2 optical fibers,
each embedded with 3 FBG sensors, along the two parallel
bending sides of a cable-driven continuum robot designed for
osteolysis treatments [59], and achieved a distal tip tracking
accuracy of 0.40 mm for free bending cases [60]. Differently, a
novel helically wrapped FBG sensor has been designed to be
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 < 4

Table 1 FBG sensing-based shape reconstruction techniques

Research Design Feature Application & Test Characteristics
Group Environment
Park et al., 3 fibers with 2 FBG nodes each; MRI-guided biopsy needle Root mean square (RMS) of tip tracking error:
[48, 49] Triangular configuration interventions (150 mm in 0.38 mm and 0.28 mm in the x–y and y–z
length); planes, respectively, with tip deflection in the
Water bath range of ±15 mm.
Henken et 3 fibers with 2 FBG nodes each; Trocar needle intervention for METE was 1.10 mm with tip deflection up to
al., [50] Triangular configuration liver tumor/cancer (191 mm in ±12.5 mm. ADTE was 8.8%.
length);
Free space
Dobbelsteen 3 fibers with 4 FBG nodes each; Tendon-driven steerable needle METE was 2.6 ± 1.1 mm with tip deflection up
et al., [55] Triangular configuration for liver biopsy and RFA; to ±30 mm at a depth of 100 mm below the
Gelatin phantom tissue phantom surface. ADTE was 8.6%.
Misra et al., 3 fibers with 4 FBG nodes each; Flexible nitinol biopsy needle METE in free space was 0.2 ± 0.21 and 0.38
[53, 54, 56] Triangular configuration with a bevel tip (172 mm); ±0.18 mm for in-plane single bend and double
Free space and gelatine phantom bend, and 1.22 mm for out-of-plane. METE in
phantom was 0.42 ± 0.11 and 0.12 ±0.11 mm
for in-plane single bend and double bend, and
0.37 ± 0.2 mm for out-of-plane. ADTE was
0.8% in free space and 1.7% in phantom.
3 fibers with 4 FBG nodes each;
Continuum robot with a single METE is 0.24 and 0.09 mm with maximum
Triangular configuration backbone for general MIS; errors of 1.37 and 0.52 mm for two cases.
Free space
Dupont et 3 soft polymer tubes with 5 Wire-driven continuum robot METE was 0.84 ± 0.62 mm (96.7% less than
al., [58, 61] FBG nodes each; with forceps for general MIS; 2 mm error).
Triangular configuration; Free space
Liu et al., 2 fibers with 3 FBG sensors Cable-driven dexterous continu METE was 0.40 ± 0.30 mm for
[60] each; -um robot for osteolysis bending/straightening cycle.
Parallel configuration; treatment;
Free space
Xu et al., 1 helically wrapped fiber with 3 Pre-curved continuum robot for Not reported
[62] FBG Sensors; neurosurgery;
Single element configuration; Free space
Shen et al., 4 fibers with 5 FBG nodes each; Colonoscope & endoscope MERS was 4.5 mm.
[63, 64] Orthogonal configuration; for intestinal diseases;
Colon in the porcine
Shi et al., 3 fibers with 8 FBG nodes each; IVUS catheter in TAVI Not reported
[52] Triangular configuration procedure;
Free space
FBG, fiber Bragg gratings; MRI, magnetic resonance imaging; RFA, radiofrequency ablation; METE, mean error of tip tracking; MERS,
mean error of robot shape; IVUS, intravascular ultrasound; TAVI, trans-catheter aortic valve implantation; ADTE, average deflection to tip
error.
embedded into helical grooves made along the continuum 4.5mm for shape reconstruction [66]. To improve the accuracy,
robot shaft for hybrid sensing. The corresponding nonlinear Shi et al. used a triplet shape sensor that consisted of three
force-curvature-strain model was developed to support optical fibers with 24 FBG sensors in the triangular
simultaneous curvature, torsion, and force measurements [65]. configuration to reconstruct the shape of Intravascular
Attempts have also been made towards shape estimation for Ultrasound (IVUS) catheter used in the trans-catheter aortic
other continuum robots with large deformations, such as valve implantation (TAVI) procedure. This design achieved a
cable-driven catheters and endoscopes. The majority of reasonable accuracy in 2D free space, but the accuracy for the
endoscopes and catheters have low stiffness, resulting in lower 3D shape estimation remains under investigation.
strain transfer efficiency onto FBG sensors. Their lengthy The comparison and summary of these applications based on
structures and complex bending requirements require FBG-enabled shape reconstruction are presented in Table 1.
employing more FBG sensors to provide sufficient curvature The commonly used fiber configurations for shape
information. Shen et al. proposed to use an orthogonal reconstruction based on FBG sensing include a single optical
configuration that consisted of four optical fibers, each with fiber configuration, an orthogonal configuration, and triangular
five FBG sensors, for shape estimation of endoscopes and configuration embedded with FBG sensors. The triplet design
colonoscopes [64, 66, 67]. This orthogonal configuration with a triangular configuration supports temperature
supported the temperature compensation and curvature profile compensation, enables to remove common terms such as noise
measurement, but suffered from poor resolution to determine and axial strain, and allows for measurement of twist angles.
the twist angle [68]. In-vivo experiments were performed in the The advantages of this approach have been validated to achieve
colon inside a live swine and achieved a large mean error of better accuracy for both force sensing and shape estimation in

0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 <
comparison to the other two configurations [45, 58]. There are
two commonly used terms to quantify the error, including the
mean error of tip tracking (METE) and the mean error of robot
shape (MERS), as well as average deflection to tip error (ADTE)
for needle shape reconstruction. Although the accuracy
requirements depend on different clinical procedures, tip
position errors of 1-2 mm are considered sufficient for most
minimally invasive surgical practice [69]. The shape
reconstruction using FBG sensing for continuum robots with a
higher stiffness can achieve reasonable accuracy for MIS
procedures. However, FBG sensing-based shape estimation
remains to produce large errors for shape reconstruction of
steerable endoscopes and catheters.
Error analysis of FBG-based shape sensing indicates that
the sensor number and placement position can significantly
affect the accuracy of shape estimation [61, 70-72]. To solve
these issues, a strategy of determining FBG sensor number
and placement positions with respect to arc length was
proposed for shape sensing of concentric tube robots [61].
This approach defined the numerical shape reconstruction
model in accordance with the sensor number and position. It
investigated optimization algorithms to determine the sensor
locations by minimizing the shape and tip errors between this
shape estimation model and a mechanics-based model.
Numerical experiments suggested that high accuracy for shape
reconstruction of concentric tube robots can be achieved with
a small number of FBG sensors. However, its effectiveness
needs to be further investigated and validated experimentally.
Payo et al. presented another method to select the appropriate
sensor number based on the deflection magnitude that depends
on the magnitude of the applied force and torque at the distal
tip. Then, the application of the Chebyshev criterion allowed
to determine the placement of FBG sensors, and achieve better
accuracy for the tip deflection [73]. Experiments were
conducted on a single-link continuum robot, which can only
deform in 2D space.
In addition, strain transfer reduction experienced in FBG
sensing-enabled shape estimation was seldom reported. The
currently existing implementations or approaches generally
assume that strain on continuum robot is perfectly transferred
to the attached FBG sensors, which is reasonable for
continuum robots with high stiffness. A strain transfer model
between a compliant concentric tube robot and the attached
soft host polymer tubers embedded with FBG sensors was
proposed to improve the accuracy of shape reconstruction
[58]. This concept needs to be further investigated to improve
shape estimation of continuum robots with lower stiffness and
large deformation, especially for lengthy steerable catheters
and endoscopes [52, 66].
III. ELECTROMAGNETIC TRACKING BASED SHAPE
RECONSTRUCTION
EM tracking systems that mainly rely on the working
principle of mutual induction, are capable of localizing EM
receivers working within their tracking workspace produced
by the EM field generator [33]. They have been employed for
continuous and real-time localization and tracking of
0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
5
Fig.1 EM tracking-based shape sensing. a)-d) shape
reconstruction for a surgical snake robot, a wire-driven
continuum robot with multiple sections, a continuum tubular
robot used in MIS, respectively; adapted and reprinted with
permission from [90, 93, 94]. e)-f) shape estimation for various
catheters used in intravascular interventions; adapted and
reprinted with permission from [101].
continuum robots, and have been successfully integrated into
several commercial products [74, 75]. The miniature size of
EM sensors and freedom from line-of-sight restrictions,
demonstrate unique capabilities to enable to track and localize
continuum robots within the human body without changing
robots’ mechanical properties [76, 77]. As a result, EM
the
tracking techniques have been widely applied to many clinical
applications such as endoscopic, orthopedic, and laparoscopic
surgeries [78-82], needle-based biopsy and tumor/cancer
ablation procedures [33], and catheter-based intravascular
interventions [83-86].
In recent work, multiple miniature EM sensors have been
attached along the continuum robots, and as a result, have
been extended to realize both distal tip tracking and shape
reconstruction [87]. To overcome the disadvantage of discrete
pose information captured from the distributed EM sensors,
the EM tracking approach has been employed in conjunction
with the robotic kinematic models to achieve accurate shape
sensing.
Choset et al. employed a single 5-DOF EM sensor at the
distal end of a highly articulated surgical snake robot (HARP
robot) used in single-incision MIS, as shown in Fig.1 a). A
probabilistic filtering approach that used an extended Kalman
filter [88, 89] to fuse the EM tracker pose data with a kinematic
model that defines the motion of the robot, was proposed to
realize shape estimation [90, 91]. Experiments were completed
on a benchtop heart phantom to validate the method, and
demonstrated an average error of 8.01 mm, compared with
23.73 mm from the kinematics-based method. Attempts have
also been made on a porcine heart under in-vivo image
guidance to demonstrate its effectiveness, but did not provide
quantitative data due to lack of ground truth. This work
demonstrated that the shape estimation problem is observable
using Lie derivative analysis, and showed that the robotic shape
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 <
can be successfully recovered and can achieve a better accuracy
with a sufficient number of EM trackers. With this knowledge,
Song et al. mounted three EM sensors along a two-section
wire-driven continuum robot, each placed at the distal end of
each section and base, as shown in Fig.1 b). A real-time shape
reconstruction and tip tracking algorithm was proposed by
fitting multiple quadratic or cubic Bézier curves based on the
pose information captured from EM tracking and the robot
section length information [87, 92]. Through both simulation
and experimentation, the proposed method was verified and
achieved a mean position error of 1.7 mm. This strategy was
also applied on a continuum tubular robot for nasopharyngeal
biopsy with three EM sensors attached to the concentric tubes
[93]. One EM sensor was mounted on the outer tube at a certain
distance from its tip, and the other two were attached to the tips
of the outer and inner tubes, respectively, as illustrated in Fig.1
c). With this configuration and the above-mentioned algorithm,
the shape reconstruction of both inner and outer tubes was
completed with a mean error of approximately 1 mm. Fig.1 d)
demonstrates a method that combined both gravitational and
gyroscopic sensing based on micro-inertial sensors to realize
shape reconstruction for a 2-DOF snake robot used in MIS [94].
Two micro-inertial sensors were attached on each universal
joint of the consecutive snake robot with three segments.
Detailed experimental results in free space have demonstrated
the high accuracy for joint angle estimation using the proposed
method.
EM tracking-based shape sensing techniques have also been
extended for other continuum robots, such as active biopsy
and ablation needles, and steerable cardiac catheters.
Fichtinger et al. incorporated a kinematic needle model with
the pose information measured from two EM trackers, which
were respectively mounted at the base and tip of the needle.
An extended Kalman filter was then applied to deal with this
nonlinear stochastic fusion process for shape estimation and
tip deflection detection [95, 96]. Through this method, the
needle tip estimation error reduction was in the range of 28%
(from 1.8 to 1.3 mm) to 74% (from 4.8 to 1.2 mm). Dore et al.
presented a catheter shape reconstruction and navigation
technique based on probabilistic fusion of in-situ real-time EM
tracking with physically-based simulation of the catheter
kinematics [97]. As shown in Fig.1 e), a 6-DOF EM sensor
was mounted at the distal tip, and six 5-DOF EM sensors were
attached along the RF ablation catheter with a uniform interval
of 125 mm. A probabilistic fusion framework based on
Kalman filtering was applied to fuse the catheter insertion
modeling algorithm and the EM tracking data. The proposed
approach was validated in the 2D in-vitro silicone aortic
phantom experiments to achieve an average error of 2.1 mm
and supported visualization of the catheter without requiring
continuous
Both offluoroscopy
theoreticalimaging.
and experimental results have
demonstrated that the distal-end deflection profile of catheters
can be estimated as a circular arc [98]. Based on this
approximation, Fu et al. attached two EM sensors at the distal
end of an active catheter actuated by shape memory alloy
(SMA) and a cardiac ablation catheter, respectively, to
determine their distal shape between these two EM sensors [99,
0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
6
100]. Condino et al. extended this method to propose a general
EM navigation framework for intravascular interventions by
developing sensorized catheters attached with two EM trackers
[101]. One EM tracker was mounted at the catheter distal tip,
while the other one was placed a few centimeters below this
point along the catheter to acquire the curvature information of
the distal part between these two EM sensors. To develop a
sensorized catheter, these sensors can be either attached on the
catheter external surface or embedded within a customized tube
utilizing a two-hole cross section, as show in Fig.1 f). These
configurations were validated for intravascular navigation
using several kinds of catheters and silicone phantoms to
realize distal shape estimation and enable reduced dose of
X-ray radiation and contrast agent injection [101, 102]. Table 2
summarizes the comparison among the different shape
reconstruction applications enabled by EM tracking.
Unfortunately, EM tracking systems also suffer from
limitations associated with their working principle. They are
prone to produce measurement errors due to magnetic field
distortions caused by magnetic and conductive objects,
especially in a dynamic clinical environment [75]. For
instance, the tracking accuracy can be compromised due to the
surrounding CT/MRI scanners and the electronic and metal
instruments used during the procedures [33]. Such generated
interference degrades the positioning precision and
deteriorates the procedure outcomes and the reliability of EM
tracking in clinical practice. In addition, EM tracking systems
have a limited workspace, and provide non-uniform
measurement accuracy throughout the tracking volume. They
can only generate the highest and uniform accuracy around the
center of the tracking volume [77].
To solve these issues, a simultaneous localization and
mapping approach has been developed to accurately estimate
the pose of the tracked instruments while creating a field
distortion map [76, 103]. A motion model that was combined
with observations of redundant EM sensors has been
developed to realize dynamic field distortion compensation
through the real-time update of distortion mapping, supporting
simultaneous localization and calibration. Experiments
performed in both research and clinical settings demonstrated
significant error reduction, and showed great potential to allow
reliable and utility EM tracking in the real clinical procedures.
Another approach attached an EM field generator onto a robot
manipulator for accurate and robust positioning to obtain
increased and uniform tracking accuracy [77]. This EM
servoing paradigm controlled the field generator to follow the
EM sensors, and kept the tracked sensors close to the center of
the tracking volume to generate the uniform measurement
accuracy. This method also enabled a large tracking
workspace for clinical use.
IV. INTRAOPERATIVE IMAGING BASED SHAPE ESTIMATION
AND GUIDANCE
Vision-based shape sensing techniques can provide direct
and accurate measurements for shape reconstruction of
continuum robots, without kinematic modeling and hardware
modifications. They have been validated to provide better
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 < 7

Table 2 EM tracking based shape sensing techniques

Research Configuration Sensing method Application & Test Characteristics
Group Environment
Song et al, 3 EM sensors with Fit multiple quadratic Bézier Wire-driven continuum robot MERS was 1.7 mm.
[92, 104] each on the distal end curves based on EM pose data for trans-oral surgery;
of every section and the robot section length Free space
Tully et al., 1 EM sensor at the Apply a filtering approach to fuse Articulated snake robot for MERS was 8.01mm for
[90, 91] distal end of the robotic kinematic models of cardiac diseases; phantom data; No data for in-
snake robot advancing, retracting, and steering A heart phantom and a vitro experiments.
with EM pose data porcine heart
Zhang et 2 magnetic inertial Incorporates both gravitational Articulated snake robot with Mean angle errors for yaw
al., [94] sensors on two and gyroscopic sensing to a hybrid motor/tendon and pitch were 0.2651° ±
robotic joints each calculate the rotation and design for endoscopy; 0.3823° and 0.4446° ±
orientation angles Free space 0.8387° respectively.
Sadjadi et 2 EM sensors Fusion between a kinematic Bevel shaped tip needle for Tip error reduction was in
al., [95, 96] installed on the needle deflection model and EM biopsy and ablation; the range of 28% (from 1.8
needle base and tip pose data Anthropomorphic prostate to 1.3 mm) to 74% (from 4.8
phantoms to 1.2 mm) with an insertion
depth of 150 mm.
Dore et al., 7 EM sensors Probabilistic fusion of EM
RFA catheter for cardiac MERS was 2.1 mm in 2D
[97] distributed along a tracking and a physically-based
interventions; compared with 3.2 mm from
catheter insertion model
Silicone phantom of the the model-based method.
Fu et al., 2 EM sensors at the aorta
Combination of a simplified
Ablation catheter & active MERS was 1.5 mm after
[99, 100] distal part of a kinematic model with EM pose
catheter for intravascular image registration.
catheter data interventions;
Silicone phantom of the
Condino et 2 EM sensors at the Combination of a simplified General catheter for MERS was 1.2 ± 0.3 mm.
al., [101, distal part of a kinematic model with EM pose intravascular interventions;
102] catheter data Abdominal aortic aneurysm
model
EM, electromagnetic tracking; RFA, radiofrequency ablation; METE, mean error of tip tracking; MERS, mean error of robot shape;
accuracy than solely using kinematic modeling [105-108]. acquired at different poses [112]. This method first extracted
Thus far, such technologies have been implemented using the centerline of the cannula in two different views, and then
non-medical imaging modalities for shape sensing of employed epipolar geometry analysis to estimate the 3D points
continuum robots. They involve employing the commonly for shape reconstruction by minimizing the back-projection
used stereo or infrared cameras and distance sensors for error using Newton’s method. The experiments were performed
experimental accuracy validation [106-110]. However, on an anthropomorphic liver phantom with tumors and vessels,
surgical continuum robots used in MIS mainly rely on and achieved a mean error of 0.473 ± 0.353mm with a
standard intraoperative imaging modalities, such as maximum value of 1.075mm. Wanger et al. improved this
endoscopy, fluoroscopy and ultrasound, to guide surgeons to bottom-up strategy using a flux driven topology to extract the
access the treatment sites and perform operations [34]. These centerlines of interventional curvilinear devices of guidewires
medical imaging modalities provide the most straightforward and catheters. Specifically, the centerlines were determined
and widely available mediums to detect surgical continuum using Dijkstra’s algorithm to minimize the curvature and
robots and anatomic tissue. This section discusses the recent distance between adjacent segments as well as the path
progress on vision-based shape sensing and guidance difference with previous frames. Then, the 3D shape was
techniques that rely on these intraoperative imaging modalities reconstructed based on epipolar geometry [113]. Hoffman et al.
for clinical applications. applied an optimal selection algorithm to determine feature
point correspondences obtained from epipolar constrains, to
A. Fluoroscopic Imaging Based Shape Reconstruction
enhance the shape reconstruction of an EP catheter [114, 115].
Biplane fluoroscopic imaging systems provide two This algorithm was able to detect complex catheters with large
simultaneous views at various poses to estimate the shape of curvatures, but was mainly designed for the detection of
surgical continuum robots based on the triangulation principle. circular mapping (CM) catheters used during EP procedures.
The biplane fluoroscopic imaging-driven shape reconstruction This approach was further improved by introducing a feature
techniques mainly consist of two methods (1) a bottom-up learning-based framework to recognize arbitrary line-shaped
strategy that detects the centerline of the continuum robots and catheters used in the EP procedures. The improved method was
(2) a top-down manner that implements initialization of a 3D validated on fluoroscopic images, and achieved mean shape
curve to approximate these features [111]. errors of 1.8 ± 1.1 mm and 2.2 ± 2.2 mm on phantom and
Burgner et al. proposed a bottom-up strategy that used a clinical data, respectively [116].
projective-invariant triangulation method for automatic shape Alternatively, Delmas et al. reported a top-down strategy
segmentation of a two-tube active cannula on biplane images that used 3D curve segments as features rather than the 3D
point

0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 <
Fig.2 a)-c) Shape estimation based on monoplane fluoroscopy
for a continuum tube robot designed for bronchoscopy, a robotic
steerable catheter and a concentric tube robot used in trans-nasal
surgery: adapted and reprinted with permission from [34, 35,
119]. d)-e) shape estimation for continuum endoscopic
instruments and a tendon-driven continuum robot designed for
arthroscopy: adapted and reprinted with permission from [124,
127].
cloud calculated from epipolar geometry analysis [117]. The
most coherent curve in terms of continuity and curvature was
determined after the optimization for shape estimation. This
strategy has been performed on a cerebral vascular phantom,
and demonstrated to effectively reconstruct the shape of
endovascular curvilinear devices using complex 3D curves
without accurate 2D image segmentation results. Using a
similar strategy for shape estimation, Mauri et al. proposed a
method using B-snakes to formulate the vascular and catheter
detection and reconstruction as an energy minimization
problem [118].
While the methods mentioned above work well, they
critically depend on biplane C-arm systems that are limited to
large costs, large radiation dosage, or operational workspace
constraints during intravascular interventions. To solve these
limitations, recent shape reconstruction methods that are based
on monoplane C-arm systems for surgical continuum robots,
have also been developed by combining with additional
information of kinematic modeling and prior feature
knowledge of surgical continuum robots.
Lobaton et al. presented a shape sensing method for a
concentric tube robot used in bronchoscopy surgery on the
basis of monoplane fluoroscopic imaging. This approach
integrated a deformable surface parameterization of kinematic
modeling over time and data extracted from an optimally
selected set of fluoroscopic images. It took advantage of
probabilistic priors and numeric optimization to determine the
optimal C-arm positioning to capture robust features [35]. The
shape estimation results are shown as red dots in Fig.2 a), and
reached a notable accuracy of around 0.8 mm. However, this
method was only evaluated on simulated data, and required
offline learning of the basis functions that model the
deformation. Vandini et al. realized shape reconstruction and
localization of a Hansen artisan robotic catheter inside a
silicone aortic phantom, as illustrated in Fig.2 b1). This
implementation used monoplane fluoroscopic images captured
0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
8
at the optimal C-arm positioning and offline appearance priors
of the catheter features [119]. The extracted 2D centerline was
incorporated with the offline database of visual appearance
priors to obtain its 3D orientation information for initial shape
estimation. This estimation was then used to determine the
optimal C-arm positioning for better feature extraction. Using
fluoroscopic imaging under this optimal positioning, this
method repeated the above steps to obtain the optimal shape
reconstruction, as shown in Fig.2 b2). However, both methods
require adjusting the C-arm positioning to achieve accurate
shape reconstruction, which is not always acceptable within the
current clinical workflow, or not available due to the absence of
robotic C-arms that support accurate control.
To address the inconvenient adjustment problem in surgical
workflow, Vandini et al. proposed an automatic and real-time
shape sensing method for a concentric tube robot designed for
trans-nasal surgery, by combining visual information extracted
from monoplane fluoroscopy with the kinematic model of this
robot [34]. A fast 2D/3D non-rigid registration was performed
to fuse 2D features extracted from the intraoperative image
and 3D shape estimation calculated from forward kinematics.
The proposed algorithm did not require any prior information
or the repositioning of the C-arm, making it applicable in the
clinical setting. It was also robust to kinematic uncertainties
and deformation induced by unknown payloads.
Consequently, this method achieved a more precise
performance for shape reconstruction with an average error of
0.88 mm on a skull phantom, compared to kinematics- and
vision-only reconstruction. The reconstructed shape is
demonstrated in Fig.2 c). Similarly, Otake presented to use the
intraoperative monoplane fluoroscopy to simultaneously
estimate the pose and kinematic modeling parameters of a
surgical snake-like robot designed for hip osteolysis treatment,
followed by a piecewise-rigid 2D/3D registration to fuse the
information together to generate 3D shape estimation [120].
Papalazaroua et al. presented shape reconstruction of
curvilinear catheters by combining a framework of non-rigid
structure-from-motion with robotic modeling that used a low-
dimensional parameterization of catheter deformation [121].
Unfortunately, this method requires multiple monoplane views
with a small view separation to retrieve the deformable 3D
pose from the 2D projections.
Despite the extensive research on shape sensing that has
been made using fluoroscopic imaging, most of these
techniques pose challenges due to interference with the
clinical workflow, and suffer from extensive exposure to
radiation and a large dose of contrast agent usage. The
robustness of these developed algorithms needs further
investigation, and the high computational expense needs to be
reduced for future clinical application.
B. Endoscopy Based Shape Reconstruction
Unlike X-ray fluoroscopy, endoscopic procedures employ a
surgical instrument called an endoscope. It is typically
integrated with video cameras at its distal tip to visually guide
the interventions. The shape and distal tip poses of flexible
endoscopes and their continuum instruments (e.g., biopsy
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 <
needles and forceps) are essential for operations during
endoscopic surgery. As a result, shape reconstruction plays an
important role to estimate the endoscope movement [122].
Basically, endoscopic shape reconstruction consists of three
main aspects (1) flexible endoscopes themselves, (2)
continuum instruments, and (3) surgical field surface.
Reconstructing the flexible endoscope shape is greatly
beneficial to guide the surgeon’s manipulation, and determine
the distal tip location in endoscopic interventions such as
bronchoscopy and colonoscopy [78, 79]. In addition to model-
based approaches, two methods for the endoscope shape
reconstruction that use FBG and EM sensor array attached
along the endoscope shaft have been described in the previous
two sections. The shape estimation of a continuum instrument
used in endoscopy is generally performed by either marker-
based or marker-less methods [123-125]. To determine the
shape and tip pose of a continuum instrument, Reilink et al.
proposed a maker-based approach that segmented the
positions of several colored markers attached along the distal
part of an endoscopic instrument (Anubis endoscope) to
update the state of its kinematic model [123], as shown in
Fig.2 d). This group also implemented another marker-less
approach that employed the positions of three feature points
captured at the distal part to update the kinematic model.
These two approaches were implemented on a colon phantom
and achieved a similar RMS error level ranging from 1.1 mm
to 1.8 mm for each direction [123]. To overcome the
uncertainties on kinematics and mechanics based modeling,
Cabras et al. extended the marker-based approach that applied
a supervised learning method for RBF network training to
approximate the 3D pose reconstruction using the image
coordinates of three colored markers and 3D position of the
instrument tip [126]. Vandini et al. explored a model-less
method that estimated the shape of a tendon-driven continuum
robot designed for arthroscopy, as displayed in Fig.2 e). This
method performed tracking on endoscopic images captured
from a knee phantom while considering its workspace
constraints [127]. Compressive tracking was employed to
model and update the appearance features in an online
learning framework, and combined with the workspace
constraints to accurately reconstruct the 3D shape of the
continuum robot by minimizing an energy function.
Surgical field surface reconstruction aims to expand the
viewing angle, produce tissue surface reconstruction and track
tissue deformation. It basically employs computer vision
techniques, such as image stitching and mosaicking, visual
simultaneous localization and mapping, stereo 3D
reconstruction [128-132]. These techniques are used to
process endoscopic video images to deal with the
disadvantages of a limited field of view, and provide
orientation and scaling of the scene image for interaction.
These techniques also support multimodal information fusion
with ultrasound, computed tomography (CT) or magnetic
resonance images to enhance intraoperative surgical guidance
and robot control [128, 133, 134].
The majority of endoscopy-based image processing
techniques have been developed for real-time large panoramic
0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
9
view and 3D surgical scene reconstruction to provide
human-machine interaction and intra-operative information for
surgical platforms [128, 135, 136]. With the improvements to
the endoscopic imaging resolution and dexterity of endoscopes,
the endoscope pose can be easily adjusted to enable direct and
high-quality visualization of continuum robots and tool-tissue
interactions. Therefore, tip estimation and shape reconstruction
of continuum instruments that rely on endoscopic imaging for
accurate control will become less important.
C. Ultrasound Imaging Based Shape Reconstruction
Ultrasound imaging provides another intraoperative
alternative to optical and fluoroscopy imaging, and supports
accurate and consistent positioning of instruments without
exposure to ionizing radiation. Although the resolution of
ultrasound imaging is low, it can detect depth information of
continuum robots and tissues in real-time. Ultrasound imaging
has been applied to support and guide certain minimally
invasive interventions, such as peripheral and central venous
access, needle-based biopsy and radiofrequency ablation for
liver and lung cancers/tumors, and cardiac catheterization
[137-140]. Recent progress involves tracking tissue
deformation and tip position of continuum robots, for
example, concentric tube robots, steerable needles, and cardiac
catheters [137, 141-144]. Furthermore, ultrasound has also
been used to realize shape estimation. Ren et al. proposed an
algorithm termed tubular enhanced geodesic active contours,
which enhanced the tubular structure of continuum robots
while reducing imaging artifacts. This algorithm was tested
with a porcine heart and reconstructed the distal shape of
continuum robots using 3D ultrasound [145, 146].
Additionally, curvature estimation of the steerable needle was
realized by integrating a biomechanical model with
ultrasound-based image processing for path planning and
further shape reconstruction [147]. The experiments have been
validated on both a gelatin phantom and chicken breast tissue.
The issues associated with ultrasound-guided interventions
remain challenging due to low resolution, signal-to-noise ratio,
and a variety of imaging artifacts generated by the continuum
robots and instruments, particularly for the metallic types.
Ultrasound-driven shape reconstruction also suffers from
heavy computation and low accuracy, which is substantially
lower than using fluoroscopy images for shape reconstruction
[35]. To address these concerns, the fusion of ultrasound and
fluoroscopic imaging modalities has been proposed to guide
the trans-catheter aortic valve implantation (TAVI) and
cardiac catheterization procedures, due to their complementary
properties [148-150]. X-ray fluoroscopy provides 2D high
contrast imaging of continuum robots, while ultrasound
supports depth information to visualize them and tissue. This
fusion depends on a 2D/3D registration procedure that online
or offline aligns the transesophageal echo (TEE) probe visible
on fluoroscopic images [148]. Wu et al. extended this fusion to
reconstruct the shape of EP catheters. After registering the
corresponding fluoroscopic images to ultrasound frames on the
basis of a previously captured TEE probe model, the extracted
catheter shape from fluoroscopy was transferred to constrain
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 < 10

Table 3 Intraoperative imaging modalities based 3D shape construction

Group Image Methods Application & Test Characteristics
modality Environment
Burgner et Biplane Triangulation; Two-tube active cannula for MERS was 0.473 ± 0.353 mm with
al., [112] fluoroscopy Bottom up strategy intravascular intervention; a maximum error of 1.075 mm on
A liver phantom phantom experiments
Wanner et Biplane Triangulation; Curvilinear devices for MERS were 0.54 ± 0.33 mm and
al., [113] fluoroscopy Bottom up strategy intravascular intervention; 0.41 ± 0.08 mm on phantom and
Phantom and cadaver cadaver experiments, respectively.
Hoffmann et Biplane Triangulation; Circular mapping catheter MERS was 0.7 mm ± 2.0 mm using
al., [114, fluoroscopy Bottom up strategy for cardiac intervention; clinical data for CM catheter only.
115] Clinical data
Hoffmann et Biplane Triangulation; All EP catheters for cardiac MERS were 1.8 ± 1.1 mm and 2.2
al., [116] fluoroscopy Bottom up strategy with a feature intervention; ± 2.2 mm on phantom and clinical
learning-based framework Phantom and clinical data experiments, respectively.
Delmas et Biplane Triangulation; Curvilinear devices for MERS was 0.46 ± 0.16 mm with a
al., [117] fluoroscopy Top-down strategy intravascular intervention; maximum error of 1.46mm.
A cerebral vascular phantom
Lobaton et Monoplane Integrate robotic kinematic Concentric tube robot for MERS was 1.05 mm, compared
al., [35] fluoroscopy models with image data extracted bronchoscopy intervention; with 3.3 from kinematic model
from optimal C-arm positioning A simulated lung phantom based approach.
Vandini et Monoplane Fusion between appearance Hansen Artisan robotic MERS was 2.52 ±1.68 mm.
al., [119] fluoroscopy feature priors and image data catheter for cardiac
from optimal C-arm positioning intervention;
A silicone phantom of the
aortic arch
Vandini et Monoplane 2D/3D non-rigid registration; Concentric tube robot for MERS was 0.88 ± 0.33 mm; METR
al., [34] fluoroscopy Fusion between a kinematic trans-nasal surgery; was 2.22 ± 1.10 mm.
model with image data A skull phantom
Otake et al., Monoplane Piecewise-rigid 2D/3D A snake-like robotic robot The joint angle error was lower than
[120] fluoroscopy registration; for hip osteolysis treatment; 0.07° with out-of-plane rotation
Fusion between a simplified Free space ranging from 0° to 60°.
kinematic model with image data
Papalazaroua Monoplane Combine non-rigid structure-
Ablation catheter for cardiac Not reported
et, al [121] fluoroscopy from-motion from multiple views
diseases;
and robotic modeling
A heart phantom
Reilink et al., Endoscopy Marker -based method that use
ANUBIS™ endoscopic RMS values for tip tracking error
[123] markers information to update the
instrument for transluminal were respectively 1.1, 1.7, and 1.5
kinematic mode endoscopic surgery; mm for three directions.
A colon phantom
Endoscopy Marker-less method that use ANUBIS™ endoscopic RMS values for tip tracking errors
feature points information to instrument for transluminal was respectively 1.5, 1.6, and 1.8
update the kinematic mode endoscopic surgery; mm for three directions.
A colon phantom
Vandini et Endoscopy Fusion of feature extraction by Tendon-driven continuum METR was 1.84 mm.
al., [127] mock camera compressive tracking and robot for arthroscopy;
workspace constraints A knee phantom
Ren et al, 3D Tubular enhanced geodesic active Curved continuum robots for Not reported
[145, 146] Ultrasound contours cardiac intervention;
A porcine heart
Misra et al, 2D Fusion of biomechanical model Steerable needle for biopsy METR was 0.42 ± 0.17 for the
[147, 151] Ultrasound and ultrasound-based image and tumor ablation; gelatin phantom and 1.63 ± 0.29 mm
processing Gelatin phantom and chicken for the chicken breast tissue.
breast tissue
Wu et al., 3D 2D/3D registration between Ablation catheter for cardiac MERS was less than 2 mm.
[152] Ultrasound & different imaging modalities diseases;
fluoroscopy Clinical data on patients
METE, mean error of tip tracking; MERS, mean error of robot shape;
the 3D catheter shape segmentation on the ultrasound images particularly significant when depth information of medical
[152]. The disadvantages of ultrasound imaging limit its continuum robots and instruments needs to be tracked or
applicability, making it a complementary technique that is interactive tissue deformation is required. The fusion between
combined with other imaging modalities (e.g., CT and MRI) for these two complementary imaging modalities demonstrate a
many intravascular interventional procedures. This is promising approach to overcome the disadvantages of their

0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 <
standalone uses, and permit them to obtain better accuracy and
improve the clinical outcomes.
V. SUMMARY AND OUTLOOK
This review presented the technical advances and state-of-
the-art applications on shape sensing of continuum robots used
in MIS based on the techniques of FBG sensing, EM tracking
and intraoperative imaging modalities. These techniques have
different strengths and drawbacks, but generally enable real-
time shape estimation and tip localization with reasonable
accuracy. Some implementations have been applied for
continuum robots as real-time feedback information to realize
closed-loop control and path planning [49, 57, 127]. Despite
the significant progress made on shape sensing, it still remains
challenging to form a general framework to accurately and
robustly reconstruct different shapes of surgical continuum
robots and achieve seamless integration into the current
clinical workflows.
FBG sensing techniques offer significant advantages for
most clinical applications due to miniature size, insensitivity
to EM interference and absence of electrical connection. These
techniques also demonstrate the increased accuracy over
kinematics and mechanics based modeling techniques, and
support real-time shape detection and closed-loop control
without the heavy computational expense [53, 57, 153].
However, the FBG-based shape reconstruction for continuum
robots with low stiffness and lengthy structures suffers from
large errors, especially when used on steerable catheters and
endoscopies. The optimal selection of the FBG sensor number
and placement location on surgical continuum robots can
reduce such errors and lead to cost effectiveness. However,
this remains unresolved and requires further investigation [61,
72]. Therefore, one promising direction is to investigate a
general framework using optimization techniques to determine
the optimal sensor number and the proper sensor placement to
meet specific accuracy requirements, and further study strain
transfer models. Moreover, FBG-based hybrid sensing for
force/torque and shape detection is another active research
topic, which offers multiple feedback sources in comparison to
other FOS-based techniques.
EM tracking has been increasingly incorporated into many
clinical procedures within the human body due to the freedom
from line-of-sight restrictions. These advantages enable to track
and navigate continuum robots after spatially registering to 3D
preoperative CT or MRI data [75]. To overcome the issue of
discrete pose information from EM sensors, EM tracking-based
shape estimation techniques are commonly fused with
kinematic models of the continuum robots to achieve higher
accuracy and robustness. This fusion can support the contact
detection and localization of contact points along the
multi-segment continuum robots, enhancing manipulation
safety in constrained spaces [22]. Multiple EM sensors-driven
shape estimation has been proven to be observable for the state
of this fusion and shape recovery [91]. However, EM tracking
suffers from the drawbacks of EM field distortions and
non-uniform tracking accuracy associated with the working
principle, limiting their surgical applications. To solve these
0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
11
problems, methods using redundant EM sensors have been
developed to simultaneously localize the tracked sensors and
compensate dynamic distortion [76, 103]. Another approach to
improve the tracking accuracy and enlarge the workspace used
a robot manipulator to automatically move the EM field
generator to ensure the EM sensors are located around the
tracking volume center [77]. These newly developed
techniques are promising, but still need to be further
investigated and improved for the clinical applications.
Intraoperative imaging based shape sensing supports direct
measurements and reconstruction for continuum robots and
instruments in MIS. Such techniques have been validated to be
more accurate than modeling-based approaches alone. Biplane
fluoroscopic imaging systems offer accurate shape
reconstruction based on triangulation or 2D/3D registration.
However, a large exposure to radiation, high cost and
workspace constraints of biplane fluoroscopy limit their
applications. The recent shape reconstruction techniques using
the monoplane C-arm systems have also been developed by
combining with prior feature knowledge or additional
information from kinematic models. However, many of them
require the repositioning of the C-arm to obtain multiple views
or optimal views, which cannot easily be fully integrated into
current clinical workflows. Ultrasound imaging provides
another intraoperative alternative and supports tracking of
depth information, but suffers from low resolution and diverse
artifacts. Fusing the complementary imaging modalities of
ultrasound and fluoroscopy has been recently developed to
allow for an improved accuracy, and shows promise to
enhance the current clinical workflows. In addition, most
intraoperative imaging modality-enabled shape reconstruction
techniques are general time-consuming procedures, and their
robustness needs to be improved to cope with dynamic
scenarios. Therefore, a large dose of radiation and contrast
agent is required for X-ray fluoroscopic imaging enabled
shape sensing. Techniques such as parallel programming
based on a graphics processing unit and highly efficient
algorithms should be developed to further speed up their
reconstruction and support real-time and robust applications.
In summary, shape sensing is significant to support
closed-loop control, path planning, surgeon–robot interaction
and manipulation safety for robotic assisted minimally invasive
surgery. Different shape sensing methods have their own
strengths and weaknesses. FBG sensing based techniques are
more suitable for integration with continuum robots with
smoother structures, such as active needles, continuum robots
with backbones and concentric tube robots. EM tracking based
methods can be suitable for continuum robots with discrete
structures, such as snake-like robots. Integrating these methods
with each other is a possible framework to improve the
accuracy and robustness instead of standalone uses. For
instance, distal shape estimation based on FBG sensing with
dense FBG sensors and proximal shape detection using EM
tracking can be a promising approach to address the issues of
shape sensing for catheters and endoscopes. Moreover, FBG
sensing and EM tracking based shape reconstruction techniques
cannot fully replace intraoperative imaging, which can provide
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 <
continuous navigation information for MIS. They can provide
the initial shape estimation for improved accuracy and
robustness of intraoperative imaging-enabled sensing, as well
as reduced computational cost. Therefore, combining
intraoperative imaging with FBG sensing or EM tracking for
robust shape estimation is a promising approach to achieve high
accuracy and create new and improved clinical workflows.
REFERENCES
[1] J. Burgner-Kahrs, D. C. Rucker, and H. Choset, "Continuum Robots for
Medical Applications: A Survey," IEEE Transactions on Robotics, vol.
31, pp. 1261-1280, 2015.
[2] V. Vitiello, S. L. Lee, T. P. Cundy, and G. Z. Yang, "Emerging robotic
platforms for minimally invasive surgery," IEEE Rev Biomed Eng, vol.
6, pp. 111-26, 2013.
[3] C. Bergeles and G. Z. Yang, "From passive tool holders to
microsurgeons: safer, smaller, smarter surgical robots," IEEE Trans
Biomed Eng, vol. 61, pp. 1565-76, May 2014.
[4] H. B. Gilbert, D. C. Rucker, and R. J. Webster III, "Concentric tube
robots: The state of the art and future directions," in Int Symp Robot Res,
2013.
[5] G. P. Moustris, S. C. Hiridis, K. M. Deliparaschos, and K. M.
Konstantinidis, "Evolution of autonomous and semi-autonomous robotic
surgical systems: a review of the literature," Int J Med Robot, vol. 7, pp.
375-92, Dec 2011.
[6] S. Tognarelli, M. Salerno, G. Tortora, C. Quaglia, P. Dario, M. O.
Schurr, et al., "A miniaturized robotic platform for natural orifice
transluminal endoscopic surgery: in vivo validation," Surg Endosc, vol.
29, pp. 3477-84, Dec 2015.
[7] G. Dogangil, B. L. Davies, and F. Rodriguez y Baena, "A review of
medical robotics for minimally invasive soft tissue surgery,"
Proceedings of the Institution of Mechanical Engineers. Part H, Journal
of engineering in medicine, vol. 224, pp. 653-79, 2010.
[8] G. Robinson and J. B. C. Davies, "Continuum robots - a state of the art,"
in Robotics and Automation, 1999. Proceedings. 1999 IEEE
International Conference on, 1999, pp. 2849-2854 vol.4.
[9] I. D. Walker, "Continuous backbone “continuum” robot manipulators,"
ISRN Robotics, vol. 2013, 2013.
[10] N. J. van de Berg, D. J. van Gerwen, J. Dankelman, and J. J. van den
Dobbelsteen, "Design Choices in Needle Steering-A Review,"
IEEE/ASME Trans. Mechatronics, vol. 20, pp. 2172-2183, 2015.
[11] X. Wang and M. Q. H. Meng, "Robotics for Natural Orifice
Transluminal Endoscopic Surgery: A Review," Journal of Robotics, vol.
2012, pp. 1-9, 2012.
[12] B. P. Yeung and P. W. Chiu, "Application of robotics in gastrointestinal
endoscopy: A review," World J Gastroenterol, vol. 22, pp. 1811-25, Feb
7 2016.
[13] C. J. Payne and G. Z. Yang, "Hand-held medical robots," Ann Biomed
Eng, vol. 42, pp. 1594-605, Aug 2014.
[14] N. Patel, C. Seneci, G. Z. Yang, A. Darzi, and J. Teare, "Flexible
platforms for natural orifice transluminal and endoluminal surgery,"
Endosc Int Open, vol. 2, pp. E117-23, Jun 2014.
[15] R. J. Webster and B. A. Jones, "Design and Kinematic Modeling of
Constant Curvature Continuum Robots: A Review," Int J Rob Res., vol.
29, pp. 1661-1683, June 10, 2010 2010.
[16] K. Xu and N. Simaan, "Analytic Formulation for Kinematics, Statics,
and Shape Restoration of Multibackbone Continuum Robots Via Elliptic
Integrals," J Mech Robot., vol. 2, p. 011006, 2010.
[17] G. S. Chirikjian, "Conformational Modeling of Continuum Structures in
Robotics and Structural Biology: A Review," Adv Robot, vol. 29, pp.
817-829, 2015/07/03 2015.
[18] B. A. Jones and I. D. Walker, "Kinematics for multisection continuum
robots," IEEE Trans Robot, vol. 22, pp. 43-55, 2006.
[19] P. Qi, C. Qiu, H. Liu, J. S. Dai, L. D. Seneviratne, and K. Althoefer, "A
Novel Continuum Manipulator Design Using Serially Connected
Double-Layer Planar Springs," IEEE/ASME Trans. Mechatronics, vol.
21, pp. 1281-1292, 2016.
[20] D. B. Camarillo, C. F. Milne, C. R. Carlson, M. R. Zinn, and J. K.
Salisbury, "Mechanics Modeling of Tendon-Driven Continuum
Manipulators," IEEE Trans Robot, vol. 24, pp. 1262-1273, 2008.
0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
12
[21] P. Qi, C. Liu, A. Ataka, H. K. Lam, and K. Althoefer, "Kinematic
Control of Continuum Manipulators Using a Fuzzy-Model-Based
Approach," IEEE Trans. Ind. Electron., vol. PP, pp. 1-1, 2016.
[22] A. Bajo and N. Simaan, "Kinematics-Based Detection and Localization
of Contacts Along Multisegment Continuum Robots," IEEE Trans
Robot, vol. 28, pp. 291-302, 2012.
[23] R. J. Webster, J. M. Romano, and N. J. Cowan, "Mechanics of
Precurved-Tube Continuum Robots," IEEE Trans Robot, vol. 25, pp. 67-
78, 2009.
[24] M. Dehghani and S. A. A. Moosavian, "Compact modeling of spatial
continuum robotic arms towards real-time control," Adv Robot, vol. 28,
pp. 15-26, 2014/01/02 2013.
[25] D. Trivedi, A. Lotfi, and C. D. Rahn, "Geometrically Exact Models for
Soft Robotic Manipulators," IEEE Trans Robot, vol. 24, pp. 773-780,
2008.
[26] B. A. Jones and I. D. Walker, "Practical Kinematics for Real-Time
Implementation of Continuum Robots," IEEE Trans Robot, vol. 22, pp.
1087-1099, 2006.
[27] K. Chunwoo, S. C. Ryu, and P. E. Dupont, "Real-time adaptive
kinematic model estimation of concentric tube robots," in Intelligent
Robots and Systems (IROS), 2015 IEEE/RSJ International Conference
on, 2015, pp. 3214-3219.
[28] D. C. Rucker, B. A. Jones, and R. J. Webster, 3rd, "A Geometrically
Exact Model for Externally Loaded Concentric-Tube Continuum
Robots," IEEE Trans Robot, vol. 26, pp. 769-780, 2010.
[29] D. C. Rucker and R. J. Webster Iii, "Statics and Dynamics of Continuum
Robots With General Tendon Routing and External Loading," IEEE
Trans Robot, vol. 27, pp. 1033-1044, 2011.
[30] W. S. Rone and P. Ben-Tzvi, "Continuum Robot Dynamics Utilizing the
Principle of Virtual Power," IEEE Trans Robot, vol. 30, pp. 275-287,
2014.
[31] P. Polygerinos, D. Zbyszewski, T. Schaeffter, R. Razavi, L. D.
Seneviratne, and K. Althoefer, "MRI-Compatible Fiber-Optic Force
Sensors for Catheterization Procedures," IEEE Sensors J., vol. 10, pp.
1598-1608, 2010.
[32] F. Taffoni, D. Formica, P. Saccomandi, G. Di Pino, and E. Schena,
"Optical fiber-based MR-compatible sensors for medical applications:
an overview," Sensors (Basel), vol. 13, pp. 14105-20, 2013.
[33] A. M. Franz, T. Haidegger, W. Birkfellner, K. Cleary, T. M. Peters, and
L. Maier-Hein, "Electromagnetic tracking in medicine--a review of
technology, validation, and applications," IEEE Trans Med Imaging,
vol. 33, pp. 1702-25, Aug 2014.
[34] A. Vandini, C. Bergeles, F.-Y. Lin, and G.-Z. Yang, "Vision-based
intraoperative shape sensing of concentric tube robots," in Intelligent
Robots and Systems (IROS), 2015 IEEE/RSJ International Conference
on, 2015, pp. 2603-2610.
[35] E. J. Lobaton, J. Fu, L. G. Torres, and R. Alterovitz, "Continuous shape
estimation of continuum robots using X-ray images," in Robotics and
Automation (ICRA), 2013 IEEE International Conference on, 2013, pp.
725-732.
[36] V. Mishra, N. Singh, U. Tiwari, and P. Kapur, "Fiber grating sensors in
medicine: Current and emerging applications," Sensors and Actuators A:
Physical, vol. 167, pp. 279-290, 6// 2011.
[37] S. Silvestri and E. Schena, Optical-fiber measurement systems for
medical applications: Citeseer, 2011.
[38] L. Dziuda, F. W. Skibniewski, M. Krej, and P. M. Baran, "Fiber Bragg
grating-based sensor for monitoring respiration and heart activity during
magnetic resonance imaging examinations," J Biomed Opt, vol. 18, p.
57006, May 2013.
[39] A. A. Abushagur, N. Arsad, M. I. Reaz, and A. Bakar, "Advances in bio-
tactile sensors for minimally invasive surgery using the Fibre Bragg
grating force sensor technique: A survey," Sensors, vol. 14, pp. 6633-
6665, 2014.
[40] B. Lee, "Review of the present status of optical fiber sensors," Optical
Fiber Technology, vol. 9, pp. 57-79, 4// 2003.
[41] P. Polygerinos, L. D. Seneviratne, R. Razavi, T. Schaeffter, and K.
Althoefer, "Triaxial Catheter-Tip Force Sensor for MRI-Guided Cardiac
Procedures," IEEE/ASME Trans. Mechatronics, vol. 18, pp. 386-396,
2013.
[42] H. Su, M. Zervas, G. A. Cole, C. Furlong, and G. S. Fischer, "Real-time
MRI-guided needle placement robot with integrated fiber optic force
sensing," in Robotics and Automation (ICRA), 2011 IEEE International
Conference on, 2011, pp. 1583-1588.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 <
[43] S. Sina, N. Yohan, L. Min, R. Tommaso, L. Hongbin, and A. Kaspar,
"Macrobend optical sensing for pose measurement in soft robot arms,"
Smart Materials and Structures, vol. 24, p. 125024, 2015.
[44] T. C. Searle, K. Althoefer, L. Seneviratne, and H. Liu, "An optical
curvature sensor for flexible manipulators," in Robotics and Automation
(ICRA), 2013 IEEE International Conference on, 2013, pp. 4415-4420.
[45] I. Iordachita, Z. Sun, M. Balicki, J. U. Kang, S. J. Phee, J. Handa, et al.,
"A sub-millimetric, 0.25 mN resolution fully integrated fiber-optic
force-sensing tool for retinal microsurgery," Int J Comput Assist Radiol
Surg, vol. 4, pp. 383-90, Jun 2009.
[46] X. He, J. Handa, P. Gehlbach, R. Taylor, and I. Iordachita, "A
submillimetric 3-DOF force sensing instrument with integrated fiber
Bragg grating for retinal microsurgery," IEEE Trans Biomed Eng, vol.
61, pp. 522-34, Feb 2014.
[47] K. O. Hill and G. Meltz, "Fiber Bragg grating technology fundamentals
and overview," Journal of Lightwave Technology, vol. 15, pp. 1263-
1276, 1997.
[48] Y. L. Park, S. Elayaperumal, B. Daniel, S. C. Ryu, M. Shin, J. Savall, et
al., "Real-Time Estimation of 3-D Needle Shape and Deflection for
MRI-Guided Interventions," IEEE ASME Trans Mechatron, vol. 15, pp.
906-915, Dec 2010.
[49] S. Elayaperumal, J. C. Plata, A. B. Holbrook, Y. L. Park, K. B. Pauly, B.
L. Daniel, et al., "Autonomous real-time interventional scan plane
control with a 3-D shape-sensing needle," IEEE Trans Med Imaging,
vol. 33, pp. 2128-39, Nov 2014.
[50] K. Henken, D. Van Gerwen, J. Dankelman, and J. Van Den
Dobbelsteen, "Accuracy of needle position measurements using fiber
Bragg gratings," Minimally invasive therapy & allied technologies :
MITAT : official journal of the Society for Minimally Invasive Therapy,
vol. 21, pp. 408-14, Nov 2012.
[51] J. P. Moore and M. D. Rogge, "Shape sensing using multi-core fiber
optic cable and parametric curve solutions," Optics Express, vol. 20, pp.
2967-2973, 2012/01/30 2012.
[52] C. Shi, S. Giannarou, S.-L. Lee, and G.-Z. Yang, "Simultaneous catheter
and environment modeling for Trans-catheter Aortic Valve
Implantation," in Intelligent Robots and Systems (IROS 2014), 2014
IEEE/RSJ International Conference on, 2014, pp. 2024-2029.
[53] R. J. Roesthuis, M. Kemp, J. J. van den Dobbelsteen, and S. Misra,
"Three-Dimensional Needle Shape Reconstruction Using an Array of
Fiber Bragg Grating Sensors," IEEE/ASME Transactions on
Mechatronics, vol. 19, pp. 1115-1126, 2014.
[54] M. Abayazid, M. Kemp, and S. Misra, "3D flexible needle steering in
soft-tissue phantoms using Fiber Bragg Grating sensors," in Robotics
and Automation (ICRA), 2013 IEEE International Conference on, 2013,
pp. 5843-5849.
[55] N. J. Van de Berg, J. Dankelman, and J. J. van den Dobbelsteen,
"Design of an actively controlled steerable needle with tendon actuation
and FBG-based shape sensing," Med Eng Phys, vol. 37, pp. 617-22, Jun
[56] R.
2015.
J. Roesthuis, S. Janssen, and S. Misra, "On using an array of fiber
Bragg grating sensors for closed-loop control of flexible minimally
invasive surgical instruments," in Intelligent Robots and Systems (IROS),
2013 IEEE/RSJ International Conference on, 2013, pp. 2545-2551.
[57] R. J. Roesthuis and S. Misra, "Steering of Multisegment Continuum
Manipulators Using Rigid-Link Modeling and FBG-Based Shape
Sensing," IEEE Transactions on Robotics, vol. 32, pp. 372-382, 2016.
[58] S. C. Ryu and P. E. Dupont, "FBG-based shape sensing tubes for
continuum robots," in Robotics and Automation (ICRA), 2014 IEEE
International Conference on, 2014, pp. 3531-3537.
[59] H. Liu, A. Farvardin, S. A. Pedram, I. Iordachita, R. H. Taylor, and M.
Armand, "Large deflection shape sensing of a continuum manipulator
for minimally-invasive surgery," in 2015 IEEE International Conference
on Robotics and Automation (ICRA), 2015, pp. 201-206.
[60] H. Liu, A. Farvardin, R. Grupp, R. J. Murphy, R. H. Taylor, I.
Iordachita, et al., "Shape Tracking of a Dexterous Continuum
Manipulator Utilizing Two Large Deflection Shape Sensors," IEEE
Sensors Journal, vol. 15, pp. 5494-5503, 2015.
[61] B. Kim, J. Ha, F. C. Park, and P. E. Dupont, "Optimizing curvature
sensor placement for fast, accurate shape sensing of continuum robots,"
in Robotics and Automation (ICRA), 2014 IEEE International
Conference on, 2014, pp. 5374-5379.
[62] R. Xu, A. Yurkewich, and R. V. Patel, "Shape sensing for torsionally
compliant concentric-tube robots," 2016, pp. 97020V-97020V-8.
[63] J. Yi, X. Zhu, H. Zhang, L. Shen, and X. Qiao, "Spatial shape
reconstruction using orthogonal fiber Bragg grating sensor array,"
Mechatronics, vol. 22, pp. 679-687, 9// 2012.
0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
13
[64] L. Zhang, J. Qian, L. Shen, and Y. Zhang, "FBG sensor devices for
spatial shape detection of intelligent colonoscope," in Robotics and
Automation, 2004. Proceedings. ICRA '04. 2004 IEEE International
Conference on, 2004, pp. 834-840 Vol.1.
[65] R. Xu, A. Yurkewich, and R. V. Patel, "Curvature, Torsion, and Force
Sensing in Continuum Robots Using Helically Wrapped FBG Sensors,"
IEEE Robotics and Automation Letters, vol. 1, pp. 1052-1059, 2016.
[66] L. Zhang, J. Qian, Y. Zhang, and L. Shen, "On SDM/WDM FBG sensor
net for shape detection of endoscope," in Mechatronics and Automation,
2005 IEEE International Conference, 2005, pp. 1986-1991 Vol. 4.
[67] J. Adv RobotIEEE ASME Trans MechatronYi, X. Zhu, L. Shen, B. Sun,
and L. Jiang, "An Orthogonal Curvature Fiber Bragg Grating Sensor
Array for Shape Reconstruction," in Life System Modeling and
Intelligent Computing: International Conference on Life System
Modeling and Simulation, LSMS 2010, Wuxi, China, September 17-20,
2010. , ed, 2010, pp. 25-31.
[68] X. Yi, J. Qian, Y. Zhang, Z. Zhen, and L. Shen, "3-D Shape Display of
Intelligent Colonoscope Based on FBG Sensor Array and Binocular
Vision," in Complex Medical Engineering, 2007. CME 2007.
IEEE/ICME International Conference on, 2007, pp. 14-19.
[69] P. Gomes, Medical robotics: Minimally invasive surgery: Elsevier,
[70] K.
2012.
R. Henken, J. Dankelman, J. J. v. d. Dobbelsteen, L. K. Cheng, and
M. S. v. d. Heiden, "Error Analysis of FBG-Based Shape Sensors for
Medical Needle Tracking," IEEE/ASME Trans. Mechatronics, vol. 19,
pp. 1523-1531, 2014.
[71] C. Ledermann, H. Pauer, O. Weede, and H. Woern, "Simulation tool for
3D shape sensors based on Fiber Bragg gratings and optimization of
measurement points," in 2013 6th IEEE Conference on Robotics,
Automation and Mechatronics (RAM), 2013, pp. 195-200.
[72] A. W. Mahoney, T. L. Bruns, P. J. Swaney, and R. J. Webster, "On the
inseparable nature of sensor selection, sensor placement, and state
estimation for continuum robots or "where to put your sensors and how
to use them"," in 2016 IEEE International Conference on Robotics and
Automation (ICRA), 2016, pp. 4472-4478.
[73] I. Payo, V. Feliu, and O. D. Cortázar, "Fibre Bragg grating (FBG) sensor
system for highly flexible single-link robots," Sensors and Actuators A:
Physical, vol. 150, pp. 24-39, 3/16/ 2009.
[74] A. P. Shah, P. A. Kupelian, T. R. Willoughby, and S. L. Meeks,
"Expanding the use of real-time electromagnetic tracking in radiation
oncology," Journal of Applied Clinical Medical Physics, vol. 12, 2011.
[75] E. Lugez, H. Sadjadi, D. R. Pichora, R. E. Ellis, S. G. Akl, and G.
Fichtinger, "Electromagnetic tracking in surgical and interventional
environments: usability study," Int J Comput Assist Radiol Surg, vol. 10,
pp. 253-62, Mar 2015.
[76] H. Sadjadi, K. Hashtrudi-Zaad, and G. Fichtinger, "Simultaneous
localization and calibration for electromagnetic tracking systems," Int J
Med Robot, vol. 12, pp. 189-98, Jun 2016.
[77] T. Reichl, J. Gardiazabal, and N. Navab, "Electromagnetic servoing-a
new tracking paradigm," IEEE Trans Med Imaging, vol. 32, pp. 1526-
35, Aug 2013.
[78] X. Luo, Y. Wan, and X. He, "Robust electromagnetically guided
endoscopic procedure using enhanced particle swarm optimization for
multimodal information fusion," Med Phys, vol. 42, pp. 1808-17, Apr
2015.
[79] X. Luo, M. Feuerstein, D. Deguchi, T. Kitasaka, H. Takabatake, and K.
Mori, "Development and comparison of new hybrid motion tracking for
bronchoscopic navigation," Med Image Anal, vol. 16, pp. 577-96, Apr
2012.
[80] M. Nakamoto, K. Nakada, Y. Sato, K. Konishi, M. Hashizume, and S.
Tamura, "Intraoperative magnetic tracker calibration using a magneto-
optic hybrid tracker for 3-D ultrasound-based navigation in laparoscopic
surgery," IEEE Trans Med Imaging, vol. 27, pp. 255-70, Feb 2008.
[81] M. Feuerstein, T. Reichl, J. Vogel, J. Traub, and N. Navab, "Magneto-
optical tracking of flexible laparoscopic ultrasound: model-based online
detection and correction of magnetic tracking errors," IEEE Trans Med
Imaging, vol. 28, pp. 951-67, Jun 2009.
[82] X. Luo, Y. Wan, X. He, and K. Mori, "Observation-driven adaptive
differential evolution and its application to accurate and smooth
bronchoscope three-dimensional motion tracking," Med Image Anal, vol.
24, pp. 282-96, Aug 2015.
[83] C. Shi, C. Tercero, S. Ikeda, K. Ooe, T. Fukuda, K. Komori, et al., "In
vitro three-dimensional aortic vasculature modeling based on sensor
fusion between intravascular ultrasound and magnetic tracker," Int J
Med Robot, vol. 8, pp. 291-9, Sep 2012.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 <
[84] C. Shi, C. Tercero, X. Wu, S. Ikeda, K. Komori, K. Yamamoto, et al.,
"Real-time in vitro intravascular reconstruction and navigation for
endovascular aortic stent grafting," Int J Med Robot, p. DOI:
10.1002/rcs.1736, Feb 9 2016.
[85] C. A. Linte, J. White, R. Eagleson, G. M. Guiraudon, and T. M. Peters,
"Virtual and augmented medical imaging environments: enabling
technology for minimally invasive cardiac interventional guidance,"
IEEE Rev Biomed Eng, vol. 3, pp. 25-47, 2010.
[86] Y. Ganji, F. Janabi-Sharifi, and A. N. Cheema, "Robot-assisted catheter
manipulation for intracardiac navigation," Int J Comput Assist Radiol
Surg, vol. 4, pp. 307-15, Jun 2009.
[87] S. Song, Z. Li, M. Q. H. Meng, H. Yu, and H. Ren, "Real-Time Shape
Estimation for Wire-Driven Flexible Robots With Multiple Bending
Sections Based on Quadratic Bezier Curves," IEEE Sensors Journal,
vol. 15, pp. 6326-6334, 2015.
[88] J. Huang, W. Huo, W. Xu, S. Mohammed, and Y. Amirat, "Control of
Upper-Limb Power-Assist Exoskeleton Using a Human-Robot Interface
Based on Motion Intention Recognition," IEEE Trans. Autom. Sci. Eng,
vol. 12, pp. 1257-1270, 2015.
[89] J. Huang, W. Xu, S. Mohammed, and Z. Shu, "Posture estimation and
human support using wearable sensors and walking-aid robot," Robotics
Auton Syst, vol. 73, pp. 24-43, 11// 2015.
[90] S. Tully, G. Kantor, M. A. Zenati, and H. Choset, "Shape estimation for
image-guided surgery with a highly articulated snake robot," in 2011
IEEE/RSJ International Conference on Intelligent Robots and Systems,
2011, pp. 1353-1358.
[91] S. Tully and H. Choset, "A Filtering Approach for Image-Guided
Surgery With a Highly Articulated Surgical Snake Robot," IEEE Trans
Biomed Eng, vol. 63, pp. 392-402, Feb 2016.
[92] S. Song, Z. Li, H. Yu, and H. Ren, "Electromagnetic Positioning for Tip
Tracking and Shape Sensing of Flexible Robots," IEEE Sensors Journal,
vol. 15, pp. 4565-4575, 2015.
[93] L. Wu, S. Song, K. Wu, C. M. Lim, and H. Ren, "Development of a
compact continuum tubular robotic system for nasopharyngeal biopsy,"
Med Biol Eng Comput, pp. 1-15, May 18 2016.
[94] Z. Zhang, J. Shang, C. Seneci, and G. Z. Yang, "Snake robot shape
sensing using micro-inertial sensors," in Intelligent Robots and Systems
(IROS), 2013 IEEE/RSJ International Conference on, 2013, pp. 831-
[95] H.
836.Sadjadi, K. Hashtrudi-Zaad, and G. Fichtinger, "Fusion of
electromagnetic trackers to improve needle deflection estimation:
simulation study," IEEE Trans Biomed Eng, vol. 60, pp. 2706-15, Oct
2013.
[96] H. Sadjadi, K. Hashtrudi-Zaad, and G. Fichtinger, "Needle deflection
estimation: prostate brachytherapy phantom experiments," Int J Comput
Assist Radiol Surg, vol. 9, pp. 921-9, Nov 2014.
[97] A. Dore, G. Smoljkic, E. V. Poorten, M. Sette, J. V. Sloten, and G.-Z.
Yang, "Catheter navigation based on probabilistic fusion of
electromagnetic tracking and physically-based simulation," in Intelligent
Robots and Systems (IROS), 2012 IEEE/RSJ International Conference
on, 2012, pp. 3806-3811.
[98] Y. Ganji and F. Janabi-Sharifi, "Catheter kinematics for intracardiac
navigation," IEEE Trans Biomed Eng, vol. 56, pp. 621-32, Mar 2009.
[99] Y. Fu, H. Liu, S. Wang, W. Deng, X. Li, and Z. Liang, "Skeleton-based
active catheter navigation," Int J Med Robot, vol. 5, pp. 125-35, Jun
[100] H.
2009.
Liu, Y. L. Fu, Y. Y. Zhou, H. X. Li, Z. G. Liang, and S. G. Wang,
"An in vitro investigation of image-guided steerable catheter
navigation," Proceedings of the Institution of Mechanical Engineers.
Part H, Journal of engineering in medicine, vol. 224, pp. 945-54, 2010.
[101] S. Condino, V. Ferrari, C. Freschi, A. Alberti, R. Berchiolli, F. Mosca, et
al., "Electromagnetic navigation platform for endovascular surgery: how
to develop sensorized catheters and guidewires," Int J Med Robot, vol. 8,
pp. 300-10, Sep 2012.
[102] S. Condino, E. M. Calabro, A. Alberti, S. Parrini, R. Cioni, R. N.
Berchiolli, et al., "Simultaneous tracking of catheters and guidewires:
comparison to standard fluoroscopic guidance for arterial cannulation,"
European journal of vascular and endovascular surgery : the official
journal of the European Society for Vascular Surgery, vol. 47, pp. 53-
60, Jan 2014.
[103] H. Sadjadi, K. Hashtrudi-Zaad, and G. Fichtinger, "Simultaneous
Electromagnetic Tracking and Calibration for Dynamic Field Distortion
Compensation," IEEE Trans Biomed Eng, vol. 63, pp. 1771-81, Aug
2016.
[104] S. Song, Z. Li, H. Yu, and H. Ren, "Shape reconstruction for wire-driven
flexible robots based on Bézier curve and electromagnetic positioning,"
Mechatronics, vol. 29, pp. 28-35, 8// 2015.
0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
14
[105] M. W. Hannan and I. D. Walker, "Real-time shape estimation for
continuum robots using vision," Robotica, vol. 23, pp. 645-651, 2005.
[106] V. K. Chitrakaran, A. Behal, D. M. Dawson, and I. D. Walker, "Setpoint
regulation of continuum robots using a fixed camera," Robotica, vol. 25,
pp. 581-586, 2007.
[107] J. M. Croom, D. C. Rucker, J. M. Romano, and R. J. Webster, "Visual
sensing of continuum robot shape using self-organizing maps," in
Robotics and Automation (ICRA), 2010 IEEE International Conference
on, 2010, pp. 4591-4596.
[108] D. B. Camarillo, K. E. Loewke, C. R. Carlson, and J. K. Salisbury,
"Vision based 3-D shape sensing of flexible manipulators," in Robotics
and Automation, 2008. ICRA 2008. IEEE International Conference on,
2008, pp. 2940-2947.
[109] B. Weber, P. Zeller, and K. Kuhnlenz, "Multi-camera based real-time
configuration estimation of continuum robots," in 2012 IEEE/RSJ
International Conference on Intelligent Robots and Systems, 2012, pp.
3350-3355.
[110] W. Xu, R. P. L. Foong, and H. Ren, "Maker based shape tracking of a
flexible serpentine manipulator," in Information and Automation, 2015
IEEE International Conference on, 2015, pp. 637-642.
[111] X. Zhong, M. Hoffmann, N. Strobel, and A. Maier, "Semi-Automatic
Basket Catheter Reconstruction from Two X-Ray Views," in Pattern
Recognition: 37th German Conference, GCPR 2015, Aachen, Germany,
October 7-10, 2015, Proceedings, J. Gall, P. Gehler, and B. Leibe, Eds.,
ed Cham: Springer International Publishing, 2015, pp. 379-389.
[112] J. Burgner, S. D. Herrell, and R. J. Webster, "Toward fluoroscopic shape
reconstruction for control of steerable medical devices," in ASME 2011
Dynamic Systems and Control Conference and Bath/ASME Symposium
on Fluid Power and Motion Control, 2011, pp. 791-794.
[113] M. Wagner, S. Schafer, C. Strother, and C. Mistretta, "4D interventional
device reconstruction from biplane fluoroscopy," Med Phys, vol. 43, pp.
1324-34, Mar 2016.
[114] M. Hoffmann, A. Brost, C. Jakob, F. Bourier, M. Koch, K. Kurzidim, et
al., "Semi-automatic Catheter Reconstruction from Two Views," in
Medical Image Computing and Computer-Assisted Intervention –
MICCAI 2012: 15th International Conference, Nice, France, October 1-
5, 2012, Proceedings, Part II, N. Ayache, H. Delingette, P. Golland, and
K. Mori, Eds., ed Berlin, Heidelberg: Springer Berlin Heidelberg, 2012,
pp. 584-591.
[115] M. Hoffmann, A. Brost, C. Jakob, M. Koch, F. Bourier, K. Kurzidim, et
al., "Reconstruction method for curvilinear structures from two views,"
in SPIE Medical Imaging, 2013, pp. 86712F-86712F-8.
[116] M. Hoffmann, A. Brost, M. Koch, F. Bourier, A. Maier, K. Kurzidim, et
al., "Electrophysiology Catheter Detection and Reconstruction From
Two Views in Fluoroscopic Images," IEEE Trans Med Imaging, vol. 35,
pp. 567-79, Feb 2016.
[117] C. Delmas, M.-O. Berger, E. Kerrien, C. Riddell, Y. Trousset, R.
Anxionnat, et al., "Three-dimensional curvilinear device reconstruction
from two fluoroscopic views," 2015, pp. 94150F-94150F-11.
[118] C. Canero, P. Radeva, R. Toledo, J. J. Villanueva, and J. Mauri, "3D
curve reconstruction by biplane snakes," in Pattern Recognition, 2000.
Proceedings. 15th International Conference on, 2000, pp. 563-566
[119] A.
vol.4.
Vandini, S. Giannarou, S.-L. Lee, and G.-Z. Yang, "3D Robotic
Catheter Shape Reconstruction and Localisation Using Appearance
Priors and Adaptive C-Arm Positioning," in Augmented Reality
Environments for Medical Imaging and Computer-Assisted
Interventions. vol. 8090, ed, 2013, pp. 172-181.
[120] Y. Otake, R. J. Murphy, M. D. Kutzer, R. H. Taylor, and M. Armand,
"Piecewise-rigid 2D-3D registration for pose estimation of snake-like
manipulator using an intraoperative x-ray projection," 2014, pp.
90360Q-90360Q-6.
[121] C. Papalazarou, P. M. J. Rongen, and P. H. N. de With, "3D catheter
reconstruction using non-rigid structure-from-motion and robotics
modeling," 2012, pp. 831620-831620-8.
[122] X. Luo, U. L. Jayarathne, S. E. Pautler, and T. M. Peters, "Binocular
Endoscopic 3-D Scene Reconstruction Using Color and Gradient-
Boosted Aggregation Stereo Matching for Robotic Surgery," in Image
and Graphics: 8th International Conference, ICIG 2015, Tianjin, China,
August 13-16, 2015, Proceedings, Part I, Y.-J. Zhang, Ed., ed Cham:
Springer International Publishing, 2015, pp. 664-676.
[123] R. Reilink, S. Stramigioli, and S. Misra, "3D position estimation of
flexible instruments: marker-less and marker-based methods," Int J
Comput Assist Radiol Surg, vol. 8, pp. 407-17, May 2013.
[124] R. Reilink, S. Stramigioli, and S. Misra, "Pose reconstruction of flexible
instruments from endoscopic images using markers," in Robotics and
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TBME.2016.2622361, IEEE
Transactions on Biomedical Engineering
> TBME-01244-2016 < 15

Automation (ICRA), 2012 IEEE International Conference on, 2012, pp. [144] Z. Neubach and M. Shoham, "Ultrasound-guided robot for flexible
2938-2943. needle steering," IEEE Trans Biomed Eng, vol. 57, pp. 799-805, Apr
[125] R. Reilink, S. Stramigioli, and S. Misra, "Three-dimensional pose [145] H.
2010.
Ren and P. E. Dupont, "Tubular structure enhancement for surgical
reconstruction of flexible instruments from endoscopic images," in 2011 instrument detection in 3D ultrasound," in 2011 Annual International
IEEE/RSJ International Conference on Intelligent Robots and Systems, Conference of the IEEE Engineering in Medicine and Biology Society,
2011, pp. 2076-2082. 2011, pp. 7203-7206.
[126] P. Cabras, D. Goyard, F. Nageotte, P. Zanne, and C. Doignon, [146] H. Ren and P. E. Dupont, "Tubular Enhanced Geodesic Active Contours
"Comparison of methods for estimating the position of actuated for continuum robot detection using 3D ultrasound," in Robotics and
instruments in flexible endoscopic surgery," in 2014 IEEE/RSJ Automation (ICRA), 2012 IEEE International Conference on, 2012, pp.
International Conference on Intelligent Robots and Systems, 2014, pp. 2907-2912.
3522-3528. [147] P. Moreira and S. Misra, "Biomechanics-Based Curvature Estimation for
[127] A. Vandini, A. Salerno, C. J. Payne, and G.-Z. Yang, "Vision-based Ultrasound-guided Flexible Needle Steering in Biological Tissues," Ann
motion control of a flexible robot for surgical applications," in Robotics Biomed Eng, vol. 43, pp. 1716-26, Aug 2015.
and Automation (ICRA), 2014 IEEE International Conference on, 2014, [148] P. Lang, P. Seslija, M. W. Chu, D. Bainbridge, G. M. Guiraudon, D. L.
pp. 6205-6211. Jones, et al., "US-fluoroscopy registration for transcatheter aortic valve
[128] T. Bergen and T. Wittenberg, "Stitching and Surface Reconstruction implantation," IEEE Trans Biomed Eng, vol. 59, pp. 1444-53, May
From Endoscopic Image Sequences: A Review of Applications and P. Mountney, R. Ionasec, M. Kaizer, S. Mamaghani, W. Wu, T. Chen, et
[149] 2012.
Methods," IEEE J Biomed Health Inform, vol. 20, pp. 304-21, Jan 2016. al., "Ultrasound and Fluoroscopic Images Fusion by Autonomous
[129] D. Stoyanov, M. V. Scarzanella, P. Pratt, and G.-Z. Yang, "Real-Time Ultrasound Probe Detection," in Medical Image Computing and
Stereo Reconstruction in Robotically Assisted Minimally Invasive Computer-Assisted Intervention – MICCAI 2012: 15th International
Surgery," in Medical Image Computing and Computer-Assisted Conference, Nice, France, October 1-5, 2012, pp. 544-551.
Intervention – MICCAI 2010: 13th International Conference, Beijing, [150] R. J. Housden, A. Arujuna, Y. Ma, N. Nijhof, G. Gijsbers, R. Bullens, et
China, September 20-24, 2010, pp. 275-282. al., "Evaluation of a Real-Time Hybrid Three-Dimensional Echo and X-
[130] P. Mountney, D. Stoyanov, and G.-Z. Yang, "Three-Dimensional Tissue Ray Imaging System for Guidance of Cardiac Catheterisation
Deformation Recovery and Tracking," IEEE Signal Process. Mag., vol. Procedures," in Medical Image Computing and Computer-Assisted
27, pp. 14-24, 2010. Intervention – MICCAI 2012: 15th International Conference, Nice,
[131] D. Stoyanov, G. P. Mylonas, F. Deligianni, A. Darzi, and G. Z. Yang, France, October 1-5, 2012, pp. 25-32.
"Soft-Tissue Motion Tracking and Structure Estimation for Robotic [151] G. J. Vrooijink, M. Abayazid, and S. Misra, "Real-time three-
Assisted MIS Procedures," in Medical Image Computing and Computer- dimensional flexible needle tracking using two-dimensional ultrasound,"
Assisted Intervention – MICCAI 2005: 8th International Conference, in Robotics and Automation (ICRA), 2013 IEEE International
Palm Springs, CA, USA, October 26-29, 2005, pp. 139-146. Conference on, 2013, pp. 1688-1693.
[132] X. Luo, A. J. McLeod, U. L. Jayarathne, S. E. Pautler, C. M. Schlacta, [152] X. Wu, J. Housden, Y. Ma, B. Razavi, K. Rhode, and D. Rueckert, "Fast
and T. M. Peters, "Towards disparity joint upsampling for robust catheter segmentation from echocardiographic sequences based on
stereoscopic endoscopic scene reconstruction in robotic prostatectomy," segmentation from corresponding X-ray fluoroscopy for cardiac
2016, pp. 97860Q-97860Q-10. catheterization interventions," IEEE Trans Med Imaging, vol. 34, pp.
[133] D. J. Mirota, M. Ishii, and G. D. Hager, "Vision-based navigation in 861-76, Apr 2015.
image-guided interventions," Annu Rev Biomed Eng, vol. 13, pp. 297- [153] M. Li, G. Li, B. Gonenc, X. Duan, and I. Iordachita, "Towards human-
319, Aug 15 2011. controlled, real-time shape sensing based flexible needle steering for
[134] P.-L. Chang, D. Stoyanov, A. J. Davison, and P. E. Edwards, "Real- MRI-guided percutaneous therapies," Int J Med Robot,
Time Dense Stereo Reconstruction Using Convex Optimisation with a 10.1002/rcs.1762, Aug 4 2016.
Cost-Volume for Image-Guided Robotic Surgery," in Medical Image
Computing and Computer-Assisted Intervention – MICCAI 2013: 16th
International Conference, Nagoya, Japan, September 22-26, 2013, pp.
42-49.
[135] B. Lin, Y. Sun, X. Qian, D. Goldgof, R. Gitlin, and Y. You, "Video-
based 3D reconstruction, laparoscope localization and deformation
recovery for abdominal minimally invasive surgery: a survey," Int J Med
Robot, vol. 12, pp. 158-78, Jun 2016.
[136] L. Maier-Hein, P. Mountney, A. Bartoli, H. Elhawary, D. Elson, A.
Groch, et al., "Optical techniques for 3D surface reconstruction in
computer-assisted laparoscopic surgery," Med Image Anal, vol. 17, pp.
974-96, Dec 2013.
[137] J. Stoll, H. Ren, and P. E. Dupont, "Passive markers for tracking surgical
instruments in real-time 3-D ultrasound imaging," IEEE Trans Med
Imaging, vol. 31, pp. 563-75, Mar 2012.
[138] E. M. Boctor, M. A. Choti, E. C. Burdette, and R. J. Webster Iii, "Three-
dimensional ultrasound-guided robotic needle placement: an
experimental evaluation," Int J Med Robot, vol. 4, pp. 180-91, Jun 2008.
[139] C. Kim, D. Chang, D. Petrisor, M. Han, and D. Stoianovici, "Robot for
ultrasound-guided prostate imaging and intervention," in 2011 IEEE/RSJ
International Conference on Intelligent Robots and Systems, 2011, pp.
943-948.
[140] A. L. Trejos, A. W. Lin, M. P. Pytel, R. V. Patel, and R. A. Malthaner,
"Robot-assisted minimally invasive lung brachytherapy," Int J Med
Robot, vol. 3, pp. 41-51, Mar 2007.
[141] C. Nadeau, H. Ren, A. Krupa, and P. Dupont, "Intensity-Based Visual
Servoing for Instrument and Tissue Tracking in 3D Ultrasound
Volumes," IEEE Trans. Autom. Sci. Eng, vol. 12, pp. 367-371, 2015.
[142] S. B. Kesner and R. D. Howe, "Position Control of Motion
Compensation Cardiac Catheters," IEEE Trans Robot, vol. PP, pp. 1-11,
[143] M.
Jul 21
Abayazid,
2011. G. J. Vrooijink, S. Patil, R. Alterovitz, and S. Misra,
"Experimental evaluation of ultrasound-guided 3D needle steering in
biological tissue," Int J Comput Assist Radiol Surg, vol. 9, pp. 931-9,
Nov 2014.

0018-9294 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.