Computer-Assisted Surgery

Medical Robotics
Medical Image Processing
LECTURE 1
1. Whats in a surgery
2. Technical tools in CS
3. CAS systems

PAST: Cut, then see

CAS, Spring 2001

L. Joskowicz

PRESENT: See, then cut

CAS, Spring 2001

L. Joskowicz

FUTURE: Combine, see, minimally cut

CAS, Spring 2001

L. Joskowicz

How do surgeries proceed?

Diagnosis
based on physical exams, images, lab tests

Preoperative planning
determine the surgical approach
elaborate intraoperative plan (path, tools, implants)

Surgery
prepare patient and assess condition
acquire intraoperative images, adapt and execute plan

Postoperative follow-up
exams, lab tests, images to be corroborated
CAS, Spring 2001

L. Joskowicz

Treatment procedures
Invasive
neurosurgery: tumor removal
hear surgery: clogged arteries, transplants
orthopaedic surgery: spine, hip replacement, knee,
fractures
gall bladder removal, prostate, various cancers

Non-invasive
radiation therapy
kidney stone pulverization
CAS, Spring 2001

L. Joskowicz

Medical imaging modalities

Preoperative
Film X-rays, Digital X-rays, Ultrasound,
Angiography, Doppler, .
Computed Tomography (CT), Magnetic Resonance
(MR), Nuclear Medicine (PET, SPECT, )

Intraoperative
X-ray fluoroscopy, ultrasound
video images (laparoscopy, arthorscopy)
Open MR
CAS, Spring 2001

L. Joskowicz

Medical imaging modalities: X-rays

Film or Digital X-ray

CAS, Spring 2001

L. Joskowicz

X-ray Fluoroscopy
9

Medical imaging modalities:

continuous X-ray angiography

CAS, Spring 2001

L. Joskowicz

10

Medical imaging modalities: Ultrasound

CAS, Spring 2001

L. Joskowicz

11

Medical imaging modalities: CT

Series of parallel
slices 2mm apart
Single slice
CAS, Spring 2001

L. Joskowicz

12

Medical imaging modalities: MRI

Good imaging of
soft tissue
CAS, Spring 2001

L. Joskowicz

13

Medical imaging modalities: Nuclear

medicine (PET, SPECT, NMR)

Functional imaging:
colors indicate
electrical activity
CAS, Spring 2001

L. Joskowicz

14

Medical imaging modalities: video

TV quality image from small camera

(laparoscope or endoscope)
CAS, Spring 2001

L. Joskowicz

15

Surgical approaches
Open surgery
area of interest directly exposed by cutting
direct sight and touch of anatomy by surgeon
direct access but causes additional damage

Closed surgery

not always feasible

indirect access to anatomical area of interest

no direct visual sight or tactile feel
catheterization, biopsies
intraoperative imaging is often required
require more skills: lengthier, more difficult

Diagnostic surgery
CAS, Spring 2001

L. Joskowicz

16

Minimally invasive surgery

Provides treatment through small incisions
Uses imaging equipment for seeing and
instruments for touching
Advantages: less damage, faster recovery
Disadvantages: hand/eye coordination, time
Examples:
brain tumor removal, laparoscopic surgery

CAS, Spring 2001

L. Joskowicz

17

Laparoscopic surgery

CAS, Spring 2001

L. Joskowicz

18

Brain surgery

CAS, Spring 2001

L. Joskowicz

19

Total Hip replacement -- principle

CAS, Spring 2001

L. Joskowicz

20

Total hip replacement procedure

CAS, Spring 2001

L. Joskowicz

21

What is required to perform surgery?

Knowledge intensive task
anatomy, procedures, cases
experience, skills, customization and generalization

Manual and cognitive skills

dexterity, precision, strength, tool manipulation
spatial orientation and navigation

Determination
information integration
judgement, decision, execution
CAS, Spring 2001

L. Joskowicz

22

Medical and surgical trends

Imaging improved dramatically diagnosis
started with X-rays last century
30% of all cases use images

Move towards minimally invasive procedures

introduced in the mid 70s, slow acceptance (laparoscopy)
the method of choice now

More precise and delicate procedures

Development of sophisticated surgical hardware
High degree of craftsmanship and skills
CAS, Spring 2001

L. Joskowicz

23

Socio-economical medical trends

Increase of aging population and associated
problems: tumors, osteoporosis, Alzheimers
Larger population volumes
Universal, first rate, highly specialized care
Health care costs reduction (managed care)
Higher patient requirements
Legal and regulatory aspects

CAS, Spring 2001

L. Joskowicz

24

Surgical Needs
Augment the surgeons capabilities with better
quantitative planning, execution, and integration
Support for image-guided surgery
Passive and active devices for accurate spatial positioning,
tracking, and execution
Modeling, planning, viewing, diagnosis systems
Systems integration: from diagnosis to post-op
Improve current practice and enable new procedures
Simulation and training systems

CAS, Spring 2001

L. Joskowicz

25

Current clinical status

Imaging
vast databases of medical images
digitized atlases
mostly uncorrelated unimodal qualitative interpretation

Devices
mostly passive and non-invasive (supports)
laparoscopic camera,
some real-time tracking

Planning, modeling, visualization

3D reconstruction, some registration
CAS, Spring 2001

L. Joskowicz

26

Part 2: Computers and Robots

Technology and algorithms
available today

How can computers help?

(or are already helping)
Image processing
single image: enhancement, noise reduction,
segmentation, quantitative measurements
image stacks: 3D reconstruction, segmentation
image sets: registration, comparison, data fusion

Planning and simulation

integrate medical images and CAD models
planning and simulation programs

Computer vision and graphics

camera modeling, image registration, rendering
CAS, Spring 2001

L. Joskowicz

28

Image processing

CAS, Spring 2001

L. Joskowicz

29

Planning and simulation

CAS, Spring 2001

L. Joskowicz

30

Virtual man project -- digital model

CAS, Spring 2001

L. Joskowicz

31

How can robots and sensors help?

(or are already helping)
Robotic devices
passive, semi-active, active devices
instrument and anatomy positioning and holding
cutting and machining

Real-time tracking
optical, video, electromagnetic devices
navigation tools

CAS, Spring 2001

L. Joskowicz

32

Robotic devices

CAS, Spring 2001

L. Joskowicz

33

Real-time tracking devices

camera

instrument

Passive markers

Instrument has infrared

LEDs attached to it
CAS, Spring 2001

L. Joskowicz

Active markers
34

Computer-Assisted Surgery (CAS)

A computer-integrated system to enhance the
dexterity, visual feedback, and information
integration of the surgeon
Key points:
The goal is NOT to replace the surgeon
A new paradigm for surgical tools
Address a real clinical need
Prove efficacy and cost-effectiveness
CAS, Spring 2001

L. Joskowicz

35

Elements of CAS systems

CAS, Spring 2001

L. Joskowicz

36

Elements of CAS systems

Preoperative planning
image acquisition, modeling, analysis, simulation
plan elaboration, tool and prosthesis selection
Output: preop images, 3D models, prosthesis type and
position, navigation and cutting plan

Intraoperative execution
passive, semi-active, active robot
real time tracking
intraoperative imaging (fluoroscopy, ultrasound)

CAS, Spring 2001

L. Joskowicz

37

State of the Art (1)

Main clinical procedures
neurosurgery: biopsies, tumor removal
orthopaedics: hip and knee replacement, spine, pelvis
and femur fractures
maxillofacial and cranofacial
laparoscopy: laparoscope holders
new fields: dentistry, ophtalmology, prostate

Mostly rigid structures: bones!!

CAS, Spring 2001

L. Joskowicz

38

State of the Art (2)

Commercial navigation systems
main uses: neurosurgery and spine surgery

Commercial robotic systems

ROBODOC for total hip replacement
laparoscope arm holders

Research
very active, very interdisciplinary
a few dozen systems tested in-vitro

CAS, Spring 2001

L. Joskowicz

39

State of the Art (3)

Major players
INRIA Sophia Antipolis, Grenoble, Johns Hopkins,
Brigham Womens H./MIT, Shadyside H./CMU,
Imperial College, many places in Germany and Japan

Interdisciplinary conferences and journals

started in 1994: MRCAS94; Orthopaedic CAS
meetings, visualization, etc,
Journals: Computer-Aided Surgery, Medical Image
Analysis
CAS, Spring 2001

L. Joskowicz

40

Examples of CAS systems in use

Image-guided navigation systems

ROBODOC: Total hip replacement surgery
LARS: Laparoscopic assistant
Radiosurgery

Brief overview follows; will be covered in detail later

CAS, Spring 2001

L. Joskowicz

41

Image-guide navigation
Purpose
accurate placement of instruments with respect to
imaged anatomy for several procedures

Problem addressed
provide 3D vision of unseen structures
replace static 2D fluoroscopy or larger openings
improve precision of biopsies, screw placements

Scope
non-invasive
creates surface model from preop images
registration of images to anatomy by direct contact
CAS, Spring 2001

L. Joskowicz

42

Image-guided navigation

CAS, Spring 2001

L. Joskowicz

43

Image-guided navigation (2)

pedicle screw insertion

CAS, Spring 2001

L. Joskowicz

44

Status
In clinical use
Over 7,000 neurosurgeries performed with
commercial systems
Gaining popularity in pedicle screw insertion
Decreased the misplacement rate from 10-40%
to 5-18% (clinical study of 700 cases)
More clinical applications under development

CAS, Spring 2001

L. Joskowicz

45

ROBODOC: Total hip replacement

Purpose
precise machining of cementless hip implant canal

Problem addressed
complications in canal preparation and implant fixation
improve positioning accuracy and surface finish

Scope

CAS, Spring 2001

invasive, numerically controled machining

plan from preop CT, registered via pins
adapted commercial robot
custom bone fixator and bone motion detection
L. Joskowicz

46

Artificial hip joint

CAS, Spring 2001

L. Joskowicz

47

Total hip replacement procedure

CAS, Spring 2001

L. Joskowicz

48

ROBODOC: Total Hip Replacement

CAS, Spring 2001

L. Joskowicz

49

ROBODOC system diagram

CAS, Spring 2001

L. Joskowicz

50

ORTHODOC Planning

CAS, Spring 2001

L. Joskowicz

51

ROBODOC robot diagram

CAS, Spring 2001

L. Joskowicz

52

ROBODOC robot

CAS, Spring 2001

L. Joskowicz

53

ROBODOC procedure

CAS, Spring 2001

L. Joskowicz

54

ROBODOC cutting

CAS, Spring 2001

L. Joskowicz

55

ROBODOC History
Developed by IBM Research and Integrated
Surgical Systems
First active surgical robot

CAS, Spring 2001

1986: feasibility study

1989: in-vitro testing of dog system
1990: 26 dog cases
1992: development of human system
1994: first human procedure in Frankfurt
1995- clinical trials in the US for FDA approval
L. Joskowicz

56

ROBODOC current status

Sold by Integrated Surgical Systems

Over 3,000 cases performed
15 systems installed in Germany, 2 in Austria
Excellent short term clinical results (3 year study)
no fractures, few failures (continue manually)

Long-term clinical results to be determined

key issue: does the artificial hip last longer?

Problems: OR time, pin insertion

CAS, Spring 2001

L. Joskowicz

57

Laparoscopic assistant: LARS

Purpose
laparocopic camera holding and precise navigation

Problem addressed
cumbersome, unintuitive, and unsteady camera
positioning

Scope
non-invasive intraoperative device
video images interpreted by surgeon

Benefits
direct camera manipulation; stability, precise targeting
CAS, Spring 2001

L. Joskowicz

58

Laparoscopic assistant: LARS

CAS, Spring 2001

L. Joskowicz

59

LARS characteristics
Designed at IBM Research, 1993. Similar
commercial devices available (AESOP)
Custom redundant 7 degree-of-freedom robot
Holds laparoscopic camera
Fulcrum motions: no motion at point of entry
Mouse-like controls on surgical scissors
Position memory and replay

CAS, Spring 2001

L. Joskowicz

60

Stereotactic Radiosurgery
Purpose
plan and deliver precise radiation doses

Problem addressed
precise positioning and dosing of radiation to avoid
healthy organ damage

Scope

CAS, Spring 2001

non-invasive intraoperative device

active beam postioning and planning
complex preoperative planning based on MRI images
registers preoperative plan with stereotactic frame
L. Joskowicz

61

Stereotactic Radiosurgery

CAS, Spring 2001

L. Joskowicz

62

CYBERKNIFE system

CAS, Spring 2001

L. Joskowicz

63

CYBERKNIFE system

CAS, Spring 2001

L. Joskowicz

64

Stereotactic Radiosurgery: planning

CAS, Spring 2001

L. Joskowicz

65

Stereotactic Radiosurgery

Developed at Stanford starting in 1992

Complex 3D radiation plans
Currently in clinical use
Frameless procedure under development
follow head with markers, video, or X-rays
Company Accuray has performed several clinical
trials with frameless procedure

CAS, Spring 2001

L. Joskowicz

66

Developing CAS systems

Similarities

understand and address real needs of surgeons

consider established procedures, context, use
work on problems that will make qualitative difference
constant feedback from user; test ideas and prototypes

Differences
system performace requirements

CAS, Spring 2001

L. Joskowicz

67

Developing CAS systems

understand and address real needs of surgeons
consider established procedures, context, use
constant feedback from user; test ideas and
prototypes
system requirements
safety and reliability
fail-safe systems: can always stop and proceed as usual
system integration
CAS, Spring 2001

L. Joskowicz

68

CAS systems design cycle

Prototype development
In-vitro experiments
system refinement

Cadaver studies
system refinement

In-vivo experiments
first animal and human trials

Clinical trials
double blind studies, Hospital and FDA protocols

Agency approval and commercial release

CAS, Spring 2001

L. Joskowicz

69

Summary
Great potential for robots and computers inside
and outside the operating room
Great research and commercial interest,
especially in the past 3 years
Just the beginning of the road: many things
remain to be invented
Great role for applied computer science:
image processing, geometric planning, registration,
graphics, vision, real-time systems, robotics, etc.
CAS, Spring 2001

L. Joskowicz

70