You are on page 1of 50

The University of

CS 4487/9587

Algorithms for Image Analysis Ontario

Image Modalities
most slides are shamelessly stolen from
Steven Seitz, Aleosha Efros, and Terry Peters
The University of

CS 4487/9587 Algorithms for Image Analysis

Image Modalities Ontario

„ Photo/Video data
• Pin-hole
• Lenses
• Digital images and volumes
„ Medical Images and Volumes
• X-ray, MRI, CT, and Ultrasound

Extra Reading: Forsyth & Ponce, Ch. 1.


Gonzalez & Woods, Ch. 1
The University of

Slide by Steve Seitz

How do we see the world? Ontario

„ Let’s design a camera


• Idea 1: put a piece of film in front of an object
• Do we get a reasonable image?
The University of

Slide by Steve Seitz

Pinhole camera Ontario

„ Add a barrier to block off most of the rays


• This reduces blurring
• The opening known as the aperture
• How does this transform the image?
The University of

Slide by Steve Seitz

Camera Obscura Ontario

„ The first camera


• Known to Aristotle
• Depth of the room is the focal length
• Pencil of rays – all rays through a point
• Can we measure distances?
The University of

Figure by David Forsyth

Distant objects are smaller Ontario


The University of

Slide by Aleosha Efros

Camera Obscura Ontario

Drawing from “The Great Art of Light and Shadow “

Jesuit Athanasius Kircher, 1646.

How does the aperture size affect the image?


The University of

Slide by Steve Seitz

Shrinking the aperture Ontario

„ Why not make the aperture as small as possible?


• Less light gets through
• Diffraction effects…
The University of

Slide by Steve Seitz

Shrinking the aperture Ontario


The University of

Slide by Aleosha Efros

Home-made pinhole camera Ontario

http://www.debevec.org/Pinhole/
The University of

Slide by Steve Seitz

The reason for lenses Ontario


The University of

Slide by Steve Seitz

Adding a lens Ontario

“circle of
confusion”

„ A lens focuses light onto the film


• There is a specific distance at which objects are “in focus”
– other points project to a “circle of confusion” in the image
• Changing the shape of the lens changes this distance
The University of

Slide by Aleosha Efros

The eye Ontario

„ The human eye is a camera!


• Iris - colored annulus with radial muscles
• Pupil - the hole (aperture) whose size is controlled by the iris
• What’s the “film”?
– photoreceptor cells (rods and cones) in the retina
The University of

Slide by Aleosha Efros

Cameras Ontario

„ Really cool
„ Not too expensive nowadays (<$200)

Canon A70
The University of

Figure by Gonzalez & Woods

Digital Image Formation Ontario

f(x,y) = reflectance(x,y) * illumination(x,y)


Reflectance in [0,1], illumination in [0,inf]
The University of

Figure by Gonzalez & Woods

Sampling and Quantization Ontario


The University of

Figure by Gonzalez & Woods

Sampling and Quantization Ontario


The University of

Slide by Aleosha Efros

What is an image? Ontario

„ We can think of an image as a function, f, from


R2 to R:
• f( x, y ) gives the intensity at position ( x, y )
• Realistically, we expect the image only to be defined
over a rectangle, with a finite range:
– f : [a,b]x[c,d] Æ [0,1]
„ A color image is just three functions pasted
together. We can write this as a “vector-valued”
⎡ r(x, y) ⎤
function: f (x, y) = ⎢ g (x, y)⎥
⎢ ⎥
⎢⎣ b ( x , y ) ⎥⎦
The University of

Slide by Aleosha Efros

Images as functions Ontario

f(x,y)
y

x
The University of

Slide by Aleosha Efros

What is a digital image? Ontario

„ We usually operate on digital (discrete) images:


• Sample the 2D space on a regular grid
• Quantize each sample (round to nearest integer)
„ If our samples are Δ apart, we can write this as:
„ f[i ,j] = Quantize{ f(i Δ, j Δ) }
„ The image can now be represented as a matrix of integer values
The University of

3D Image Volumes? Ontario

„ Video - 3D = X*Y*Time
„ Medical volumetric data (MRI, CT) - 3D = X*Y*Z

Combine multiple images (slices) into a volume


The University of

CS 4487/9587 Algorithms for Image Analysis

Image Modalities Ontario

PART II:
Medical images and volumes
• X-ray
• CT
• MRI
• Ultrasound
The University of

Slides from Terry Peters

In the beginning…..X-rays Ontario

„ Discovered in 1895
„ Mainstay of medical imaging till 1970’s
„ 1971 – Computed Tomography (CAT, CT) scanning
„ 1978 - Digital Radiography
………
„ 1980 Magnetic Resonance Imaging
The University of

X-rays Ontario

„ Wilhelm Conrad Röntgen (1845-1923)


Nobel Prise in Physics, 1901

• “X” stands for “unknown”


• X-ray imaging is also known as
- radiograph
- Röntgen imaging
The University of

X-rays Ontario

Bertha Röntgen’s Hand 8 Nov, 1895 A modern radiograph of a hand

• Calcium in bones absorbs X-rays the most


• Fat and other soft tissues absorb less, and look gray
• Air absorbs the least, so lungs look black on a radiograph
The University of

X-rays Ontario

2D “projection” imaging 1895 - 1970’s


The University of

From Projection Imaging


Towards True 3D Imaging Ontario

X-ray imaging
1895

Mathematical results: Development of CT


Radon transformation (computed tomography)
1917 1972
•Image reconstruction from
projection
Computers can perform •Also known as CAT
complex mathematics to (Computerized Axial Tomography)
reconstruct and process images •"tomos" means "slice" (Greek)

Late 1960’s:
The University of

Radon Transformation Ontario

•Mathematical transformation (related to Fourier)


•Reconstruction of the shape of object (distribution f(x,y)) from
the multitude of 2D projections g ( s, θ )
The University of

Figure from www.imaginis.com/ct-scan/how_ct.asp

CT imaging Ontario
The University of

CT imaging, inventing (1972) Ontario

„ Sir Godfrey Hounsfield


Engineer for EMI PLC
1972
„ Nobel Prize 1979 (with
Alan Cormack)
The University of

CT imaging, availability (since 1975) Ontario

25 years later

1974

Original axial CT image from the Axial CT image of a normal brain using a
dedicated Siretom CT scanner circa 1975. state-of-the-art CT system and a 512 x
The EMI-Scanner
This image is a coarse 128 x 128 matrix; 512 matrix image. Note the two black
however, in 1975 physicians were "pea-shaped" ventricles in the middle of
fascinated by the ability to see the soft the brain and the subtle delineation of
tissue structures of the brain, gray and white matter
including the black ventricles for the first (Courtesy: Siemens)
time (enlarged in this patient)
The University of

Slides from Terry Peters

Clinical Acceptance of CT!? Ontario

„ Dr James Ambrose 1972


• Radiologist, Atkinson -
Morley’s Hospital London
• Recognised potential of EMI-
scanner

„ “Pretty pictures, but they


will never replace
radiographs” –
Neuroradiologist 1972
The University of

Slides from Terry Peters

Then ……………and Now Ontario

„ 80 x 80 image „ 512 x 512 image


„ 3 mm pixels „ <1mm slice thickness
„ <0.5mm pixels
„ 13 mm thick slices
„ 0.5 sec rotation
„ Two simultaneous
„ 0.5 sec recon per slice
slices!!!
„ Isotropic resolution
„ 80 sec scan time per
„ Spiral scanning - up to 16
slice
slices simultaneously
„ 80 sec recon time
The University of

Slides from Terry Peters

30 Years of CT Ontario
The University of

Slides from Terry Peters

Birth of MRI Ontario

„ Paul Lautebur 1975


• Presented at Stanford CT
meeting
• “Zeugmatography”
„ Raymond Damadian 1977 –
„ Sir Peter Mansfield early
1980’s

Early Thorax Image


Nottingham
The University of

Slides from Terry Peters

Birth of MRI Ontario

Early Thorax Image


Nottingham
• Electro Marnetic signal emitted (in harmless radio frequensy)
is acquired in the time domain
• image has to be reconstructed (Fourier transform)
The University of

Birth of MRI Ontario

In 1978, Mansfield presented his first


image through the abdomen.

Lauterbur
and the
first
magnetic
resonance
images
(from
Nature)
The University of

Slides from Terry Peters

30 Years of MRI Ontario

First brain MR image Typical T2-weighted MR image


The University of

Slides from Terry Peters

MR Imaging Ontario

„ “Interesting images, but will never be as


useful as CT”

• (A different) neuroradiologist, 1982


The University of

Slides from Terry Peters

MR Imaging …more than T1 and T2 Ontario

„ MRA - Magnetic resonance angiography


• images of vessels
„ MRS - Magnetic resonance spectroscopy
• images of chemistry of the brain and muscle metabolism
„ fMRI - functional magnetic resonance imaging
• image of brain function
„ PW MRI – Perfusion-weighted imaging
„ DW MRI – Diffusion-weighted MRI
• images of nerve pathways
The University of

Slides from Terry Peters

Magnetic Resonance Angiography Ontario

„ MR scanner tuned to
measure only moving
structures
„ “Sees” only blood - no static
structure
„ Generate 3-D image of
vasculature system
„ May be enhanced with
contrast agent (e.g. Gd-DTPA)
The University of

Slides from Terry Peters

MR Angiography Ontario

GD-enhanced
Phase-contrast

GD-enhanced

In-flow
The University of

Slides from Terry Peters

Dynamic 3-D MRI of the thorax Ontario

1 www.atamai.com
The University of

Slides from Terry Peters

Diffusion-Weighted MRI Ontario

„ Image diffuse fluid motion in


brain
„ Construct “Tensor image” –
extent of diffusion in each
direction in each voxel in
image
„ Diffusion along nerve sheaths
defines nerve tracts.
„ Create images of nerve
connections/pathways
The University of

Slides from Terry Peters

Tractography Ontario

„ Data analysed after


scanning
„ Identify “streamlines”
of vectors
„ Connect to form fibre
tracts Internal Capsule
„ 14 min scan time

- Dr. D Jones, NIH


The University of

Slides from Terry Peters

Tractography Ontario

Wernicke’s area
Broca’s area

Short fibres

Long fibres
Insula
fibres

Temporal fibres
“just like Gray’s Anatomy”!
Superior
Longitudinal Fasciculus
- Dr. D Jones, NIH USA
The University of

Slides from Terry Peters

Functional MRI (fMRI) Ontario

„ Active brain regions demand more fuel


(oxygen)
„ Extra oxygen in blood changes MRI signal
„ Activate brain regions with specific tasks
„ Oxygenated blood generates small (~1%) signal
change
„ Correlate signal intensity change with task
„ Represent changes on anatomical images
The University of

Slides from Terry Peters

fMRI Ontario

Subject looks at Stimulus


flashing disk while
being scanned
“Activated” sites
detected and
merged with 3-D
MR image Activation
The University of

Slides from Terry Peters

fMRI in Neurosurgery Planning Ontario

Hand Activation

Tumour
Face Activation
The University of

Slides from Terry Peters

Ultrasound Ontario

Images courtesy GE Medical