Lecture 3: Digital Acquisition Ilan Davis November 2006 Advanced topics to be covered

Light path for digital acquisition -multiwavelength, multi-Z sections - e.g. DeltaVision microscopy OMX microscope light path Kohler illumination / critical illumination / fibre optic delivery Field diaphragm, Aperture diaphragm, neutral density filters, stops. Filter wheels, motorized XYZ stages, piezoelectric focus drives. High resolution DIC and how to image fluorescence and DIC at the same time The use of fluorescent beads to test lens and microscope performance Determination of resolution of the microscope (full width half Max measurement) Airy rings (out of focus light) Point spread function (PSF) and optical transfer function (OTF) The concept of convolution Deconvolution New methods of deconvolution in development: Pupal function, 3D variations in psf in a specimen CCD cameras including Intensified CCD and EMCCD. TIRF and high resolution DIC

Last lecture How do fluorescence microscopes work ?
Eye piece camera Emmision Filter Dichroic mirror Objective lens specimen Excitation filter

Mercury arc Light source

w to use DIC and Fluorescence without loss signal by polarizing filter?
Emmision Filter Eye piece camera

Dichroic mirror Polarizing filter Wollastom Prism Objective lens specimen Wollastom Prism Polarizing filter Brightfield Halogen lamp Excitation filter

Mercury arc Light source

us DIC and fluorescence imaging with no loss of fluorescence intensity

ube where dichroic mirror acts as polariser only in red light instead of the analyser. only for FITC now available for FITC/rhodamine/DIC and other flavours emoved to emission filter wheel (poorer quality DIC)

ution DIC

right field illumination (image of magnified central part of UV bulb focused directly o Kohler). d IR filters used. nser with oil immersion lens. resolution (low contrast) Wollaston prism

Elements that make up the DeltaVision widefield fluorescence Microscope (Based on design by John Sedat and David Agard)
Specimen XYZ Motorized stage

mirror IR (heat) filter Excitation Shutter

Mercury vapour arc lamp
Collector lens (focusable)

Neutral density Filters (in filter wheel) Excitation Filter (in filter wheel)
Collector lens (light into fibre)

Objective lens Quad Dichroic mirror

Eye piece

AD

Condenser

FD

Kohler or critical Partial mirror illumination

1mm Quartz fibre optic cable (to scramble the light)

CCD
Shutter

Photosensor measures The intensity of light
Mirror Emmision filter wheel

Special features of the DeltaVision Microscope
Nanomoover XYZ stage with ramping system - the most accurate on the market 10nm error. 100nm repeatability Quartz fiber optic light scrambling -Achromatic optics -Efficient transmission at 340nm and approx 20mW delivered at objective at 546nm -Can be very carefully aligned for Kohler or critical illumination Intensity Correction system -adjusts for lamp fluctuations Filter wheel accuracy to 3.5microns -ensures no shift in image in successive optical planes (multichannel imaging) 7 Neutral density filters Dry Nitrogen gas supplied to all filters to slow deterioration due to oxydation and moisture Shutters are spring mounted - isolate vibrations

Excitation shutter mounted at an angle -increases bulb life (reduced heating from reflection) -prevents reflections into image light path Camera imaging chip placed at side port with no intermediate glass (which absorbs some light and causes chromatic aberration) -optimal position for resolution -deconvolution on the fly on the camera chip (DV-RT) Software deals with 5D datasets in one file and runs on Unix platform (more stable, better networking and deals with large files better). Probably the

OMX - light path
Light source
Rotating ground Neutral density Diode lasers Rapid shutters Glass make Change intensity Choose line non-coherent

XYZ nanomover XYZ Piezo

Specimen

Objective lens

Tube lens Fibre optic
EM-CCD EM-CCD

Simultaneous acquisition of 4 channels

EM-CCD EM-CCD

Precisely machined Metal block with internal sculpturi That absorbs stray light.

So far we have really considered image formation by objective lenses in an over-simplified manner.

Out of focus light -Airy rings in 3D view
Different Focal plains

Detectors

otographic emulsion. ry high resolution, poor sensitivity but low noise. t used now and replaced by 3-CCD colour cameras, more sensitive than film, low noise if cooled.

oton Multiplier Tube (PMT). Noisy and non-linear, but very good with laser scanning confocals ndy’s lecture).

arge Coupled Device (CCD) -silicon. Very sensitive, linear. Excellent with widefield systems and inning disc confocal.

Representative pixel of CCD

Front illuminated

vs. Backthinned

CCDs (Charge Coupled Device)
Types of readout Full frame Frame transfer Interline transfer
Backthinned vs. Front illuminated Full-well capacity Number of maximum electrons that can be stored per well

Analog-Digital converter Usually 12bit signal (4096 in decimal) Some 14 bit or 16bit Amplifier e.g. Coolsnap (Roper) approx 16,000e- wel Depth Or 4e- per 1 gray scale unit Pixel size: 6.45µm Reader Ixon pixel size 17µm, 16bits, A-to-D converter well depth 100,000e-.

Properties of CCD cameras

(most important when pixel values are low) of number of electrons captured on each pixel). - due to counting stochastic process. e=10 S/N=10 oise=100, S/N=100

(most important when taking short exposures <1sec) the electronics of the camera and increases with speed of camera.

t noise (most important when taking long exposures >1sec) dependent (reduced by factor of 2 for every 8-9 degrees cooling) Peltier effect (current between different metals and heat sink)

iciency etectible electrons produced per incident photon. Varies considerably with wavelength. tings increase UV sensitivity. (Film, QE=0.01)

rs ze of pixels (generally 7-15 microns) i.e. maximum number of electrons that can be captured in a pixel) ad out of block of pixels pooled -eg 2x2 binning increases sensitivity x4. But reduced resolution). adout 1MHz, 5MHz, 20MHz -not the only factor in how fast you can capture images. grades of CCDs (usually scientific grades are used -only very few dud pixels) , masks, gating. and other references for more details)

Scientific CCD cameras -Jargon Buster (Based on a document from Opto and Laser Europe http://optics.org/articles/ole/8/2/3/1) CCD cameras may be a popular tool for scientfic imaging, but lots of technical jargon and a wide range of models can make it very difficult to select the most suitable product. Oliver Graydon offers some useful advice for the novice. The bewildering array of products on the market and reams of associated technical jargon can make choosing the right CCD camera a daunting task. This aims to simplify the task by indicating the key performance criteria to consider and translating the jargon. How CCDs work At the heart of all CCD cameras is a charge-coupled device (CCD) image chip. This is an array of light-sensitive pixels that are electrically biased so that they generate and store electrons - electric charge - when illuminated with light. The amount of charge trapped beneath each pixel directly relates to the number of photons illuminating the pixel. This charge is then "read out" by changing the electrical bias of an adjacent pixel so that the charge travels out of the sensor, is converted into a voltage and is then digitized into an intensity value. This action is performed for each pixel, to create an electronic image of the scene. Electronics inside the camera control the read-out process. Criteria for CCD use: • The wavelength region you wish to image. • The lighting level. Are you trying to detect single photons or bright events? • The frame-rate required. Are you intending to perform very fast imaging? Note that the performance characteristics of a CCD camera are often interrelated, and trying to optimize one parameter will often compromise another. What wavelength response do you need? The wavelength sensitivity of a CCD camera is usually determined by the quantum efficiency (QE) of the CCD chip. Most CCD chips with no special coatings have a QE in excess of 30% in the visible and near-infrared (400-850 nm). However, if you need to perform imaging at slightly shorter or longer wavelengths, it is possible to obtain CCD chips that are coated with phosphors to increase their sensitivity in the infrared, blue and ultraviolet. Buying a CCD camera with a back-thinned or deep-depleted CCD chip can also enhance a camera's sensitivity in the ultraviolet, visible and nearinfrared, and may be useful if you are performing low-light imaging in these regions. It is important to note that the wavelength sensitivity of a CCD camera is temperature-dependent and will change if the camera is cooled (cooling is a popular way to reduce the dark current of a camera). By controlling the temperature of the CCD chip by thermoelectric cooling (rather than cooling with liquid nitrogen) it is possible to optimize the CCD's QE in a given wavelength range. How strong is the lighting level? If you want to image very low light-levels, for instance in experiments involving fluorescence or bio/chemoluminescence, it is essential to use a CCD camera with an optimized signal-to-noise ratio. In reality that means choosing between a cooled high-QE CCD camera, which has very low noise, and an intensified camera (ICCD), which makes use of an image intensifier and can image single photons. Vendors that sell both types of devices indicate that CCD sensor technology is now so good that in many cases they recommend using cooled CCD cameras unless you need a measurement with a nanosecond response.

CCD Jargon Buster - page 2 When imaging in good light conditions or attempting to capture bright events, the signal-to-noise ratio is no longer a pressing issue and a CCD camera will not need to be cooled or require an image intensifier. Instead, make sure that the CCD camera has good dynamic range, linearity and saturation characteristics. To obtain good-quality images it is important that the CCD camera does not saturate. The latter occurs when a pixel is close to storing its maximum amount of charge (known as its full-well capacity). When saturation starts to occur, the CCD's image becomes distorted as its response to light is no longer linear. In extreme cases, blooming occurs when the charge from a pixel starts to overflow into adjacent pixels. This shows up as a white streak around a bright point on the image. Many CCD cameras now come with sophisticated antiblooming features. What frame-rate and time resolution? If you need to image very fast events, such as laser-induced breakdown spectroscopy, laser ablation or time-resolved fluorescence, you will need to use an intensified CCD camera. This offers very fast gating (shutter) times of a few nanoseconds. It is also important to consider the required framerate of the CCD or ICCD camera. If you want to perform imaging at video frame-rates or above then fast read-out cameras are necessary, often in the form of frame-transfer of interline CCD formats. What does "dynamic range" mean? The dynamic range (DR) of a CCD camera is a measure of its ability to accurately record bright and dim parts of an image. A large DR (ideally 12 bits or more) is required to obtain high-quality images. The cause of the confusion is that different parts of the camera system, namely the CCD image sensor and the camera's electronics, often have a different DR. The important point to remember is that it is the weaker value of the two that ultimately dictates the true DR of the camera. • Image sensor DR This is defined as the image sensor's full-well capacity divided by its read noise. It is quoted in bits: commercial CCD image sensors often have values ranging from 10 to 16 bits. • Electronic dynamic range This refers to the resolution of the analogue-to-digital (A/D) converter in the camera's electronics, which is used to convert a pixel's electron count into an intensity value in an image. A 12-bit A/D gives 4,096 intensity levels and a 16-bit A/D gives 16,384 levels. On-chip multiplication/electron-multiplying CCD A relatively new technology that has another stage of electronics in the read-out circuit of the CCD sensor. This amplifies the signal so that it dwarfs the read-out noise of the sensor. Vendors of this technology say that the result is that read-out noise is essentially eliminated. Back-filled The housing of the CCD is filled with argon to prevent condensation from forming on the image sensor. Back-thinned/back-illuminated A highly sensitive CCD chip image sensor that has been etched by acid so that it is just 10 µm thick. A backthinned chip is illuminated from the rear so that incident light does not have to pass through the polysilicon gate electrode that is deposited on the top surface of a pixel. The process can enhance the QE through the visible, ultraviolet and near-infrared. Binning The process of combining the charge from several adjacent pixels to create "superpixels" prior to read-out. Binning enhances the frame-rate and signal-to-noise ratio of the camera at the expense of reduced spatial resolution. Blooming/saturation Saturation relates to the maximum light intensity that a pixel can cope with. If a pixel is thought of as a bucket of photoelectrons, saturation occurs when a pixel's bucket is full.Blooming occurs when a bucket starts to overflow and charge spreads into neighbouring pixels, causing them to report false light levels. Blooming shows up as a white streak or blob around a bright point on the image.

CCD Jargon Buster - page 3 CCD noise/dark current This refers to spurious electrons that are generated in the CCD chip by thermal and other effects in the absence of any illumination. The effects of CCD noise can be dramatically reduced by cooling the image sensor. Deep-depletion CCD A CCD chip that is designed to offer superior sensitivity in the near-infrared and high-energy X-ray region. It contains a biased area of high-resistivity silicon for capturing photons that would not normally be absorbed. Etaloning In a back-illuminated CCD, reflections between the front and back surfaces can lead to interference effects that degrade the performance of the camera in the near-infrared. Anti-etaloning technology in deep-depletion CCDs can overcome this effect. Image intensifier A vacuum tube, usually 18-25 mm in diameter, that gives a significant boost in light sensitivity. It is placed in front of the CCD chip and converts incoming photons into electrons, which are multiplied before being converted back into photons. The intensifier can also be gated for time-resolved experiments. Megapixel A term used to describe a CCD sensor that contains at least one million pixels. Multipinned phase (MPP) A way of biasing the CCD chip so that the CCD's dark current is dramatically reduced. As a result, less cooling is required to reduce the noise of the camera, and compact thermoelectric coolers can be used instead of liquid nitrogen. Photon/shot noise The laws of physics dictate that the number of photons striking a detector is inherently uncertain. This uncertainty is known as photon, or shot, noise and varies with the square root of the signal level. High QE cameras improve the signal to shot noise ratio. Quantum efficiency (QE) The probability of a CCD chip converting an incoming photon of a given wavelength into an electron. It is measured as a percentage, and the higher the value, the more sensitive the camera. Read-out noise This is noise that is generated by the electronics that convert the charge from each pixel into a digitized light-intensity value that is displayed in the image. Read-out noise increases with the speed of the read-out and consequently the frame-rate. It is reduced by technologies such as on-chip multiplication. Read-out rate This is often quoted in MHz and refers to the rate of read out of data per second. It should not be confused with frame-rate. Smearing If light is still falling on the CCD chip during read-out, image distortion called smearing can result. The use of physical shutters and frame-transfer or interline CCDs minimizes the effect.

ous types of CCD cameras

apHQ.

Interline chip, fast readout es per second). High QE, due microlenses, low noise.

ull frame readout. Even lower noise and high QE, but much slower readout.

ned - very expensive, higher QE.

ax intensified CCD (I-CCD) - very expensive, very sensitive but high noise, poor resolution.

nologies:

gain (amplifier) before reading the signal and converting from analogue to digital. E.g. Ixon from An ch pixel has its own amplifier. So far only on very cheap mass produced CCD cameras (not good in low

Examples of CCD cameras

Now largely Superceded by EMCCD technology

Electron Multiplication CCDs (EMCCD)

(same as on chip multiplication or on chip gain)

Back thinned or front illuminated Dual amplifier (conventional / electron multiplicat On chip amplification -makes read noise irrelevant (<1e-)

Amplifier Reader A-to-D converter

Cascade: http://www.photomet.com/cascade.html

erent manufacturers (lots of interesting information of web sites):

su C9100: http://www.globalspec.com/FeaturedProducts/ Detail?ExhibitID=16167 Ixon: http://www.andor-tech.com/products/brand.cfm?marketsegment=1&brand=3

Appendix I

 air    η 1 =1.0

coverslip glass η 2 =1.515 α2

α1

Numerical Aperture NA = η  sin α  (η = refractive index α = cone angle ) x­y Resolution d = 0.61 λ / ΝΑ (d = smallest resolvable distance) Brightness I α NA4 / mag2 (I = intensity mag = magnification)

Snell’s law:   η 1  sin α 1 = η 2  sin α 2 

Appendix II

ences

ric focusing theory and practice. PI (www.pi.ws) http://www.physikinstrumente.com/

ue - Excellent history of subject and current detailed explanations of latest equipment

calization by Fluorescence Microscopy, A practical Approach Ed. V.J. Allan OUP ISBN 0-19-963740-7 y Ilan Davis: How to make and use fluorescent bead slides and how to correct spherical aberration. davis@ed.ac.uk for a copy)

y Bill Amos: Instuments for fluorescence imaging (CCD information)

., Schaefer, L. H., Swedlow, J. R. (2001) A working person’s guide to deconvolution in oscopy. Biotechniques 31, 1076

. R., Hu, K., Andrews, P. D., Roos, D. S., Murray, J. M. (2002) Measuring tubulin content in gondii: a comparison of laser-scanning confocal and wide-field fluorescence microscopy. 014

., Sedat, J., Agard, D. (1990). Biophysics Journal 57 325. Determination of three-dimentional imaging microscope system, Partial confocal behaviur in epifluorescnece microscopy

nufacturer of CCD cameras http://www.roperscientific.com/

od application notes and information on their site. E.g.

.roperscientific.com/library.shtml http://www.roperscientific.com/pdfs/technotes/ccd_grading.pdf

http://www.roperscientific.com/library.shtm

ading manufacturer of CCDs: http://www.andor.com/

different deconvolution methods used in Astronomy http://jstarck.free.fr/pasp02.pdf

and Davis, I. (2005) Deconvolution: Lifting the fog. Book chapter, in Cell Biology, a laboratory han E.Celis. Academic Press. (Ask Richard.Parton@ed.ac.uk for a copy)

Thanks Richard Parton Dave Kelly Ken Sawin Alejandra Clark John Sedat

al Internal reflection fluorescence microscopy (TIR 60X/NA1.45 oil TIRF; 60X/NA1.65 oil TIRF

Evanescent Wave (only penetrates 100nm below coverslip)

Objective lens

Laser light At an angle

Spinning disc confocal microscope

Laser scanning confocal microscope

Airy rings - 2D view

The basis of Airy ring formation Is diffraction through a slit (aperture of the lens)

e shape of the Airy rings vary according to velength and size of the aperture in a reproducible t depend in details on the individual objective len

Airy rings - 3D view Point Spread Function (psf)

Objective lens α

Z X Y X
Planes of focus (z series)

le lenses (single element) have spherical aberratio

ives are made of many elements to correct spherical aberration.

bjectives are designed to image correctly at the surface of a cover articular thickness (usually number 1.5, or 0.17mm (0.15-0.19mm).

z

x

y

x

Bead slide: 0.1micron and 0.5 micron Surface of slide z microns thick Surface of cover slip
Tetraspeck beads: chromatic registration DAPI/FITC/Rhodamine/Cy5 Beads (PS Spec): Single fluorochrome Brighter -better for generating point spread functions for deconvolution Inspec Intensity beads: Measure dynamic range

Affects of deep imaging (90µm) and collar settings on spherical aberration and psf of 60X/NA1.2w

0.13 surf 0.13deep f

0.15 deep 0.15 surf Data from Alejandra Clark

0.13 surf 0.17 surf.17 deep 0

0.19 surf

0.19 deep

0 0.21 surf.21 deep

z

y

x

0.21 setting surface bead z y x
Intensit y Intensity
3500

0.21 setting deep bead z y x

4000
4500 4500

Intensit y

3000

3500

4000

2500

3000

3500

2500

3000

2000

2000

2500

1500

2000

1500

1000

1500

1000

500

1000

500
500

Intensity
7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45

4000

3500

3000

2500

2000

1500

1000

500

0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 x-axis (pixels)

0 1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141 148 155 162 169 176
0 1 3 5 0 1 7 13 19 25 31 37 43 49 55 61 67 73 79 85 91 97 103 109 115 121 127 133 139 145 151 157 163 169 175 181

x-axis (pixels)

z-axis (pixels)

x-axis (pixels)

z-axis (pixels)

defield Deconvolution estoring out of focus light to its point of origin

fore Deconvolution

After Deconvolution

0.1µm fluorescent  bead

real image

deconvolution (+psf otf)

convolution (objective  lens)

Planes of focus of real image

infocus light z y

observed  image

out of focus light  (airy rings) x Planes of focus of observed image

Increase in resolution (XY and Z) after deconvoluti
a
x-y focal plane

intensity (in gray scale values)

a

b
6000 5000

6000

b

z axis

5000

4000

4000

3000

0.18 0.26

3000

0.34 0.65

observed deconvolved fwhm

2000

2000

1000

1000

5

25

10

20

5

20

x or y (pixels)

z (pixels)

15

25

10

15

30

30

0

0

c

0

0

d

convolution

lculations done in Fourrier (frequency) space not XYZ space. f is converted to optical transfer function (only information in X and Z) veral methods that vary in their implementation

Point Spread Function PSF (XYZ space)

Z

Optical Transfer Function OTF (XZ frequency space)
Frequency (X or Y)

X Y X
New methods (Sedat)

Frequency (Z)

•Pupal functions (used to sharpen Hubble telescope) include information in otf in X, Y and Z and phase. •Mapping psf in 3D in specimen by measuring RI using DIC to correct for specimen aberrations (dispersion, reflection, absorption, diffraction) •OMX Structural illumination- much higher resolution / Faster microscope: 3 CCD cameras + 4 computers Pifocal -Piezoelectric focusing on the YXZ stage (objective collar piezofocusing changes the psf and affects DIC). Laser illumination instead of mercury arc lamp -must make non-coherant to avoid speckles in illumination (spining ground glass) •Programmable Array microscope -soon to be commercialized by Cairn. Separates

Laser Scanning Confocal

Widefield Deconvolution

Reassigns out of focus light Removes out of focus light To Using a pinhole in the light path the point of origin Less sensitive -looses light More convenient Image one section More sensitive and quantitative Less convenient -requires lots of Z sections and time consuming calculations on expensive computers Better signal to noise ratio Better for point sources of light

PMT is more noisy than CCD Better for diffuse signal with a lot of out of focus light