Procesamiento de Imágenes

para Biología y Medicina IMA INA

Centro de Obtención y Análisis de Imágenes Biomédicas

Deconvolución

EQUATIONS
 Convolución

f (x)

g(x)

f (x) ⇤ g(x)
 Convolución

f (x)

g(x)

f (x) ⇤ g(x)
 Convolución

f (x)

g(x)

f (x) ⇤ g(x)
 Convolución

f (x)

g(x)

f (x) ⇤ g(x)
 Convolución

f (x)

g(x)

f (x) ⇤ g(x)
 Convolución

newImage("PSF", "8-bit black", 512, 512, 1);

makeOval(236, 236, 40, 40);
setColor(255);
fill();
run("Select None");
run("Gaussian Blur...", "radius=8");
run("Invert");

newImage("Objeto", "8-bit black", 512, 512, 1);

makeRectangle(206, 206, 100, 100);
setColor(255);
fill();
run("Select None");
run("Invert");

run("FD Math...", "image1=Objeto operation=Convolve image2=PSF result=Blurred do");

run("Invert");
 Convolución

PSF Objeto Convolución

132 CHAPTER 15. BLUR & THE P

Convolución
 Deconvolución

http://microscopy.duke.edu/learn/introtomicroscopy/deconvolutionexample.jpg
https://svi.nl/wikiimg/composite.gif

¡No es magia, es Ciencia!

http://sicf.tau.ac.il/imaging/wp-content/uploads/2013/01/decon.jpg
“Any sufficiently advanced technology is
indistinguishable from magic.”

Arthur C. Clark

http://www.biology.wustl.edu/imaging-facility/images/deconbeforeafter.jpg
 Modelo de formación de la imagen
14.1. THE BIG PICTURE OF FLUORESCENCE IMAGING 129 14.1. THE BIG PICTURE OF FLUORESCENCE IMAGING 129

˜ y) = I(x, y) ⇤ h(x, y) + n(x, y)

I(x,
! !

(a) Ideal imaging – a direct view of ‘reality’ (a) Ideal imaging – a direct view of ‘reality’

The universal reflected light vertical illuminator is interposed between the ob-
servation viewing tubes and the nosepiece carrying the objectives. (Figs. I & K)

˜ y) ⇤=!h(x,
Fig. I

˜ y) =! I(x, ˜ y) != I(x, y) ⇤
Universal Reflected Light
Illuminator mounted on an

y) ⇤ h(x, y)
I(x,y)
upright microscope

I(x, The illuminator is designed to direct light onto the specimen by first passing the
!
I(x,
light through the microscope objective on the way toward the specimen and then
using that same objective to capture the light being emitted by the specimen.
(Fig. J)

Fig. J
Fluorescence Reflected
Light (Diagrammatic)

(b) More realistic imaging – a blurred and noisy view (b) More realistic imaging – a blurred and noisy view

Figure 14.2: The di↵erence between what we might wish to image, and what we actually Figure 14.2: The di↵erence between what we might wish to image, and what we actually
PAGE 9 / REFLECTED LIGHT FLUORESCENCE ILLUMINATOR

can image. In both cases (a and b), the ‘real’ scene is shown on the left. Ideally, the can image. In both cases (a and b), the ‘real’ scene is shown on the left. Ideally, the
small coloured spots in reality would directly map to coloured spots of a related size and small coloured spots in reality would directly map to coloured spots of a related size and
separation in the image (a). However, the light emitted from these spots would actually separation in the image (a). However, the light emitted from these spots would actually
end up producing larger objects in the image, which can then blur together (b, centre). end up producing larger objects in the image, which can then blur together (b, centre).
Noise is further added to this blurriness to give us the best image we can really record (b, Noise is further added to this blurriness to give us the best image we can really record (b,
right). right).

these quantifications pixel values are determined. A larger charge indicates these quantifications pixel values are determined. A larger charge indicates
more photons, which translates into a higher pixel value. more photons, which translates into a higher pixel value.
˜ y) = I(x, y) ⇤ h(x, y) + n(x, y)
I(x,
14.1.2 Errors and imprecisions 14.1.2 Errors and imprecisions
From the above summary, it is clear that we are quite some distance away from From the above summary, it is clear that we are quite some distance away from
knowing exactly how much light is emitted from the specimen: most photons do knowing exactly how much light is emitted from the specimen: most photons do
not reach the detector, and many that do are still not registered. But since we not reach the detector, and many that do are still not registered. But since we
can expect to always lose track of a similar proportion of emitted light – perhaps can expect to always lose track of a similar proportion of emitted light – perhaps
90% – this does not matter much: we can expect all parts of the image to be 90% – this does not matter much: we can expect all parts of the image to be
similarly a↵ected, so relative di↵erences in brightness would still be reflected in similarly a↵ected, so relative di↵erences in brightness would still be reflected in
our pixel values. However, there are two more critical ways in which the images our pixel values. However, there are two more critical ways in which the images
we can record are less good than the images we would like: we can record are less good than the images we would like:

1. Uncertainty in space. Ideally, all light originating from a particular point 1. Uncertainty in space. Ideally, all light originating from a particular point
in the specimen would strike the detector in exactly the same place, and in the specimen would strike the detector in exactly the same place, and
 Deconvolución

˜ y) = I(x, y) ⇤ h(x, y) + n(x, y)

I(x,

˜ = y)
I(x,
y) =y)
I(x, I(x, y) ⇤y)
⇤ h(x, h(x, y) ˆ y)
I(x,
˜ y) = I(x, y) ⇤ h(x, y)
I(x,

˜ y) = I(x, y) ⇤ h(x, y) + n(x, y)

I(x,
 ¿Cómo se implementa la deconvolución?

• Dos grandes familias de algoritmos (conociendo la PSF)

• Filtrado (en Fourier)

• Cálculo de un filtro inverso para recuperar la mejor imagen bajo ciertas

condiciones.

• Iterativos

• Secuencia de pasos repetitivos que (en ciertas condiciones) se aproximan a

una buena restauración.

• Más costosos computacionalmente, pero mejores (PSF variable en la

imagen).

• Pueden incluir restricciones (no-negatividad)

• Regularización para manejar y no sobreajustarse al ruido.

• Otro enfoque de algoritmos recuperan la imagen y la PSF (desconocida)

simultáneamente (blind deconvolution).

• PSF

• Teórica (sin ruido)

• Experimental (a partir de puntos en la imagen)

 Filtrado inverso

˜ y) = I(x, y) ⇤ h(x, y) + n(x, y)

I(x,

˜ = y)
I(x,
y) =y)
I(x, I(x, y) ⇤y)
⇤ h(x, h(x, y) g(x, y) ˆ y)
I(x,
˜ y) = I(x, y) ⇤ h(x, y)
I(x,

˜ y) = I(x, y) ⇤ h(x, y) + n(x, y)

I(x,

ˆ y) = I(x,
I(x, ˜ y) ⇤ g(x, y)
 Filtrado inverso

˜ y) = I(x, y) ⇤ h(x, y) + n(x, y)

I(x,

˜ = y)
I(x,
y) =y)
I(x, I(x, y) ⇤y)
⇤ h(x, h(x, y) g(x, y) ˆ y)
I(x,
˜ y) = I(x, y) ⇤ h(x, y)
I(x,

ˆ y) = I(x, y) ⇤ h(x, y) ⇤ g(x, y) + n(x, y) ⇤ g(x, y)

I(x,
Filtrado de la imagen …que también actúa
no visible… sobre el ruido.

Teorema de Convolución
en Fourier
g(r) ⇤ h(r) , G(f )H(f )
 Filtrado inverso

Noise power

H(f )
 Filtrado inverso

1
Inverse filter H(f )

Noise power

H(f )
 Filtrado inverso

1
Inverse filter H(f )

lter
in g fi
th
oo
Sm
Noise power

H(f )
 Filtrado inverso

1
Inverse filter H(f )

Wiener filter

lter
in g fi
th
oo
Sm
Noise power

H(f )
 Filtrado de Wiener

I˜ = h ⇤ I
Debería ser cero
(no lo es por el ruido). I˜ h ⇤ I
Diferencia (error) en la
I˜ h ⇤ Iˆ aproximación de I con Iˆ.

˜
I(r) ˆ
h(r) ⇤ I(r)
X⇣ ⌘2
Error cuadrático
ˆ⇤
I (r) = arg min ˜
I(r) ˆ
h(r) ⇤ I(r) medio.
Iˆ r
X⇣ ⌘2
ˆ⇤
I (r) = arg min ˜
I(r) ˆ
h(r) ⇤ I(r) Buscamos el
mínimo error.
Iˆ r
X⇣ ⌘2
ˆ⇤
I (r) = arg min ˜
I(r) ˆ
h(r) ⇤ I(r)
Iˆ r
 Filtrado de Wiener

X
ˆ⇤
I = arg min (h ⇤ Iˆ ˜2
I)
• Wiener deconvolution filter
Iˆ r
• Hallar el mejor filtro G(f ) que minimiza el error cuadrático medio

0 1 “Espectro” del ruido. !

1 @ |H(f )|2 A= 1 |H(f )| 2
G(f ) = 1
H(f ) |H(f )|2 + N (f ) H(f ) |H(f )|2 + SNR(f )
S(f )
Transformada de “Espectro” de la señal.
Fourier del filtro de Transformada de
deconvolución. Fourier de la PSF.
• Si no hay ruido N (f ) = 0
1
G(f ) =
H(f )

ˆ ) = I(f )H(f )G(f ) = I(f )H(f ) 1

I(f = I(f )
H(f )

En el caso que no hay ruido se recupera la imagen original.

 Deconvolución iterativa

Convolución
Iˆ(0) (r) Condición inicial Nueva imagen

estimada con PSF

PSF h(r)

ˆ
I(r)
Restricciones
Calcular
˜
en la imagen corrección
Calcular el error Imagen obtenida I(r)
Iˆ(t+1) > 0

Iˆ(t+1) = Iˆ(t) + (I˜ Iˆ(t) ⇤ h)

I˜
Iˆ(t+1) = Iˆ(t)
Iˆ(t) ⇤ h

[Deconvolution Microscopy (David Agard) iBiology.org]

 Algoritmo de Richardson-Lucy

X
• Richardson-Lucy I˜i = hij Ij Valores reales
(Asume Poisson).
j
Valores
observados. PSF

• Procedimiento iterativo (EM), basado en un enfoque probabilístico;

asumiendo distribución Poisson para I

P (I|h ⇤ I)P (h ⇤ I)
P (h ⇤ I|I) =
P (I)

• Iteración:

! !
(t+1) (t)
X I˜i hij I˜
ˆ
Ij = ˆ
Ij P ˆ(t+1)
I ˆ(t)
=I ⇤ h̄
i hik Iˆ
k
(t)
k
Iˆ(t) ⇤ h

Va aproximándose
a la unidad con las
iteraciones
(si converge).
 Blind deconvolution

Condición

inicial

Nueva imagen Convolución PSF Nueva PSF

Convolución
estimada con PSF estimada con PSF

Calcular
Calcular

Calcular el error Calcular el error

corrección corrección

Restricciones

en la imagen Iteración en la imagen Restricciones

Iteración en la PSF
en PSF

Imagen obtenida

• Son necesarias buenas condiciones iniciales.

 Regularización

X
ˆ⇤
I (r) = arg min ˆ
(h(r) ⇤ I(r) ˜
I(r)) 2 ˆ
+ ↵R(I(r))
Iˆ r

ER-Decon: Entropy Regularized Deconvolution Regularización:

Incluido en el paquete de software IVE/Priism define la forma de
http://www.msg.ucsf.edu/IVE/ manejar el ruido.

Raw ER-Decon Huygens Deconv.Lab Raw ER-Decon Huygens Deconv.Lab

Exposure 100%

Exposure 3%
Exposure 1.5%
Exposure 33%

Yeast vacuole. Scale bar: 2µm.

[Arigovindan, 2013]
 Software para deconvolución y PSF

• DeconvolutionLab by Biomedical Imaging Group, EPFL (http://bigwww.epfl.ch/algorithms/deconvolutionlab/)

• Parallel Iterative Deconvolution by Piotr Wendykier (http://imagej.net/Parallel_Iterative_Deconvolution)
• Parallel Spectral Deconvolution by Piotr Wendykier (imagej.net/Parallel_Spectral_Deconvolution)
• Iterative Deconvolve 3D by Bob Dougherty (http://www.optinav.com/Iterative-Deconvolve-3D.htm)
• FDMath (¿Robert Dougherty?)
• Colour Deconvolution by Gabriel Landini (http://www.mecourse.com/landinig/software/cdeconv/cdeconv.html)
• PSF Tool by the ETH Computational Biophysics Lab (http://www.mosaic.ethz.ch/Downloads/psftool)
• Diffraction PSF 3D by Robert Dougherty (http://imagej.net/Diffraction_PSF_3D)
• PSF Generator by the Biomedical Imaging Group, EPFL (http://bigwww.epfl.ch/algorithms/psfgenerator/)

Para obtener la PSF de forma teórica, es necesario conocer

varios parámetros de la adquisición (NA, índice refracción,
longitud de onda, entre otros).

• MitivDeconvolution by CRAL (http://icy.bioimageanalysis.org/plugin/MitivDeconvolution)

• MitivBlindDeconvolution (https://mitiv.univ-lyon1.fr/)
• PSF Generator by Praveen Pankajakshan (http://icy.bioimageanalysis.org/plugin/
Widefield_Fluorescence_Microscope_PSF)

• Huygens by Scientific Volume Imaging (http://svi.nl/)

• Microscopy Nyquist rate and PSF calculator (https://svi.nl/NyquistCalculator)
¿Cómo obtener la PSF?
 DeconvolutionLab

Synthetic hollow bars

These synthetic data consist of six parallel hollow bars. The

images have been blurred using a theoretical microscopic
PSF, and corrupted by Gaussian noise and Poisson noise
with several signal to noise ratios (SNR = 15, 30 dB). The
knowledge of the ground-truth allows one to quantatively
compare the deconvolution tools.

http://bigwww.epfl.ch/deconvolution/?p=bars
DeconvolutionLab

Deconvolución con 10 iteraciones de

Datos sintéticos originales.
Richardson-Lucy.
 DeconvolutionLab

Richardson-Lucy Richardson-Lucy Richardson-Lucy

“Imagen” obtenida
Iteración 1 Iteración 6 Iteración 10
 DeconvolutionLab

Fluorescent bead

This test volume contains a fluorescent

bead of known dimension—its diameter is
precisely 2.5 μm. These data have the
advantage of offering a simple object on
which it is easy to perform a quantitative
validation of the recovering of shape and
dimension, before and after deconvolution.

http://bigwww.epfl.ch/deconvolution/?p=bead
 DeconvolutionLab
C. elegans embryo
This real dataset is composed of three stacks of images of a C.
Elegans embryo. The deconvolution effects can be evaluated on
different kinds of structures: extended objects (the chromosomes in
the nuclei), filaments (the microtubules), and point-wise spots (a
protein stained with CY3).

Microscope Acquisition: Olympus CellR

• Objective: UPlanSApo 100X/1.4 oil(n.i. 1.518); image pixel size

(binning 1X1): 64.5 nm

• Laser intensity: 100% - Camera pixel size: 6450 nm - Gain: 0

• z step: 200 nm (experiment 10, 1344x1024x111; crop

672x712x104)

• Channel 1: DAPI, lamda excitation: 377/50 nm, lamda emission:

447/60 nm; Channel 2: FITC, lamda excitation: 485/20, lamda
emission: 531/22; Channel 3: CY3,lamda excitation: 560/25,lamda
emission: 634/40 Composite original

Composite luego de 10 iteraciones

de Richardson-Lucy

http://bigwww.epfl.ch/deconvolution/?p=bio
 Parallel Iterative Deconvolution

MRNSD

WPL 5 WPL 30 CGLS HyBR

Parallel Iterative Deconvolution
Icy: MitivBlindDeconvolution
Icy: MitivBlindDeconvolution
 References

• DeconvolutionLab: http://bigwww.epfl.ch/algorithms/deconvolutionlab/

• Deconvolution http://imagej.net/Deconvolution

• Icy: an open community platform for bioimage informatics: http://

icy.bioimageanalysis.org/

• Microscopy: Deconvolution Microscopy (David Agard) iBioEducation

(YouTube)

• Fidorra, M., Garcia, A., Ipsen, J. H., Härtel, S., & Bagatolli, L. A. (2009).
Lipid domains in giant unilamellar vesicles and their correspondence with
equilibrium thermodynamic phases: A quantitative fluorescence
microscopy imaging approach. Biochimica et Biophysica Acta -
Biomembranes, 1788(10), 2142–2149. doi:10.1016/j.bbamem.2009.08.006

• Arigovindan, M., Fung, J. C., Elnatan, D., Mennella, V., Chan, Y. M.,
Pollard, M., … Agard, D. a. (2013). High-resolution restoration of 3D
structures from widefield images with extreme low signal-to-noise-ratio.
Proceedings of the National Academy of Sciences of the United States of
America, 110(43), 17344–17349. doi:10.1073/pnas.1315675110