Dissertation UnalSakoglu

Signal-processing strategies for spectral
tuning and chromatic nonuniformity

correction for quantum-dot IR sensors
by
Unal Sako glu

B.S., EEE, Bilkent University, Ankara, Turkey, 2000
M.S., Electrical and Computer Engineering,
University of New Mexico, 2002
DISSERTATION
Submitted in Partial Fulllment of the
Requirements for the Degree of
Doctor of Philosophy
Engineering
The University of New Mexico
Albuquerque, New Mexico
July, 2006
c _2006,

Unal Sako glu
iii
Dedication
Canm annecigim, senin icin. Bu doktora aslnda senin eserin.
1
1
Beyhan (
Ozer) Sakoglu
iv
v
Acknowledgments
I would like to start noting that this dissertation would not be written at all if I
havent had my dear wife Dalin always beside me. I would like to thank her for
being really patient, and for being the main reason to stay and nish my PhD in
Albuquerque; birtanem you are the best thing that has happened to me. I would like
to thank my kaynco Ali Alper and my other kayncos, sisters-in-law, mother-in-law
and father-in-law, who are my family in Albuquerque, for the support and warmth
they have provided.
I am grateful to my mom, dad and my only brother Cihan, who live in beautiful
T urkiye, but they have always been here with me in my heart. Since my early
childhood, my mom had an extraordinary care about my education; she thought me
how to read and write when I was just four; therefore, she was my rst teacher. She
frequently communicated with my teachers and checked on me until I started college
and nally got away from home. Annecigim, senin emeklerin sayesinde buradaym.
I would like to thank my main adviser Professor Majeed M. Hayat for his contin-
uous support and patience throughout my PhD studies. I would also my co-advisors
professors J. Scott Tyo and Sanjay Krishna for their valuable advice and discussions
on my research. I would like to extend my gratitude to Dr. Phil Dowd, Dr. Amtout
Abdenour, Kalyan T. Posani, Senthil Annamalai and Zhipeng Wang for valuable
help during my experiments at the UNM Center for High Technology Materials. I
would like to express my appreciation to my committee members, Professors Balu
Santhanam and Sudhakar Prasad, for serving on my committee. I have taken two of
my important core courses from Professor Santhanam, who is an excellent teacher.
I would also extend special thanks to Professor Russell Hardie at the University of
Dayton for his idea of the matrix-based solution to the nonuniformity correction
problem. Finally, I thank Dr. Brad Ratli for providing valuable help throughout
this dissertation.
I would like to thank professors Edl Schamiloglu and Chaouki Abdallah, for
encouraging me to pursue PhD study after my masters. I would also like to thank
vi
Professor Anna Scaglione for her supervision during the early stages of my graduate
studies. I would like to thank our departments always-smiling graduate coordinator
Maryellen Missik Tow, as well as Roger Ragoonanan and other sta for their kindness
and help. I would like to thank Dr. Nader Vadiee and Professor Andy Salazar, for
their help during my teaching assistantship.
I would like to thank

Omer Dedeoglu (sadk dostum), Mehmet Fatih Su (baba),
Serhat Altunc, Ali Oguz, T unay Oguz,

Omer F. G uvener, muhtar Ahmet Tiryaki,
and Erdogan Altunok, who were my good friends in Albuquerque.
I would also like to thank my professors at Bilkent University, especially professors
Enis C etin, Billur Barshan and B ulent

Ozg uler for encouraging me to pursue graduate
degrees.
I would like to thank the great people who inspire me, most of whom are in the
eternity, but their voices still echo through the ages.
Last but not least, I would like to thank God for giving me the mind and health.
I hope the work done in this dissertation can contribute even a little bit to the lives
of the people in the world.
This work was supported by the National Science Foundation under awards IIS-
0434102 and ECS-0401154. It was also supported by the National Reconnaissance
Organization under a Directors Innovative Initiative Award.
vii
Ilim ilim bilmektir,
Ilim kendin bilmektir,

Sen kendini bilmezsen,
Ya nice okumaktr?
(Knowledge is to understand,
To understand who you are,
If you know not who you are,
What is the use of learning?)
-Yunus Emre, 13th century humanist mystic poet from Anatolia.
viii
by
Unal Sako glu

ABSTRACT OF DISSERTATION
Submitted in Partial Fulllment of the
Requirements for the Degree of
Doctor of Philosophy
Engineering
The University of New Mexico
Albuquerque, New Mexico
July, 2006
by
Unal Sakoglu
B.S., EEE, Bilkent University, Ankara, Turkey, 2000
M.S., Electrical and Computer Engineering,
University of New Mexico, 2002
Ph.D., Engineering, University of New Mexico, 2006
Abstract
Quantum-dot infrared photodetectors (QDIPs) are emerging mid-IR sensors which
exhibit bias-dependent change in their spectral response. They can be viewed as dif-
ferent detectors, or dierent spectrally overlapping bands of a sensor, under dierent
applied bias voltages. Widths of their bands, however, are wider than 2m, and they
do not provide spectral resolution comparable to those of common hyperspectral or
multispectral sensors. In this work, we have developed a signal-processingbased
spectral-tuning (ST) algorithm, which exploits the spectral overlap in the bands,
corresponding to dierent bias voltages of the same detector, in order to simulate
the outputs of arbitrary desired bands with higher spectral resolution and arbitrary
shape. This is done by linearly combining the detector photocurrents at dierent
x
bias voltages, with positive and negative weights, which are calculated to provide
the minimum-mean-square-error approximation of the output of the desired arbi-
trary band. We have shown that with certain limits on the resolution and range of
the center wavelength, which are dictated by the diversity of the detectors respon-
sivities, continuous tuning of the detector, with higher resolution than those of the
individual detectors, can be achieved. We have applied this algorithm to bias-tunable
QDIPs that have been developed by our group at the Center for High Technology
Materials at the University of New Mexico. Our approach has provided continuous
spectral tuning between 3 8m, and 8 12m, with resolution up to 0.5m. It
is further shown that photocurrent noise degrades the tuning capability of the ST
algorithm, and considering the noise is critically important for the successful oper-
ation of the ST algorithm. In particular, the interplay between signal-to-noise ratio
of the QDIPs photocurrent and the accuracy of tuning is thoroughly investigated.
We also considered an array conguration of QDIPs for the infrared spectral
imaging. It is well-known that pixel-to-pixel dissimilarities in the photodetectors re-
sponses result in nonuniformity (NU) noise, which appears as an undesirable spatial
pattern on the image. Moreover, NU also varies as the detector/sensor is tuned to
dierent regions of the spectrum, resulting in a phenomenon often termed chromatic
NU. Thus, the successful and practical operation of tunable QDIP arrays require a
means for mitigating chromatic NU noise. Recently, nonuniformity correction (NUC)
algorithms that mitigate NU noise by using scene-based, signal-processing techniques
have been developed; however, very little work has been reported on mitigating chro-
matic NU. In this dissertation, a recently reported and credible scene-based NUC
algorithm that is based on discriminating between NU noise and scene by detecting
and exploiting global scene motion has been reformulated in a matrix-based frame-
work. This approach addresses both oset NU as well as gain NU. The proposed
NUC technique is applied to real mid-infrared imagery. The matrix-based framework
allows one to use versatile matrix-algebra techniques such as regularization and ma-
xi
trix sparsity. It also gives greater control over the mitigation of the NUC problem
through selection of algorithm parameters. It is shown that the approach of the ST
algorithm can be eectively adopted to perform chromatic NUC that is adaptive
to algorithmic spectral-tuning process. In particular, the ST algorithm provides a
recipe for combining NUC maps, corresponding to QDIP array operated at each bias
voltage, to generate a chromatic NUC map customized for specic tuning require-
ments. Despite the focus on QDIP sensors, the work reported in this dissertation is
general and it provides a framework for using a collection of sensors that have over-
lapping bands and dierent noise characteristics to synthesize outputs corresponding
to a desired arbitrary band.
xii
Contents
List of Figures xviii
List of Tables xxv
Glossary xxvi
1 Introduction 1
1.1 Background and motivation . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Rationale and overview for the algorithmic tuning approach . 5
1.1.2 Rationale and overview nonuniformity correction for spectrally
tunable QDIP arrays . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 Organization of the dissertation . . . . . . . . . . . . . . . . . . . . . 11
1.3 Publications resulting from the dissertation . . . . . . . . . . . . . . . 12
2 Quantum-Dot Infrared Photodetectors 14
2.1 Infrared detectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 Quantum-dot-based structures . . . . . . . . . . . . . . . . . . . . . . 15
xiii
Contents
3 Spectral-Tuning Algorithm 20
3.1 Spectral-tuning algorithm . . . . . . . . . . . . . . . . . . . . . . . . 20
3.1.1 Algorithm development . . . . . . . . . . . . . . . . . . . . . . 21
3.1.2 Regularized version of the algorithm . . . . . . . . . . . . . . 24
3.2 Application of ST algorithm to QDIPs . . . . . . . . . . . . . . . . . 26
3.3 Noise-modied spectral-tuning algorithm . . . . . . . . . . . . . . . . 32
3.3.1 Photocurrent noise model . . . . . . . . . . . . . . . . . . . . 32
3.4 Application of noise-modied ST algorithm to QDIPs . . . . . . . . . 38
3.4.1 Experiment and measurement process . . . . . . . . . . . . . . 39
3.4.2 Experiments with simulated noise . . . . . . . . . . . . . . . . 41
3.5 Multispectral-based target classication . . . . . . . . . . . . . . . . . 45
3.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4 Chromatic Nonuniformity Correction for Detectors 63
4.1 Nonuniformity model . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.2 Chromatic nonuniformity and the ST algorithm . . . . . . . . . . . . 66
4.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.3.1 Application to simulated imagery . . . . . . . . . . . . . . . . 70
4.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5 Matrix-based Nonuniformity Correction Algorithm 75
xiv
Contents
5.1 One-point oset correction model . . . . . . . . . . . . . . . . . . . . 75
5.1.1 Modeling the two-dimensional global shift . . . . . . . . . . . 77
5.1.2 Matrix equation model for the non-radiometric case . . . . . . 79
5.1.3 Solution to the matrix equation . . . . . . . . . . . . . . . . . 83
5.2 Two-point gain and oset correction model . . . . . . . . . . . . . . . 87
5.3 Applications of the NUC algorithm . . . . . . . . . . . . . . . . . . . 87
5.3.1 One-point NUC: Application to simulated images . . . . . . . 88
5.3.2 One-point NUC: Application to real images . . . . . . . . . . 89
5.3.3 Two-point NUC: Application to simulated images . . . . . . . 90
5.3.4 Two-point NUC: Application to actual images . . . . . . . . . 92
5.4 Computational issues: Sparse matrices . . . . . . . . . . . . . . . . . 94
5.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6 Conclusions 108
7 Future Work 111
7.1 ST algorithm: Application on dierent QDIPs . . . . . . . . . . . . . 111
7.2 ST Algorithm: Selection of the best detectors in a given set . . . . 112
7.3 NUC algorithm: Dependence on accuracy of shift estimation . . . . . 113
7.4 NUC algorithm: Memory issues and sparse matrices . . . . . . . . . . 113
7.5 Chromatic NUC algorithm: Application on real QDIP imagery . . . . 114
xv
Contents
Appendices 117
A Poisson Noise Model 118
References 121
xvi
List of Figures
1.1 Bias-dependent spectral response of a QDIP, namely QDIP 1999,
produced in CHTM. Twenty one dierent response corresponding to
various bias voltages from the set 1, 0.9, ..., 0, ...0.9, 1V . . . . . 3
1.2 Schematic of the projection algorithm for the spectral tuning of QDIPs. 6
1.3 Left: Spectral-tuning of desired triangular lter with FWHM of
0.5m, between 3m10.5m, by using bias-tunable QDIP 1999, de-
veloped at CHTM. Right: A reconstruction of 3mm polystyrene lter
spectrum with the QDIP. . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1 Peak-normalized spectral response of a quantum-dot detector at 17
dierent applied bias voltages. . . . . . . . . . . . . . . . . . . . . . 18
2.2 Schematic of the 15-layer asymmetric InAs/In
0.15
Ga
0.85
As DWELL
detector structure (# 554, CHTM) sandwiched between two highly
doped n-GaAs contact layers, grown on a semi-insulating GaAs sub-
strate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1 Bias-dependent spectral response of (a)QDIP 1198 (b)QDIP 1199,
for 21 dierent bias voltages. The devices are manufactured at the
CHTM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
xvii
List of Figures
3.2 Examples of three (narrow, medium and coarse) approximated (nor-
malized) responsivity by the ST algorithm. QDIP 1199 spectra are
used as basis spectra. . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.3 Attained post-processing spectral resolution (FWHM) as a function
of desired center wavelength (tuning parameter), and spectral band-
width (linewidth-control parameter) for (a)QDIP 1198 (b)QDIP 1199. 50
3.4 Approximation of a desired triangular responsivity with FWHM of
0.5m as a function of desired center frequency for the two QDIPs.
A post-processing FWHM< 1.0m can be achieved for most of the
tuning parameter values especially for the QDIP 1199. . . . . . . . . 51
3.5 Spectral tuning results: Reconstruction of unltered blackbody source
spectrum by using approximated triangular bands with (a) FWHM
= 0.5m, (b) FWHM = 0.25m. . . . . . . . . . . . . . . . . . . . . 52
3.6 Spectral tuning results: Reconstruction of 3mm polystyrene-ltered
scene spectrum by using approximated triangular bands with (a)
FWHM = 0.5m, (b) FWHM = 0.25m. . . . . . . . . . . . . . . . 53
3.7 (a) Example lter approximation: desired triangular responsivity (l-
ter) with a desired FWHM of 0.5m and with centers of (a)4.0m (a
bad approximation) and (b)7.65m (a good approximation). . . . . 54
3.8 (a) Example lter approximation: desired triangular responsivity (l-
ter) with a desired FWHM of 0.25m and with centers of (a)4.0m
(a bad approximation) and (b)7.65m (a good approximation). . . . 55
3.9 Flowchart of the entire noise-modied spectral-tuning (NMST) algo-
rithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
xviii
List of Figures
3.10 (a) Peak-normalized spectral response of the QDIP detector in Fig. 2.2
for dierent bias-voltages. (b) Example desired triangular lter ap-
proximation with the center 9.5m and full-width at half-maximum
(FWHM) of 0.5m. . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.11 Total and dark current proles of the DWELL detector with 300
m-diameter at 50 K. The and 2 symbols represent the total
current with and without the 3mm polystyrene lter, respectively.
The represents the dark current at 50 K. Inset: Noise standard
deviation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.12 Performance of the algorithm in synthesizing the relative power spec-
trum of (a) black-body source (b) 3-mm polystyrene lter, by using
desired responsivity of ideal triangular lters of FWHM of 0.5m,
under moderately low noise (with SNR of 100). The and +
symbols represent the original tuning algorithm that does not ac-
commodate the noise and the new noise-modied tuning algorithm,
respectively. The symbols s represent the reconstruction by us-
ing ideal responsivity. Under very low noise (for SNR of more than
2000), the ideal reconstruction () and algorithms reconstruction
(+,) overlap. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.13 Same as Fig.3.12 but with a wider desired triangular responsivity
with FWHM of 1.5 m. . . . . . . . . . . . . . . . . . . . . . . . . . 60
xix
List of Figures
3.14 Comparison of the original algorithm that does not accommodate
noise (thin curves) and the noise-modied algorithm (thick curves);
average normalized root-mean-square error (NRMSE) vs. the pho-
tocurrent SNR, for desired resolution of (a) narrow FWHMs of 0.5m
(dashed curves) and 1m (solid curves) (b) wide FWHMs of 1.5m
(dashed curves) and 2m (solid curves). . . . . . . . . . . . . . . . . 61
4.1 (a)Response of dierent detectors to incident temperature, (b)Two-
point (linear) approximation of the response which characterizes the
response with gain and oset. . . . . . . . . . . . . . . . . . . . . . . 65
4.2 (a) Image frame corrupted with NU. (b) Image frame after two-point
(gain and oset) hard calibration is applied. . . . . . . . . . . . . . . 66
4.3 Normalized root-mean-square error (NRMSE) versus the temper-
ature for (a) high SNR (b) low SNR. The solid curve represents
two-point calibration and the dashed curve represents the two-point
CNUC by using spectral-tuning algorithm. . . . . . . . . . . . . . . 73
xx
List of Figures
4.4 Eect of spectral radiance on NUC. (a) Simulated image without
NU. The perimeter temperature level is simulated to be 273 K, back-
ground 303 K, numbers 1-6 represent 273, 283, 293, 313, 323, 333
K, respectively. The average level of temperature is 296 K for the
image. (b) Image with simulated NU, plus temporal noise. The gain
and oset NU both have standard deviation 10% (of mean values)
and the additive temporal noise corresponds to an SNR of 100. Image
after (c) two-point NUC, (d) combining two-point calibration maps
of detectors with ST algorithm weights, (e) combining matrix-based
NUC algorithm maps of detectors with ST algorithm weights. The
temperatures used for calibration were 293 and 313 K. (f) NUC re-
sult when the source spectral radiance is dierent than assumed. (g)
Image in (f) scaled to its full dynamic range. . . . . . . . . . . . . . 74
5.1 (a) First frame. (b) Second frame, which is a shifted version of the
rst frame resulting from 1.55-right, 0.70-downward camera motion.
(c) First frame with bias non-uniformity added. The standard devi-
ation of the oset NU is 4. (d) Estimation of the rst frame with
radiometric calibration applied to the perimeter pixels. . . . . . . . . 97
5.2 Non-radiometric estimation for the same shift values and same level
of oset NU as in Fig. 5.1 for: (a) = 10 (b) = 2 (c) = 0.5 (d)
= 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.3 (a) First frame. (b) Second frame, shifted as a result of 0.365-right,
0.276-downward camera motion. (c) Correction of the rst frame
using a two-point calibration. (d) Correction using the algorithm in
the non-radiometric mode. (e) Correction using the algorithm in the
radiometric mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
xxi
List of Figures
5.4 Simulated radiometric spatial two-point (gain and oset) NUC ex-
ample: (a) 1st frame, true irradiance; (b)2nd frame, simulates a
super-pixel-shifted version of the rst frame scene, obtained by bi-
linear shift model, with shift (,)=(-1.3,+2.35); (c,d) NU-degraded
1st and 2nd frames, with oset-NU std. of 20 and gain-NU std. of
10%; (e) 1st frame after two-point NUC; (f) error in NUC. . . . . . 100
5.5 Simulated non-radiometric spatial two-point (gain and oset) NUC
example: (a) 1st frame, true irradiance; (b)2nd frame, simulates a
super-pixel shifted version of the rst frame scene, obtained by bi-
linear shift model, with shift (,)=(+2.25,-1.25); (c,d) NU-degraded
1st and 2nd frames, with oset-NU-std. of 20 and gain-NU-std. of
10%; (e) 1st frame after two-point NUC; (f) error in NUC. . . . . . 101
5.6 Simulated non-radiometric spatial two-point (gain and oset) NUC
example, sub-pixel shift case: (a) 1st frame, true irradiance; (b)2nd
frame, simulates a sub-pixel-shifted version of the rst frame scene,
obtained by bi-linear shift model, with shift (,)=(+0.914,+0.569);
(c,d) NU-degraded 1st and 2nd frames, with oset-NU std. of 20 and
gain-NU std. of 10%; (e) 1st frame after two-point NUC; (f) error in
NUC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.7 (a) Simulated non-radiometric spatial two-point (gain and oset)
NUC example, after averaging over 10 frame-pairs; (b) error in NUC
averaged over 10 frame-pairs. The statistics of the NU was the same
as in previous gures. Oset-NU std. is 20 and gain-NU std. is 10%. 103
5.8 Root-mean-square error(RMSE) in the non-radiometric NUC aver-
aged over many frame-pairs versus number of frame-pairs. Oset-NU
std. is 20 and gain-NU std. is 10%. . . . . . . . . . . . . . . . . . . 104
xxii
List of Figures
5.9 (a,b) Frames 392 and 393 of a 512-frame sequence obtained by InSb
camera, as observed. (c,d) Frames after two-point calibration. . . . . 105
5.10 Radiometric NUC application results, in which the parameters of the
perimeter pixels (the top row and the rightmost two columns) are
assumed to be known. (a) Frame 512 from the same sequence used
in Fig. 5.9. (b) Frame after two-point calibration. (c) One-point
(oset) and (d) two-point (oset and gain) NUC results. . . . . . . . 106
5.11 Non-radiometric NUC application results, in which no information
about the gain and oset parameters of the pixels is assumed to be
known. (a) Frame 512 from the same sequence used in Fig. 5.9. (b)
Frame after two-point calibration. (c) One-point (oset) and (d)
two-point (oset and gain) NUC results. . . . . . . . . . . . . . . . . 107
7.1 (a) Approximation of ideal triangular lter with FWHM of 1m cen-
tered at 8.5 and 10.5m. (b) Approximation at the center 9.5m
but with fwhm of 0.5, 1.5 and 3.5m. Basis spectra used for the
approximation belongs to the QDIP device #1780 of CHTM. . . . . 116
xxiii
List of Tables
3.1 Parameters used for the three simulated sensing modes of the ST
algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2 Seven-band multispectral reconstruction (normalized) . . . . . . . . 30
3.3 Three-band multispectral reconstruction (normalized) . . . . . . . . 31
3.4 Five-band multispectral performance of the algorithm in synthesizing
the relative power spectrum of (a) black-body source and (b) 3-mm
polystyrene lter in the case of moderate noise (SNR = 20) using a
synthesized triangular lter with FWHM=1.0 m. . . . . . . . . . . 62
5.1 List of pre- and post-operations for dierent directions of shift be-
tween the frames. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
xxiv
Glossary
1-D one-dimensional
2-D two-dimensional
CNU chromatic nonuniformity
CNUC chromatic nonuniformity correction
DWELL dots-in-a-well
D
detectivity
FPA focal plane array
FPN xed-pattern noise
FTIR Fourier-transform infrared spectrometer
FWHM full-width at half-maximum
GaAs Gallium Arsenide
HgCdTe Mercury Cadmium Telluride
InGaAs Indium Gallium Arsenide
InAs Indium Arsenide
xxv
Glossary
InSb Indium Antimonide
IR infrared
MMSE minimum mean-square error
MWIR mid-wave infrared
NMST noise-modied spectral-tuning
NRMSE normalized root-mean-square error
NU nonuniformity
NUC nonuniformity correction
QD quantum-dot
QDIP quantum-dot infrared photodetector
QWIP quantum-well infrared photodetector
SNR signal-to-noise ratio
ST spectral-tuning
SWIR short-wave infrared
VNIR very-near infrared
xxvi
Chapter 1
Introduction
1.1 Background and motivation
The goal of a sensor system is to provide outputs so that information can be de-
duced about certain properties of the target to be sensed. For infrared (IR) sensing
in particular, the properties of the targets to be sensed are usually their tempera-
tures and/or their spectral signatures based on the IR radiation they emit and/or
reect. Obtaining spectral characteristics of targets requires sensing in dierent
spectral bands by using dierent detectors, dierent lters, and/or optical systems
that provide separation of the signal based on its spectral content. Usually, the
goals of the application dictate the number, location, and the spectral resolution
of the bands. For some applications, several bands, with coarse spectral resolution
( 0.5m) may be sucient. These applications fall in the category of multispec-
tral sensing/imaging. For other applications, the number of bands needed can be
on the order of hundreds, with far ner spectral resolution (10-100nm), and these
applications fall in the category of hyperspectral sensing/imaging. For hyperspectral
sensors, the desired spectral resolution is very high, as narrow as nanometers.
1
Chapter 1. Introduction
Modern hyperspectral-sensor systems, such as AVIRIS or HYDICE, are capable
of sensing the spectrum of a source in upwards of 200 bands of width 10-20nm [1, 2, 3]
in the very-near infrared (VNIR) and short-wave infrared (SWIR) regime. These sys-
tems are sophisticated and costly; they either use a dispersive optic element to spread
the spectral content for each pixel to multiple line arrays [3] or use interferometric
techniques that involve computation of interference pattern and Fourier transform
to obtain the spectral content [4, 5, 6, 7, 8]. In these systems, the spectrum or the
interferogram, corresponding to each pixel, is captured on a single column of the
array, leaving only one dimension of the array for gathering the spatial information.
Multispectral systems, such as the Multispectral Thermal Imager (MTI) [9], are
dierent from hyperspectral systems in that they use much fewer but wider bands,
which results in reduced spectral resolution. However, multispectral-sensing systems
are simpler and easier to realize than hyperspectral-sensing systems, as they require
far less sophisticated optical front ends.
A key characteristic common to existing hyperspectral and multispectral sensors
is that they cannot oer spectral resolution below the resolution of their bands, which
is a consequence of the spectrally nonoverlapping nature of the bands. On the other
hand, multispectral detectors often have much wider responsivities than the desired
resolution. Nonetheless, one can extract higher spectral resolution information from
multiple detectors provided that the detectors exhibit spectrally overlapping respon-
sivities [10]. Indeed, the use of spectral overlap in detectors in order to extract higher
spectral resolution information is seen in nature. For example, the human eye uses
overlapping spectral bands of three (in some cases four) dierent kinds of cone cells
to sense color. Also, it has been shown that the use of overlapping spectral ltering
in technology can improve performance of spectral sensing systems [11].
An example of detectors that can be utilized to produce spectrally overlapping
bands is a subclass of quantum-dot infrared detectors (QDIPs) [12, 13, 14, 15]. A
2
QDIP may display a bias-dependent spectral responsivity due to quantum-conned
Stark eect, which is caused by an asymmetric potential prole, which, in turn, is
from the asymmetric nature of its physical structure. As the detectors operating
bias is varied, its responsivity changes by exhibiting a shift in its peak, and to a
lesser extent, a variation in its shape. For this type of QDIP, a single detector can be
operated at multiple biases sequentially, whereby the detectors responsivity changes
each time the bias is varied. Therefore, a single QDIP detector can be exploited as
dierent detectors. Figure 1.1 presents spectral response of a QDIP under dierent
bias voltages [10]; depending on the QDIP physical structure, other response shapes
are possible.
Figure 1.1: Bias-dependent spectral response of a QDIP, namely QDIP 1999, pro-
duced in CHTM. Twenty one dierent response corresponding to various bias volt-
ages from the set 1, 0.9, ..., 0, ...0.9, 1V .
3
Based on the spectral overlap and diversity of QDIP detectors, approaches that
use principal-components analysis and canonical-correlation analysis have been de-
veloped by Wang et al. [16, 17, 18] and Paskaleva et al. [19, 20, 21] in order to
study the performance of sensing, applied to matched ltering and rock classica-
tion algorithms. In such approaches, detectors responsivities, which are functions of
wavelength, are transformed into an uncorrelated function space. Since the spectra
are practically sampled in wavelength, these approaches view sensing as an inner
product between the sampled scene spectrum vector and the detector responsivity
vector. They have transformed the vectors into an uncorrelated vector space in which
the detectors responsivities are decorrelated, and only the principal contributers
are chosen to be used in sensing. Then, inner product outputs for each transformed
detector constitute a new space. They have shown that sensing in this new space
can give comparable results with a reduced data set. The new responsivites in the
transformed space can still be overlapping in wavelength.
This dissertation reports the rst eort in utilizing sensors having spectrally
overlapping bands to achieve higher spectral resolutions. While subsequent eorts
by Wang et al. and Paskaleva et al. are based on projecting the detectors responsiv-
ities onto an uncorrelated space, this dissertation aims at projecting them onto a new
responsivity space which has certain properties: the new responsivity functions are
desired to have certain center wavelength and certain resolution, as well as certain
shape. These properties may be desirable for certain applications in which the spec-
tral information from certain bands are desired. We have shown that approximation
of desired responsivity functions that span certain central wavelength and resolution
range can be achieved, based on the diversity and overlap of detectors responsivities.
4
1.1.1 Rationale and overview for the algorithmic tuning ap-
proach
As described earlier, in this dissertation we have developed a post-processing algo-
rithm that exploits the spectral overlap of multiple detectors to perform spectral
sensing, and we have applied it to QDIPs in the spectral range 3 m 8 m [10, 22]
and 5 m 10 m [23], with a resolution down to 0.50 m, which is one fourth of
the spectral resolution of the detector. We have also shown theoretically that this
technique is capable of performing some level of multispectral sensing. The idea of
the algorithm can be described as follows. Suppose that we operate the detector
at K dierent biases, and assume that for the kth bias the detector responsivity
is R
k
() (A/W). Now suppose that we are interested in combining the outputs of
the detectors at various biases so as to mimic the response of a desired spectral l-
ter, or band, which we denote by r
c
(). The rst step of the algorithm is to form
a weighted superposition of K responsivities to optimally approximate the multi-
spectral band represented by r
c
() (in the minimum mean-square sense). Then, by
exploiting the linearity of the detector, these very weights can be used in forming a
weighted sum of the individual photocurrents from the detector, one for each oper-
ating bias. The resulting superposition would yield the best approximation of the
output of the sought-after band represented by r
c
(). Synthesized responsivities
obtained in this fashion can be continuously varied, within certain limits, in shape
(unimodal or multimodal), width (narrowband or wideband) and location, depending
on the requirements of each application. In other words, continuous spectral-tuning
is obtained [10]. This makes it possible to performing some level of multispectral
sensing [10, 22] without using any physical lters.
Figure 1.2 presents a schematic of the spectral-tuning algorithm; Fig. 1.3 presents
example spectral tuning between 38m by using the spectral response of the QDIP
shown in Fig. 1.1, and shows an example spectrum reconstruction [10].
5
database containing all
responsivitiesRV
1
,, RV
M
and
bias voltages V
1
, , V
M
.
projection algorithm:
compute the weights w
1
, , w
M
QDIP
detector
bias
controller
V
1
V
2
:
:
V
M
Y
1
Y
2
:
:
Y
M
X
w
1
w
2
:
:
w
M
desired
responsivity
spectral
irradiance
of the object
shaded area
=photocurrent due to
desired responsivity
Algorithms
approximation of Y
database containing all
responsivitiesRV
1
,, RV
M
and
bias voltages V
1
, , V
M
.
projection algorithm:
compute the weights w
1
, , w
M
QDIP
detector
bias
controller
V
1
V
2
:
:
V
M
Y
1
Y
2
:
:
Y
M
X
w
1
w
2
:
:
w
M
desired
responsivity
spectral
irradiance
of the object
shaded area
=photocurrent due to
desired responsivity
Algorithms
approximation of Y
Figure 1.2: Schematic of the projection algorithm for the spectral tuning of QDIPs.
From a practical standpoint, however, the success of the aforementioned post-
processing-based tunable sensing strategy depends upon the level of noise in the
current, which is mostly attributable to dark current and the associated generation-
recombination noise [24, 25, 26]. Since the algorithm is based on forming a weighted
sum of photocurrents (usually with positive and negative signs), the noise variances
corresponding to each weighted photocurrent accumulate, resulting in approximation
error and poor signal-to-noise ratio (SNR) [22]. Moreover, the eect of noise on the
performance is shown to vary with the spectral width of the desired responsivity. In
particular, as the full-width at half-maximum (FWHM) of the desired responsivity
decreases, the degradation due to noise eect becomes more severe. Intuitively,
this is because the magnitudes of the weights needed to approximate a narrow
responsivity become large and the noise accumulation becomes more pronounced.
Therefore, there is a fundamental tradeo between the spectral resolution sought by
6
Figure 1.3: Left: Spectral-tuning of desired triangular lter with FWHM of 0.5m,
between 3m10.5m, by using bias-tunable QDIP 1999, developed at CHTM. Right:
A reconstruction of 3mm polystyrene lter spectrum with the QDIP.
the spectral-tuning algorithm and the noise-induced error.
In this dissertation, we derive a generalization of the spectral-tuning algorithm [10,
22] reported earlier to accommodate noise in the current [23]. More generally, this
work provides a framework for using a collection of sensors that have overlapping
bands and dierent noise characteristics to synthesize outputs corresponding to a
desired arbitrary band. The theory is applied to mid-infrared QDIPs fabricated by
our group.
1.1.2 Rationale and overview nonuniformity correction for
spectrally tunable QDIP arrays
At the sensor array level, pixel-to-pixel dissimilarities among the spectral response
of detectors exist, even though extreme care is taken in order to manufacture them
to be identical to each other. Changes in lithographic dimensions and other man-
7
ufacturing factors contribute to this. As a result, an undesirable spatial mesh-like
pattern is superimposed on the scene. This is known as xed-pattern noise (FPN)
or nonuniformity (NU). Combinations of Johnson noise, generation-recombination
noise, signal-induced shot noise, 1/f noise, uctuations in the optical background,
electronic interface noise such as changes in resistance and other electrical charac-
teristics, clock pickup noise contribute to NU [27, 28, 29, 30].
Background and existing work on NUC
In order to combat NU noise during the imaging application, calibration of the
detector array is performed. The calibration is done by placing a blackbody radiation
source at known and uniform temperature(s) in front of the array, measuring the
response of the array detectors, and obtaining a correction map that provides a
relation between the true temperature and the detector output. If this is done for
one temperature level, then it is called a one-point (oset) correction since the NU
is characterized only by deviation by an oset from the uniform-to-be response level.
One-point correction is useful if the dynamic range of the application is small (up to
10 K), and, a two-point (gain and oset) calibration is used to achieve good results in
wider dynamic ranges. Two-point calibration involves use of a blackbody source at
two dierent known temperatures in the range of operation, and the correction map
characterizes NU noise by a gain and an oset parameter. The more temperature
levels are used, the more accurate is the calibration.
One-time calibration is not enough to mitigate NU noise. Temporal drift in NU
noise makes it necessary to repeat the calibration. Although temporal drift is slow
in cryogenically cooled cameras (with time constant of few hours), it can be as fast
as fraction of a minute for uncooled cameras. Since during each calibration opera-
tion of the camera is halted, such repetitive interruptions of the camera operation
are undesirable. And, at times, this may be critical or even catastrophic for certain
8
applications, such as night vision systems in automobiles. Calibration also increases
the cost of the imaging system since it requires mechanical parts, blackbody source
or sources, a cooling/control system to keep the blackbody source(s) at xed tem-
perature(s). There is also the possibility of breakdown of mechanical parts, further
adding to the cost of maintenance of the camera. If the imaging system is used in a
satellite for example, such mechanical failures may not be repaired.
Techniques which aim to replace the calibration process or reduce the its fre-
quency have been long desired. Consequently, a number of so-called scene-based
post-processing techniques have been developed. Post-processing nonuniformity cor-
rection (NUC) techniques can be classied into two main categories: (1) statistical
techniques and (2) registration-based, or, algebraic techniques. The statistical tech-
niques are based on modeling NU noise as a random spatial or spatio-temporal noise
and estimating its statistics, and thereby compensating for NU noise. Many statisti-
cal techniques have been explored extensively [31, 32, 33, 34, 35, 36]. Some assume
that the scene statistics are constant, and others even capture the drift in statistics
of the NU noise. Many of these statistical techniques require a signicant number
of image frames (on the order of thousands) before reliable NU compensation is
achieved.
Recently, algebraic techniques that exploit motion in the camera/scene have been
developed. Among these is the algebraic NUC technique developed by Ratli et al.,
which utilizes global motion in the camera/scene [37, 38, 39, 40, 41, 42, 43, 44, 45].
Algebraic techniques usually require lower number of frames and they are more robust
than the statistical techniques [45, 46, 47, 48].
The foundation of the algebraic technique proposed in this work was laid by
Ratli et al. They used a bilinear-interpolation model for motion in order to unify
all the biases of the array. The NUC they have proposed and applied on real imagery
was for one-point (oset, or bias) correction. Although they have proposed a solution
9
for the two-point (gain and oset) NUC, they have not applied it to real imagery. In
this dissertation, we put this technique in a general matrix-algebra framework that
allows the utilization of minimum-mean-square-error principles and regularization
techniques. A matrix-based framework has also the potential for exploiting sparse
techniques due to the sparse nature of the matrices involved. We later extended
the matrix-based framework for the two-point (gain and oset) correction. Similarly
to Ratli et al., the NUC can be operated in two modes: (a) radiometric mode,
where partial calibration information at the perimeter pixels of the frame is used (b)
non-radiometric mode, in which no calibration information is used, i.e., NU in all of
the pixels is estimated, by making some general statistical assumptions about the
perimeter pixels.
The issue of chromatic NU
It is known that NU is dependent upon the spectrum of the scene and responsivity of
the detector. This type of NU is referred to as chromatic nonuniformity (CNU). In
the case of a tunable QDIP array, as the array is tuned to dierent spectra, the NUC
must be adjusted accordingly so as to combat CNU. Although the eect of chromatic
nonuniformity was studied before [29, 49], to the best of our knowledge, no NUC
algorithm (either calibration-based or scene-based) that is adaptive to the changes in
scene spectrum or detector spectrum has been developed. In other words, no adaptive
chromatic nonuniformity correction technique exists for tunable sensors. Ideally, it
would be desirable to have NUC maps for every spectral response, thereby covering
all tuning possibilities. However, this is clearly impractical from the implementation
perspective. In this dissertation, a post-processing technique that makes use of our
spectral-tuning algorithm is developed to mitigate CNU noise resulting from tuning
the sensor. Our approach is based on the following novel idea: We rst obtain NUC
maps under dierent biases and establish a NUC map library for all the operational
10
biases. Next, as we tune the sensor to a particular band, by forming a superposition
of sensor outputs according to the weights rendered by our tuning algorithm, we use
the very same weights to form superposition of the NUC maps. This superposition
yields a NUC customized for that particular band.
1.2 Organization of the dissertation
This dissertation is organized as follows. Chapter 2 provides necessary background on
QDIPs, as well as IR detectors in general. The main contribution of this dissertation
is included in Chapter 3; it includes the spectral-tuning algorithm as well as specic
examples. We explain the degrading eect of photocurrent noise on the spectral-
tuning algorithm and quantitatively compare the performance of algorithms that
do and do not accommodate noise. We also present some multispectral sensing
examples. In Chapter 4, we introduce our approach for the mitigation of chromatic
nonuniformity (CNU) in the infrared detector arrays by using the spectral-tuning
algorithm. We describe how the ST-algorithm can be combined with NUC techniques
in order to mitigate CNU. We apply our chromatic NUC algorithm on simulated
imagery based on QDIPs. In Chapter 5 we describe our matrix-based NUC approach
in detail and present applications of our NUC technique on simulated and actual
infrared images. We rst introduce one-point (oset) NUC, and then proceed to
two-point (gain and oset) NUC. Conclusions of the work done in this dissertation
are presented in Chapter 6, and possible directions for future work are presented in
Chapter 7.
11
1.3 Publications resulting from the dissertation
The publications related to this work include 2 journal papers, 3 conference presen-
tations and a U.S. patent application.
1. U. Sakoglu, M. M. Hayat, J. S. Tyo, P. Dowd, S. Annamalai, K. T. Posani, and
S. Krishna, Statistical adaptive sensing using detectors with spectrally overlapping
bands, Applied Optics, accepted (2006).
2. U. Sakoglu, Z. Wang, M. M. Hayat, J. S. Tyo, S. Annamalai, P. Dowd, and
S. Krishna, Quantum dot detectors for infrared sensing: Bias-controlled spectral
tuning and matched ltering, in Nanosensing - Materials and Devices, M. S. Islam
and A. K. Dutta, eds., vol. 5593, pp. 396407 (Proceedings of the SPIE, 2004).
Invited.
3. U. Sakoglu, R. C. Hardie, M. M. Hayat, B. M. Ratli, and J. S. Tyo, An
algebraic restoration method for estimating xed-pattern noise in infrared imagery
from a video sequence, in Applications of Digital Image Processing XXVII, A. G.
Tescher, ed., vol. 5558, pp. 6979 (Proceedings of the SPIE, 2004).
4. Z. Wang, U. Sakoglu, S. Annamalai, N.-R. Weisse-Bernstein, P. Dowd, J. S.
Tyo, M. M. Hayat, and S. Krishna, Real-time implementation of spectral matched
ltering algorithms using adaptive focal plane array technology, in Imaging Spec-
trometry X, A. G. Tescher, ed., vol. 5546, pp. 7383 (Proceedings of the SPIE,
2004).
5. U. Sakoglu, J. S. Tyo, M. M. Hayat, S. Raghavan, and S. Krishna, Spectrally
adaptive infrared photodetectors with bias-tunable quantum dots, Journal of the
Optical Society of America B 21(1), 717 (2004).
A U.S. patent application related to this work was led on 9/22/2004:
Detector with Tunable Spectral Response, Sanjay Krishna, J. Scott Tyo, Majeed
12
M. Hayat, Sunil Raghavan and Unal Sakoglu.
13
Chapter 2
Quantum-Dot Infrared
Photodetectors
In this chapter we provide necessary background on infrared (IR) detectors in general,
and we provide the advantages of quantum-dot infrared photodetectors (QDIPs) with
respect to other IR detectors. We review their bias-dependent spectral diversity,
drawing freely from recent literature. This spectral-diversity property makes them
applicable for our spectral-tuning algorithm.
2.1 Infrared detectors
Infrared photodetectors, and particularly mid-IR (MIR) photodetectors, which allow
detection in the wavelength range of 3-20 m, are desirable for many applications.
These applications include thermal imaging, night-vision, mine-detection, chemical
analysis and euent detection. However, the operating temperatures of the present
MIR detectors have to be lowered to cryogenic temperatures (40-80 K) in order to
obtain practically low noise levels, since they cannot perform well at the air-cooled
14
Chapter 2. Quantum-Dot Infrared Photodetectors
temperatures (200 K).
The performance of a detector is measured by several gures of merit. Among
these are responsivity and detectivity (D
). Among the cryogenically-cooled detec-

tors, low band-gap material-based detectors such as HgCdTe-based ones, have high
detectivity and responsivity; however, they have high uncertainty in these parame-
ters due to problems in their epitaxial growth [50]. Similarly, superlattice structures
based on InAsSb-InSb or InGaSb-InAs, which are other alternatives at the cryogenic
temperatures, also have epitaxial-growth problems [51]. Overall, the manufacturing
cost of these technologies is high.
The demand for higher responsivity and detectivity in photodetectors and lower
cost pushed for producing devices based on more mature manufacturing technologies
that lead to higher structure uniformity. As a result, quantum-well infrared photode-
tectors (QWIP) were developed [52]. Although dierent QWIP spectral response can
be obtained within 3-20 m range by changing the well size and material compounds,
their detectivity is low due to mainly short lifetime of electrons in excited state [53].
In order to obtain higher detectivity, lower dark-current and associated current noise,
they have to be cooled to liquid-nitrogen temperatures or below.
2.2 Quantum-dot-based structures
Recently, quantum-dot-based structures, which have more favorable carrier dynam-
ics than their quantum-well-based counterparts, have been developed [54, 55, 56].
QDIPs possess normal-incidence operation, have less dark-current and associated
current noise, thus, higher detectivity than the QWIPs. Quantum-dots are structures
that contain small number of electrons and are sometimes referred to as articial
atoms [57]. They possess discrete atom-like density of states, due to connement of
electrons in small 3-D space. In order to be useful for photodetection, their dimension
15
has to be less than 15-20 nanometers. This had been extremely dicult to fabri-
cate, until the development of self-assembled quantum-dots, which has emerged re-
cently [58]. Under high strain and certain other growth conditions, the dots assemble
themselves in three-dimensional (3-D) pyramid-like structures instead of a 2-D layer,
and build the quantum-dot structure. They are usually formed from InGaAs/GaAs,
SiGeSi and InGaAs/InP systems [59]. They have energy spacing in 5 20m range
that allows fabrication of devices for MIR sensing applications. They have longer
photo-generated carrier lifetime, i.e. the photoelectrons stay in the excited state for
longer time, and thus contribute to the photocurrent more eciently [12, 13]. This
increases the signal-to-noise level of the detector.
In order to improve their carrier dynamics, the dots have been placed in dots-
in-a-well (DWELL) structures, which, in turn, lowers the thermionic emission and
thus increases their operating temperatures. Changing the dimension size of the well
makes them exhibit dierent spectral and other response characteristics. However,
a very promising property comes from the change in their spectral response under
dierent applied bias voltages, allowing us to tune their spectral response electroni-
cally.
Bias-dependent spectral response
QDIPs exhibit bias-voltage dependent change in their spectral characteristics [12, 13,
14, 15] in terms of change in shape, width and peak location, due to quantum-conned
Stark eect, which is caused by an asymmetric potential prole. The reason for the
asymmetric potential prole of the QDIP is their asymmetric physical structure.
The bias-voltage-dependent diversity in spectral response is promising since it can
be exploited to tune the detector to respond within desired wavelength range. Bias-
dependent spectral response of a quantum-dot device, normalized with respect to
the their peaks, produced at the UNMs Center for High Technology Materials, is
16
presented in Fig. 2.1. It can be seen that for the bias voltage values between 2.25V-
4V, the device has a main peak at 7.9m and a secondary peak in the range 5.1 m
5.6 m, exhibiting bi-modal response at some of the bias-voltage values. For applied
bias-voltage values between 2.25 m 0 m (not shown here) the peak around 5.6
m remains dominant; however, the overall responsivity decreases for these small
bias-voltage values. This means that for these bias-voltage values, the detectivity
and the output photocurrent is small. For negative applied bias-voltage values, the
device has a similar behavior. It exhibits main peak around 9.5 m 10 m with a
secondary peak around 6m, at applied bias-voltage values of -3.5V to -2.5V. As the
voltage values are changed to values in the range -2.25V to -1.75V, the peak around
6m dominates. For voltage values between -1.75V to 0V, the responsivity of the
detector decreases (not shown here), also making the detectivity of the device lower.
For bias voltages greater than the maximum values shown here, the device goes
into saturation. The range of the bias voltages for which the device provides high
responsivity can be determined by making detectivity or responsivity measurements.
A simple schematic of the quantum-dot device structure mentioned above is
shown in Fig. 2.2. The device possesses asymmetric dots-in-well (DWELL) struc-
ture, sandwiched between highly n-doped GaAs layers and grown on a semi-insulating
GaAs substrate. The active region of the detector consists of 15 layers of InAs quan-
tum dots in an In
0.15
Ga
0.85
As well [60]. The well is asymmetric in the sense that
the width of the top and bottom well layers are dierent (60 and 50

A), which leads
to photo-responsivities that can be spectrally tuned through changing the applied
bias voltage. The width of the quantum-dot layer is only 50
A80
A, and the total

thickness of the detector is about 3.5 m. Devices with dierent diameter apertures,
ranging from 25 m to 300 m, were fabricated on arrays.
17
3 4 5 6 7 8 9 10 11
0
0.2
0.4
0.6
0.8
1
WAVELENEGTH, m
P
E
A
K
N
O
R
M
A
L
I
Z
E
D

R
E
S
P
O
N
S
I
V
I
T
Y
,

f
k
(
)
4
3.75
3.5
3.25
3
2.75
2.5
2.25
2
1.75
2
2.25
2.5
2.75
3
3.25
3.5
2V
3.5V
4V
Figure 2.1: Peak-normalized spectral response of a quantum-dot detector at 17 dif-
ferent applied bias voltages.
18
Figure 2.2: Schematic of the 15-layer asymmetric InAs/In
0.15
Ga
0.85
As DWELL de-
tector structure (# 554, CHTM) sandwiched between two highly doped n-GaAs
contact layers, grown on a semi-insulating GaAs substrate.
19
Chapter 3
Spectral-Tuning Algorithm
In this chapter, the spectral-tuning (ST) algorithm and its application to QDIPs are
presented. The degrading eect of noise in the detector outputs to the algorithm
is shown and a noise-modied version of the algorithm is developed and applied
to QDIPs. Finally, the interplay between photocurrent-noise power, desired tuning
resolution and the tuning performance is investigated.
3.1 Spectral-tuning algorithm
Suppose that we would like to have a detector with certain desired spectral response,
and its corresponding photocurrent output. We rst approximate this desired re-
sponse by using the best weighted superposition of the detectors responses, this
is called the projection step, in which we basically project the desired spectrum
onto our basis spectra. Then, we use the same weights to combine the outputs of the
detectors, the photocurrents, to obtain the output photocurrent that would approx-
imate the output of the desired response. We provide the mathematical justication
of this idea is in this section.
20
Chapter 3. Spectral-Tuning Algorithm
3.1.1 Algorithm development
Let us assume that we have a collection of K detectors, T
k
= D
1
, ..., D
K
, each
having spectral responsivity R
k
() (measured in A./W.), k = 1, ..., K, in the wave-
length range
min
to
max
, which are measured a-priori. Suppose that each detector
has photocurrent output, y
p,k
, k = 1, ..., K. The photocurrent output of a detector
T
k
can be related to its spectral responsivity by using the linear detector response
model
y
p,k
= A
e
_

max
min
g()R
k
()d, (3.1)
where g() (W cm
2
m
1
) is the spectral irradiance of the source (combined with any
other lters and/or eects in the scene) and A
e
is the eective source-to-detector
area that combines all geometric factors. For a detector at certain location in a
detector array, A
e
can be assumed xed. Let us denote the spectral response of the
desired detector responsivity to be approximated as r
c
(). The shape and width
of r
c
() and the location of its center,
c
(c = 1, ..., C, say), all depend on the
application.
Our goal is to nd the set of weights, w
c
[w
c,1
, ..., w
c,K
]
T
, so that we can best
approximate, in the minimum mean-square error (MMSE) sense, the ideal output
y
c
= A
e
_

max
min
g()r
c
()d (3.2)
by the weighted combination of the individual detector outputs,
y
c
=
K
k=1
w
c,k
y
p,k
(3.3)
so that the mean-square error,
[y
c
y
c
[
2
, (3.4)
is minimized. Note that, the error depends on the choice of the desired responsivity,
r
c
() and the set of basis spectra, T
k
. We show below that, in the absence of
21
noise, the best approximate response y
c
, in the MMSE sense, is equivalent to the
output of the detector whose spectral response is
r
c
() =
K
k=1
w
c,k
R
k
() (3.5)
that best approximates (again, in the MMSE sense) r
c
() [10].
Based on Eqs. (3.4), (3.3), (3.2) and (3.1), the error can be cast as
[y
c
y
c
[
2
=
max
min
g()
_
r
c
()
K
k=1
w
c,k
R
k
()
_
d
_
_
max
min
g()
_
r
c
()
K
k=1
w
c,k
R
k
()
_
d
_
2
,
The inequality is based on triangle inequality. The last integral can be further upper-
bound with the Schwarz inequality [61] to obtain
[y
c
y
c
[
2
__

max
min
g
2
()d
_
_
_

max
min
[r
c
()
K
k=1
w
c,k
R
k
()]
2
d
_
. (3.6)
Without loss of generality, we can assume normalized scene irradiance function g(),
so that the rst integral becomes unity. Then, minimization of the maximum of [y
c
y
c
[
2
over all possible normalized irradiance functions g() is equivalent to minimizing
the second integral in Eq. 4.4. The minimization of maximum [y
c
y
c
[
2
over all
possible scene irradiance is useful since we would not like the weights to be dependent
on the scene irradiance. If there is some knowledge about the scene irradiance in an
application, it can be further used to improve the performance of the minimization.
We now dene the second integral of Eq. (4.4) as the squared-error,
2
(r
c
, T)
_

max
min
[r
c
()
K
k=1
w
c,k
R
k
()]
2
d, (3.7)
and we discretize it in order to use standard quadratic techniques to solve for the
weights,
2
(r
c
, T) L
1
L
l=1
_
r
c
(
l
)
K
k=1
w
c,k
R
k
(
l
)
_
2
, (3.8)
22
where =
max
min
,
1
=
min
, ...,
L
=
max
,
k+1
k
= /L, and L is the
size of the wavelength grid used in the approximation of the integral. We denote
the discretized versions of the functions with their bold counterparts and dene the
column vectors
r
c
[r
c
(
1
), ..., r
c
(
L
)]
T
and R
k
[R
k
(
1
), ..., R
k
(
L
)]
T
(3.9)
and form the matrix including in its columns all the discretized basis spectra
A [R
1
, ..., R
K
]. (3.10)
We now can express Eq. (3.8) in matrix equation form as
2
L
1
[|r
c
Aw
c
|
2
], (3.11)
which we can expand as
2
L
1
[(r
c
Aw
c
)
T
(r
c
Aw
c
)]
= L
1
[r
T
c
r
c
w
T
c
A
T
r
c
r
T
c
Aw
c
+w
T
c
A
T
Aw
c
].
(3.12)
The above equation is quadratic in w
c
, and in order to solve for the weights, w
c
,
that minimize the it, we dierentiate with respect to w
c
and equate it zero to nd
its minimum. We then obtain
w
c
(
2
) [2A
T
r
c
+ 2A
T
Aw
c
] = 0, (3.13)
from which we obtain the weights,
w
c
= [A
T
A]
1
A
T
r
c
, (3.14)
in the MMSE sense. This solution actually gives the MMSE projection of the desired
spectrum r
c
onto the nite-dimensional function space spanned by the basis detector
spectra A = [R
1
, ..., R
K
], or minimizing r
c
()
K
k=1
w
c,k
R
k
(). Therefore nding
the weights that approximates r
c
() in the MMSE sense, so that
r
c

K
k=1
w
c,k
R
k
(), (3.15)
23
would provide us with the same weights needed to obtain the tuned current output
with the MMSE. Therefore, our problem reduces to best approximation of the
desired responsivity r
c
(), or r
c
in its discrete from, in the MMSE sense.
The shape, spectral width and center wavelength of the desired responsivity is
arbitrary. Thus, the algorithm can be applied to tune the detectors response to
arbitrary spectral width and center wavelength. The range of center wavelength
and the minimum spectral width (resolution) that the algorithm can obtain good
performance are dictated by the range, center and diversity of the basis spectra. The
error toleration between the approximation and the actual desired responsivity is
dictated by the goal of the application that uses the ST algorithm. The algorithm is
summarized in Fig. 1.2. We next introduce the regularized version of the algorithm.
3.1.2 Regularized version of the algorithm
In our ST algorithm developed above, we have not imposed any constraints on the
approximation of r
c
(). In practical applications of the ST algorithm, the approxi-
mation r
c
() can be very undesirably rough (having sharp peaks and dips) in ,
even though it achieves the minimum mean-square error. This is because of rough-
ness of the basis spectra themselves; they might contain sharp peaks, which are
due to sharp transitions at the spectra due to atmospheric-absorbtion bands and
sharp transition characteristic of the devices themselves. Stringent requirements on
the desired responsivity such as very good desired resolution, which corresponds to
narrow spectral width, would result in signicant amplication of certain parts of
basis spectra that contain high amount of roughness. We introduce a smoothing
criterion that penalizes the amount of roughness in the approximated r
c
() to rem-
edy this problem, at the expense of tolerable spectral resolution. The regularized
24
mean-square error, which includes the penalization can be written as
2
(r
c
, T)
_

max
min
[r
c
()
K
k=1
w
c,k
R
k
()]
2
d+
_

max
min
[
d
2
d
2
K
k=1
w
c,k
R
k
()]
2
d,
(3.16)
where the Laplacian operator, d
2
/d
2
is used to measure the amount of roughness in
approximation of desired responsivity, and the regularization parameter, , controls
the amount of penalization in the roughness. Its value depends on users experience
(for example, in our applications, = 0.04 gave good results).
We can write the discretized version of the above equation as
2
(r
c
, T)

L
L
l=1
_
r
c
(
l
)
K
k=1
w
c,k
R
k
(
l
)
_
2
+
_
K
k=1
w
c,k
[R
k
(
l1
) + 2R
k
(
l
) R
k
(
l+1
)]
_
2
.
(3.17)
We can then write in matrix notation
2
L
1
[|r
c
Aw
c
|
2
+ |QAw
c
|
2
], (3.18)
where
Q=
_
_
1 1
1 2 1
.
.
.
1 2 1
1 1
_
_
. (3.19)
With steps similar to minimization of the unregularized problem before, by dier-
entiating Eq. (3.18) with respect to w
c
and equating to zero, we obtain the MMSE
weights,
w
c
= [A
T
A+ A
T
QQ
T
A]
1
A
T
r
c
. (3.20)
25
3.2 Application of ST algorithm to QDIPs
For the application of the ST algorithm two separate QDIPs (namely, QDIP 1198 and
QDIP 1199) that were fabricated at the CHTM was considered. The sets of biased-
dependent responses for these two devices were measured and subsequently used in
the post-processing technique to generate simulated tuned responses with varying
FWHMs and center wavelengths. The two devices have slightly dierent responses,
as shown in Fig.3.1. For QDIP 1198, 20 spectra (K=20) which correspond bias volt-
ages of [V
1
, ..., V
20
] = [1.0, 0.9, ..., 0.1, 0.1, , 0.9, 1.0]V were used. For QDIP 1199,
21 spectra, corresponding to [V
1
, ..., V
21
] = [1.0, 0.9, ..., 0.1, 0, 0.1, ..., 0.9, 1.0]V ,
were used.
Figure 3.2 shows examples of simulated tuned responses for narrow (0.5m),
medium (1.0m), and coarse (3.0m) spectral resolution at various center wave-
lengths across the sensitivity range of the QDIP 1199. Figure 3.3 show the variation
in the spectral resolution (viz., FWHM) attained by the post-processing algorithm
as the tuning wavelength center (
0
) and the desired FWHM are changed for the two
dierent QDIPs. For each selection of the tuning parameter and desired FWHM,
the shape of the desired responsivity is taken as a triangular function, whose base
is twice as wide as the desired FWHM. Note that as we move up the vertical axis
scale, the resulting FWHM rendered by the post-processing algorithm decreases.
Hence, the regions in red represent the narrowest FWHM. The transparent planes in
the gures represent spectral resolution of 1.0m (recall that the FWHM associated
with QDIPs spectral response is in excess of 2.0m). Moreover, the data in Fig. 3.4
indicates that spectral resolution (FWHM) on the order of 0.5m can be mostly
obtained continuously across the wavelength range of 3m-8m for both detectors.
Also, the spectral resolution can be tuned from 0.5m to over 3.0m regardless of
center wavelength. For both QDIPs, a FWHM of 0.5m is the practical lower limit
that we were able to obtain in these examples.
26
Table 3.1: Parameters used for the three simulated sensing modes of the ST algorithm
Mode Number of bands Band centers Desired resolution (FWHM)
Hyperspectral 200 3.5-10.5m 0.5m
(every 0.5m)
Multispectral 7 4-10m 1m
(every 1m)
Multispectral 3 5m 2m
7.5m 1m
9m 2m
In order to demonstrate the potential of the spectrally adaptive QDIPs, we have
performed experiments to predict the performance in a variety of spectral sensing
modes. We tested the ability of the QDIP technology to dierentiate the transmission
spectrum of a 3mm polystyrene lter from a blackbody spectrum obtained from an
unltered blackbody source. The QDIPs were operated as a hyperspectral sensor, a
7-band multispectral sensor, and a 3-band multispectral sensor.
The parameters of each of the spectral sensing modes are presented in Table 3.1,
including the number of spectral bands and desired spectral resolution of each band.
For each spectral sensing mode, the parameters in Table 3.1 were supplied to the
post-processing algorithm described above. The weights required to realize each
spectral band were computed based on the measured response spectrum, and the
resulting spectral sensitivity functions were computed through linear combination
of the photocurrents corresponding to the various biases. The photocurrent of the
QDIP at each bias value was obtained by multiplying and integrating the known
desired spectrum (either blackbody or polystyrene) with the spectral response of the
QDIP at the prescribed bias. The desired spectra were measured using a Nicolet
FTIR spectrometer with 4cm
1
resolution in the wavelength range 2.5-20m, and
converted into wavelength domain, and reconstructed in a linear grid in wavelength.
27
After the QDIP photocurrents were predicted for each applied bias, a reconstructed
photocurrent, corresponding to each multispectral lter (band) was constructed by
linearly combining the QDIP photocurrents according to the weights corresponding
to each desired multispectral band (as computed in the projection step of the al-
gorithm). The results for the hyperspectral-sensing mode are presented in Fig. 3.5,
along with actual spectra measured by the FTIR device. We used triangular lters of
desired FWHM of 0.5m, and 0.25m. As might be expected, the procedure yields
very good results in reconstructing the blackbody spectrum especially with trian-
gular lters of parameter 0.5m (Fig. 3.5(a)). With a desired FWHM of 0.25m,
however, the approximation has some spurious perturbations that are especially vis-
ible between 7-8m, as shown in Fig. 3.5(b). Note that as the blackbody spectrum
is a slowly varying function of wavelength, the relatively broad hyperspectral lter
(0.5m) does not adversely aect the hyperspectral tuning. On the other hand, the
polystyrene spectrum has both large-scale spectral features and narrow absorption
lines. In Fig. 3.6, in the reconstruction of polystyrene lter spectrum, we see that
the hyperspectral data does capture the overall envelope of the spectrum, but it
is unable to resolve spectral features narrower than the realized resolution of the
bands (0.5m), as seen in Fig. 3.6(a). Notably, the hyperspectral data is able to
resolve the H
2
0 and CO
2
absorption features between 3.5 and 4m, when lters with
a FWHM of 0.25m are employed, as shown using arrows in Fig. 3.6(b). Note that
even though the ne spectral structure of the polystyrene lm was not resolved, the
hyperspectral-tuning result clearly indicates that there are important dierences in
the spectra of the black body and that of the polystyrene lm. This information may
not be as detailed as that obtained with a high-resolution spectrometer, but it is ad-
equate to indicate that a second material is present and demonstrates the power of
the proposed post-processing technique and its potential utility as a algorithm-based
spectrometer.
In order put the hyperspectral data oered by our post-processing technique in
28
better perspective, we generated Figs. 3.7 and 3.8, where two bad and good
representative triangular lter approximations are shown. In Fig. 3.7 the FWHM of
the desired triangle responsivity is 0.5m, and the centers are at 4.0m and 7.65m
in (a) and (b), respectively. Note that the yielded post-processing FWHM is ap-
proximately 1.0m, which is twice as much the desired FWHM. By our observations
from all the approximations and also from Fig.3.3, FWHM of 0.5m is near the
resolution limit of the algorithm, depending on the center wavelength of the lter
to be approximated. The use of a desired FWHM of 0.5m is partially successful
in capturing ner spectral details. However, this comes at the expense, at some
wavelengths, of reduced ability to have an isolated peak and nding the peak in the
approximation. The approximations and the reconstructions become more noisy and
less-dependable as we force the FWHM below 0.5m. For example, if we force the
desired FWHM to 0.25m, as shown in Fig. 3.8, then we not only observe that the
main peak becomes wider but we also start to see the emergence of spurious compet-
ing peaks. One possible remedy to the spurious peaks problem is to dened measure
(such as the ratio between the main peak and the rst competing peak) that can be
incorporated in the penalization in the error expression.
The results for the two multispectral modes are presented in Tables 3.2 and 3.3.
For the desired parameters in Table 3.1, the actual values of the spectra, the recon-
structed spectra by using ideal (triangular) lters, and the reconstructed spectra by
using approximated lters are shown altogether for comparison. At these multispec-
tral settings, with much wider spectral resolutions, the QDIP can realize spectra that
are excellent approximations to the desired spectra. We see from the data in Table
2 that both multispectral modes can be used to easily dierentiate the polystyrene
spectrum from the blackbody spectrum. The two modes were chosen to demonstrate
one system with broad spectral coverage and uniform bands and a second system
with variable spectral coverage and bandwidths. These two examples reinforce the
full adaptivity of the QDIP sensors. We also see that the reconstruction by either
29
Table 3.2: Seven-band multispectral reconstruction (normalized)
Center(m) Actual Ideal Approximation
Reconstruction of 3mm polystyrene spectrum (W/cm
1
)
4 0.6887 0.6083 0.5969
5 0.7905 0.7391 0.7358
6 0.6211 0.5657 0.5753
7 0.3315 0.3705 0.3920
8 0.5330 0.3529 0.3349
9 0.5745 0.4791 0.5080
10 0.2967 0.4019 0.3958
Reconstruction of blackbody spectrum (W/cm
1
)
4 0.7029 0.6926 0.6807
5 0.8484 0.8548 0.8510
6 0.9532 0.9420 0.9580
7 0.9994 0.9525 1.0077
8 0.9881 1.0280 0.9753
9 0.9012 0.8672 0.9194
10 0.8452 0.8709 0.8575
using the ideal or approximated spectra are very close, which means that the lim-
itation in accuracy of the reconstruction stems from the mathematical limitation.
Even if we use ideal lters (in this case, with parameter 0.5), the ideal lter is not
thin enough to resolve the ne parts of the target spectra. Theoretically, we have to
have innitesimally thin lters (delta function) to perfectly reconstruct the spectra.
All the reconstruction was done by using the regularization parameter, = 0.04,
as it appeared to give the best results. If too large an -parameter is used, then our
reconstruction loses resolution, since the penalty on uctuations are increased (see
Eq. 3.18). If we use too small an , then the reconstruction becomes very noisy,
30
Table 3.3: Three-band multispectral reconstruction (normalized)
Center(m) Actual Ideal Approximation
Reconstruction of 3mm polystyrene spectrum (W/cm
1
)
5.0 0.7905 0.7244 0.7133
7.5 0.2758 0.3330 0.3223
9.5 0.3840 0.4340 0.4478
Reconstruction of blackbody spectrum (W/cm
1
)
5.0 0.8424 0.8424 0.8296
7.5 0.9998 1.0271 0.9940
9.5 0.8669 0.8599 0.8872
which results in noisy FWHM measurement, shift of the peak due to the noise, and
therefore a poor and erroneous reconstruction. By itself, penalizing the noise is not
enough and we had to use a median lter to smoothen the reconstructed algorithm
in order to be able to nd measure FWHM consistently. The median-ltered spec-
tral response was not, however, used in the tuning. The ltering was solely used to
obtain an accurate estimate of the yielded FWHM. In the bias-dependent response
measurements (between 3-11m), we had 1257 data points, and the length of the
median lter was chosen to be 50 (4% of the grid size), which provided good results
in terms of the measuring the FWHM. Other manifestations of roughness/noise pe-
nalization and lter shapes (instead of a triangular shape, for example) can be used
in order to generate better approximations in the projection step, which may further
improve the overall performance of the algorithm.
31
3.3 Noise-modied spectral-tuning algorithm
3.3.1 Photocurrent noise model
Under illumination, the total amount of current produced by the detector includes
two basic components: (a) the photocurrent, generated by the photo-excited elec-
trons; and (b) noise current (containing one or more of the following components de-
pending on the operation conditions: generation-recombination noise (signal-induced
shot noise), Johnson (thermal) noise, 1/f noise, etc.). The noise is dominantly gen-
erated by thermally-excited electrons, and the it is commonly referred to as dark
current. Dark current is the current that ows through the detector under no
photo-illumination. The dark-current increases as the temperature increases. It is
also dependent on the bias-voltage applied across the detector. Having a convention
of dening random quantities with capital letters, we denote the total noise current
by D
k
, the d.c. component (i.e. time-average) of it by y
d,k
, and the zero-mean ran-
dom component of it by N
k
. We nally dene y
p,k
as the true (noiseless) photocurrent
resulting from illumination.
The current is proportional to the electrical charge, and since electrical charge
is discrete, so is the noise in the current. Therefore due to its discrete nature, the
noise in current is called shot noise. The statistics of this random component is
also dependent on the detector temperature and the applied bias-voltage. It can
be derived by assuming a Poisson process model for the statistics of the generated
electrons that contribute to dark current [25, 62]. We develop an analysis of the
Poisson model of the noise for the detector in Appendix A.
Based on the above denitions, the total current generated by an infrared detector
T
k
can be written as [23]
Y
k
= y
p,k
+ y
d,k
+ N
k
. (3.21)
32
We assume that the noise components of dierent detectors are statistically indepen-
dent.
Under the assumption of a linear detector, we can write
y
p,k
A
e
_

max
min
R
k
()g(, Z)d, (3.22)
where g(, Z) (W cm
2
m
1
) is the spectral irradiance of the source (combined with
any other lters and/or eects in the scene) at temperature Z (Kelvin) and A
e
is
the eective source-to-detector area that combines all the geometric factors, and
is xed throughout our analysis (it only changes with the location of the detector in
the array). For the sake of the analysis to follow, it would be convenient to embed
the noise into the integral above and obtain an equivalent expression for the noisy
photocurrent. By using Eqns. (A5) and (3.22), we can write
Y
p,k
y
p,k
+ N
k
(3.23)
= A
e
_

max
min
_
R
k
() +
N
k
A
e
_
max
min
g(
, Z)d
_
g(, Z)d (3.24)
A
e
_

max
min
_
R
k
() +
R
k
()N
k
A
e
_
max
min
R
k
(
)g(
, Z)d
_
g(, Z)d (3.25)
= A
e
_

max
min
R
k
()
_
1 +
N
k
N,k
SNR
k
_
g(, Z)d. (3.26)
In the approximation shown via Eq. (2.6), we can identify the noisy responsivity
as
R
k
() R
k
()
_
1 +
N
k
N,k
SNR
k
_
. (3.27)
The approximation introduced in Eq. (2.6) is done in order to get rid of the de-
pendence on source irradiance, g(, Z), i.e., in order to make the algorithm source-
independent. This approximation turns into an equality when the variation in R
k
()
33
is negligible over the integration range. However, in that case it would not serve our
algorithm since there would not be any spectral diversity. The approximation can
be equivalently restated as
R
k
()
_

max
min
g(
)d

_

max
min
R
k
(
)g(
)d
.
From the rst mean value theorem for integration, we know that at least one
satises equality in the approximation. The theorem states that
if R(
) : [
min
,
max
] R is continuous and g(
) : [
min
,
max
] R is integrable
positive function, then there exists a number in (
min
,
max
) such that
_

max
min
R
k
(
)g(
)d
= R
k
()
_

max
min
g(
)d
. (3.28)
Moreover, since the detectors response are close to 0 near
min
and
max
, and have
peak(s) in between, there exist at least two values for which the equality holds
since this condition leads to existence of a set of =
1
,
2
, .. such that R
k
(
1
) =
R
k
(
2
) = .... Furthermore, due to nature of our spectral-tuning algorithm, which
combines (with positive and negative weights) the detectors spectral response, the
associated approximation error also gets averaged. Thus, when we compare the
spectral-tuning results of the original and the noise-modied algorithm, we see that
this approximation does not seem to aect the noise-modied algorithm so badly, as
we present in the applications section.
We now incorporate noise into the algorithm by seeking the vector w
c
which
minimizes the average of the mean-square error
2
(r
c
,T)

= E[[y
c

Y
c
[
2
], where E
denotes ensemble averaging. By substituting Eqns. (3.2) and (3.4) in this denition,
we obtain
2
(r
c
,T) = E
_
A
e
_

max
min
g(, Z)[r
c
()
K
k=1
w
c,k
R
k
()(1+
N
k
N,k
SNR
k
)]d
2
_
. (3.29)
In order to remove the dependence of the error expression on the spectral irradiance
g(), we compute an upper bound to this expression using the Schwarz inequality [61].
34
We assume that the noise N
k
has zero mean for all detectors, and that noise for
dierent detectors are independent. We then discretize the integral and convert
Eq. (3.29) into matrix form (see Appendix A for a detailed derivation) and obtain
the expression to be minimized as
L
L
j=1
_
r
2
c
(
j
)2r
c
(
j
)[
K
k=1
w
c,k
R
k
(
j
)] + [
K
k=1
w
c,k
(R
k
(
j
)]
2
+[
K
k=1
w
2
c,k
R
2
k
(
j
)
SNR
2
k
]
_
.
(3.30)
In the above, L is the number of wavelength locations at which the spectrum was
sampled,
1
=
min
,
L
=
max
, and =
max
min
. We dene the vector forms of the
discretized spectrum as R
k
[R
k
(
1
), . . . , R
k
(
L
)]
T
and r
c
[r
c
(
1
), . . . , r
c
(
L
)]
T
,
and the matrix of spectral response A [R
1
, ..., R
K
].
From Eqs. (3.2), (3.4) and (3.29), the expectation of the square of the error
between the actual and approximated outputs can be written as
E[[y
c
Y
c
[
2
]=E[[A
e
_

max
min
g(, Z)[r
c
()
K
k=1
w
c,k
R
k
()(1 +
N
k
SNR
k
N,k
)]d[
2
]. (3.31)
After applying Schwarz inequality [61], we can write
E[[y
c
Y
c
[
2
]E[A
e
_

max
min
[g(, Z)[
2
d][
_

max
min
[r
c
()
K
k=1
w
c,k
R
k
()(1 +
N
k
SNR
k
N,k
)d[
2
].
Since the term in the rst brackets is a constant, the minimization reduces to the
minimization of
E[
_

max
min
[r
c
()
K
k=1
w
c,k
R
k
()(1 +
N
k
SNR
k
N,k
)d[
2
]. (3.33)
Expanding the equation and moving the expectation through the deterministic vari-
ables, we obtain
_

max
min
[r
2
c
()2r
c
()
K
k=1
w
c,k
R
k
()(1 +
E[N
k
]
SNR
k
N,k
)]d
35
+
_

max
min
E[[
K
k=1
w
c,k
R
k
()(1 +
N
k
SNR
k
N,k
)[
2
]d. (3.34)
In order to complete the terms in the rst integral into a quadratic, we add and
subtract
[
K
k=1
w
c,k
R
k
()(1 +
E[N
k
]
SNR
k
N,k
)d)]
2
,
expand the square of summation into multiplication of double summation, and obtain
_

max
min
[r
c
()
K
k=1
w
c,k
R
k
()(1 +
E[N
k
]
SNR
k
N,k
)]
2
d
+
_

max
min
[
K
k=1
K
l=1
w
c,k
w
c,l
(E[N
k
N
l
]E[N
k
]E[N
l
])R
k
()R
l
()
N,k
N,l
SNR
k
SNR
l
]d. (3.35)
Since we assume independence among noise components from dierent detectors, the
term inside the parenthesis in the second integral reduces to
K
k=1
w
2
c,k
R
2
k
()
SNR
2
k
, (3.36)
and the discretized minimization problem then becomes
L
L
j=1
_
[r
c
(
j
)
K
k=1
w
c,k
R
k
(
j
)(1 +
E[N
k
]
SNR
k
N,k
)]
2
+[
K
k=1
w
2
c,k
R
2
k
(
j
)
SNR
2
k
]
_
. (3.37)
By invoking the assumption E[N
k
] = 0 for all k, the above quantity simplies to
L
L
j=1
_
r
2
c
(
j
)2r
c
(
j
)[
K
k=1
w
c,k
R
k
(
j
)] + [
K
k=1
w
c,k
(R
k
(
j
)]
2
+[
K
k=1
w
2
c,k
R
2
k
(
j
)
SNR
2
k
]
_
.
(3.38)
We now dierentiate with respect to w
c,s
, set the result to zero and obtain
L
j=1
r
c
(
j
)R
s
(
j
) =
L
j=1
_
R
s
(
j
)[
K
k=1
w
c,k
R
k
(
j
)] + w
c,s
R
2
s
(
j
)
SNR
2
s
_
. (3.39)
36
Moving R
k
(
j
) into the summation and interchanging the order of summations, we
have
L
j=1
r
c
(
j
)R
s
(
j
) = [
K
k=1
w
c,k
L
j=1
R
s
(
j
)R
k
(
j
)] +
w
c,s
SNR
2
s
[
L
j=1
R
2
s
(
j
)]. (3.40)
Dening
c,s
r
T
c
R
s
=
L
j=1
r
c
(
j
)R
s
(
j
), and
s,k
R
T
s
R
k
=
L
j=1
R
s
(
j
)R
k
(
j
),
we can write the equation as
c,s
= [
K
k=1
w
c,k
s,k
] +
w
c,s
SNR
2
s
s,s
. (3.41)
If we repeat this equation for s = 1, ..., K, we have a linear system of K equations
with K unknowns w
c,k
, which can be stated in matrix form as
c
= [A
T
A+]w
c
, (3.42)
where
=
_
_
R
T
1
R
1
SNR
2
1
0 . . . 0
0
R
T
2
R
2
SNR
2
2
. . . 0
.
.
.
.
.
.
.
.
.
.
.
.
0 0 . . .
R
T
K
R
K
SNR
2
K
_
_
, (3.43)
and
c
= A
T
r
c
. (3.44)
From linear algebra, the solution to Eq. (3.42) is
w
c
= [A
T
A+]
1
c
. (3.45)
It can be seen from Eq. (3.30) that in the absence of noise, the minimization
problem reduces to the minimization of the error in the approximation of the lter
r
c
() (or equivalently r
c
). However, such minimization can produce a solution that
37
is highly uctuating as a function of ; therefore, it is desirable to include a regular-
ization term that penalizes non-smoothness in the approximation [10]. As such,
we include a regularization term, |QAw
c
|
2
, in the mean-square error, so that the
regularized version of Eq.(3.30) becomes
L
1
_
|r
c
Aw
c
|
2
+w
T
c
w
c
+ |QAw
c
|
2
, (3.46)
where Q is a regularization matrix based on the Laplacian operator. Q was dened
in Eq. 3.18, and is the corresponding weight for the penalization term and can
be adjusted according to the desired level of roughness penalization [10]. We nally
obtain an expression for the weights that minimize (3.46):
w
c
= [A
T
A++ Q
T
A
T
AQ]
1
c
. (3.47)
A owchart summarizing the entire algorithm is shown in Fig. 3.9.
3.4 Application of noise-modied ST algorithm to
QDIPs
In this section, we show how the noise can degrade the performance of the ST al-
gorithm, and how the noise-modied ST algorithm performs, by applying the algo-
rithms to QDIPs. Peak-normalized responsivity of a QDIP device manufactured at
the CHTM is shown in Fig. 3.10(a) for certain bias voltages, and an example ap-
proximation of desired band-specic lter centered at 9.5 m by using the weighted
combination of these responsivities is shown in Fig. 3.10(b).
In this section we apply the noise-modied spectral-tuning (NMST) algorithm to
QDIPs that have been recently fabricated. The QDIP device used in our application
has 300 m-diameter aperture. This relatively large aperture area was chosen since
it yields a large photocurrent.
38
3.4.1 Experiment and measurement process
The spectral response of the QDIP detector was measured at a temperature of 50 K
by using a Fourier transform infrared (FTIR) spectrometer and was averaged over 64
measurements for each bias voltage, [V
1
, ..., V
17
] =[4, 3.75, ..., 2, 1.75, 2.25, ...,3.5]
V. The set of responses were obtained in wavenumber domain ranging from 500
to 5000 cm
1
, corresponding to 2 m 20 m. The response were re-sampled
to t in linear wavelength grid, with 0.01-m intervals in the range 2.51 m to
11.00 m. The parameters L and , used in the algorithm, are therefore 850
and 8.5 m, respectively. The spectral response was also normalized so that the
peak value is equal to unity, and the resulting peak-normalized spectral responses,
denoted by f
k
(), were obtained. The average total current, y
k
, and average dark
current, d
k
, of the detector for the same set of voltages were also measured. The
average photocurrents, y
p,k
, under illumination from a black-body source, were then
computed by taking the dierence between the measured total current and the dark
current. In total 30 and 110 measurements were used for the average total and dark
current response, respectively. The standard deviation of the current noise,
N,k
,
was also estimated empirically from the dark-current measurements.
Next, the bias-dependent peak responsivity, R
p
k
, of the device was calculated by
using the relation [52, 54]
y
p,k
= A
e
_

max
min
R
p
k
f
k
()g(, Z)d. (3.48)
In our devices, the eective source-to-detector area was found to be A
e
= 10
6
cm
2
.
The responsivity of the detector can now be obtained by multiplying the peak-
responsivity with the peak-normalized spectral response
R
k
() = R
p
k
f
k
(). (3.49)
We dene the transmittance of the source-to-detector optical system, or the rel-
39
ative spectrum (with respect to a black-body source), as
G(, Z) =
g(, Z)
W(, Z)
, (3.50)
where, W(, Z) = [2hc
2
/
5
]/[exp(hc/k
B
Z) 1] (W cm
2
m
1
) is the black-body
spectral density determined by Plancks law at the source temperature Z (h is
Plancks constant, k
B
is Boltzmann constant, and c is the speed of light). One
can pick either g(), the total irradiance, or G(), the transmittance, as the spec-
trum to be reconstructed, since one can be calculated from the other (as long as
the source temperature is known). In our application, we consider two cases: (1)
Placing no lter in front of the detector so that G() 1, or equivalently, g(, Z) is
the black-body irradiance at the temperature Z; and (2) placing a 3-mm polystyrene
lter in front of the detector so that G() is the transmittance of the lter. In our
experiment, the black-body source incident on the detector was at room temperature
(300 K).
The bias-dependent total current prole, y
k
, k = 1, ..., K = 17, with and without
the 3-mm-polystyrene lter in the view of the detector, and the dark-current prole,
d
k
, and its standard deviation are all shown in Fig. 3.11. These were measured at the
detector temperature of 50 K. The peak-normalized spectral responses of the device,
f
k
(), measured at the same temperature for the prescribed set of bias voltages,
are shown in Fig. 3.10(a). It can be seen in the gure that the peak location and
the shape of the spectral response change as the bias voltage is varied. In addition
to the change in shape of the responsivity, the location of the main peak covers
the ranges 5.3 m 5.8 m (corresponding to low bias voltages), 7.7 m 7.9 m
(corresponding to high positive bias voltages), and 9.5 m 10.0 m (corresponding
to high negative bias voltages). As it turns out, the bias-dependent change in shape
of the responsivity yields sucient diversity so that the reconstruction algorithm
results are satisfactory over a wide spectral range between 5m-10m. Also note
that the FWHM of the main peaks of the response change from 1.5 m (at 4V) to
40
2 m (at -2V). The narrower the FWHM of the responsivities, the better can the
algorithm approximate narrower responsivities r
c
.
For the devices considered, much of the range of the spectral tuning lays in the
atmospheric absorbtion region; however, we will ignore this limitation in this paper
since our focus is to prove the concept of algorithmic spectral tuning. We have
recently demonstrated QDIPs with peaks of the bias-dependent spectral responses
spanning the range 8 12 m.
3.4.2 Experiments with simulated noise
To demonstrate the eect of noisy data on the old version of the algorithm [10] (which
does not account for the noise), we calculated the weights, w
c
, according to Eq. (3.45)
with the noise variances,
2
N,k
, or equivalently 1/SNR
2
k
, set to zero in Eq. (3.43). We
then simply added noise (Poisson-distributed random variables [25] with specied
variances) to the photocurrents before using them in the weighted superposition. In
order to demonstrate the eectiveness of the noise-modied algorithm in mitigating
noise, we used Eq. (3.45), with the actual noise parameters, to arrive at the noise-
modied weights. The average value of the SNR, as dened in Eq. (A5) and taken
over the 17 bias voltages, was calculated for the two sources used; these SNRs are
226 for the unltered black-body source, and 163 for the 3-mm polystyrene ltered
black-body source. These values correspond to our dark-current measurements at a
detector temperature of 50 K. The dierence in the SNR, which is due to the change
in the signal (as the lter was introduced), was compensated by a multiplicative
factor of 0.72 in order to obtain same level of SNR for both cases.
41
Tradeo between spectral resolution and approximation error
We applied the algorithm to obtain spectral tuning and consequently the reconstruc-
tion of the spectra of the two dierent sources. In each case, we performed spectral
reconstruction of the sources from the photocurrents by repeatedly applying the al-
gorithm, with a triangular tuning responsivity and a specied desired FWHM, while
varying center wavelengths of the triangular responsivity from the collection 5.0 m,
4.2 m, . . ., 9.8 m, 10.0 m. The spectral reconstruction was repeated for dierent
SNRs, ranging from 0.02 (low) to 20000 (high), and for dierent desired FWHMs of
the triangular responsivity in the range 0.5 m (narrow) to 2.5 m (wide).
Representative spectral reconstructions are shown in Fig. 3.12 for a narrow tri-
angular responsivity with a FWHM of 0.5 m, and in Fig. 3.13 for a wide desired
responsivity of 1.5 m. Included in these gures are: (i) the true spectral irradiance
viewed by the detector normalized by the black-body spectrum (solid line); (ii) spec-
tral reconstruction of the normalized source spectrum, using ideal triangular lters of
the desired FWHM, shown as ; this is used as a benchmark for the performance
as there is no error in approximating the tuning lters; (iii) spectral reconstruction
of the normalized source spectrum using the old spectral-tuning algorithm, shown
as ; and (iv) spectral reconstruction of the normalized spectrum using the new,
noise-modied spectral-tuning algorithm, shown as +. The average SNR used in
generating Fig. 3.12 is 100. For this level of SNR, we see that the old algorithm
is aected severely by the presence of noise in the data, while the new algorithm
performs reasonably well despite the noise. Spectral reconstructions with higher
SNRs (> 2000) were also obtained (results not shown); in this case, both old and
noise-modied algorithms performs well yielding a reconstruction that is very close
to the ideal reconstruction. High SNR levels are achievable at very low detector
temperatures, or with other noise-reduction techniques such as time-averaging of the
photocurrent, which we discuss in the next subsection.
42
The results in Fig. 3.13 show a slightly reduced overall error in the spectral recon-
struction at the expense of degradation in spectral resolution. Intuitively, achieving
a high spectral resolution requires the algorithm to generate weights with large mag-
nitudes, possibly with varying signs, as the algorithm attempts to meet the challenge
of approximating a narrow lter using the relatively broad bias-dependent re-
sponsivities of the QDIP. This, in turn, results in accentuated noise accumulation as
the magnitudes of the weights increase. Thus, there is a fundamental tradeo be-
tween synthesized spectral resolution and robustness to noise. Indeed, calculations
conrm that there is a strong trend of increase in the sum of absolute values of the
weights (over all bias voltages) as the desired FWHM is decreased. For example, in
the case of the old algorithm with a triangular lter centered at
c
= 6.0 m, the
sum of absolute values of the weights takes the values of 55751, 62666 and 246340,
for FWHM values of 2.5, 1.5 and 0.5 m, respectively, which directly translates
to an increase in noise accumulation in the weighted superposition as the FWHM
is reduced. In contrast, the noise-modied algorithm shows a similar trend in the
magnitudes of the weights only at high photocurrent SNRs (> 200), but the trend
weakens and eventually disappears as the SNR decreases. For example, for an SNR
value of 0.2, the noise-modied algorithm renders the values of 4604, 4897 and 4763,
for the sum of the absolute values of the weights, for FWHM values of 2.5, 1.5 and
0.5 m, respectively. The decrease in the weights magnitudes in the noise-modied
algorithm is a direct consequence of the algorithms attempt to combat noise. The
sample standard deviation of the weights, which is another measure of the magnitude
of the weights, also shows a similar trend as the sum of absolute values of the weights
(results not shown).
43
Role of photocurrent SNR
For each SNR level, the tuning algorithm was applied 1000 times, with the noise
added to the photocurrent varying randomly from trial to trial. The errors of the
spectral reconstruction, normalized with respect to the true value of the target spec-
trum, were then calculated and averaged over the 1000 trials and over the construc-
tion wavelengths. We then took the square root of this empirical error to obtain the
average normalized root-mean-square error (NRMSE). This gives an average normal-
ized error for each SNR for a given FWHM of the triangular responsivity, as shown
in Figs. 3.14(a) and 3.14(b). In these gures, we present results for the case of recon-
structing the black-body spectrum for: (a) narrow (FWHM=0.5 and 1.0 m) and
(b) wide (FWHM=1.5 and 2.0 m) desired spectral resolutions. The performance of
the algorithm was studied for both the old (thin curves) as well as the noise-modied
algorithm (thick curves). Note that the maximum value of the NRMSE is 1 (i.e.,
100%) for the noise-modied algorithm, since the weights approach zero as the noise
power increases.
We observe that the average NRMSE is signicantly less for the noise-modied
algorithm than that for the old algorithm. For example, for an average SNR level
of 2, the NRMSE is reduced by 18dB. In particular, the results shown in Fig. 3.14
demonstrate that the noise-modied algorithm requires much less photocurrent SNR
than the original algorithm. For example, for the case when the FWHM is 1.0 m,
the noise-modied algorithm can attain average error of 20%, for an average SNR
level of 63, while the old algorithm will require an average SNR level of 316 (a 5-time
increase in SNR) to achieve the same performance. For the case when FWHM is
1.5 m, the required average SNR levels for the same error level are 210 and 16,
respectively for the old and the noise-modied algorithms.
We also observe in these gures that the average NRMSE in the reconstruction
44
decreases as the FWHM is increased, signifying the tradeo, as described earlier, be-
tween spectral resolution and robustness to noise. For a xed operating temperature,
the SNR can be increased by either increasing the integration time of the detector
or by repeating the reconstruction and averaging the results. Note that averaging
the results over M photocurrent current samples increases the SNR eectively by
factor of M
1/2
. However, such averaging will slow the speed of the reconstruction.
The noise-modied algorithm, albeit, will help reduce this speed overhead by a factor
ranging from 25 to 175 for the devices considered, depending on the desired accuracy.
3.5 Multispectral-based target classication
We used the tuning algorithm to perform multispectral target discrimination based
on 5-band sensing. Table 1 lists the details of the 5-band representation of the two
source spectra considered (the black-body source with and without the polystyrene
lm) with the old and noise-modied spectral-tuning algorithms. Individual bands
correspond to triangular lters with a FWHM of 1 m, centered at various wave-
lengths. The SNR is assumed as 200. The rst column of the table shows the centers
of the ve bands, ranging from 5 m to 9 m. The second column depicts the actual
values of the target spectrum, sampled at the center wavelengths, while the third
column is the ideal reconstruction using the triangular lters with a FWHM of 1 m.
The fourth and fth columns are the reconstruction results using the old algorithm
and the noise-modied algorithm, labeled as OLD and NEW, respectively. Sepa-
rability between the two target spectra can be examined by comparing the Euclidean
distance between the 5-band reconstruction (columns four or ve in Table 1) and the
actual samples of the spectra (column 2 in Table 1) in the 5-dimensional multispectral
feature space, comprising the reconstructed outputs of each band. More precisely,
45
we have adopted a simple performance metric, dened by [63]
d
nor
=
d
0
d
1
0.5(d
0
+ d
1
)
, (3.51)
where d
1
is the distance between the 5-band reconstructed 5-band feature vector
and the true spectrum samples, and d
0
is the distance between the reconstructed
5-band feature vector and the wrong spectrum. Note that the best separation is
achieved when d
nor
= 2, which corresponds to d
1
= 0. On the other hand, the worst
performance occurs when d
nor
= 0), which corresponds to the case when separation
is impossible under this simple distance-based metric.
Tables 1(a) and 1(b) correspond, respectively, to the cases for which the true
hypothesis is the unltered black-body source and the 3-mm polystyrene-ltered
black-body source. The results are for a noisy case in which SNR = 20 (low SNR).
We can see that the noise-modied algorithm results in much less distance to the
true spectrum, and hence, larger d
nor
. For example, as shown in Table 1(a), the
noise-modied algorithm yields d
nor
= 0.9311, which is much closer to that of the
ideal reconstruction (for which d
nor
= 1.9075) than that of the old algorithm (for
which d
nor
= 0.0217). These results show that under low SNR conditions, only
the noise-modied algorithm can be used for successful target discrimination, and
the separation oered by the noise-modied algorithm performs is quite close to an
ideal multispectral sensor. When the noise power is small (e.g., when the SNR is
greater than 2000), the reconstruction results for the old and new algorithm become
comparable (results not shown here).
3.6 Conclusions
We have developed a spectral-tuning algorithm that is based on exploiting the pres-
ence of spectral overlap and spectral diversity in the responsivities of a collection
46
of detectors to synthesize the output of a desired, arbitrary band. This work is an
important generalization of an earlier version of the algorithm to accommodate and
compensate for noise in the outputs of the detectors. We have applied the algorithm
to quantum-dot mid-infrared photodetectors (QDIPs) developed by our group, and
have shown approximate, continuous spectral-tuning for two dierent source spectra,
viz., a black-body source with and without 3-mm polystyrene lter, in the range 5
m 10 m. As a measure of performance, we used the normalized root-mean-square
error as a function of dierent noise levels and desired tuning resolutions. We have
shown that the SNR requirements for the noise-modied algorithm are signicantly
reduced when compared to the old algorithm, which did not accommodate photocur-
rent noise. This promises robustness to photocurrent noise with certain limitations
depending on the spectral diversity in the detectors responsivities. Also, as the de-
sired spectral resolution of the tuning is reduced, the required SNR becomes less.
Therefore, as the desired width of the tuning lter is increased, which is the case
in many wide-band applications, the algorithm becomes more robust to noise and
the overall tuning error decreases. This shows a fundamental trade-o between the
resolution of spectral tuning and robustness to noise.
47
(a)
(b)
Figure 3.1: Bias-dependent spectral response of (a)QDIP 1198 (b)QDIP 1199, for 21
dierent bias voltages. The devices are manufactured at the CHTM.
48
Figure 3.2: Examples of three (narrow, medium and coarse) approximated (nor-
malized) responsivity by the ST algorithm. QDIP 1199 spectra are used as basis
spectra.
49
(a)
(b)
Figure 3.3: Attained post-processing spectral resolution (FWHM) as a function of
desired center wavelength (tuning parameter), and spectral bandwidth (linewidth-
control parameter) for (a)QDIP 1198 (b)QDIP 1199.
50
(a)
(b)
Figure 3.4: Approximation of a desired triangular responsivity with FWHM of 0.5m
as a function of desired center frequency for the two QDIPs. A post-processing
FWHM< 1.0m can be achieved for most of the tuning parameter values especially
for the QDIP 1199.
51
4 5 6 7 8 9 10
0
10
20
30
40
50
60
70
80
90
100
WAVELENGTH, m
P
O
W
E
R

S
P
E
C
T
R
U
M

(
W
/
c
m
1
)
APPROXIMATION
(a)
4 5 6 7 8 9 10
0
10
20
30
40
50
60
70
80
90
100
WAVELENGTH (m)
P
O
W
E
R

S
P
E
C
T
R
U
M

(
W
/
c
m
1
)
APPROXIMATION
(b)
Figure 3.5: Spectral tuning results: Reconstruction of unltered blackbody source
spectrum by using approximated triangular bands with (a) FWHM = 0.5m, (b)
FWHM = 0.25m.
52
4 5 6 7 8 9 10
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
WAVELENGTH (m)
P
O
W
E
R

S
P
E
C
T
R
U
M

(
W
/
c
m
1
)
APPROXIMATION
(a)
4 5 6 7 8 9 10
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
WAVELENGTH (m)
P
O
W
E
R

S
P
E
C
T
R
U
M

(
W
/
c
m
1
)
APPROXIMATION
(b)
Figure 3.6: Spectral tuning results: Reconstruction of 3mm polystyrene-ltered scene
spectrum by using approximated triangular bands with (a) FWHM = 0.5m, (b)
FWHM = 0.25m.
53
3 4 5 6 7 8 9 10
20
10
0
10
20
30
40
WAVELENGTH, (m)
R
E
S
P
O
N
S
E

(
W
/
c
m
1
)
DESIRED FILTER RESPONSE
APPROXIMATION
MEDIAN FILTERED APPROX.
(a)
3 4 5 6 7 8 9 10
20
0
20
40
60
80
100
120
WAVELENGTH, (m)
R
E
S
P
O
N
S
E

(
W
/
c
m
1
)
APPROXIMATION
(b)
Figure 3.7: (a) Example lter approximation: desired triangular responsivity (lter)
with a desired FWHM of 0.5m and with centers of (a)4.0m (a bad approximation)
and (b)7.65m (a good approximation).
54
3 4 5 6 7 8 9 10
30
20
10
0
10
20
30
40
50
60
70
R
E
S
P
O
N
S
E

(
W
/
c
m
1
)
APPROXIMATION
(a)
3 4 5 6 7 8 9 10
50
0
50
100
150
200
250
WAVELENGTH, (m)
R
E
S
P
O
N
S
E

(
W
/
c
m
1
)
APPROXIMATION
(b)
Figure 3.8: (a) Example lter approximation: desired triangular responsivity (lter)
with a desired FWHM of 0.25m and with centers of (a)4.0m (a bad approximation)
and (b)7.65m (a good approximation).
55
Figure 3.9: Flowchart of the entire noise-modied spectral-tuning (NMST) algo-
rithm.
56
3 4 5 6 7 8 9 10 11
0
0.2
0.4
0.6
0.8
1
WAVELENEGTH, m
P
E
A
K
N
O
R
M
A
L
I
Z
E
D

R
E
S
P
O
N
S
I
V
I
T
Y
,

f
k
(
)
4.0V
3.75
3.5
3.25
3.0
2.75
2.5
2.25
2.0
1.75
2.0
2.25
2.5
2.75
3.0
3.25
3.5
2V
3.5V
4V
(a)
(b)
Figure 3.10: (a) Peak-normalized spectral response of the QDIP detector in Fig. 2.2
for dierent bias-voltages. (b) Example desired triangular lter approximation with
the center 9.5m and full-width at half-maximum (FWHM) of 0.5m.
57
3 2 1 0 1 2 3 4
10
8
10
7
10
6
10
5
10
4
BIAS VOLTAGE (V.)
C
U
R
R
E
N
T

(
A
.
)
3 2 1 0 1 2 3 4
10
9
10
8
10
7
10
6
y; average total current
d; average dark current
y; average total current
with 3mm polystyrene filter
NOISE STD.
Figure 3.11: Total and dark current proles of the DWELL detector with 300 m-
diameter at 50 K. The and 2 symbols represent the total current with and
without the 3mm polystyrene lter, respectively. The represents the dark current
at 50 K. Inset: Noise standard deviation.
58
5 6 7 8 9 10
2
1
0
1
2
3
4
WAVELENGTH (m)
R
E
L
A
T
I
V
E

S
P
E
C
T
R
U
M
(a)
5 6 7 8 9 10
1.5
1
0.5
0
0.5
1
WAVELENGTH (m)
R
E
L
A
T
I
V
E

S
P
E
C
T
R
U
M
(b)
Figure 3.12: Performance of the algorithm in synthesizing the relative power spec-
trum of (a) black-body source (b) 3-mm polystyrene lter, by using desired respon-
sivity of ideal triangular lters of FWHM of 0.5m, under moderately low noise
(with SNR of 100). The and + symbols represent the original tuning al-
gorithm that does not accommodate the noise and the new noise-modied tuning
algorithm, respectively. The symbols s represent the reconstruction by using
ideal responsivity. Under very low noise (for SNR of more than 2000), the ideal
reconstruction () and algorithms reconstruction (+,) overlap.
59
5 6 7 8 9 10
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
WAVELENGTH (m)
R
E
L
A
T
I
V
E

S
P
E
C
T
R
U
M
(a)
5 6 7 8 9 10
0
0.2
0.4
0.6
0.8
1
1.2
WAVELENGTH (m)
R
E
L
A
T
I
V
E

S
P
E
C
T
R
U
M
(b)
Figure 3.13: Same as Fig.3.12 but with a wider desired triangular responsivity with
FWHM of 1.5 m.
60
0.02 0.2 2.0 20 200 2000 20000
10
2
10
1
10
0
10
1
10
2
10
3
10
4
AVERAGE SNR
A
V
E
R
A
G
E

|
|
N
M
S
E
|
|
2
FWHM=0.5m, OLD
FWHM=0.5m, NEW
FWHM=1.0m, OLD
FWHM=1.0m, NEW
(a)
0.02 0.2 2.0 20 200 2000 20000
10
2
10
1
10
0
10
1
10
2
10
3
10
4
A
V
E
R
A
G
E

|
|
N
M
S
E
|
|
2
FWHM=1.5m, OLD
FWHM=1.5m, NEW
FWHM=2.0m, OLD
FWHM=2.0m, NEW
AVERAGE SNR
(b)
Figure 3.14: Comparison of the original algorithm that does not accommodate noise
(thin curves) and the noise-modied algorithm (thick curves); average normalized
root-mean-square error (NRMSE) vs. the photocurrent SNR, for desired resolution
of (a) narrow FWHMs of 0.5m (dashed curves) and 1m (solid curves) (b) wide
FWHMs of 1.5m (dashed curves) and 2m (solid curves).
61
Table 3.4: Five-band multispectral performance of the algorithm in synthesizing the
relative power spectrum of (a) black-body source and (b) 3-mm polystyrene lter
in the case of moderate noise (SNR = 20) using a synthesized triangular lter with
FWHM=1.0 m.
(a)
c
ACTUAL IDEAL OLD NEW
5 1.0000 1.0180 -5.6348 1.3027
6 1.0000 0.9933 -1.9762 1.0701
7 1.0000 0.9901 12.654 0.8619
8 1.0000 0.9918 -0.2698 0.9624
9 1.0000 0.9943 -1.0783 0.8660
d
1
0 0.0238 13.9514 0.3674
d
0
1.0166 1.0054 14.2562 1.0073
d
nor
2 1.9075 0.0217 0.9311
(b)
c
ACTUAL IDEAL OLD NEW
5 0.7905 0.6962 1.0854 0.7258
6 0.6210 0.5533 -0.6804 0.6115
7 0.3315 0.3224 2.0281 0.3962
8 0.5330 0.4249 0.1977 0.3902
9 0.5745 0.4480 0.4894 0.2898
d
1
0 0.2051 2.1861 0.3315
d
0
1.0166 1.1789 2.1892 1.2112
d
nor
2 1.4072 0.0013 1.1405
62
Chapter 4
Chromatic Nonuniformity
Correction for Detectors
In this chapter, we present the nonuniformity (NU) noise and we model it. We
then show the chromatic (wavelength-dependent) nature of NU noise and apply the
spectral-tuning algorithm to mitigate chromatic nonuniformity (CNU).
4.1 Nonuniformity model
An infrared detector array, also called focal plane array (FPA), consists of many
detectors, which constitute the pixels. Each photo-detector element of the FPA
converts the photonic energy into electrical energy by converting the irradiance in-
formation into voltage or current information. As we presented in Chapter 3, the
detector output can be modeled as a function of the target spectral radiance, which,
in turn, is a function of target temperature. The main goal of infrared detector sys-
tems is to determine the temperature of the target; if it is a spectral sensor, the goal
is to determine the spectral radiance of the target. Let us assume that the output
63
Chapter 4. Chromatic Nonuniformity Correction for Detectors
of any detector, denoted by y, can be expressed as a function of the temperature Z
of the target
y = f(Z) (4.1)
Unfortunately, such input-output characteristics of detectors vary from detector to
detector in the array, even though they are fabricated under identical conditions. Al-
though extreme care is taken to manufacture detectors that have similar properties
(such as response transfer function, spectral response characteristics, etc.) each de-
tector has dierent characteristics, i.e., dierent function f. Even the characteristics
of the same detector might drift over time because of changes in external conditions
and it generally cannot be modeled accurately in a deterministic fashion; therefore, f
has a random component. Figure 4.1(a) illustrates a snapshot of this; it shows pho-
tocurrent response of dierent detectors versus the temperature of the scene they
see. Fig.4.2(a) shows an example infrared image which has NU [59]. Knowing the
detectors precise characteristics, i.e., the function f, would ideally solve the spatial
NU problem since it enables us to extract the real irradiance, viz., the real image,
from the measured FPA readout. The process of exposing the detectors with a target
source of given temperature and measuring their output, and obtaining the input-
output map for certain temperatures is called calibration, and it solves the NU
problem mostly. Since f is usually a smooth function of Z, it is not necessary to
obtain the calibration for many temperature levels. As commonly practiced, the
calibration is done usually at two temperatures; the input-output relation is approx-
imated using a linear-response model, which is characterized by a gain and oset
parameter for the photodetector. For detectors having smooth-enough function,
we can write
y = aZ + b,
where Z is the true irradiance, a is the gain, b is the oset, and y is the observed
quantity (usually, current or voltage). Figure 4.1(b) illustrates two-point approxima-
64
Figure 4.1: (a)Response of dierent detectors to incident temperature, (b)Two-point
(linear) approximation of the response which characterizes the response with gain
and oset.
tions of detectors response in (a). If the calibration temperatures, which are known
and set to be in the range of operation of the camera are denoted with Z
1
and Z
2
,
and if the corresponding detector outputs are denoted with y
1
, y
2
, respectively, then
the gain and oset values can be calculated for each detector as
a =
y
2
y
1
Z
2
Z
1
and b = Y
1
y
2
y
1
Z
2
Z
1
Z
1
. (4.2)
They can later be used to estimate the true temperature, Z, given the response y,
by using the two-point model equation:
y = aZ + b. (4.3)
Fig.4.2(b) shows a two-point calibrated version of the image frame in (a) [59].
65
Figure 4.2: (a) Image frame corrupted with NU. (b) Image frame after two-point
(gain and oset) hard calibration is applied.
4.2 Chromatic nonuniformity and the ST algo-
rithm
From the ST algorithm, we have shown that an arbitrary desired lter band output
can be approximated by dierent detector outputs, that is
Y
c
(Z) =
K
k=1
w
c,k
Y
k
(Z), (4.4)
where Z denotes the source temperature. To proceed, we would like to start by
explicitly writing linear detector model that represents the relation between the total
output current and the spectral response of the detector and the scene.
Y
i,j,k
= A
i,j
_

max
min
R
i,j,k
()G(Z
i,j
)W(Z
i,j
)d +y
d,{i,j,k}
+N
{i,j,k}
. (4.5)
Here i and j are row and column indices in the detector array denoting the pixel
location; k is the index for detector T
k
. In the particular case of QDIPs k denotes
66
the index for the bias-voltage, V
k
. Y
i,j,k
is the total current output of the pixel
i, j for the detector T
k
(or bias voltage V
k
). A
i,j
(cm
2
) is the eective source-to-
detector area of the pixel i, j, it includes all the coupling factors; R
k
() is detector
response; Z
i,j
(K.) is the temperature of the pixel i, j; G(Z
i,j
) is the relative
spectral emittance (w.r.t blackbody emittance) of the source (unitless); W(Z
i,j
) is
the blackbody radiant emittance at temperature Z
i,j
seen by the pixel i, j, so
that the product GW is the eective radiant emittance of the source seen by pixel
i, j; y
d,{i,j,k}
is detectors d.c. component of the dark current, which depends on
the detector temperature Z
d
(which is assumed to be independent from the source
temperature); N
i,j
is the random uctuation of this dark current. We clearly see
that the output current depends on the source spectral irradiance, as well as the
detectors response, which are wavelength-dependent. Therefore, NU noise shows
itself as a wavelength-dependent (chromatic) noise. Fluctuations and changes in the
detectors response or the source/scene irradiance requires NUC maps to be adjusted
accordingly. Then, a chromatic nonuniformity correction (CNUC) would be achieved.
Dropping the indexes for pixels for convenience, let us assume that for a xed
pixel and for each detector T
k
, some nonuniformity correction was performed. As
a result, let us assume that a two-point correction map was obtained in order to
approximate the true scene temperature
Z
k
=
Y
k
b
k
a
k
. (4.6)
In the particular case of a QDIP, T
k
represent the QDIP under bias voltage V
k
. If we
had a detector with arbitrary response r
c
() and if we had done two-point correction,
we would obtain
Z
c
=
Y
c
b
c
a
c
. (4.7)
From Eq. (4.4), which we can approximate the current response with weighted su-
67
perposition of current response of individual detectors, we can write
Z
c

[
K
k=1
w
c,k
Y
k
] b
c
a
c
We can substitute for Y
k
from Eq. 4.6, and obtain
Z
c

[
K
k=1
w
c,k
(a
k
Z
k
+ b
k
)] b
c
a
c
. (4.8)
Even if we change the detector, since we dont change the source characteristics
(the temperature and the spectral content for a xed pixel), the two-point correction
maps for dierent types of detectors should lead to the same source temperature for
a pixel. Therefore,
Z Z
c
Z
k
. (4.9)
Using this, we can rearrange Eq. 4.8 as
Z
[
K
k=1
w
c,k
a
k
]Z + [
K
k=1
w
c,k
b
k
] b
c
a
c
,
which can be rearranged as
[
K
k=1
w
c,k
a
k
]Z + [
K
k=1
w
c,k
b
k
] a
c
Z +b
c
,
from which we can obtain
a
c
[
K
k=1
w
c,k
a
k
], (4.10)
b
c
[
K
k=1
w
c,k
b
k
]. (4.11)
(4.12)
This means that an approximation of the amount of target temperature sensed by
a spectral band with an arbitrary shape can be obtained by using the weighted
68
superposition of the gain and oset values that approximate the temperature for
individual detectors, whose spectral response is used to approximate that spectral
band. The weights to approximate the desired band and to obtain the corresponding
gain and oset map are the same weights.
Based on this, we propose the following chromatic nonuniformity correction al-
gorithm:
Description of the proposed chromatic NUC algorithm
A-priori procedure:
1) Determine the bias voltages to operate the detector V
k
, k = 1, ..., K,
2) Measure the total current response of the detector Y
k
, k = 1, ..., K,
3) Measure the dark current and obtain the statistics of the dark current, i.e., the
standard deviation,
N,k
and the mean (d.c. component) y
d,k
, based on multiple
measurements,
4) Compute the photocurrent, Y
p,k
, based on (2) and (3),
5) Measure the spectral response of the detector, f
k
(), k = 1, ..., K, and compute
the spectral responsivity R
k
()
During imaging application:
6) Apply the NUC algorithm to the M by N detector array, obtain the gain
a
k
(i, j) and oset b
k
(i, j), and the observation Y
k
(i, j), i = 1, ..., M, j = 1, ..., N, for
each bias voltage V
k
, k = 1, ..., K,
7) Determine the center and shape of the spectral lter to be approximated, r
c
(),
8) Approximate the lter in the MMSE sense by using the spectral-tuning algorithm
and obtain the weight vector w
c
= [w
c,1
, ..., w
c,K
],
9) Compute the weighted average of the gain coecients using the weights, i.e.,
a
c
(i, j) =
K
k=1
w
c,k
a
k
(i, j), and compute the weighted average of oset coecients
b
c
(i, j) =
K
k=1
w
c,k
b
k
(i, j),
10) Compute the estimate of the source temperature, Z
c
(i, j) that corresponds to
69
the desired band r
c
(), by using
Z
c
(i, j) =
Y
c
(i, j) b
c
(i, j)
a
c
(i, j)
. (4.13)
11) Repeat steps 6 10 for dierent lter centers,
c
, c = 1, ..., C, ultimately having
reconstruction in C number of bands. Obtain the temperature prole (or, spectral
prole) of the scene.
4.3 Applications
In Fig. 4.3, we show the normalized root-mean-square error (NRMSE) versus the
temperature after two-point calibration (solid curve) and the two-point chromatic
nonuniformity correction (CNUC) by using the ST algorithm (dashed curve). In (a)
we present high SNR case (SNR=200) and in (b) we present low SNR case (SNR=20).
285 K and 317 K are chosen to be the two calibration temperatures. By denition, the
two-point calibration is has zero error at the calibration temperatures. The estimated
temperature using the ST algorithm is close to the estimates obtained by two-point
calibration for the high SNR case. As the noise level is increased, the performance
of both the two-point calibration and the ST algorithm decrease. However, at the
calibration temperatures, the two-point calibration performs excellent, as expected.
For other temperatures, the ST algorithm result is slightly better, resulting from the
averaging nature of the algorithm as it feel the weighted average of 17 independent
noise components, each coming from 17 dierent detectors T
k
, corresponding to 17
bias voltages, V
k
.
4.3.1 Application to simulated imagery
We apply two-point correction on a simulated 1616 pixel image with simulated gain
and oset NU and temporal noise. We show the eect of CNU on the calibration
70
and the NUC algorithms. We generated an image with known temperature values,
Z
i,j
, in the range 273-333. Then, we simulated the output according to Eq. (4.5),
by assuming a detector with triangular responsivity with FWHM=1m, center at
10.5m and responsivity peak 0.025 A/W. The spatial multiplicative coupling factor,
A
i,j
, had a spatial standard deviation of 10
7
and a mean value of 10
6
, leading
to a gain NU standard deviation of 10%. The additive (oset) NU had spatial
standard deviation of 10% of the mean A
e
Z value. In addition to spatial NU, additive
temporal noise, D
k
with SNR of approximately 100, was added. The d.c. component
of the additive noise, y
d,k
, was xed for a pixel, and was set to be the average
dark-current measurement from QDIP. We see in Fig. 4.4(a) the clean image; in
(b) the image corrupted with simulated NU and temporal noise; in (c) the image
corrected by simulation of a matrix-based two-point NUC algorithm that we have
developed (which will be explained in the next chapter) applied directly by assuming
ideal detector response assumed above and based on calibration via Eq. (4.5) at
temperatures 293 and 313. The source radiance was set to be that of the blackbodys.
The normalized root-mean-square error is 0.0092, or, 0.92%, which corresponds
to an error of about 2.7 K. The residual error from the temporal noise is present
in the corrected image; however, the NU pattern is corrected, and the correction is
especially felt for the patterns 3 and 4 whose temperature values are close to
temperature value of the background. The NUC algorithm was based on a simu-
lated 2-D sub-pixel shift. The result is very close to two point calibration result (not
shown here). In (d) we have shown the result after combining the two-point cali-
bration (293 and 313 K) with the ST algorithm weights in order to approximate the
detector responsivity. In (e) we have shown the result after combining matrix-based
NUC algorithm maps of detectors with ST algorithm weights. In (f) we have shown
NUC result when the source has a dierent spectral radiance (triangular lter with
FWHM=1m and center at 8.0m). The deterious eect of change in the source
irradiance is apparent. Even when we scale this image to its full dynamic range
71
(248.8-255.1 K), we obtain the image in (g), which is highly degraded.
4.4 Conclusions
We have shown that the chromatic NU in QDIP arrays can be mitigated by combining
the spectral-tuning algorithm with any NUC algorithm. We considered a simulated
image based on the QDIPs and applied the ST algorithm with our NUC technique.
We compared the results when the ST algorithm is combined with the chromatic
calibration. The results with our NUC technique are visually almost as good as the
ones obtained by a calibration procedure.
72
280 290 300 310 320 330
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
Temperature, Kelvin
N
R
M
S
E
2PT. CALIBRATION
CNUC
(a)
280 290 300 310 320 330
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
Temperature, Kelvin
N
R
M
S
E
2 PT. CALIBRATION
CNUC
(b)
Figure 4.3: Normalized root-mean-square error (NRMSE) versus the temperature for
(a) high SNR (b) low SNR. The solid curve represents two-point calibration and the
dashed curve represents the two-point CNUC by using spectral-tuning algorithm.
73
IMAGE WITHOUT NU
IMAGE WITH NU
(a) (b)
IMAGE AFTER 2POINT NUC
CHROMATIC 2PT. CALIBRATION
CHROMATIC 2PT NUC ALGORITHM
(c) (d) (e)
NUC APPLIED WITH UNMATCHING SPECTRAL MAP
(f) (g)
Figure 4.4: Eect of spectral radiance on NUC. (a) Simulated image without NU. The
perimeter temperature level is simulated to be 273 K, background 303 K, numbers
1-6 represent 273, 283, 293, 313, 323, 333 K, respectively. The average level of
temperature is 296 K for the image. (b) Image with simulated NU, plus temporal
noise. The gain and oset NU both have standard deviation 10% (of mean values)
and the additive temporal noise corresponds to an SNR of 100. Image after (c) two-
point NUC, (d) combining two-point calibration maps of detectors with ST algorithm
weights, (e) combining matrix-based NUC algorithm maps of detectors with ST
algorithm weights. The temperatures used for calibration were 293 and 313 K. (f)
NUC result when the source spectral radiance is dierent than assumed. (g) Image
in (f) scaled to its full dynamic range.
74
Chapter 5
Matrix-based Nonuniformity
Correction Algorithm
In this chapter, we develop a matrix-based version of a recently reported algebraic
nonuniformity correction algorithm developed by Ratli et al.[41, 43, 44]
5.1 One-point oset correction model
Assume that an FPA has MxN detectors, where M is the number of rows and N
is the number of columns. For nth frame in the video sequence 1 and for the i, jth
pixel, we can write
y
n
(i, j) = a(i, j)Z
n
(i, j) + b(i, j) (5.1)
Note that in the above model we have not included the temporal noise. Also note
that the gain and biases in each pixel, a(i, j) and b(i, j), do not change within the
sequence 1.
If we assume that the gain a is constant among all detectors in Eq. (5.1), then we
75
Chapter 5. Matrix-based Nonuniformity Correction Algorithm
can set it to unity without loss of generality. Hence, the two-point xed pattern noise
model reduces to one-point model that only includes the oset NU. This assumption
is commonly practiced and it is due to the observation that in many operational
conditions, the oset NU typically dominates gain NU. With these simplifying as-
sumptions, our model reduces to
y
n
(i, j) = Z
n
(i, j) +b(i, j), i = 1, ..., M, j = 1, ..., N. (5.2)
Observe that in the above equation, there are MN equations and the only known
quantity are the measurements y
n
(i, j). Thus, there are a total of 2MN unknowns.
In one-point calibration, we rst apply a known constant illumination Z on all
pixels and calculate the bias response b(i, j) = Z
n
(i, j) y
n
(i, j). To do this, the
cameras operation is halted, a black-body source is inserted in front of the cameras
eld-of-view, and the b(i, j) are calculated for all (i, j). These calculated biases are
subtracted from the imagery in the sequence accordingly yielding bias correction.
However, if calibration is to be avoided, the general question is how can Eq. (5.2) be
solved to get Z
n
(i, j)s without having any a-priori information about b(i, j)s. Realiz-
ing that we have 2MN unknowns and only MN equations, to solve for the Z
n
(i, j)s,
we need another set of equations. Such new set can be y
m
(i, j) = L(Z
n
+ b), where
L is some new linear transformation of Z
n
(i, j) and b(i, j), i = 1, ..., M, j = 1, ..., N.
The transformation (or operator) L is assumed to represent a physical process, and
y
m
(i, j) are the corresponding results that we can physically observe. An exam-
ple of such physical process is translational movement of the camera, which yields
global translational motion of the scene. In the next section, we will exploit this
representation to yield estimates of the unknown biases.
76
5.1.1 Modeling the two-dimensional global shift
We will assume that the motion between the frames n and m, where m > n, is
a simple translational shift. Usually IR-cameras are operated at rates of 30 or 60
frames/sec. Assuming a realistic motion of the user, the shifts between frames are
usually sub-pixel shifts. We will discuss this case in the next sub-section and the
super-pixel case is discussed in 5.1.1. In our paper, we assume that the algorithm is
provided with frames that exhibit global translational motion and that we know the
amount of vertical and horizontal translation. This knowledge of motion is obtained
through the use of standard motion estimation algorithms. In this paper we use the
gradient-based shift estimation algorithm reported in [64, 65].
Sub-pixel shift model
The algebraic shift model was introduced by Ratli et al. [43]. We begin with
the frame n and assume (without loss of generality) downward-right motion of the
camera. Denoting the vertical shift between two frames with , and horizontal shift
with , we can express the shifted frame m as
y
m
(i, j) = (1 )(1 )Z
n
(i, j) + (1 )Z
n
(i + 1, j)
+ (1 )Z
n
(i, j + 1) + Z
n
(i + 1, j + 1) + b(i, j), (5.3)
for i = 1, ..., M 1 and j = 1, ..., N 1. Therefore we obtain (M 1)(N 1)
additional equations in addition to the MN equations in Eq. (5.2), which yields a
total of 2MN (M +N 1) equations. Recall that we had 2MN known quantities,
namely, the y
n
(i, j)s and the y
m
(i, j)s. Also, when we reach the boundaries, where
i = M, or j = N, we cannot use Eq. (5.3), and we are therefore in need of M +N1
additional equations. We can overcome this situation in two ways. (1) The non-
radiometric way and (2) the radiometric way. In the former, we impose a constraint
77
on the bias values. For example, by relying on the physical fact that the average bias
over all detector elements is approximately is zero, we may introduce the constraint
that the bias at the end of each row is the sum of the negative of all the remaining
biases in the row. A similar constraint can be applied to biases at the bottom of each
column. This constraint is non-radiometric since we do not utilize any radiometric
calibration, but make a physical, statistical assumption for the biases of the perimeter
pixels. Accordingly, we introduce the following constraints:
b(M, j) =
M1
i=1
b(i, j), j = 1, ..., N 1,
b(i, N) =
N1
j=1
b(i, j), i = 1, ..., M 1, and
b(M, N) =
M1
i=1
N1
j=1
b(i, j). (5.4)
These equations provide us extra M +N 1 equations we need. Thus, by means of
the above constraints we reduce the number of unknown biases by M +N 1, which
results in the same number of equations and unknowns in the system described by
Eqns.5.2 through 5.4. We can therefore solve for the perimeter irradiance, that is
Z
n
(i, j), for i = M and j = N, by means of Eq. (5.2).
For the radiometric mode, we may use the fact that biases on the perimeter of
the array have already been calibrated a priori (see Ratli et al. [43] for details
of the notion of partial perimeter calibration) from radiometric-calibration so that
b(M, j) = b(i, N) = b(M, N) = 0 for i = 1, . . . , M 1 and j = 1, . . . , N 1.
We next discuss the case for which the shifts are general (i.e., not only sub-pixel).
78
The general case: Super-pixel shift model
If the camera motion is suciently fast, there can be global super-pixel shift between
the frames for which > 1 or > 1. In this case Eq. (5.4) must be modied. For
example, if the horizontal shift is = 1.55, and the vertical shift is = 2.70,
then the last 3 rows and last 2 rows are the perimeter pixels. This means, for
the non-radiometric case, the constraint must be applied to the biases b(i, j) for
i = M2, M1, M and j = N 1, N. For the radiometric case, on the other hand,
the radiometrically estimated bias values for these perimeter pixels are needed. We
dene | and | to be the nearest integers, towards zero, to and , respectively.
With this notation, we can write the modied version of Eq. (5.4) as:
b(M|, j) = ... = b(M, j) =
M1
i=1
b(i, j), j = 1, ..., N|1,
b(i, N|) = ... = b(i, N) =
N1
j=1
b(i, j), i = 1, ..., M|1, and
b(M| : M , N| : N) =
M1
i=1
N1
j=1
b(i, j), (5.5)
where (M| : M , N| : N) denotes all pairs (i, j) for i = M|, ..., M and
j = N|, ..., N.
5.1.2 Matrix equation model for the non-radiometric case
The system of equations 5.3-5.5 can be written in matrix form as
Y = AD, (5.6)
where
Y=
_
_
Y
n
Y
m
_
_
, D=
_
_
Z
B
_
_
, A=
_
_
I
MNxMN
R
MNx(M1)(N1)
A
(M1)(N1)xMN
I
(M1)(N1)x(M1)(N1)
_
_
,
79
(5.7)
Y
n
=
_
_
y
n
(1, 1)
.
.
.
y
n
(M, 1)
y
n
(1, 2)
.
.
.
y
n
(M, 2)
.
.
.
y
n
(1, N)
.
.
.
y
n
(M, N)
_
_
MNx1
, Y
m
=
_
_
y
m
(1, 1)
.
.
.
y
m
(M|1, 1)
y
m
(1, 2)
.
.
.
y
m
(M|1, 2)
.
.
.
y
m
(1,N|1)
.
.
.
y
m
(M|1,N|1)
_
_
(M1)(N1)x1
,
Z =
_
_
Z(1, 1)
.
.
.
Z(M, 1)
Z(1, 2)
.
.
.
Z(M, 2)
.
.
.
Z(1, N)
.
.
.
Z(M, N)
_
_
MNx1
, and B=
_
_
b(1, 1)
.
.
.
b(M|1, 1)
b(1, 2)
.
.
.
b(M|1, 2)
.
.
.
b(1,N|1)
.
.
.
b(M|1,N|1)
_
_
(M1)(N1)x1
.
80
In Eq. (5.7), A is the weight matrix that governs the shift estimation and R is the
appropriate matrix that governs the zero-mean constraint. Dening
= 1 and
= 1 , these matrices can be recast as

A=
_
0 ... 0 0
0 ... 0 0 0 0 0 0 0 0
0
0 ... 0 0
0 ... 0 0 0 0 0 0 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 ... 0
0 ... 0 0
0 0 0 0 0 0
.
.
.
.
.
.
.
.
.
0 0 0 0 0 0
0 ... 0 0
0 ... 0 0
0 0 0 0 0 0 0
0 ... 0 0
0 ... 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0 0 0 0 0 0 ... 0
0 ... 0 0
N1
row
blocks
of
width
M1
(5.8)
81
The matrix that governs the constraint for non-radiometric mode is
R =
_
_
1 ... 0 0 ... 0 0 ... 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 ... 1 0 ... 0 0 ... 0
1 ... 1 0 ... 0 0 ... 0

1 ... 1 0 ... 0 0 ... 0
0 ... 0 1 ... 0 0 ... 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0
.
.
.
0 ... 0
0 ... 0 0 ... 1 0 ... 0
0 ... 0 1 ... 1 0 ... 0

0 ... 0 1 ... 1 0 ... 0
.
.
.
.
.
.
.
.
.
.
.
.
0 ... 0 0 ... 0 1 ... 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 ... 0 0 ... 0 0 ... 1
0 ... 0 0 ... 0 1 ... 1

0 ... 0 0 ... 0 1 ... 1
1 ... 0 1 ... 0 1 ... 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 ... 1 0 ... 1 0 ... 1
1 ... 1 1 ... 1 1 ... 1

1 ... 1 1 ... 1 1 ... 1
.
.
.
.
.
.
.
.
.
.
.
.
1 ... 0 1 ... 0 1 ... 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 ... 1 0 ... 1 0 ... 1
1 ... 1 1 ... 1 1 ... 1

1 ... 1 1 ... 1 1 ... 1
_
_
.
M1 rows of 1
+1 rows of 1
|
|
this block repeats + 1 times
|
|
N1 column blocks of width M1

(5.9)
82
Matrix equation model for the radiometric case
In the radiometric mode of the algorithm, the bias values in the perimeter pixels
are assumed to be known through radiometric calibration and there is no need for
constraint as in the non-radiometric case discussed earlier. Although in most of
the imaging applications the absolute value of the scene value is not necessary, for
certain applications, obtaining the true values (radiometrically accurate values) of
the scene might be important, such as for temperature measurement systems. The
NUC algorithm that was applied can be combined with dierent partial calibration
mechanisms. One can use dierent partial calibration techniques, depending on the
cost and obstruction requirements (remember that calibration obstruct the camera).
The most useful one can be thought as obtaining the perimeter calibration of the
array[44]. Depending on the chosen perimeter thickness (usually 1-3 pixels of thick-
ness will be enough), the calibration system can be integrated to the perimeter of the
array, while the inner arrays remain unobstructed. The gain and oset parameters
obtained from this perimeter calibration can then be used by the NUC algorithm to
perform correction for the inner arrays. Denoting perimeter thickness with P, let us
assume that the NUC algorithm does not use frame-pairs which exhibit shift values
that are greater than P Therefore, the 2-D motion can be restricted to
< P and < P. (5.10)
Under the above assumptions, the entries that are 1 in the rst (N|1)M rows
and all the non-zero entries in the last (|+ 1)M rows become 0. Aside from this,
the matrix equation model is same as in Eq. (5.6).
5.1.3 Solution to the matrix equation
Clearly, the solution to Eq. (5.6) is

D = A
1
Y, which also trivially minimizes
the error represented by the cost function |Y AD| (which is zero in this case).
83
However, for the non-radiometric case, we introduce an additional penalization term
on the error in the estimation of the bias values. We dene regularized least-squares
cost function as
C(D) = |Y AD|
2
+|FD|
2
, (5.11)
where
F =
_
_
F
Z
0
0 F
B
_
_
, (5.12)
F
Z
is a high-pass (Laplacian) operator matrix that operates on Z, is the corre-
sponding weight, F
B
is a low-pass (smoothing) operator matrix that operates on B,
and is the corresponding weight. Note that by adding the term |FD|
2
to the
cost function, the following types of solutions are penalized: (1) The uctuation in
the estimated irradiance, since we expect irradiance to have low spatial variance, (2)
smoothness in the estimated bias, since we expect bias to have high spatial variance.
The solution that minimizes Eq. (5.11) is
D = (A
T
A+F
T
F)
1
A
T
Y. (5.13)
Thus, by utilizing Eq. (5.4), one can solve for the bias values at the perimeters and
can correct for the future frames (whose indices are in ) by using
z
k
(i, j) = y
k
(i, j) b(i, j), i = 1, ..., M, j = 1, ..., N, k .
The algorithm can be repeated for many set of frames.
To improve the estimation within a frames set, one can repeat the estimation
for many pairs of frames and average the resulting estimated bias. A multi-frame
approach can also be used in which more than two frames are used for estimation. In
the latter case, using all of the possible frame combinations would give the optimal
result; however, it would require extensive computation as the matrices become
prohibitively large .
84
Finally, note that the matrix equations that are written so far are for the right-
downward camera motion. However, the estimation for the motions in the other three
directions can be solved using the same set of equations after rotating the frames
in an appropriate manner, swapping the vertical and horizontal shift parameters
if necessary, applying the algorithm, and rotating back the frames. In Table 4.1,
the pre- and post-operations to be done prior to and after the application of the
algorithm are listed.
Table 5.1: List of pre- and post-operations for dierent directions of shift between
the frames.
Direction Pre-operation Post-operation
> 0, > 0 none none
> 0, < 0 rotate frames counterclockwise 90
, swap(, ) rotate frames clockwise 90
< 0, > 0 rotate frames clockwise 90
, swap(, ) rotate frames counterclockwise 90
< 0, < 0 rotate frames 180
rotate frames 180
Correction with the known gain nonuniformity

For completeness, we provide here the matrix equations that provide the oset uni-
formity correction in case the gain nonuniformity parameters, i.e., a(i, j), are given.
If we include the known gain NU parameters in the model, described by Eq. (5.1),
then there has to be modications in the matrix equation, Eq. 5.6. For the sake of
compactness, we dene
M
= M|1 and N
= N|1. (5.14)
The upper-left matrix I
MNxMN
of A, given by Eq. (5.7), has to be modied to include
the reshaped gain coecient vector [a
1,1
, ..., a
M,N
] on the diagonal. name this matrix
85
G. Also, the lower-left matrix A, given by Eq.(5.1.2), must be modied to include
gain coecients. The new A matrix, modied for two-point correction model, is:
A =
_
_
G
MNxMN
R
MNxM
A
M
xMN
I
M
xM
_
_
, (5.15)
where
G =
_
_
a
1,1
.
.
.
a
M,1
.
.
.
a
1,N
.
.
.
a
M,N
_
_
, (5.16)
the A matrix, which governs the bilinear shift estimation, is modied to include the
gain coecients so that
A =
_
_
a
1,1
a
1,1
0 ... 0 a
1,1
a
1,1
0 ... 0 0 0 0 0 0
0 a
2,1
a
2,1
... 0 0 a
2,1
a
2,1
... 0 0 0 0 0 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 ... a
M
,1
a
M
,1
0 ... 0 a
M
,1
a
M
,1
0 0 0 0 0
.
.
.
.
.
.
.
.
.
0 0 0 0 0 a
1,N
a
1,N
0 ... 0 a
1,N
a
1,N
0 ... 0
0 0 0 0 0 0 a
2,N
a
2,N
... 0 0 a
2,N
a
2,N
... 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0 0 0 0 0 ... a
M
,N
a
M
,N
0 ... 0 a
M
,N
a
M
,N
_
.
(5.17)
It has N
N bi-diagonal Toeplitz blocks of size M
M and itself is bi-diagonal

block Toeplitz. R, the matrix that governs the zero-mean constraint, is the same as
in one-point model.
86
5.2 Two-point gain and oset correction model
Similar to the idea that based on Ratli et al., as proposed in [45], once one-point
(oset) NUC algorithm is applied and the oset-NU-free estimates are obtained, the
one-point NUC algorithm can be applied to the estimates after taking the logarithm
of the estimates. After the one-point NUC, the linear model for the oset-NU-free
detector response can be written as
y
n
(i, j) = a
n
(i, j)Z
n
(i, j), i = 1, ..., M, j = 1, ..., N. (5.18)
If the logarithm of the observation is taken, we obtain
log(y
n
(i, j)) = log(Z
n
(i, j)) + log(a
n
(i, j)), i = 1, ..., M, j = 1, ..., N. (5.19)
If we feed the log-converted observation into the one-point NUC algorithm, we will
obtain estimate of log(a
n
(i, j)) and hence estimate of the true irradiance (tempera-
ture) log(Z
n
(i, j)). After taking the exponential, we obtain the true estimates.
5.3 Applications of the NUC algorithm
In this section, we will provide the results of the matrix-based NUC algorithm.
In the rst subsection we will present the application results of one-point correction
algorithm on frames with simulated NU and simulated shifts based on our linear-shift
model. Application to actual infrared image sequences will follow this subsection.
In later subsections we will present the application results of two-point correction
algorithm on simulated and actual image sequences.
87
5.3.1 One-point NUC: Application to simulated images
The algorithm was rst applied to a simulated frame pair. A 32-by-32 gray level
image, shown in Fig. 5.1(a), was considered. The image pixels have values between 0
and 255. The gray level of the background is 192, and the gray level just around the
letters NUC has value 225. The darkest level of the letters is 76. In Fig. 5.1(b),
a frame simulated with bilinear interpolation model for down-rightward shift with
parameters = 0.70 and = 1.55 is shown. We picked these shift parameters to
simulate both sub-pixel and super-pixel shift in the same example. We then added
bias non-uniformity as shown in Fig. 5.1(c), utilizing zero-mean Gaussian-distributed
random variables with each with a standard deviation of 4. We used the true shift
values in the algorithm, and the results are discussed below.
For the radiometric case, we obtained perfect results as expected without the
need for any regularization, as shown in Fig. 5.1(d). For the non-radiometric case,
our algorithm estimated the biases very well too, as presented in Fig. 5.2, which
should be compared to Fig. 5.1(c). Estimation results with dierent values of the
regularization parameter , for = 10, 2, 0.5 and 0, are shown in Fig. 5.2. We see
from Fig. 5.2(d) that the regularization is necessary for the estimation with non-
radiometric perimeter calibration. As a measure of error, we calculated the standard
deviation of the error in estimating the true irradiance (hence the bias values) as
2.67, 2.63, 5.40 and 124.12 for = 10, 2, 0.5 and 0, respectively. In general, the
estimation results for 2 were the best visually and had the least error standard
deviation, as we see in Fig. 5.2(b). For the case = 0.5, as shown in Fig. 5.2(c),
we can see a pattern in the direction of the camera motion. This is characteristic of
error in predicting the perimeter bias values through the constraint equations.
88
5.3.2 One-point NUC: Application to real images
We applied the algorithm on a 48-by-48 pixel sub-image of video sequence taken
with an IR-camera (InSb FPA). The video sequence had 512 frames, and we picked
frames 487 and 488, as shown in Figs. 5.3(a) and (b), respectively. The estimated
shifts between the frames were 0.276-downward and 0.365-rightward.
We applied our algorithm in both the radiometric and non-radiometric modes.
From our experience on simulated data, we picked = 2 for the non-radiometric
version. In Figs. 5.3(a) and (b) we see the observed frames, corrupted with the
oset NU. The one-pixel-thick perimeter of frame 1 is calibrated. As a bench mark,
we show in Fig. 5.3(c) the estimate of the true scene obtained through radiometric
calibration (namely, a two-point calibration). In Fig. 5.3(d), we show result of our
non-radiometric mode of algorithm. In contrast, in Fig. 5.3(e) we show the results of
the radiometric mode of the algorithm. In both corrected images shown in Fig. 5.3(d)
and (e), the non-uniformity has almost disappeared. However, in Fig. 5.3(d) we see a
pattern in the direction of the motion, which stems from the fact that the error in the
perimeter biases propagate in the error of the inner bias values along the direction of
the motion. The main reason is that the bilinear-interpolation model for shift does
not model real shifts exactly, therefore it leads to errors that further propagate along
the direction of the motion. This eect also appears in other registration-based NUC
algorithms [41, 43] and can be reduced through diversity in the motion direction by
averaging the corrections from multiple frames. Note that this is an extreme test for
the algorithm since we only utilize one pair of frames. The performance is expected
to improve as corrections from dierent pairs of frames are obtained and averaged.
89
5.3.3 Two-point NUC: Application to simulated images
Radiometric mode
To test whether the algorithm works properly, we applied it in the radiometric mode
rst. In the radiometric mode, the perimeter NU parameters (i.e., the gain and
oset values) are assumed to be known. This can realize the case where there is
a xed calibration source on the perimeter pixels which is continuously calibrating
them. Thickness of the perimeter pixels will limit the amount of the shift that can be
used by the NUC algorithm (i.e., 1-pixel thickness only allow sub-pixel shift, where
two-pixel thickness allow shifts up to 2 and so on). Once the perimeter thickness
is set, one can set the algorithm so that it chooses the frame-pairs that have shifts
that satisfy the desired limitation. The thicker the perimeter, the less number of
pixels of the camera will be used for imaging. However, in the radiometric mode, a
perimeter of thickness 1, which corresponds to a sub-pixel global motion, is found to
be enough for almost-perfect NUC, as we will demonstrate in the following examples.
In this example, the two-point correction algorithm is applied on a 32-by-32 pixel test
image frame-pair; the frames are shown in Fig. 5.4 (a) and (b). The second frame
is a simulation of the rst frame with the 2-D shift that corresponds to = 1.30,
= +2.35. These shift values correspond to upward-right motion of the camera,
or equivalently, downward-left global motion of the scene. The simulation is based
on the bi-linear approximation. The gain NU, a(i, j), i = 1...M, j = 1, ..., N, is
simulated as a Gaussian-distributed random variable with mean 1 and the standard
deviation 0.1 (10%), and the oset NU b(i, j), i =1, ..., M, j =1, ..., N, is simulated
as a Gaussian-distributed random variable with mean 0 and standard deviation
20. The intensity range of the image is between 0 and 255. The coecient of the
regularization parameter, , was set to 0.2. NU-corrupted image frames are shown
in (c) and (d).
90
The result of radiometric NUC is shown in (e). The error in the approximation
is theoretically zero; however, there is some error for the lower-left pixels, which is
on the order of 10
6
, as shown in (f). This is due to roundo errors in the matrix
inversion.
Non-radiometric mode
When the algorithm is applied in the non-radiometric mode, the error in the esti-
mation of the perimeter gain and oset values manifest itself as a residual striping
artifact inside the inner pixels. In Fig. 5.5, we present a NUC example with simulated
frames. The frame in (a) shows the true irradiance, same as in Fig. 5.4(a). In (b), we
show the simulated second frame which has simulated 2-D shift (,)=(+2.25,-1.25).
NU-corrupted frames are shown in (c,d); the oset NU has zero-mean and standard
deviation of 20, while gain NU has mean one and standard deviation of 0.1 (10%).
We see in (e) that NUC has mitigated the NU fairly; however, the striping artifact
can be seen on the image. The readability of the letters have been greatly improved;
the letter U, which is impossible to distinguish in the NU-corrupted frames, is
easily readable after NUC. As expected, we see that the perimeter pixels have much
NU, since the NUC algorithm can only make a statistical assumptions about those
pixels. NUC performance is sensitive to the value of . For example, using = 0.2
degraded the NUC performance a lot and made it practically useless. After some
experimenting, we have decided to use = 1. The NUC algorithm can perform well
for the sub-pixel shift case, as we can see in Fig. 5.6. The shift amount used for the
frame-pair in this example is (,)=(+0.914,+0.569). The striping artifact in the
direction of the shift is more visible Fig. 5.6(e).
91
Using many frame-pairs
As suggested by Ratli et al., the eect of striping artifact can be easily removed
by applying NUC to several frame-pairs and averaging the NU estimation results.
In Fig. 5.7(a), we present NUC result after using 10 frame-pairs. The rst image
frame used in the pairs is the same as in Fig. 5.6. The 2-D shifts had uniform
random distribution between [0,2]. We see that after averaging, both the NU and the
striping artifact have almost disappeared. In (b), we see a decrease in the error, when
compared with Fig. 5.6(f). The root-mean-square error (RMSE) in the estimation
of the true frame is calculated to be 4.78, which is less than a third value of the
RMSE (15.6) corresponding to single-pair NUC presented Fig. 5.6. In Fig. 5.8, we
present the RMSE in the non-radiometric NUC averaged over many frame-pairs
versus number of frame-pairs used.
5.3.4 Two-point NUC: Application to actual images
The two-point correction algorithm was applied to images taken by the MIR InSb
camera. The sequence had 512 frames. First, we determined the consecutive frame-
pairs that possess certain amount of 2-D motion. We picked the pairs that had
motion satisfying the condition [[ > 0.5 and [[ > 0.5. We picked a consecu-
tive frame-pair, pair (392-393), that satises this condition with the 2-D shift of
(, ) = (0.56, 1.18). This 2-D shift, which corresponds to right-upward motion
of the camera, includes sub-pixel motion in vertical direction and super-pixel mo-
tion in horizontal direction, which helps us test the algorithm for both sub-pixel and
super-pixel motions at once.
In Fig. 5.9(a) and (b), we present these frames. We provide the two-point cal-
ibration results in (c) and (d) as a benchmark. We have used the regularization
coecients = 1 and = 0 for the NUC algorithm. We applied the algorithm in
92
radiometric and non-radiometric modes, and for each mode we applied the two-point
(gain and oset) NUC algorithm. We also provided one-point (oset) NUC result
for overall comparison. Once we applied the one-point NUC algorithm on the chosen
frame-pair and obtained estimate of the oset NU, we then subtracted the estimated
oset NU from all the frames and obtained one-point corrected version of the im-
age sequence. After the one-point NUC, we applied the algorithm again by taking
the logarithm of the one-point corrected frame-pair (392,393) and feeding it to the
algorithm. We then obtained the two point-corrected image sequence by using the
estimates of the gain and oset correction parameters.
Radiometric mode
In the radiometric case, the parameters (oset nonuniformity coecients for one-
point correction and both oset and gain nonuniformity coecients for two-point
correction) for the perimeter pixels are assumed to be given via the 2-point calibration
process. This partial calibration information is then used in the algorithm. In
Fig. 5.10, we present the radiometric NUC results on one of the frames, Frame 512,
of the sequence. In (a) we present the frame as observed, i.e., with nonuniformity.
In (b) we present the two-point calibrated frame. In (c) and (d) the results of NUC
algorithm for one-point and two-point correction are presented, respectively. We see
that the NUC algorithm results, which are based on only one frame-pair and only
perimeter pixel information, are comparable to that of two-point calibration. Note
that the camera motion, which is characterized by the shifts (,)=(-0.56, 1.18), is
right-upward. Therefore, the perimeter pixels are the top row and the two rightmost
columns.
93
Non-radiometric mode
In Fig. 5.11, we present the non-radiometric NUC algorithm results, for which no
information about the pixel parameters is assumed to be known. We see in (c)
and (d) that the NUC algorithm mitigated the NU fairly, the NU pattern, which is
heavily present on the original frame in (a), is not visible in (c) and (d). However,
the contrast ratio of the estimated image has decreased slightly. The NUC results
are more appreciable when the whole corrected image sequence is watched as a video,
due to dierential (contrast) adaptivity of the eye. Since the camera motion is in the
right-upward direction, we can see the degradation in the top row and the rightmost
column of the image, as expected. This is our perimeter for this frame-pair; it
includes the pixels for which the NUC algorithm does not have any bi-linear shift
equation and it has to make assumptions. An artifact that is visible in (c) and (d) is
the striping artifact in the direction of motion, which is seen mostly in the pixels that
are neighboring the perimeter pixels. The bilinear interpolation model spreads the
assumption error from the perimeter pixels onto the neighboring inner pixels along
the direction of motion.
5.4 Computational issues: Sparse matrices
In this section we provide discussion on the computational complexity of the algo-
rithm. The computational complexity of the algorithm is considerably high due to
the large sizes of the system matrices and the inversions involved. For an image of
size M-by-N, the matrix to be inverted in Eq. (5.13) is on the order of 2MN-by-2MN.
For an image of size 128-by-128, the dimension of A is approximately 2
15
-by-2
15
.
One remedy to the computational complexity can be exploiting the sparsity of the
matrix A. Normally, with double precision, the storage of A for 128-by-128 image
would require about 2
33
8 GB of memory. However, A is not only sparse, but also
94
many of its entries are 1s. The number of nonzero elements is MN+6(M-1)(N-1) and
MN+2(M-1)(N-1) of them are 1s. For instance, in the sub-pixel-shift case, exam-
ples of the fraction r(M,N) of the number of nonzero elements to all elements are:
r(32,32)0.00172, r(48,48)0.000764, r(64,64)0.000429, r(128,128)0.000107, and
r(256,256)0.0000267.
Another remedy to the computational complexity problem can be the structure of
the system matrix. The matrix R has a multiple-bordered block-diagonal form [66]
and it can be represented as a Kronecker product of two simple multiple-bordered
diagonal-form matrices. The matrix A is a bi-diagonal block Toeplitz matrix [67],
which is obtained via addition of two Kronecker products of smaller bi-diagonal
Toeplitz matrices. The aforementioned structural and sparseness properties of the
system matrix motivate us to exploit sparse-matrix techniques to remedy the com-
putational burden [68]. Use of direct-inversion techniques decrease the computation
time. Additionally, we would also obtain a reduction in the memory requirements if
an iterative inversion method is used [69].
5.5 Conclusions
In this chapter, we developed a matrix-based technique for two-point (gain and o-
set) NUC that uses knowledge of global motion in a video sequence. In the proposed
setting, the entire video sequence is regarded as an output of a motion-dependent
linear transformation, which acts collectively on the true scene and the unknown
bias elements in each detector. The developed approach exploits the rationale pro-
posed earlier by Ratli et al. [41, 43, 44, 45] and puts it in a general matrix-algebra
framework that allows the utilization of minimum-mean-square-error principles as
well as the employment of versatile regularization techniques. The true scene is then
estimated from the video sequence according to a minimum mean-square-error cri-
95
terion. Two modes of operation are considered: a radiometric and non-radiometric
modes. The algorithms performs perfectly on simulated data for which the global
motion is induced according to a bilinear-interpolation model, as we have shown on
simulated images with simulated shifts. The algorithm works very well for the radio-
metric mode, for which it is assumed that the perimeter pixel parameters are given.
Two-point NUC in this mode provides slightly improved result than the one-point
NUC. In the non-radiometric mode, for which information about the parameters is
not assumed to be given, the correction results were not as good as the radiometric
mode; but, at the expense of some contrast reduction, the NU was mitigated.
96
10 20 30
5
10
15
20
25
30
10 20 30
5
10
15
20
25
30
10 20 30
5
10
15
20
25
30
10 20 30
5
10
15
20
25
30
(a) (b)
(c) (d)
Figure 5.1: (a) First frame. (b) Second frame, which is a shifted version of the
rst frame resulting from 1.55-right, 0.70-downward camera motion. (c) First frame
with bias non-uniformity added. The standard deviation of the oset NU is 4. (d)
Estimation of the rst frame with radiometric calibration applied to the perimeter
pixels.
97
10 20 30
5
10
15
20
25
30
10 20 30
5
10
15
20
25
30
10 20 30
5
10
15
20
25
30
10 20 30
5
10
15
20
25
30
(a) (b)
(c) (d)
Figure 5.2: Non-radiometric estimation for the same shift values and same level of
oset NU as in Fig. 5.1 for: (a) = 10 (b) = 2 (c) = 0.5 (d) = 0
98
20 40
10
20
30
40
20 40
10
20
30
40
(a)
20 40
10
20
30
40
(c)
20 40
10
20
30
40
20 40
10
20
30
40
(d)
(e)
(b)
Figure 5.3: (a) First frame. (b) Second frame, shifted as a result of 0.365-right,
0.276-downward camera motion. (c) Correction of the rst frame using a two-point
calibration. (d) Correction using the algorithm in the non-radiometric mode. (e)
Correction using the algorithm in the radiometric mode.
99
10 20 30
10
20
30
10 20 30
10
20
30
10 20 30
10
20
30
10 20 30
10
20
30
10 20 30
10
20
30
10
20
30
10
20
30
2
0
2
x 10
6
horizontal
vertical
(a)
(f) (e)
(d) (c)
(b)
Figure 5.4: Simulated radiometric spatial two-point (gain and oset) NUC example:
(a) 1st frame, true irradiance; (b)2nd frame, simulates a super-pixel-shifted version of
the rst frame scene, obtained by bi-linear shift model, with shift (,)=(-1.3,+2.35);
(c,d) NU-degraded 1st and 2nd frames, with oset-NU std. of 20 and gain-NU std.
of 10%; (e) 1st frame after two-point NUC; (f) error in NUC.
100
10 20 30
10
20
30
10 20 30
10
20
30
10 20 30
10
20
30
10 20 30
10
20
30
10 20 30
10
20
30
10
20
30
10
20
30
50
0
50
horizontal
vertical
error
(a) (b)
(c) (d)
(e) (f)
Figure 5.5: Simulated non-radiometric spatial two-point (gain and oset) NUC
example: (a) 1st frame, true irradiance; (b)2nd frame, simulates a super-pixel
shifted version of the rst frame scene, obtained by bi-linear shift model, with shift
(,)=(+2.25,-1.25); (c,d) NU-degraded 1st and 2nd frames, with oset-NU-std. of
20 and gain-NU-std. of 10%; (e) 1st frame after two-point NUC; (f) error in NUC.
101
10 20 30
10
20
30
10 20 30
10
20
30
10 20 30
10
20
30
10 20 30
10
20
30
10 20 30
10
20
30
10
20
30 10
20
30
100
0
100
horizontal
vertical
Error, rmse=15.31
(a) (b)
(c) (d)
(e)
(f)
Figure 5.6: Simulated non-radiometric spatial two-point (gain and oset) NUC exam-
ple, sub-pixel shift case: (a) 1st frame, true irradiance; (b)2nd frame, simulates a sub-
pixel-shifted version of the rst frame scene, obtained by bi-linear shift model, with
shift (,)=(+0.914,+0.569); (c,d) NU-degraded 1st and 2nd frames, with oset-NU
std. of 20 and gain-NU std. of 10%; (e) 1st frame after two-point NUC; (f) error in
NUC.
102
estimate of first frame true irradiance, after 10 pairs
10 20 30
5
10
15
20
25
30
10
20
30
10
20
30
20
0
20
horizontal
Error after 10 pairs rmse=4.7808
vertical
(a)
(b)
Figure 5.7: (a) Simulated non-radiometric spatial two-point (gain and oset) NUC
example, after averaging over 10 frame-pairs; (b) error in NUC averaged over 10
frame-pairs. The statistics of the NU was the same as in previous gures. Oset-NU
std. is 20 and gain-NU std. is 10%.
103
1 2 3 4 5 6 7 8 9 10
4
6
8
10
12
14
16
NUMBER OF FRAMEPAIRS
N
U
C

R
M
S
E
Figure 5.8: Root-mean-square error(RMSE) in the non-radiometric NUC averaged
over many frame-pairs versus number of frame-pairs. Oset-NU std. is 20 and
gain-NU std. is 10%.
104
Frame 392, after calibration
(c)
10 20 30 40
5
10
15
20
25
30
35
40
(d)
10 20 30 40
5
10
15
20
25
30
35
40
Frame 392, observation
(a)
10 20 30 40
5
10
15
20
25
30
35
40
(,)= (0.56,1.18)
(b)
10 20 30 40
5
10
15
20
25
30
35
40
Figure 5.9: (a,b) Frames 392 and 393 of a 512-frame sequence obtained by InSb
camera, as observed. (c,d) Frames after two-point calibration.
105
(a)
10 20 30 40
10
20
30
40
(b)
10 20 30 40
10
20
30
40
(c)
Frame 512, after 1pt NUC
10 20 30 40
10
20
30
40
(d)
10 20 30 40
10
20
30
40
Figure 5.10: Radiometric NUC application results, in which the parameters of the
perimeter pixels (the top row and the rightmost two columns) are assumed to be
known. (a) Frame 512 from the same sequence used in Fig. 5.9. (b) Frame after two-
point calibration. (c) One-point (oset) and (d) two-point (oset and gain) NUC
results.
106
(a)
10 20 30
5
10
15
20
25
30
35
(b)
10 20 30
5
10
15
20
25
30
35
(c)
10 20 30
5
10
15
20
25
30
35
(d)
10 20 30
5
10
15
20
25
30
35
Figure 5.11: Non-radiometric NUC application results, in which no information
about the gain and oset parameters of the pixels is assumed to be known. (a)
Frame 512 from the same sequence used in Fig. 5.9. (b) Frame after two-point
calibration. (c) One-point (oset) and (d) two-point (oset and gain) NUC results.
107
Chapter 6
Conclusions
This dissertation addressed the problem of extracting reliable high-spectral-resolution
information from sensors that have overlapping bands. The goal is to extract high
spectral resolution information from the spectral redundancy present in such sensors.
In particular, we have developed a spectral-tuning algorithm that is based on exploit-
ing the presence of spectral overlap and spectral diversity in the responsivities of a
collection of detectors to synthesize the output (current) of a desired, arbitrary band
or optical lter. We have shown that the desired band/lter could have higher spec-
tral resolution (narrower width) than those the individual detectors have. We have
established that the weights that approximate certain desired band/lter, obtained
by using weighted superposition of the detectors responsivities in the minimum-
mean-square error(MMSE) sense, also yield the MMSE-solution for combining the
outputs of the detectors so as to synthesize the output that corresponds to the desired
band.
We have applied the tuning algorithm to quantum-dot mid-infrared photode-
tectors (QDIPs) which were fabricated and characterized at the Center for High
Technology Materials at the University of New Mexico. We have shown approximate
108
Chapter 6. Conclusions
continuous spectral-tuning capability for two dierent source spectra, viz., a black-
body source with and without 3-mm polystyrene lter, in the ranges 3 m 8 m
and 5 m 10 m by using these QDIPs. Spectral resolution of up to 0.5 m is
achieved, constituting a four-fold enhancement in comparison to that obtained by
the detectors without the use of any post-processing.
We have shown that noise in measured outputs of the detectors, which is primarily
associated with the dark-current, does degrade the tuning algorithm. Moreover, the
superposition weights have to be calculated in a dierent way with the photocurrents
signal-to-noise ratio (SNR) in mind so as to minimize the eect of noise. We have
developed a noise-modied version of the tuning algorithm so as to accommodate
and compensates for noise by using the SNR level of each detector. As a measure of
performance, we used the normalized root-mean-square error (NRMSE) as a function
of dierent noise levels and desired tuning resolutions. We have shown that the SNR
requirements for reliable operation for the noise-modied algorithm are signicantly
reduced when compared to the algorithm that does not accommodate noise. This
promises robustness to noise in detector outputs with certain limitations depending
on the spectral diversity in the detectors responsivities. To this end, the interplay
between SNR and spectral tunability was thoroughly investigated. We have shown
that as the desired spectral resolution of the tuning is reduced (i.e., the spectral
width of the desired band is increased), the required SNR becomes less. Therefore,
as the desired width of the tuning band/lter is increased, which is the case in many
wide-band applications, the algorithm becomes more robust to noise and the overall
tuning error decreases. This shows a fundamental trade-o between the resolution
of spectral tuning and robustness to noise. We have also demonstrated multispectral
sensing examples in which the noise-modied algorithm performs reliable sensing in
moderate-low SNR conditions.
For infrared detector arrays, pixel-to-pixel dierences in detectors result in nonuni-
109
Chapter 6. Conclusions
formity (NU). Many nonuniformity correction (NUC) approaches have been proposed
in the past in order to obtain correction maps that estimate the true temperature
of the scene from the observed outputs. Dependence of the detectorss outputs on
detector responsivity and scene irradiance makes the nonuniformity noise dependent
on tuning conditions such as the center wavelength and the spectral resolution of
the band. This dependence is referred to as chromatic nonuniformity (CNU). The
uctuations in the detector responsivity and the scene requires new nonuniformity
correction (NUC) maps to be calculated under new spectral-tuning conditions. By
using the tuning capability of the spectral-tuning algorithm, a solution to mitigate
CNU, which is inherently present in infrared detector arrays, was proposed and
tested. The degrading eect of CNU on the nonuniformity correction (NUC) tech-
niques is shown by simulated imagery based on QDIPs, and, the proposed chromatic
nonuniformity correction (CNUC) solution was applied to these imagery.
Finally, to complement our CNUC solution, we considered a recently reported
algebraic NUC algorithm and formulated it in a versatile matrix-based framework.
This algorithm was then applied to real mid-infrared imagery for two-point gain and
oset correction. We have also utilized this NUC algorithm in conjunction with the
CNUC solution.
110
Chapter 7
Future Work
In this chapter, we provide possible directions for future work.
7.1 ST algorithm: Application on dierent QDIPs
Most of the range of the QDIPs spectral response that has been used in our ST al-
gorithm coincide with an atmospheric absorbtion region between 4.28.0m, which
is due to water in the atmosphere. However, the QDIPs have been used to show
the proof of concept, since they were the only available QDIPs manufactured by
our group. Development of other QDIPs which have dierent and more favorable
characteristics are in progress. Application of the algorithm requires complete spec-
tral and noise characterization, and the complete characterization was only done for
one QDIP, for which we have applied our ST algorithm results in the ST algorithm
chapter. Characterization of other QDIPs are ongoing, and here we provide ST for
another QDIP, namely QDIP 1780 produced in CHTM, which has spectral response
between 414m and its spectral diversity enables us to tune it between 812.0m.
example tuning results are presented in Fig. 7.1. The desired response has a triangu-
111
Chapter 7. Future Work
lar shape width FWHM of 0.5m. QDIP 1780 has peaks mainly in the 8 m12 m
range. Figure 7.1(a) shows approximation of an ideal triangular lter with full-width
at half-maximum (FWHM) of 1 m, at two dierent wavelength centers, 8.5m and
10.5m. Figure 7.1(b) shows approximation of triangular lters centered at 9.5 m,
but has FWHM of 0.5, 1.5 and 3.5 m. Development of detectors which have more
diversity in their spectral response will directly increase the tuning capability of the
ST algorithm. The tuning range of the ST algorithm depends on the range of the
spectral response of the spectrum; the tuning resolution depends on the peak diver-
sity of the detectors as well as their individual resolution. Set of detectors having
spectrally overlapping response with peaks at dierent wavelengths is desirable.
7.2 ST Algorithm: Selection of the best detec-
tors in a given set
We have done some preliminary studies on the selection of the best detectors. We
have shown that although there is some amount of linear dependence among the
detectors responses, which can allow us to reduce the number of spectra. Although
reducing the number detector responses reduces the computation time, it does not
help improve the performance of the ST algorithm. Removing some response reduces
the spectral space that can be spanned by the set of the responses. One can develop
techniques based on singular value decomposition to linearly transform the responses
into another space, remove the vectors that correspond to little singular values and
perform the algorithm with the reduced set. When noise is involved in the model,
the statistics of noise corresponding to each response also determines this selection.
112
7.3 NUC algorithm: Dependence on accuracy of
shift estimation
Performance of the NUC algorithm depends on the accuracy of the shifts assumed
between the frame pairs used. In this work, we have assumed the shift values were
given accurately. However, shift estimation technique is a well-developed area, where
good and fast shift estimation techniques exist. Ratli et al. studied the eect of shift
estimation on the algebraic NUC algorithm [45] and provided a background on shift
estimation techniques that can be useful for NUC algorithms. They developed a shift
estimation technique based on algebraic NUC algorithm. Although not presented in
this work, we have done some preliminary studies to see the eect of uncertainty
of the shift estimates on the NUC performance, and saw that for shift estimation
errors more than 5%, the performance of the algorithm decreases rapidly. Existing
shift estimation techniques can obtain much lower estimation error levels, within
reasonable amount of computation time. However, a shift estimation technique that
can be developed specically for the matrix-based NUC algorithm is an open future
problem, and will help minimize the computation time of the overall NUC system.
7.4 NUC algorithm: Memory issues and sparse
matrices
Due to large dimensions of matrices involved, NUC algorithm requires a large amount
of memory. The memory can be easily consumed when the image dimensions are
large. One remedy to this problem can be splitting the image into smaller images, and
performing the NUC for each of them in a parallel fashion. Running the algorithm
in the non-radiometric mode would allow this. However, in the radiometric mode, in
113
which perimeter pixels are reserved for the calibration, this splitting would be more
challenging; the perimeter pixels of the split images, which lay inside the inner pixels
of the greater image, would require to be calibrated. For example, performing NUC
for an array of dimension M-by-N image sequence requires a system matrix on the
order of 2MN-by-2MN, which, in turn, requires storage of 4M
2
N
2
elements. For
100-by-100 image, the number of elements is 410
8
, which roughly requires 400 MB
of space in 8-bit precision format. If the image is divided into 4 50-by-50 sub-images,
then the total number of elements in 4 system matrices is 4 4 50
2
50
2
= 10
8
.
The number of elements, thus the storage requirements decrease 4 times.
As we discussed before, another possible remedy to the memory and computa-
tional requirements of the algorithm is the sparsity of the matrices involved. In
this work, we have not used any techniques that uses this sparsity to reduce the
computation time. Sparse matrix techniques are specic to matrix structure, and
specic sparse techniques have to be developed for the system matrices in our NUC
algorithm. Some general sparse techniques which are applicable for similar matrices
might be applicable to our case with compromising some error, however, we should
note that the problem itself is already very sensitive to numerical errors due to matrix
inversion involved.
7.5 Chromatic NUC algorithm: Application on
real QDIP imagery
Application of the CNUC algorithm on real imagery requires image sequences taken
by either dierent detectors which have dierent spectral responses, and/or with
the same detector with dierent lters. The CNUC idea is presented in this work
theoretically and veried on simulated imagery. A bias-tunable QDIP camera is still
114
to be developed at the CHTM, and application of the algorithm on imagery obtained
by it would be the next step.
115
4 6 8 10 12
0
0.2
0.4
0.6
0.8
1
Wavelength (m)
R
e
s
p
o
n
s
i
v
i
t
y

(
a
.
u
.
)
(a)
4 6 8 10 12
0.2
0
0.2
0.4
0.6
0.8
Wavelength (m)
R
e
s
p
o
n
s
i
v
i
t
y

(
a
.
u
.
)
=0.5m
=1.5m
=3.5m
(b)
Figure 7.1: (a) Approximation of ideal triangular lter with FWHM of 1m centered
at 8.5 and 10.5m. (b) Approximation at the center 9.5m but with fwhm of 0.5, 1.5
and 3.5m. Basis spectra used for the approximation belongs to the QDIP device
#1780 of CHTM.
116
Appendices
A Poisson noise model
117
Appendix A
Poisson Noise Model
In order model the noise in current of an infrared photodetector, it is important to un-
derstand how the measurements are performed. The spectral response of an infrared
photodetector is usually measured by a Fourier transform infrared (FTIR) spectrom-
eter, for each bias voltage V
k
. FTIR spectrometer rst creates an interference pattern
and measures an estimate of the autocorrelation function of the photocurrent, and
then converts it into a power spectrum by taking the Fourier transform.
Let us dene (under bias voltage V
k
) the electrons generated at the detector at
time t with ^
k
(t), and without loss of generality let us assume ^
k
(0) = 0. Let us
assume that each dark-generated electron contributes to dark current, and that the
generation rate (per unit time) is equal to
k
. Let us assume that our photodetector
counts (integrates) these electrons for a duration of . Since the electron generation
times are independent from each other, ^
k
(t) can be modeled as a Poisson counting
process with parameter (rate)
k
. We can write for the mean and the variance of
this process (in time) [70]
E[^
k
(t)] =
k
t, Var[^
k
(t)] =
k
t. (A1)
Let us denote the amount of dark-electrons that the detector accumulates at time t,
118
Appendix A. Poisson Noise Model
for a duration of , with ^(t), where:
^
k
:= ^
k
(t) ^
k
(t ), t . (A2)
The dark current, D
k
, therefore can be written as
D
k
= q
^
, (A3)
where q is the electron charge. ^
k
is Poisson random variable with parameter
(mean and variance)
k
. Thus, the dark current, D
k
, and N
k
has variance of
2
N,k
=
2
D,k
Var[D
k
] =
q
2
2
Var[^
k
] = q
2
, (A4)
We dene the noise power as the standard deviation of the noise. We dene the
measurement signal-to-noise ratio (SNR), as the ratio of the average photocurrent,
y
p,k
, to the noise power, i.e.,
SNR
k

y
p,k
N,k
, (A5)
for kth detector.
It can be assumed that the measured dark currents, corresponding to dierent
interference patterns, are statistically independent [24]. Thus, the autocorrelation
function of the measured dark current is
2
N,k
(t) +y
2
d,k
. (A6)
The rst component is due to the measurement-to-measurement variation of the
dark current, which is white noise, and the second component is due to the d.c.
component (mean) of the dark current. Let us dene the power spectral response
of the detector under bias voltage V
k
, with S
k
(), which is the Fourier transform
of the autocorrelation function of the noiseless photocurrent, y
p,k
. After the Fourier
transform of the autocorrelation function of the dark-current is taken, for each bias
voltage V
k
, we obtain the total power spectral response
S
k
() S
k
() +
2
FT[
2
N,k
(t) +y
2
d,k
]
= S
k
() +
2
(
2
N,k
+y
2
d,k
()),
(A7)
119
Appendix A. Poisson Noise Model
where is the width of the range of integration for the spectral response in microns,
and FT[.] denotes the Fourier transform operator in time. The d.c. component
of the photocurrent is blocked by the FTIR spectrometer prior to the Fourier trans-
formation, therefore the delta-function term after the transform vanishes. Thus, we
nally obtain:
S
k
() = S
k
() +
2
2
N,k
= S
k
() +
2
q
2
, (A8)
which depends on the integration time of the detector, the integration range and the
rate of the process of photo-generation.
120
References
[1] G. Vane and A. F. H. Goetz, Terrestrial Imaging Spectroscopy, Remote Sens-
ing of Environment 24(1), 129 (1988).
[2] G. Vane, R. Green, T. Chrien, H. Enmark, E. Hansen, and W. Porter, The air-
borne visible infrared imaging spectrometer, Remote Sensing of Environment
44, 127143 (1993).
[3] R. W. Basedow, D. C. Carmer, and M. E. Anderson, HYDICE System: Imple-
mentation and Performance, in Proceedings of the SPIE: Imaging Spectrometry,
M. R. Descour, J. M. Mooney, D. L. Perry, and L. R. Illing, eds., vol. 2480, pp.
258267 (The International Society for Optical Engineering, 1995).
[4] T. H. Barnes, Photodiode Array Fourier Transform Spectrometer with Im-
proved Dynamic Range, Applied Optics 24(22), 37023706 (1985).
[5] G. B. Rafert, R. G. Sellar, and I. L. J. Otten, An interactive performance
model for spatially modulated Fourier transform spectrometers, in Imaging
Spectrometry, M. R. Descour, J. M. Mooney, D. L. Perry, and L. R. Illing, eds.,
vol. 2480, pp. 410417 (Proceedings of the SPIE, 1995).
[6] L. J. Otten, III, G. B. Rafert, and R. G. Sellar, The design of an airborne
Fourier transform visible hyperspectral imaging system for light aircraft envi-
ronmental remote sensing, in Proceedings of the SPIE: Imaging Spectrometry
121
References
V, M. R. Descour, J. M. Mooney, D. L. Perry, and L. R. Illing, eds., vol. 2480,
pp. 418424 (The International Society for Optical Engineering, 1995).
[7] L. J. Otten, III, A. D. Meigs, A. J. Franklin, R. D. Sears, M. W. Robison,
B. Rafert, D. C. Fronterhouse, and R. L. Grotbeck, Onboard Spectral Imager
Data Processor, in Proceedings of the SPIE: Imaging Spectrometry V, M. R.
Descour and S. S. Shen, eds., vol. 3753, pp. 8694 (The International Society
for Optical Engineering, 1999).
[8] J. S. Tyo and T. S. Turner, Variable-Retardance, Fourier-Transform Imag-
ing Spectropolarimeters for Visible Spectrum Remote Sensing, Applied Optics
40(9), 14501458 (2001).
[9] W. R. Bell and P. G. Weber, Multispectral Thermal Imager: Overview, in
Proceedings of the SPIE: Algorithms for Multispectral, Hyperspectral, and Ultra-
spectral Imagery VII, S. S. Shen and M. R. Descour, eds., vol. 4381, pp. 173183
(The International Society for Optical Engineering, 2001).
[10] U. Sakoglu, J. S. Tyo, M. M. Hayat, S. Raghavan, and S. Krishna, Spectrally
adaptive infrared photodetectors with bias-tunable quantum dots, Journal of
the Optical Society of America B 21(1), 717 (2004).
[11] H. J. Cauleld, Articial Color. Neurocomputing 51, 463465 (2003).
[12] P. Bhattacharya, S. Krishna, J. D. Phillips, D. Klotzkin, and P. J. McCann,
Quantum dot carrier dynamics and far infrared devices, in Optoelectronics
Materials and Devices II, Y.-K. Su and P. Bhattacharya, eds., vol. 4078, pp.
8489 (Proceedings of the SPIE, 2000).
[13] P. Bhattacharya, S. Krishna, J. Phillips, P. J. McCann, and K. Namjou, Car-
rier dynamics in self-organized quantum dots and their application to long-
122
References
wavelength sources and detectors, Journal of Crystal Growth 227, 2735
(2001).
[14] S. Krishna, P. Rotella, S. Raghavan, A. Stintz, M. M. Hayat, J. S. Tyo, and S. W.
Kennerly, Bias-dependent tunable response of normal incidence long wave in-
frared quantum-dot photodetectors, in Proc. IEEE/LEOS Annual Meeting
2002, New York, vol. 2, p. 754 (Proceedings of the IEEE, 2002).
[15] S. Krishna, Quantum dots-in-a-well infrared photodetectors, Journal of
Physics D: Applied Physics 38, 21422150 (2005).
[16] Z. Wang, U. Sakoglu, S. Annamalai, N.-R. Weisse-Bernstein, P. Dowd, J. S. Tyo,
M. M. Hayat, and S. Krishna, Real-time implementation of spectral matched
ltering algorithms using adaptive focal plane array technology, in Imaging
Spectrometry X, A. G. Tescher, ed., vol. 5546, pp. 7383 (Proceedings of the
SPIE, 2004).
[17] Z. Wang, B. Paskaleva, J. S. Tyo, and M. M. Hayat, Canonical correlations
analysis for assessing the performance of adaptive spectral imagers, in SPIE
Defense and Security Symposium, Algorithms and Technologies for Multispec-
tral, Hyperspectral, and Ultraspectral Imagery XI, S. S. Shen and P. E. Lewis,
eds., vol. 5806, pp. 2334 (Proceedings of the SPIE, 2005).
[18] Z. Wang, B. Paskaleva, M. M. Hayat, and J. S. Tyo, Analyzing spectral sen-
sors with highly overlapping bands, in SPIE Defense and Security Symposium,
Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral
Imagery XII (Proceedings of the SPIE, 2006).
[19] B. Paskaleva, M. M. Hayat, M. M. Moya, and R. J. Fogler, Multispectral rock
type separation and classication, in Infrared Spaceborne Remote Sensing XII,
vol. 5543, pp. 152163 (Proceedings of the SPIE, 2004).
123
References
[20] B. Paskaleva and M. M. Hayat, Optimized algorithm for spectral band se-
lection for rock-type classication, in SPIE Defense and Security Symposium,
Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral
Imagery XI, S. S. Shen and P. E. Lewis, eds., vol. 5806, pp. 131138 (Proceedings
of the SPIE, 2005).
[21] B. Paskaleva, M. M. Hayat, J. S. Tyo, Z. Wang, and M. Martinez, Feature se-
lection for spectral sensors with overlapping noisy spectral bands, accepted, in
SPIE Defense and Security Symposium, Algorithms and Technologies for Multi-
spectral, Hyperspectral, and Ultraspectral Imagery XII (Proceedings of the SPIE,
2006).
[22] U. Sakoglu, Z. Wang, M. M. Hayat, J. S. Tyo, S. Annamalai, P. Dowd, and
S. Krishna, Quantum dot detectors for infrared sensing: Bias-controlled spec-
tral tuning and matched ltering, in Nanosensing - Materials and Devices,
M. S. Islam and A. K. Dutta, eds., vol. 5593, pp. 396407 (Proceedings of the
SPIE, 2004).
[23] U. Sakoglu, M. M. Hayat, J. S. Tyo, P. Dowd, S. Annamalai, K. T. Posani,
and S. Krishna, Statistical adaptive sensing using detectors with spectrally
overlapping bands, Applied Optics, accepted (2006).
[24] V. Ryzhii, Physical model and analysis of quantum-dot infrared photodetectors
with blocking layer, Journal of Applied Physics 89, 5117 (2001).
[25] P. Bhattacharya, Semiconductor Optoelectronic Devices, 2nd ed. (Prentice Hall,
Englewood Clis Press, N.J., 1996).
[26] E. L. Dereniak and G. D. Boreman, Infrared Detectors and Systems (John Wiley
and Sons, Inc., New York, N.Y., 1996).
124
References
[27] A. F. Milton, F. R. Barone, and M. R. Kruer, Inuence of Nonuniformity on
Infrared Focal Plane Array Performance, Optical Engineering 24(5), 855862
(1985).
[28] D. A. Scribner, K. A. Sarkady, M. R. Kruer, and C. J. Gridley, Measurement,
characterization, and modeling of noisie in staring infrared focal plane arrays,
in Proceedings of the SPIE: Infrared Sensors and Sensor Fusion, vol. 782, pp.
147160 (The International Society for Optical Engineering, 1987).
[29] J. M. Mooney, F. D. Shepherd, W. S. Ewing, J. E. Murguia, and J. Silverman,
Responsivity Nonuniformity Limited Performance of Infrared Staring Cam-
eras, Optical Engineering 28, 11511161 (1989).
[30] G. C. Holst, CCD Arrays, Cameras, and Displays (SPIE Optical Engineering
Press, Bellingham, 1996).
[31] P. M. Narendra, Reference-free Nonuniformity Compensation for IR Imaging
Arrays, in Proceedings of the SPIE: Smart Sensors II, vol. 252, pp. 1017 (The
International Society for Optical Engineering, 1980).
[32] P. M. Narendra and N. A. Foss, Shutterless Fixed Pattern Noise Correction for
Infrared Imaging Arrays, in Proceedings of the SPIE: Technical Issues in Focal
Plane Development, vol. 282, pp. 4451 (The International Society for Optical
Engineering, 1981).
[33] J. G. Harris, Continuous-time Calibration of VLSI Sensors for Gain and Oset
Variations, in Proceedings of the SPIE: Smart Focal Plane Arrays and Focal
Plane Array Testing, M. Wigdor and M. A. Massie, eds., vol. 2474, pp. 2333
(The International Society for Optical Engineering, 1995).
[34] J. G. Harris and Y.-M. Chiang, Minimizing the Ghosting Artifact in Scene-
based Nonuniformity Correction, in Proceedings of the SPIE: Infrared Imaging
125
References
Systems: Design, Analysis, Modeling, and Testing IX, G. C. Holst, ed., vol.
3377, pp. 106113 (The International Society for Optical Engineering, 1998).
[35] M. M. Hayat, S. N. Torres, S. C. Cain, and E. Armstrong, Model-based Real-
time Nonuniformity Correction in Focal Plane Array Detectors, in Proceedings
of the SPIE: Infrared Imaging Systems: Design, Analysis, Modeling, and Testing
IX, G. C. Holst, ed., vol. 3377, pp. 122132 (The International Society for
Optical Engineering, 1998).
[36] M. M. Hayat, S. N. Torres, E. E. Armstrong, S. C. Cain, and B. J. Yasuda,
Statistical Algorithm for Nonuniformity Correction in Focal-plane Arrays,
Applied Optics 38(5), 772780 (1999).
[37] B. M. Ratli, An Algebraic Scene-based Algorithm for Nonuniformity Correc-
tion in Focal Plane Arrays, Masters thesis, University of Dayton, Dayton, OH
(2001).
[38] B. M. Ratli, M. M. Hayat, and R. C. Hardie, Algebraic Scene-based Nonuni-
formity Correction in Focal Plane Arrays, in Proceedings of the SPIE: Infrared
Imaging Systems: Design, Analysis, Modeling, and Testing XII, G. C. Holst,
ed., vol. 4372, pp. 114124 (The International Society for Optical Engineering,
Orlando, FL, 2001).
[39] B. M. Ratli, M. M. Hayat, and J. S. Tyo, On the Performance of a
Radiometrically-calibrated Nonuniformity Correction Algorithm, vol. IX, pp.
1418 (Proceedings of the 6th World Multiconference on Systemics, Cybernetics
and Informatics, Orlando, FL, 2002). Invited.
[40] B. M. Ratli, M. M. Hayat, and J. S. Tyo, Radiometrically Calibrated Scene-
based Nonuniformity Correction for Infrared Array Sensors, in Proceedings of
the SPIE: Infrared Technology and Applications XXVIII, B. F. Andresen, G. F.
126
References
Fulop, and M. Strojnik, eds., vol. 4820, pp. 359367 (The International Society
for Optical Engineering, Seattle, WA, 2003).
[41] B. M. Ratli, M. M. Hayat, and R. C. Hardie, An Algebraic Algorithm for
Nonuniformity Correction in Focal Plane Arrays, Journal of the Optical Society
of America A 19(9), 17371747 (2002).
[42] B. M. Ratli, M. M. Hayat, and J. S. Tyo, Algorithm for Radiometrically Ac-
curate Nonuniformity Correction with Arbitrary Scene Motion, in Proceedings
of the SPIE: Infrared Imaging Systems: Design, Analysis, Modeling, and Test-
ing XIV, G. C. Holst, ed., vol. 5076, pp. 8291 (The International Society for
Optical Engineering, Orlando, FL, 2003).
[43] B. M. Ratli, M. M. Hayat, and J. S. Tyo, Radiometrically Accurate Scene-
based Nonuniformity Correction for Array Sensors, Journal of the Optical So-
ciety of America A 20(10), 18901899 (2003).
[44] B. M. Ratli, M. M. Hayat, and J. S. Tyo, Generalized Algebraic Nonunifor-
mity Correction Algorithm, Journal of the Optical Society of America A 22,
239249 (2005).
[45] B. M. Ratli, A Generalized Algebraic Scene-based Nonuniformity Correction
Algorithm for Infrared Focal Plane Arrays, Ph.D. Dissertation, University of
New Mexico, Albuquerque, NM (2004).
[46] W. F. ONeil, Dithered Scan Detector Compensation, in Proceedings of the
Infrared Information Symposium (IRIS) Specialty Group on Passive Sensors
(1993).
[47] W. F. ONeil, Experimental Verication of Dithered Scan Nonuniformity Cor-
rection, in Proceedings of the Infrared Information Symposium (IRIS) Specialty
Group on Passive Sensors, vol. 1, pp. 329339 (1997).
127
References
[48] R. C. Hardie, M. M. Hayat, E. E. Armstrong, and B. J. Yasuda, Scene-based
Nonuniformity Correction with Video Sequences and Registration, Applied Op-
tics 39(8), 12411250 (2000).
[49] D. L. Perry and E. L. Dereniak, Linear Theory of Nonuniformity Correction
in Infrared Staring Sensors, Optical Engineering 32(8), 18531859 (1993).
[50] J. M. Arias, M. Z. J. Bajaj, J. G. Pasko, L. Bubulac, S. H. Shin, and R. E.
DeWarnes, Molecular beam epitaxy HgCdTe growth-induced void defects and
their eect on infrared photodiodes, Journal of Electronic Materials 24, 521
524 (1995).
[51] M. H. Young, D. H. Chow, A. T. Hunter, and R. H. Miles, Recent advances in
Ga
1x
In
x
Sb/InAs superlattice IR detector materials, Applied Surface Science
123, 395399 (1998).
[52] B. F. Levine, Quantum-well Infrared Photodetectors, Journal of Applied
Physics 74(8), R1R81 (1993).
[53] V. Ryzhii, I. Khmyrova, V. Mitin, M. Stroscio, and M. Willander, On the
detectivity of quantum-dot infrared photodetectors, 78, 35233525 (2001).
[54] J. Phillips, P. Bhattacharya, S. W. Kennerly, D. W. Beekman, and M. Dutta,
Self-Assembled InAs-GaAs Quantum-Dot Intersubband Detectors, IEEE
Journal of Quantum Electronics 35, 936942 (1999).
[55] H. C. Liu, M. Gao, J. McCafrey, Z. R. Wasilewski, and S. Fafard, Quantum
dot infrared photodetectors, 78, 7981 (2001).
[56] S. Raghavan, P. Rotella, A. Stintz, B. Fuchs, S. Krishna, C. Morath, D. A.
Cardimona, and S. W. Kennerly, High responsivity, normal-incidence long-
wave infrared ( 7.2m) InAs/In
0.15
Ga
0.85
As dots-in-well detector, 81, 1369
1371 (2002).
128
References
[57] P. M. Petro, A. Lorke, and A. Imamoglu, Epitaxially self-assembled quantum-
dots, Physics Today 54, 46 (2001).
[58] D. Bimberg, M. Grundmann, and N. N. Ledenstov, Quantum Dot Heterostruc-
tures, 1st ed. (Wiley, New York, 1999).
[59] U. Sakoglu, R. C. Hardie, M. M. Hayat, B. M. Ratli, and J. S. Tyo, An alge-
braic restoration method for estimating xed-pattern noise in infrared imagery
from a video sequence, in Applications of Digital Image Processing XXVII,
A. G. Tescher, ed., vol. 5558, pp. 6979 (Proceedings of the SPIE, 2004).
[60] S. Krishna, D. Forman, S. Annamalai, P. Dowd, P. Varangis, J. T. Tumolillo,
A. Gray, J. Zilko, K. Sun, M. Liu, J. Campbell, and D. Carothers, Demonstra-
tion of a 320x256 two-color focal plane array using InAs/InGaAs quantum dots
in well detectors, Applied Physics Letters 86, 193,5013 (2005).
[61] H. Stark and J. Woods, Probability and Random Processes with Applications to
Signal Processing, 3rd ed. (Prentice-Hall, Englewood Clis Press, N.J., 2002).
[62] C. Beenakker and C. Schoenenberger, Quantum shot noise, Physics Today
56, 37 (2003).
[63] F. F. Sabins, Remote Sensing, Principles and Interpretation, 3rd ed.
(W. H. Freeman and Co., New York, N.Y., 1997).
[64] M. Irani and S. Peleg, Improving Resolution by Image Registration, CVGIP
- Graphical Models and Image Processing 53(3), 231239 (1991).
[65] R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Wat-
son, High-resolution Image Reconstruction from a Sequence of Rotated and
Translated Frames and its Application to an Infrared Imaging System, Optical
Engineering 37(1), 247260 (1998).
129
References
[66] J. K. Reid, ed., Large Sparse Sets of Linear Equations (Academic Press, London,
UK, 1971).
[67] G. H. Golub and C. F. V. Loan, Matrix Computations, 3rd ed. (The Johns
Hopkins University Press, Baltimore, Maryland, 1996).
[68] D. S. Watkins, Fundamentals of Matrix Computations, 2nd ed. (John Wiley and
Sons, Inc., New York, 2002).
[69] I. S. Du, ed., Sparse Matrices and their Uses (Academic Press, London, UK,
1981).
[70] F. C. Klebaner, Introduction to Stochastic Calculus with Applications (Imperial
College Press, London, UK, 1998).
130

Dissertation UnalSakoglu

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Dissertation UnalSakoglu

Uploaded by

Copyright:

Available Formats

Signal-processing strategies for spectral

tuning and chromatic nonuniformity

Unal Sako glu

Ilim ilim bilmektir,

Ilim kendin bilmektir,

Unal Sako glu

). Among the cryogenically-cooled detec-

A, and the total

= 1 , these matrices can be recast as

N1 column blocks of width M1

, swap(, ) rotate frames clockwise 90

< 0, > 0 rotate frames clockwise 90

, swap(, ) rotate frames counterclockwise 90

< 0, < 0 rotate frames 180

rotate frames 180

Correction with the known gain nonuniformity

N bi-diagonal Toeplitz blocks of size M

M and itself is bi-diagonal

You might also like