Measurement Automation Monitoring, Jun. 2015, vol. 61, no.
06 
Robert ŚWITA, Zbigniew SUSZYŃSKI
UNIVERSITY OF TECHNOLOGY IN KOSZALIN, FACULTY OF MULTIMEDIA SYSTEMS AND ARTIFICIAL INTELLIGENCE,
Śniadeckich St. 2, 75-453 Koszalin, POLAND
Processing of thermographic sequence using
Principal Component Analysis
Abstract
This paper describes application of the principal component analysis in
relation to the thermal imaging contrast sequences, recorded with pulse
excitation for three different objects. The aim of the study was to
demonstrate that thermographic sequences contain an excessive number of
data-distorting information about the characteristics of an object and that it
is possible to reduce them. It has been shown that PCA can improve SNR,
simplifies separation of areas with distinct features and allows
determining their count, which is important, inter alia, with infrared image
segmentation. The study shows examples of the results for a sequence of
infrared registered for thin-layer-chromatography plate (SiO2 on glass)
with separated analytes, the high power Si-eutectic-Mo thyristor structure
with defects in eutectic, and the Al disk with cavities of different diameter
and depth.
Keywords: filtering, correlation, PCA, segmentation, thermal sequences.
1. Introduction
Active thermography aims to visualize thermal singularities and
density of the power dissipated in the test object. The most
common result of measurements is a sequence of images
illustrating the variability of temperature field as a function of
time or frequency. Generally, the methods and algorithms of IR
sequences do not defer in comparison to other types of 2D and 3D
signal processing [1, 2]. However, there are few specific
transformations, e.g. temperature sequence to contrast sequence
[3] (temperature, amplitude, phase or complex). Temperature
sequence allows detection and visualization of singularities, while
for quantitative analysis one uses primarily contrast sequences,
which have the advantage of independence of observations from
the absolute value of the excitation energy and the elimination of
the reference area signal background. Their analysis allows
detection of structural defects and improper functioning of diverse
devices. However, in many cases, infrared measurements provide
data to measure thermal and optical characteristics [4] and even to
determine their depth profiles (thermal tomography) [5, 6]. The
accuracy of the analysis depends on proper selection of the
stimulation frequency spectrum [7], thermal models, signal
processing methods and analytical methods adapted to transform
the parameters describing the signal to the parameters which
define characteristics of the object.
A characteristic feature of the thermal sequences is that the
detection area is fixed and viewed scene does not change its
position - only temperature value of individual regions (objects,
pixels) varies with time.
For this reason, each pixel of thermographic sequence can be
treated as an independent observation and it’s intensity
representing temperature or contrast signal (amplitude, phase, or
complex contrast in case of the Fourier Transform sequence) is
a parameter, which indirectly encodes features of the object’s
fragment, corresponding to the pixel. Number of observations
equals to the number of pixels (M  N) and the number of
parameters to be recorded corresponds to the number of frames of
the sequence (K). One can also assume that the information
contained in the neighboring pixels (in observations) often
correlates with each other. Also, the thermal parameters of the
particular observation in adjacent frames may have similar values.
This means that the amount of data contained in the temperature
sequence is normally in excess to the information contained
therein, which unnecessarily increases the time of possible
processing operations and analysis. However, bigger problem is,
that a large number of parameters can hinder analysis of the
215
features of the individual observations, especially since they differ
in significance – they have diverse values from the point of view
of the amount of information they provide in the process of
identifying features. In particular, they may be redundant or
unnecessary data and thus, distort the information about the
features of the object.
One can see that, using Principal Component Analysis, there is
possible to achieve two important goals - orthogonalization and
reducing the number of parameters that define the object [8]. This
should simplify the analysis of its properties and thereby simplify
segmentation and identification of the thermal parameters. PCA
allows finding new parameters, important from the representation
of objects features point of view, which are linear combination of
original parameters.
2. Experiment
Thermal imaging sequences were recorded for the three objects:
thin-layer-chromatography plate (SiO2 on glass) with separated
analytes (fullerenes C70 and C60), the silicon-eutectic-
molybdenum high power thyristor structure with defects in the
eutectic and the Al disk with cavities of different diameter and
depth. Optical pulse excitation emitted by the flash lamp with
configuration presented in Figure 1 was applied.
I
v
1
I
v, max
0
0 20 40 60
t, ms
Fig. 1. Thermal measurements setup
IR sequence was recorded with a 128128 pixels resolution,
a 500 frames/s frequency frame rate, and acquisition time up to
2 s. For PCA analysis, sequences no longer than 128 frames were
taken into account, in respect to cubic growth of computation
time. All results were obtained with Soft4Image application,
requiring about 3 min calculations on i7-compatible processor for
the thermal sequence with dimensions 128128128.
3. PCA
For sequence of observations matrix X(M, N, K), which stores
MN pixel values of consecutive frames, the PCA performs shift of
pixel values by the mean value in each frame, making
transformation independent from the average intensity in each
frame.
:, = :, − ∑ , , = 1. . (1)
where colon denotes the whole range of pixel indexes 1..MN and
K is a number of frames.
216 
If the parameters could not be compared, one would need to
normalize them. Each column of the matrix A would then be
divided by the standard deviation of the parameter in the
column. New parameter in the PCA is determined by the
direction of vector v, for which the projected data show the
greatest variance:
‖ ‖ = (2)
The expression (2) for the unit vector v determines Rayleigh
quotient that takes the maximum value for the largest eigenvalue
of symmetric covariance matrix ATA (in the standardization it is
the correlation matrix of data). Vector v is the dominant
eigenvector of this matrix. The pixel values recorded by the
thermal camera are in the same space and can be compared with
each other. Normalization of each frame with the value of the
standard deviation is therefore unnecessary and the PCA
algorithm is strictly connected with the problem of finding
eigenvectors of the data covariance matrix, and not for the
correlation matrix.
The standard method for determining components of PCA is
the SVD (Singular Value Decomposition) matrix factorization:
= (3)
where: matrix U is an orthogonal matrix of the left singular
vectors, V is an orthogonal matrix of the right singular vectors,
and matrix Σ is a diagonal matrix of the singular values.
If the matrix is centered by subtracting mean values in the
columns of the observation matrix, then the product ATA forms
a covariance matrix of the data. Note that this matrix can be
defined as:
= = (4)
The matrix V therefore is a matrix of eigenvectors of the
symmetric covariance matrix ATA and the diagonal matrix Σ2 is an
eigenvalues’ matrix of covariance matrix. Knowing matrix V and
Σ, matrix U may be determined from the definition of the SVD
factorization (3):
= (5)
PCA decomposition using SVD does not requires explicit
formation of the covariance matrix, which for large and dense
matrices is computationally expensive. The right singular vectors
in columns of matrix V represent weight vectors of principal
components and eigenvectors of covariance matrix. The product
of the matrices UΣ (equivalent to the AV product) determine
observation coordinates in PCA space.
Practical approach for determining the SVD decomposition
consists most often of two stages [9]. The first step is to determine
triangular or bidiagonal matrix using Householder reflections,
which retains similarity to matrix A (matrix with the same
eigenvalues as A). The second stage is an iterative numerical
method that reduces matrix to a diagonal form using different
varieties of QR algorithm. It is worth noting that despite that the
second stage is iterative, its fast convergence means that the first
stage is more computationally expensive - O(n3).
Much older SVD factorization method, but still popular because
of its simplicity and possibility to parallelize computations, is one-
sided Jacobi method. Despite the fact that this is not the fastest
method, singular values could be determined with the highest
known accuracy with this method. One-sided transform implicitly
forms symmetric matrix ATA and using sequence of Jacobi
rotation (alias Givens rotations), and thus keeping matrix
similarity, changes it into a diagonal matrix.
Measurement Automation Monitoring, Jun. 2015, vol. 61, no. 06
4. The results of PCA analysis.
Dimensionality reduction, grouping
and filtering
One of the greatest advantages of using the PCA is
orthogonalization of parameter space, which generally leads to
a concentration of information in a much smaller number of
parameters then the initial count. This allows visualization of
individual observation locations (pixels) in provided space of
orthogonal parameters. Fig. 2 shows a selection of images (1, 2, 4,
8, 16, 32, 64, 128, 256) from the IR sequence (upper images)
registered for an aluminum disk with drilled holes (Fig. 4) and the
first six images of a PCA sequence – from the rest of the images in
PCA sequence only three contained information related to
singularities and the other had the character of noise images. At
the same time the intensity of the remaining frames were many
times smaller than the pixel values of the first frames of the PCA
sequence. It follows that the PCA sequence "encoded" all the
information about the object in only few frames. This is well
illustrated with so-called “scree plot” (sorted eigenvalues of
successive frames as a function of the frame number), also shown
in Fig. 2. Number of frames after rejecting principal components
with noise levels is very accurate and reliable indicator of optimal
segments number in segmentation process.
a)
b)
Fig. 2. a) Images of IR sequence and principal images of PCA sequence,
b) scree plot of PCA
Concentration of information in the principal images of the PCA
sequence simplifies the separation of areas with different
characteristics by working with smaller number of dimensions
(frames). Fig. 3 shows projection graphs of the principal images
parameters on the principal projection surfaces XY, XZ, YZ (X, Y
and Z being the first three principal components) and the graph in
XYZ space. These graphs show the possibility of separation of the
sequence pixels in the principal component space by using linear
combination of the images in IR sequence. This can be used both
in the process of segmentation and recognition of singularities.
Fig. 4 presents the same 3D graph with indexed areas related to
specific singularities.
The above example relates to an object with eight singularities
differing in terms of depth and diameter. These differences were
visible, among others, through the grouping of pixels in the
principal components for the various singularities in separate
groups.
In many cases, the number of singularity types is reduced. Such
two examples will be described below. The first one relates to
thin-layer-chromatography plate (SiO2 on glass) with separated
analytes (fullerenes C70 and C60) [10] and the second to silicon-
eutectic-molybdenum thyristor structure. Both the measuring
Measurement Automation Monitoring, Jun. 2015, vol. 61, no. 06  217

system and the parameters of the stimulation and recording were details of fullerenes. Their contrast and signal-to-noise ratio is
the same as in the previously discussed example. clearly better than in the best image of the temperature sequence.
The second image of PCA sequence contains mostly information
about changing the thickness of the SiO2 layer (stationary phase)
XY YZ and the third and subsequent images contain thermal background
noise signal. Likewise, the coordinates image of the first three
principal components (Fig. 7) shows three types of clusters -
concentrated points representing an area of increased thickness of
the SiO2 stationary phase, scattered but separating themselves
group of pixels corresponding to the areas of concentrated
fullerenes and points related to the thermal background (noise).

XZ XYZ 2-nd PCA Image 3-rd PCA Image

The best image of 1-st PCA Image (thickness (background
thermal sequence (fullerenes) irregularities) noises)

Fig. 3. Principal images of PCA sequence in 2D and 3D projections

Fig. 6. The best image of IR sequence and three successive images of PCA sequence
for TLC plate

1 2
4
6
3
10 11 2
5 9 7
8 5
4
3
7
8 6

Fig. 7. 3D distribution of pixels for TLC plate

Fig. 4. Grouping of thermal heterogeneities pixels in principal component 3D space
(black dots refer to the thermal background)
The outcome of the third analysis relates to the structure of
Si-eutectic-Mo. In this case, there are also evident two types of
singularities that are represented in two consecutive images in
Fullerenes SiO2 PCA sequence (Fig. 8). The third PCA image has already been
a noise image.

Glass

a)

Silicon
Fig. 8. The best image of IR sequence and three successive images of PCA sequence
for thyristor structure Si-eutectic-Mo with two types of non-uniformities

Eutectic Image of pixels distribution in three principal coordinates of the

Molybdenum
layer
b)
Fig. 5. Thin-layer-chromatography plate (SiO2 on glass) with separated analytes,
a) fullerenes C70 and C60, b) structure Si-eutectic-Mo
For both structures the scree plot happened to be even steeper.
This was reflected in an even smaller number of principal images,
containing in successive frames correlated content, which could
also be found in the thermal images. In case of the plates with
fullerene (Fig. 6) the first image of the PCA sequence contains
PCA (Fig. 9) shows a clear separation of singularity no. 2 and
overlapping sets of inhomogeneity no.1 and background. This is
understandable, because the heterogeneity no.1 involves smooth
change of the eutectic thickness and it is the reason why the border
between area no.1 and the background is difficult to define.
Heterogeneity no.2, in turn, involves structural changes in eutectic
area (cracks) having a strong effect on the thermal impedance.
Assuming, in this case, that only two first images of the PCA
sequence contain information featuring characteristics of the
temperature contrast (Fig. 8), and the rest is primarily
a representation of the noise, one can remove them from the
sequence of PCA. Restoring original sequence of temperature
218 
should then improve signal-to-noise ratio. This process is very
similar to filtering the signal using Fourier analysis or wavelet
decomposition.
Fig. 9. 3D distribution of pixels for thyristor Si-eutectic-Mo structure
Temperature
sequence before
PCA
Temperature sequence
reconstructed from
two principal images
of PCA sequence
frame
Fig. 10. Temperature images and temperature signals of a single pixel for original
IR sequence and for sequence reconstructed from two first images of the
PCA sequence
Elimination (zero filling) of all PCA sequence frames with an
index greater than two and reconstructing temperature sequence
using this basis, resulted in a very favorable improvement in
temperature signal-to-noise ratio. Filtration was done in the time
domain and hardly affected signal-to-noise ratio on each
individual frame of the sequence.
5. Conclusions
The Principal Components Analysis applied to the
thermographic image sequence allows the concentration of
information about the characteristics of the object in the principal
images. The main result of the analysis is a sequence of so-called
principal images, which are linear combination of the images of
temperature sequence. They are ordered accordingly to the amount
of the information contained in them - in many cases for a very
good representation of the characteristics of the object it is
sufficient to take into account only few first images of the PCA
sequence. Degree of information concentration depends on the
diversity of the test object characteristics. Observation of the
individual images of this sequence allows elimination of images
that do not contain important information about the characteristics
of the object, which in turn improves the signal-to-noise ratio
without significant loss of information about the object. In this
way PCA enables convenient interpretation of heterogeneities and
their separation.
This work was supported by the European Union under European Social Fund and
the Polish State Budget, Operational Programme Human Capital Priority VIII,
Measure 8.2 Transfer of knowledge Measure 8.2.2 "Regional Innovation Strategies"
system project implemented by the Regional Labour Office in Szczecin, "Investment
Measurement Automation Monitoring, Jun. 2015, vol. 61, no. 06
in knowledge as an engine for development innovation in the region - the third
edition”
6. References
[1] Więcek B., Strzelecki M., Jakubowska T., Wysocki M., Peszyński-
Drews C.: Advanced Thermal Image Processing. The Biomedical
Engineering Handbook, vol.2 (Medical devices and systems) pp.28_1-
28_13, 2006
[2] Arsoba R., Suszyński Z.: Application of Photoacoustic Method and
Evolutionary Algorithm for Determination of Thermal Properties of
Layered Structure. Journal de Physique, vol.117, pp. 1-6, 2004.
[3] Seidel U., Lan T.T., Heinz-Guenter W., Schmitz B. and others:
Quantitative characterization of material inhomogeneities by thermal
waves. Society of Photo-Optical Instrumentation Engineers, vol. 36,
(376), pp.376-390, 1997.
[4] Suszyński Z., Maliński M., Bychto L.: Thermal Parameters
Measurement Method of Electronics Materials. IEEE Trans. On
Comp., Pack. And Manufacturing Techn., Part A, vol.21, no.3,
pp.424-433, 1998.
[5] Ravi J., Lu Y., Longuemart S.: Optothermal depth profiling by neural
network infrared radiometry signal recognition. Journal of Applied
Physics, 97, 014701, 2005.
[6] Dudzik S.: Characterization of material defects using active
thermography and an artificial neural network. Metrology &
Measurement Systems, vol.XX, no.3 pp.491-500, 2013.
[7] Majchrzak P., Suszyński Z.: New Approach in Analysis of Sensitivity
of Temperature Response to Selected Parameters of Two-Layer
Structure. International Journal of Thermophysics, vol.32, pp. 836-
843, 2010.
[8] Rajic N.: Principal component thermography for flaw contrast
enhancement and flaw depth characterization in composite structures.
Composite Structures, vol. 58, pp. 521-8, 2002.
[9] Demmel J., Kahan W.: Accurate singular values of bidiagonal
matrices. Society for Industrial and Applied Mathematics. Journal on
Scientific and Statistical Computing 11 (5), pp.873–912, 1990.
[10] Suszyński Z., Zarzycki P.K.: New approach for sensitive photothermal
detection of C60 and C70 fullerens on micro-thin-layer
chromatographic plates. Analitica Chimica Acta, vol.863, pp.70-77,
2015.
_____________________________________________________
Received: 14.03.2015 Paper reviewed Accepted: 05.05.2015
Prof. Zbigniew SUSZYŃSKI
Head of Multimedia Systems and Artificial Intelligence
Faculty in Koszalin University of Technology. Studies
and PhD - Department of Electron Technology at the
Technical University of Kiev. Habilitation colloquium at
the Institute of Electron Technology Academy of
Sciences in Warsaw. Interests: active thermography,
thermal tomography, signal and image processing and
use of the GA and ANN.
e-mail: zbigniew.suszynski@tu.koszalin.pl
M.Sc. Robert ŚWITA
Graduated from the Faculty of Electronics,
Telecommunications and Computer Science in
Gdansk University of Technology. PhD student at the
Faculty of Electronics and Computer Science in
Koszalin University of Technology. Assistant in the
Department of Multimedia Systems and Artificial
Intelligence. Research interests: linear algebra, 3D
graphics, audio and video processing, multimedia
technologies.
e-mail: robert.swita@wp.pl