90
IEEE TRANSACTIONS ON COMPUTERS, JANUARY 1974
_{I}_{f} _{o}_{n}_{l}_{y} _{t}_{h}_{e} _{L} _{v}_{a}_{l}_{u}_{e}_{s} _{R}_{o} _{t}_{h}_{r}_{o}_{u}_{g}_{h} RL_1 are required, then it can be _{s}_{e}_{e}_{n} by comparing (15) and (11) that the new algorithm will require fewer multiplications than theFFT method if
< 11.2 (1 + log2 N) _{a}_{n}_{d} willrequirefewer additions if
L
_{L}
< _{5}_{.}_{6} _{(}_{1} + log2 N).
(16a)
(16b)
_{T}_{h}_{e}_{r}_{e}_{f}_{o}_{r}_{e}_{,} _{w}_{e} _{c}_{o}_{n}_{c}_{l}_{u}_{d}_{e} _{t}_{h}_{a}_{t} the new algorithm will generally be more efficientthan theFFT method if
or
_{N} _{<} _{1}_{2}_{8}
_{L} _{<} _{1}_{0}_{(}_{1} + _{l}_{o}_{g}_{2} _{N}_{)}_{.}
CONCLUSION
(17a)
(17b)
_{A} _{n}_{e}_{w} _{a}_{l}_{g}_{o}_{r}_{i}_{t}_{h}_{m} for computingthe correlation ofablock of sampled _{d}_{a}_{t}_{a} has been presented. It is a direct method which trades an in _{c}_{r}_{e}_{a}_{s}_{e}_{d} _{n}_{u}_{m}_{b}_{e}_{r} _{o}_{f} _{a}_{d}_{d}_{i}_{t}_{i}_{o}_{n}_{s} _{f}_{o}_{r} _{a}_{d}_{e}_{c}_{r}_{e}_{a}_{s}_{e}_{d} _{n}_{u}_{m}_{b}_{e}_{r} ofmultiplications. _{F}_{o}_{r} _{a}_{p}_{p}_{l}_{i}_{c}_{a}_{t}_{i}_{o}_{n}_{s} _{w}_{h}_{e}_{r}_{e} _{t}_{h}_{e} _{"}_{c}_{o}_{s}_{t}_{"} _{(}_{e}_{.}_{g}_{.}_{,}_{t}_{h}_{e} _{t}_{i}_{m}_{e}_{)} ofamultiplication is _{g}_{r}_{e}_{a}_{t}_{e}_{r} _{t}_{h}_{a}_{n} _{t}_{h}_{a}_{t} ofan addition, the new algorithm isalways more com _{p}_{u}_{t}_{a}_{t}_{i}_{o}_{n}_{a}_{l}_{l}_{y} efficient than direct evaluation ofthe correlation, and itis _{g}_{e}_{n}_{e}_{r}_{a}_{l}_{l}_{y} _{m}_{o}_{r}_{e} _{e}_{f}_{f}_{i}_{c}_{i}_{e}_{n}_{t} _{t}_{h}_{a}_{n} _{F}_{F}_{T} methods forprocessing 128 or fewer datapoints,orforcalculatingonlythe firstL "lags"forL < 10 log2 2N.
REFERENCES
[1] _{S}_{.}
Winograd,
Comput.
"A new
algorithm
C17,
_{f}_{o}_{r} inner
(Corresp.), vol.
pp. 693694,
product,"
July
1968.
IEEE Trans.
DiscreteCosineTransfonn
N. _{A}_{H}_{M}_{E}_{D}_{,} T. NATARAJAN, AND
K. R. RAO
AbstractA discrete cosine transform (DCT) is defined and an algo _{r}_{i}_{t}_{h}_{m} _{t}_{o} _{c}_{o}_{m}_{p}_{u}_{t}_{e} _{i}_{t} _{u}_{s}_{i}_{n}_{g} _{t}_{h}_{e} _{f}_{a}_{s}_{t}_{F}_{o}_{u}_{r}_{i}_{e}_{r} _{t}_{r}_{a}_{n}_{s}_{f}_{o}_{r}_{m} isdeveloped. Itis
shown that the discrete cosine transform can be used in the area of _{d}_{i}_{g}_{i}_{t}_{a}_{l} _{p}_{r}_{o}_{c}_{e}_{s}_{s}_{i}_{n}_{g} for the purposes of pattern recognition and Wiener _{f}_{i}_{l}_{t}_{e}_{r}_{i}_{n}_{g}_{.} _{I}_{t}_{s}_{p}_{e}_{r}_{f}_{o}_{r}_{m}_{a}_{n}_{c}_{e} _{i}_{s}_{c}_{o}_{m}_{p}_{a}_{r}_{e}_{d} with thatofaclassoforthogonal _{t}_{r}_{a}_{n}_{s}_{f}_{o}_{r}_{m}_{s} and _{i}_{s} found to compare closely to that of the Karhunen _{L}_{o}_{'}_{e}_{v}_{e} _{t}_{r}_{a}_{n}_{s}_{f}_{o}_{r}_{m}_{,} _{w}_{h}_{i}_{c}_{h} _{i}_{s}_{k}_{n}_{o}_{w}_{n} _{t}_{o} _{b}_{e} _{o}_{p}_{t}_{i}_{m}_{a}_{l}_{.} The performancesof the KarhunenLo'eve and discrete cosine transforms are also found to _{c}_{o}_{m}_{p}_{a}_{r}_{e} closelywithrespect to theratedistortion criterion.
Index TermsDiscrete cosine transform, discrete Fourier transform, _{f}_{e}_{a}_{t}_{u}_{r}_{e} _{s}_{e}_{l}_{e}_{c}_{t}_{i}_{o}_{n}_{,} _{H}_{a}_{a}_{r} _{t}_{r}_{a}_{n}_{s}_{f}_{o}_{r}_{m}_{,} KarhunenLoeve transform, ratedis
tortion,WalshHadamard transform, Wiener vector and scalarfiltering.
INTRODUCTION
_{I}_{n} _{r}_{e}_{c}_{e}_{n}_{t} _{y}_{e}_{a}_{r}_{s} _{t}_{h}_{e}_{r}_{e} _{h}_{a}_{s} _{b}_{e}_{e}_{n} an increasing interest with respect to _{u}_{s}_{i}_{n}_{g} _{a} class of orthogonal transforms in the general area of digital _{s}_{i}_{g}_{n}_{a}_{l} _{p}_{r}_{o}_{c}_{e}_{s}_{s}_{i}_{n}_{g}_{.} This correspondence addresses itself towards two
_{p}_{r}_{o}_{b}_{l}_{e}_{m}_{s} _{a}_{s}_{s}_{o}_{c}_{i}_{a}_{t}_{e}_{d} with _{i}_{m}_{a}_{g}_{e} processing, namely, pattem recogni
_{t}_{i}_{o}_{n} _{[}_{1} ] andWiener filtering [2]. In _{p}_{a}_{t}_{t}_{e}_{r}_{n} recognition, orthogonal transforms enable a noninvertible transformation from the pattern space to a reduced dimensionality _{f}_{e}_{a}_{t}_{u}_{r}_{e} _{s}_{p}_{a}_{c}_{e}_{.} This _{a}_{l}_{l}_{o}_{w}_{s} a classification scheme to be implemented
with substantially less features, with only a small increase inclassifica tion error.
_{I}_{n} _{d}_{i}_{s}_{c}_{r}_{e}_{t}_{e}_{W}_{i}_{e}_{n}_{e}_{r} filteringapplications, the filterisrepresented by an
_{(}_{M} _{X} _{M}_{)} _{m}_{a}_{t}_{r}_{i}_{x} _{G}_{.} _{T}_{h}_{e} _{e}_{s}_{t}_{i}_{m}_{a}_{t}_{e} _{X} of data vector X is _{g}_{i}_{v}_{e}_{n} _{b}_{y} _{G}_{Z}_{,} _{w}_{h}_{e}_{r}_{e} _{Z} = _{X} + _{N} _{a}_{n}_{d} _{N} _{i}_{s} _{t}_{h}_{e} _{n}_{o}_{i}_{s}_{e} _{v}_{e}_{c}_{t}_{o}_{r}_{.} Thisimplies that _{a}_{p}_{p}_{r}_{o}_{x}_{i}_{}
mately
2M arithmetic
2A
operations
are
a _{G} in
required
which
to
compute
_{a} substantial
X.
Use of
orthogonal transforms yields
number of
_{e}_{l}_{e}_{m}_{e}_{n}_{t}_{s} are relatively small in magnitude, and hence can be set equal
_{t}_{o} _{z}_{e}_{r}_{o}_{.} Thus a significantreduction in computation loadisrealized at
_{t}_{h}_{e} _{e}_{x}_{p}_{e}_{n}_{s}_{e} ofa smallincrease in the meansquare estimation error. The WalshHadamard transform (WHT), discrete Fourier transform _{(}_{D}_{F}_{T}_{)}_{,} _{t}_{h}_{e} _{H}_{a}_{a}_{r} _{t}_{r}_{a}_{n}_{s}_{f}_{o}_{r}_{m} _{(}_{H}_{T}_{)}_{,} _{a}_{n}_{d} _{t}_{h}_{e} slant transform (ST), have _{b}_{e}_{e}_{n} _{c}_{o}_{n}_{s}_{i}_{d}_{e}_{r}_{e}_{d} _{f}_{o}_{r} various applications [1], [2], [4][9 since these
_{a}_{r}_{e} _{o}_{r}_{t}_{h}_{o}_{g}_{o}_{n}_{a}_{l} _{t}_{r}_{a}_{n}_{s}_{f}_{o}_{r}_{m}_{s} _{t}_{h}_{a}_{t} _{c}_{a}_{n} _{b}_{e} _{c}_{o}_{m}_{p}_{u}_{t}_{e}_{d} _{u}_{s}_{i}_{n}_{g} fastalgorithms.
_{T}_{h}_{e} _{p}_{e}_{r}_{f}_{o}_{r}_{m}_{a}_{n}_{c}_{e} of these transforms is generally compared with that
of the KarhunenLoeve transform (KLT) which isknown to be optimal with respect to the following performance measures: variance distribu tion _{[}_{1} ] , estimationusing the meansquare error criterion [2], [4],and the ratedistortion function [5]. Although the KLT isoptimal, there is _{n}_{o} _{g}_{e}_{n}_{e}_{r}_{a}_{l} _{a}_{l}_{g}_{o}_{r}_{i}_{t}_{h}_{m} thatenablesitsfastcomputation [1] _{I}_{n} _{t}_{h}_{i}_{s} correspondence, a discrete cosine transform (DCT) is intro
_{d}_{u}_{c}_{e}_{d} _{a}_{l}_{o}_{n}_{g} _{w}_{i}_{t}_{h} _{a}_{n} _{a}_{l}_{g}_{o}_{r}_{i}_{t}_{h}_{m} that enables its fast computation. Itis
_{s}_{h}_{o}_{w}_{n} that the performance ofthe DCT compares more closely to that
_{o}_{f} the KLT
relative to the performances of the DFT, WHT, and HT.
_{D}_{I}_{S}_{C}_{R}_{E}_{T}_{E} _{C}_{O}_{S}_{I}_{N}_{E} TRANSFORM
_{T}_{h}_{e} _{D}_{C}_{T}
_{o}_{f} _{a} _{d}_{a}_{t}_{a} _{s}_{e}_{q}_{u}_{e}_{n}_{c}_{e} _{X}_{(}_{m}_{)}_{,} _{m} = _{0}_{,} _{1}_{,} _{*}_{*}_{,} (M  1)isdefmed
as
/2Mi
Gx(O)"Z
X(m)
m=o
_{2} Mi
Gx(k)=
_{M} m0
X(m)Cos
(2m+1)kn
2M
k ^{=} 1,2,

 , (M 1)
(1)
where Gx(k)
the set of
_{i}_{s} _{t}_{h}_{e} _{k}_{t}_{h} DCT coefficient. It is worthwhile
noting
that
_{b}_{a}_{s}_{i}_{s} vectors {1/12, cos ((2m + 1) kfl)I(2AM)} is actually a
_{c}_{l}_{a}_{s}_{s} _{o}_{f} _{d}_{i}_{s}_{c}_{r}_{e}_{t}_{e} _{C}_{h}_{e}_{b}_{y}_{s}_{h}_{e}_{v} polynomials. This can be seen by recalling thatChebyshev polynomials can be defined as [3]
1
To(Qp) =
TkQp) = cos (k cos1 _{p}_{)}_{,}
k,p = 1, 2,
,M
(2)
where Tk(Qp)
Now, in
by [31
isthe kth Chebyshev polynomial.
_{(}_{2}_{)}_{,} _{t}_{p} _{i}_{s} _{c}_{h}_{o}_{s}_{e}_{n} _{t}_{o} _{b}_{e} _{t}_{h}_{e}_{p}_{t}_{h} _{z}_{e}_{r}_{o} _{o}_{f} _{T}_{M}_{(}_{t}_{)}_{,} which isgiven
tp = Cos ^{(}^{2}^{p}
^{O}^{l}^{l}
2M
^{I}
^{t}^{,}
= 13 21 ...
M.
(3)
_{S}_{u}_{b}_{s}_{t}_{i}_{t}_{u}_{t}_{i}_{n}_{g} _{(}_{3}_{)} _{i}_{n} _{(}_{2}_{)}_{,} _{o}_{n}_{e} obtains the set of Chebyshev polynomials
A
1
To(P) =
_
A
Tk(p) ^{=} cos
(2pl)kFI
k,p=1, 2,"* ,M.
_{(}_{4}_{)}
From (4)itfollowsthatthe Tk(p) can equivalently be defmedas
1
Manuscript
N.
received January 29, 1973;
the
Departments
revised June
of Electrical
Manhattan,
7, 1973.
Ahmed is with
R. Rao is
Engineering and
Kans.
Computer Science,
T.
State
K.
Natarajan University,
Kansas State
University,
iswith the
Manhattan,
Department
Kans.
of Electrical
Engineering, Kansas
with the Department of
Electrical Engineering, Uni
_{v}_{e}_{r}_{s}_{i}_{t}_{y} of _{T}_{e}_{x}_{a}_{s} at Arlington,
Arlington, Tex. 76010.
Tk(m) = cos
(2m
+
l)krI
_{2}_{M}
k = 1,2,*  *,(M 1),
_{C}_{o}_{m}_{p}_{a}_{r}_{i}_{n}_{g} _{(}_{5}_{)} _{w}_{i}_{t}_{h} _{(}_{1}_{)} _{w}_{e} _{c}_{o}_{n}_{c}_{l}_{u}_{d}_{e} thatthebasismember cos ((2m +
CORRESPONDENCE
91
1)kll)/(2M) is
the kth Chebyshev polynomial _{T}_{k}_{(}_{)} evaluated at the
mth zero ofTM{t). Again, the inverse cosine discrete transform (ICDT) is _{d}_{e}_{f}_{i}_{n}_{e}_{d} _{a}_{s}
_{1}
X(m) = >2 Gx(°) +
2
M1
k=1
Gx(k) cos
(2m + 1)kfl
2M
m=0,1,** ,(M1). (6) We note thatapplyingthe _{o}_{r}_{t}_{h}_{o}_{g}_{o}_{n}_{a}_{l} _{p}_{r}_{o}_{p}_{e}_{r}_{t}_{y} _{[}_{3}_{]}
M1
_{2}_{2}
m=o
M/2,
Tk(m)Tj (m) = ^ M/2,
0,
to (6)yieldsthe DCT in _{(}_{1}_{)}_{.}
k
=I= O
k=l=
k = _{I} _{O} 0
kM*I
(7)
If(6) iswritten in matrix _{f}_{o}_{r}_{n} _{a}_{n}_{d} A _{i}_{s}_{t}_{h}_{e} _{(}_{M} X _{M}_{)} _{m}_{a}_{t}_{r}_{i}_{x} _{t}_{h}_{a}_{t}_{d}_{e}_{} notes the cosine transformation, then the _{o}_{r}_{t}_{h}_{o}_{g}_{o}_{n}_{a}_{l} _{p}_{r}_{o}_{p}_{e}_{r}_{t}_{y} can be expressed as
A
T
A=
M
2[I]
(8)
Fig. 1. Eigenvectors of _{(}_{8} X
8) Toeplitz
matrix (p = 0.9) and basis
vectors of the DCT.
where ATis the transpose ofA and [I] isthe (M X Al)identitymatrix.
MOTIVATION
The motivation for defining the DCT is that it can be _{d}_{e}_{m}_{o}_{n}_{s}_{t}_{r}_{a}_{t}_{e}_{d} that its basis set provides a good approximation to the eigenvectors _{o}_{f} the class of _{T}_{o}_{e}_{p}_{l}_{i}_{t}_{z} matrices defined as
[1
p
_{p}
1
p
2
p
'p
^{=}
IM1i
p
*
*
Mi
p
_{p}
M2
.
.
J
O<p<1. (9)
For the purposes of illustration, the eigenvectors of _{'}_{p} forM = 8 _{a}_{n}_{d} _{p} = 0.9areplotted _{(}_{s}_{e}_{e}_{F}_{i}_{g}_{.} _{1}_{)}_{a}_{g}_{a}_{i}_{n}_{s}_{t}
,cos(2m+1)k, k= _{1}_{,}_{2}_{,}_{}_{}_{,}_{7}_{,}_{m}_{=}_{0}_{,} _{1}
_{7}
_{(}_{1}_{0}_{)}
IOD"
)X DISCRETE COSINE
1.0
1.%.f
WI_
1.
_{\}
.1
_{0}_{.}_{1}
_{}_{1}
\
\N40 _{"}_{,}
HAAR
FOURIER
WALSHHADAMARD
KARHLINENLOeVE

0
2
_{4}
6
8
10
12
14
_{1}_{6}
TRANSFORM COMPONENT _{#}
Fig. 2. Transform domain variance;M = 16, _{p} = 0.95.
placed by Im _{{}_{*}_{}}_{,} which denotes the _{i}_{m}_{a}_{g}_{i}_{n}_{a}_{r}_{y} _{p}_{a}_{r}_{t} of the _{t}_{e}_{r}_{m} enclosed.
which constitute the basis set for the _{D}_{C}_{T}_{.} _{T}_{h}_{e} _{c}_{l}_{o}_{s}_{e} _{r}_{e}_{s}_{e}_{m}_{b}_{l}_{a}_{n}_{c}_{e} (aside from the 1800 phase shift) between the eigenvectors and the set
definedin(10)isapparent.
ALGORITHMS
Itcanbe shown that(1) can be expressed as
Gx(O) ^{=}
2 X(m)
m=o
2
2M1
'2E
Gx(k) =Re je'")
k
X(m) wkm}
COMPUTATIONAL _{R}_{E}_{S}_{U}_{L}_{T}_{S}
In image processing applications, ' in (9)providesausefulmodel for the _{d}_{a}_{t}_{a} covariance matrix _{c}_{o}_{r}_{r}_{e}_{s}_{p}_{o}_{n}_{d}_{i}_{n}_{g} to the rows and columns of an _{i}_{m}_{a}_{g}_{e} matrix _{[}_{6}_{1}_{,} _{[}_{7}_{]}_{.} The covariance matrix in the transform domain isdenotedby I and isgivenby
T=A'pA*T
(12)
_{w}_{h}_{e}_{r}_{e} A _{i}_{s} the _{m}_{a}_{t}_{r}_{i}_{x} _{r}_{e}_{p}_{r}_{e}_{s}_{e}_{n}_{t}_{a}_{t}_{i}_{o}_{n} of an _{o}_{r}_{t}_{h}_{o}_{g}_{o}_{n}_{a}_{l} transformation and A* is the _{c}_{o}_{m}_{p}_{l}_{e}_{x} _{c}_{o}_{n}_{j}_{u}_{g}_{a}_{t}_{e} of A. From _{(}_{1}_{2}_{)} it follows that T
can be computed asatwodimensional transform _{o}_{f}_{'}_{p}_{.}
FeatureSelection
where
k=1,2,
,(Ml) _{(}_{1}_{1}_{)}
A criterion for _{e}_{l}_{i}_{m}_{i}_{n}_{a}_{t}_{i}_{n}_{g} features _{(}_{i}_{.}_{e}_{.}_{,} _{c}_{o}_{m}_{p}_{o}_{n}_{e}_{n}_{t}_{s} of a transform _{v}_{e}_{c}_{t}_{o}_{r}_{)}_{,}_{w}_{h}_{i}_{c}_{h} areleastuseful forclassificationpurposes,was _{d}_{e}_{v}_{e}_{l}_{o}_{p}_{e}_{d} _{b}_{y} Andrews _{[}_{1}_{]}_{.} It states that features whose variances _{(}_{i}_{.}_{e}_{.}_{,} main
_{W} = ee21/2M _{i}_{=}_{j}_{i}_{/}_{j}
X(m) = 0,
m =M, (M+ 1),
,(2M  _{1}_{)}
and Re {} implies the real part of the term enclosed. From _{(}_{1}_{1}_{)} it
follows that all theM DCT coefficients can be _{c}_{o}_{m}_{p}_{u}_{t}_{e}_{d} _{u}_{s}_{i}_{n}_{g} a 2M _{p}_{o}_{i}_{n}_{t} fast Fourier transform _{(}_{F}_{F}_{T}_{)}_{.} _{S}_{i}_{n}_{c}_{e} _{(}_{1}_{)} _{a}_{n}_{d} _{(}_{6}_{)} _{a}_{r}_{e} _{o}_{f}_{t}_{h}_{e}_{s}_{a}_{m}_{e}
_{d}_{i}_{a}_{g}_{o}_{n}_{a}_{l} _{e}_{l}_{e}_{m}_{e}_{n}_{t}_{s} _{o}_{f} _{I}_{)} _{a}_{r}_{e} _{r}_{e}_{l}_{a}_{t}_{i}_{v}_{e}_{l}_{y} _{l}_{a}_{r}_{g}_{e} shouldbe retained. _{(}_{F}_{i}_{g}_{.}_{2} should be retained.) Fig. 2 shows the various variances ranked in de
_{c}_{r}_{e}_{a}_{s}_{i}_{n}_{g} order of _{m}_{a}_{g}_{n}_{i}_{t}_{u}_{d}_{e}_{.}
From the information in _{F}_{i}_{g}_{.} 2 it is
_{a}_{p}_{p}_{a}_{r}_{e}_{n}_{t} that relative to the set of _{o}_{r}_{t}_{h}_{o}_{g}_{o}_{n}_{a}_{l} _{t}_{r}_{a}_{n}_{s}_{f}_{o}_{r}_{m}_{s} _{s}_{h}_{o}_{w}_{n}_{,} the DCT comparesmost closelyto theKLT.
WienerFiltering
_{f}_{o}_{r}_{m}_{,} FFT can also be used to _{c}_{o}_{m}_{p}_{u}_{t}_{e} crete sine transform were _{d}_{e}_{f}_{i}_{n}_{e}_{d}_{,} _{t}_{h}_{e}_{n}
the IDCT. Similarly,
ifadis The role of orthogonal transforms played in filteringapplicationsis
illustrated _{i}_{n} _{F}_{i}_{g}_{.} _{3} _{[}_{2}_{]}_{.} Z _{i}_{s} _{a}_{n} _{(}_{M} X _{1}_{)} vector which isthesum ofa
theRe {Iin(11)would bere
92
IEEE TRANSACTIONS ON _{C}_{O}_{M}_{P}_{U}_{T}_{E}_{R}_{S}_{,} JANUARY 1974
Fig. 3. Wiener filtering model.
TABLE I _{M}_{E}_{A}_{N}_{}_{S}_{Q}_{U}_{A}_{R}_{E} ERROR PEFORMANCE OF VARIOUS TRANSFORMS FOR SCALAR WIENER _{F}_{I}_{L}_{T}_{E}_{R}_{I}_{N}_{G}_{;} _{P} = _{0}_{.}_{9}
M
Transform 
2 
4 
8 
16 
32 
_{6}_{4} 
KarhunenLoeve 
0.3730 
0.2915 
_{0}_{.}_{2}_{5}_{3}_{3} 
0.2356 
0.2268 
_{0}_{.}_{2}_{2}_{2}_{4} 
Discrete cosine 
_{0}_{.}_{3}_{7}_{3}_{0} 
_{0}_{.}_{2}_{9}_{2}_{0} 
_{0}_{.}_{2}_{5}_{4}_{6} 
0.2374 
0.2282 
_{0}_{.}_{2}_{2}_{3}_{2} 
Discrete Fourier 
0.3730 
0.2964 
0.2706 
0.2592 
0.2441 
0.2320 
WalshHadamard 
_{0}_{.}_{3}_{7}_{3}_{0} 
0.2942 
_{0}_{.}_{2}_{6}_{4}_{9} 
0.2582 
0.2582 
_{0}_{.}_{2}_{5}_{5}_{9} 
Haar 
0.3730 
0.2942 
0.2650 
0.2589 
0.2582 
0.2581 
0a 

: 

cr 

cE 

:D 

z 

4Lhi 

SIZE OF 0 
_{f} 
_{F}_{i}_{g}_{.} 4. _{M}_{e}_{a}_{n}_{}_{s}_{q}_{u}_{a}_{r}_{e} error _{p}_{e}_{r}_{f}_{o}_{r}_{m}_{a}_{n}_{c}_{e} ofvarious transforms for scalar Wiener _{f}_{i}_{l}_{t}_{e}_{r}_{i}_{n}_{g}_{;}_{p} _{=} 0.9.
vector X and a noise vectorN. X is considered to belong to arandom
process whose covariance matrix isgiven _{b}_{y} _{i}_{)} _{w}_{h}_{i}_{c}_{h} _{i}_{s}_{d}_{e}_{f}_{i}_{n}_{e}_{d} _{i}_{n} _{(}_{9}_{)}_{.} The Wiener filter G is in the form of an (M X _{M}_{)} matrix. A and _{A}_{}_{'}
represent an orthonormal transform and _{i}_{t}_{s} _{i}_{n}_{v}_{e}_{r}_{s}_{e}_{,} _{r}_{e}_{s}_{p}_{e}_{c}_{t}_{i}_{v}_{e}_{l}_{y}_{,} _{w}_{h}_{i}_{l}_{e}
X denotes theestimate ofX, _{u}_{s}_{i}_{n}_{g} themeansquare _{e}_{r}_{r}_{o}_{r} _{c}_{r}_{i}_{t}_{e}_{r}_{i}_{o}_{n}_{.} We restrict our attention to the case when G is constrained to be a
diagonal matrix Q. This class ofWiener filtersis _{r}_{e}_{f}_{e}_{r}_{r}_{e}_{d} _{t}_{o} _{a}_{s} _{s}_{c}_{a}_{l}_{a}_{r} filters while the more _{g}_{e}_{n}_{e}_{r}_{a}_{l} _{c}_{l}_{a}_{s}_{s} _{(}_{d}_{e}_{n}_{o}_{t}_{e}_{d} _{b}_{y} _{G}_{)} _{i}_{s} _{r}_{e}_{f}_{e}_{r}_{r}_{e}_{d} _{t}_{o} _{a}_{s}
vector filters. The additive noise (see Fig. 3) isconsidered tobe white,
zero _{m}_{e}_{a}_{n}_{,} and uncorrelated with the _{d}_{a}_{t}_{a}_{.} Ifthemeansquare _{e}_{s}_{t}_{i}_{m}_{a}_{}
tion error due
expressed as [4]
to scalar filtering is denoted _{b}_{y} _{e}_{Q}_{,} _{t}_{h}_{e}_{n} _{e}_{(} _{c}_{a}_{n} _{b}_{e}
1 M
1
_{E}
e=
'Ix(s,
x(s,S)+
s)
In(s,s)
(13)
where _{'}_{V}_{x} and _{T}_{n} denote the transform domain covariancematricesof the _{d}_{a}_{t}_{a}_{a}_{n}_{d} _{n}_{o}_{i}_{s}_{e}_{,} _{r}_{e}_{s}_{p}_{e}_{c}_{t}_{i}_{v}_{e}_{l}_{y}_{.} _{T}_{a}_{b}_{l}_{e} Iliststhevaluesof _{e}_{Q} _{f}_{o}_{r}_{d}_{i}_{f}_{f}_{e}_{r}_{}
ent _{v}_{a}_{l}_{u}_{e}_{s} ofM for _{t}_{h}_{e} _{c}_{a}_{s}_{e} _{p} = 0.9 and a _{s}_{i}_{g}_{n}_{a}_{l}_{}_{t}_{o}_{}_{n}_{o}_{i}_{s}_{e} ratio ofunity. From Table I it is evident that the DCT comes closest to the KLT which is optimal. This information is _{p}_{r}_{e}_{s}_{e}_{n}_{t}_{e}_{d} in terms of _{a} set of
_{p}_{e}_{r}_{f}_{o}_{r}_{m}_{a}_{n}_{c}_{e} curves inFig.4.
ADDITIONAL CONSIDERATIONS In conclusion, we compare the performance of the DCT with _{K}_{L}_{T}_{,}
DFT, and the identity transforms, using the ratedistortion criterion _{[}_{5}_{1}_{.} This _{p}_{e}_{r}_{f}_{o}_{r}_{m}_{a}_{n}_{c}_{e} criterionprovidesa measure oftheinformation rate R that can be achievedwhile stillmaintaining a fixeddistortionD, for encoding purposes. Considering Gaussian sources _{a}_{l}_{o}_{n}_{g} with the
_{m}_{e}_{a}_{n}_{}_{s}_{q}_{u}_{a}_{r}_{e} error _{c}_{r}_{i}_{t}_{e}_{r}_{i}_{o}_{n}_{,} the ratedistortion _{p}_{e}_{r}_{f}_{o}_{r}_{m}_{a}_{n}_{c}_{e} measure
isgivenby _{[}_{5}_{]}
_{1}
MD
^{2}^{M} i=1
(/{ I
l
(14a)
CORRESPONDENCE
93
Minimization ofLinearSequentialMachines
CHITSONG CHEN
AbstractAn algorithm is presented to minimize linear _{s}_{e}_{q}_{u}_{e}_{n}_{t}_{i}_{a}_{l} machinesto reduced _{f}_{o}_{r}_{m}_{.}
Index _{T}_{e}_{r}_{m}_{s}_{}_{A}_{l}_{g}_{o}_{r}_{i}_{t}_{h}_{m}_{,} linear _{m}_{a}_{c}_{h}_{i}_{n}_{e}_{,} _{m}_{i}_{n}_{i}_{m}_{i}_{z}_{a}_{t}_{i}_{o}_{n}_{,} reduced _{m}_{a}_{} chine,reduction, sequential machine.
Fig. 5. Rate versus distortion forM = 16 and p = _{0}_{.}_{9}_{.}
^{D}
1 M
^{=} ^{}
)
(14b)
where A denotes the orthogonal transformation and the aj _{a}_{r}_{e} _{t}_{h}_{e} main diagonal terms of the _{t}_{r}_{a}_{n}_{s}_{f}_{o}_{r}_{m} _{d}_{o}_{m}_{a}_{i}_{n} _{c}_{o}_{v}_{a}_{r}_{i}_{a}_{n}_{c}_{e} _{m}_{a}_{t}_{r}_{i}_{x} '1
in(12). 

The ratedistortion pertaining to M = 16 _{a}_{n}_{d} _{p} = _{0}_{.}_{9} _{i}_{s} _{s}_{h}_{o}_{w}_{n} _{i}_{n} 

Fig. 5, 
from which it is evident that the KLT and DCT _{c}_{o}_{m}_{p}_{a}_{r}_{e} _{m}_{o}_{r}_{e} 
closely than theKLT and _{D}_{F}_{T}_{.}
SUMMARY
It has been shown that the DCT can be used in the area of _{i}_{m}_{a}_{g}_{e} _{p}_{r}_{o}_{c}_{e}_{s}_{s}_{i}_{n}_{g} _{f}_{o}_{r} the _{p}_{u}_{r}_{p}_{o}_{s}_{e}_{s} of feature selection in _{p}_{a}_{t}_{t}_{e}_{r}_{n} _{r}_{e}_{c}_{o}_{g}_{n}_{i}_{t}_{i}_{o}_{n}_{;} _{a}_{n}_{d} _{s}_{c}_{a}_{l}_{a}_{r}_{}_{t}_{y}_{p}_{e} _{W}_{i}_{e}_{n}_{e}_{r} _{f}_{i}_{l}_{t}_{e}_{r}_{i}_{n}_{g}_{.} Itsperformancecompares _{c}_{l}_{o}_{s}_{e}_{l}_{y}_{w}_{i}_{t}_{h} that of the _{K}_{L}_{T}_{,} which isconsidered to be _{o}_{p}_{t}_{i}_{m}_{a}_{l}_{.} The _{p}_{e}_{r}_{f}_{o}_{r}_{m}_{a}_{n}_{c}_{e}_{s} of _{t}_{h}_{e} KLT and DCT are also found to _{c}_{o}_{m}_{p}_{a}_{r}_{e} _{c}_{l}_{o}_{s}_{e}_{l}_{y}_{,} with _{r}_{e}_{s}_{p}_{e}_{c}_{t}
to _{t}_{h}_{e} ratedistortion criterion.
ACKNOWLEDGMENT
The authors wish to thankDr. H. C. _{A}_{n}_{d}_{r}_{e}_{w}_{s}_{,} _{U}_{n}_{i}_{v}_{e}_{r}_{s}_{i}_{t}_{y} ofSouthern
California, for hisvaluable suggestions _{p}_{e}_{r}_{t}_{a}_{i}_{n}_{i}_{n}_{g} to the ratedistortion
computation.
REFERENCES
_{l}_{1}_{l} H. C. Andrews, "Multidimensional _{r}_{o}_{t}_{a}_{t}_{i}_{o}_{n}_{s} in feature _{s}_{e}_{l}_{e}_{c}_{t}_{i}_{o}_{n}_{,}_{"}
IEEE _{[}_{2}_{1} W. K.
[3]
niques,"
C.
141
151
[61 W. K.
171
181
Trans. Comput.,
T.
Pratt,
IEEE Trans.
vol.
C20, pp.
10451051, Sept. 1971.
filtering computation
C21,
of
pp. 636641,
Mathematical
July
"Generalized Wiener
Comput.,
vol.
Evaluation
PrenticeHall,
of random
tech
1972.
Fike, Computer
Cliffs,
N. J.:
Functions.
Englewood
J.
Pearl,
tromag.
J.
Pearl,
1968, ch. 6 and 7.
"Walsh
processing
signals," IEEE Trans. Elec
Compat., vol.
H.
EMC13, pp.
and
W. K.
IEEE
137141, Aug. 1971.
C. Andrews,
Pratt, "Performance measures Trans. Commun. _{T}_{e}_{c}_{h}_{n}_{o}_{l}_{.}_{,}
image processing
and two
for transform _{d}_{a}_{t}_{a}
vol.
COM20, pp.
Pratt,
coding,"
411415, June 1972.
"Walsh functions in
_{6}_{5}_{0}_{,} _{p}_{p}_{.} 1422.
dimensional filtering," in 1972
Functions, AD744
W.
K.
coding,"
pp.
229234.
W.
Proc. Symp. Applicationsof Walsh
transforms in image
Pratt, L. R. Welch, and W. Chen, _{"}_{S}_{l}_{a}_{n}_{t}
in
1972 Proc. Symp. Applications of Walsh Functions,
Pratt,
"Color
image coding
with the slant
Walsh Functions,
Symp. Applicationsof
coding of color _{i}_{m}_{a}_{g}_{e}_{s}_{,}_{"} _{I}_{E}_{E}_{E}
pp. 980992, Dec. 1971.
H. Chen and W. K.
transform," in 1973 Proc.
AD763000, pp.
155161.
Pratt, "Spatial
[91 W. K.
transform
Trans. Commun. Technol., vol. COM19,
I. INTRODUCTION
The minimization of a linear sequential machine to areduced form _{i}_{s}
an important topic and is discussed in _{m}_{a}_{n}_{y} texts _{[}_{1} _{}_{[}_{4}_{]}_{.} The _{m}_{i}_{n}_{i}_{}
mization procedure presented in _{[}_{1}_{]}_{}_{[}_{4}_{]} isasfollows. Let _{{}_{A}_{,}_{B}_{,} _{C}_{,}_{D}_{}}
be an ndimensional linear machine over _{G}_{F}_{(}_{p}_{)}_{,} and let _{r}_{,} with _{r} < _{n}_{,} be the rank of the diagnostic matrix K A [C'A'C' ***(A')1nC'T,
where the _{p}_{r}_{i}_{m}_{e}
_{s}_{t}_{a}_{n}_{d}_{s} for _{t}_{h}_{e} _{t}_{r}_{a}_{n}_{s}_{p}_{o}_{s}_{e}_{.} Define _{a}_{n} _{r} _{X} _{n} matrix _{T}
consisting of the first r linearly independent rows ofK, and an n X _{r} matrixR denoting the rightinverse of T so that TR =L Then the _{l}_{i}_{n}_{e}_{a}_{r}
machine _{{}_{T}_{A}_{R}_{,} _{T}_{B}_{,} _{C}_{R}_{,} _{D}_{}} is a reduced form
_{o}_{f}_{{}_{A}_{,}_{B}_{,}_{C}_{,}_{D}_{}}_{.} _{I}_{n} _{t}_{h}_{i}_{s}
correspondence, an algorithm will be introduced tofinda specialsetof
r _{l}_{i}_{n}_{e}_{a}_{r}_{l}_{y} _{i}_{n}_{d}_{e}_{p}_{e}_{n}_{d}_{e}_{n}_{t} rows in K. _{A} reduced _{m}_{a}_{c}_{h}_{i}_{n}_{e} can then be _{r}_{e}_{a}_{d} out from this algorithm without the need of inverting any matrix.
Furthermore, thereduced machine willbe _{i}_{n}_{a} _{c}_{a}_{n}_{o}_{n}_{i}_{c}_{a}_{l} _{f}_{o}_{r}_{m}_{.}
II. ALGORITHM
Let C be aq X n matrix, and let _{c}_{i} be the ithrow ofC. The _{d}_{i}_{a}_{g}_{n}_{o}_{s}_{t}_{i}_{c} matrix of{A,B, C,D} willbe arrangedin thefollowing order:
An1
Cq
cqA
_{c}_{q} An1
(1)
Now an _{a}_{l}_{g}_{o}_{r}_{i}_{t}_{h}_{m} willbe introduced to find thefirstrlinearlyindepen
_{d}_{e}_{n}_{t} rows inP. _{T}_{h}_{i}_{s} _{i}_{s} _{a}_{c}_{h}_{i}_{e}_{v}_{e}_{d} _{b}_{y} _{a} series ofelementary _{t}_{r}_{a}_{n}_{s}_{f}_{o}_{r}_{m}_{a}_{}
tions. Let
_{K}_{r}_{K}_{r}_{l}
. ..
K2K1PA KP
_{(}_{2}_{)}
where _{K}_{i} are lower triangularmatriceswith alldiagonalelementsunity,
and are obtained in the _{f}_{o}_{l}_{l}_{o}_{w}_{i}_{n}_{g} _{m}_{a}_{n}_{n}_{e}_{r}_{.} _{L}_{e}_{t} _{p}_{i}_{(}_{j}_{)} be the firstnon
zero element from the left in the first row ofP, where/ denotes the
position. ThenK1 ischosen so thatall,except the first,elements ofthe
jth column of K1P are zero. Let p2(1) be the first nonzero _{e}_{l}_{e}_{m}_{e}_{n}_{t}
from the left of the second row of _{K}_{I}_{P}_{.} ThenK2 ischosen so _{t}_{h}_{a}_{t}_{a}_{l}_{l}_{,}
except the first Proceed in this
two, elements of the Ith column ofK2K1P are _{z}_{e}_{r}_{o}_{.}
manner until _{a}_{l}_{l}
_{l}_{i}_{n}_{e}_{a}_{r}_{l}_{y} _{i}_{n}_{d}_{e}_{p}_{e}_{n}_{d}_{e}_{n}_{t} _{r}_{o}_{w}_{s} of P _{a}_{r}_{e}
found. Note that in this _{p}_{r}_{o}_{c}_{e}_{s}_{s}_{,} ifone row _{i}_{s} _{i}_{d}_{e}_{n}_{t}_{i}_{c}_{a}_{l}_{l}_{y} _{z}_{e}_{r}_{o}_{,} _{t}_{h}_{e}_{n}
proceed to the next nonzero row. By _{m}_{u}_{l}_{t}_{i}_{p}_{l}_{y}_{i}_{n}_{g} these _{K}_{i}_{,} we obtain
K = _{K}_{r}_{K}_{r}_{i}_{l}
...
_{K}_{2}_{K}_{A}_{.} Note thatKP willbe finally ofform
Manuscript received
The
February 21, 1973;
Department
Brook,
of
revised
Electrical
May 21,
1973.
author is with the
Sciences, State Uni
versity of New York, Stony
N.Y. 11790.