Discreet Haar Transform

Todays Schedule Building the Haar Matrix Coding the Haar Transform 2D Haar Transform Iterating In the Classroom
Discrete Haar Wavelet Transforms

Patrick J. Van Fleet
Center for Applied Mathematics
University of St. Thomas
St. Paul, MN USA
PREP - Wavelet Workshop, 2006
Wednesday, 7 June, 2006 Lecture 4
Outline
Todays Schedule
Building the Haar Matrix
Putting Two Filters Together
Why the Word Wavelet?
Examples
Coding the Haar Transform
Implementing W
N
Implementing W
T
N
2D Haar Transform
Building the 2D Transform
Coding the 2D Transform
Iterating
In the Classroom
Teaching Ideas
Computer Usage
Todays Schedule
9:00-10:15 Lecture One: Why Wavelets?
10:15-10:30 Coffee Break (OSS 235)
10:30-11:45 Lecture Two: Digital Images, Measures, and
Huffman Codes
12:00-1:00 Lunch (Cafeteria)
1:30-2:45 Lecture Three: Fourier Series, Convolution and
Filters
3:00-4:15 Lecture Four: 1D and 2D Haar Transforms
5:30-6:30 Dinner (Cafeteria)
Consider again the lter h = (h

0
, h
1
) = (
1
2
,
1
2
).
If we compute y = h x, we obtain the components

y
n
=
1
2
x
n
+
1
2
x
n1
We could write down the convolution matrix


0
, h
1
) = (
1
2
,
1
2
).

y
n
=
1
2
x
n
+
1
2
x
n1


0
, h
1
) = (
1
2
,
1
2
).

y
n
=
1
2
x
n
+
1
2
x
n1


0
, h
1
) = (
1
2
,
1
2
).

y
n
=
1
2
x
n
+
1
2
x
n1

H =
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0
1
2
1
2
0 0 0 0 0 0 0 0
0 0 0 0
1
2
1
2
0 0 0 0 0 0 0
. . . 0 0 0 0 0
1
2
1
2
0 0 0 0 0 0 . . .
0 0 0 0 0 0
1
2
1
2
0 0 0 0 0
0 0 0 0 0 0 0
1
2
1
2
0 0 0 0
.
.
.
.
.
.
.
.
.

. . . but we cant invert the process.
What would we need to be able to invert the process?
We have averages of consecutive numbers - if we had the

directed distance between these averages and the
consecutive numbers, then we could invert.
The directed distance is exactly the sequence x convolved

with the lter g = (
1
2
,
1
2
).


with the lter g = (
1
2
,
1
2
).


with the lter g = (
1
2
,
1
2
).


with the lter g = (
1
2
,
1
2
).


with the lter g = (
1
2
,
1
2
).
Indeed if
y
n
=
1
2
x
n
+
1
2
x
n1
and z
n
=
1
2
x
n
1
2
x
n1
then
x
n
= y
n
+ z
n
and x
n1
= y
n
z
n
Perhaps we could invert the process if we used both lters. We
know that G is
G =
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0
1
2
1
2
0 0 0 0 0 0 0 0
0 0 0 0
1
2
1
2
0 0 0 0 0 0 0
. . . 0 0 0 0 0
1
2
1
2
0 0 0 0 0 0 . . .
0 0 0 0 0 0
1
2
1
2
0 0 0 0 0
0 0 0 0 0 0 0
1
2
1
2
0 0 0 0
.
.
.
.
.
.
.
.
.

So that
H
G
x =
y
z

If we think about inverting, we can write down:
.
.
.
.
.
.
y
1
z
1
=
x
1
+x
0
2

x
1
x
0
2
= x
0
y
1
+z
1
=
x
1
+x
0
2
+
x
1
x
0
2
= x
1
y
2
z
2
=
x
2
+x
1
2

x
2
x
1
2
= x
1
y
2
+z
2
=
x
2
+x
1
2
+
x
2
x
1
2
= x
2
y
3
z
3
=
x
3
+x
2
2

x
3
x
2
2
= x
2
y
3
+z
3
=
x
3
+x
2
2
+
x
3
x
2
2
= x
3
.
.
.
.
.
.
But there is some redundancy here - we do not need all the
values of y
n
, z
n
to recover x
n
:
.
.
.
.
.
.
y
1
z
1
=
x
1
+x
0
2

x
1
x
0
2
= x
0
y
1
+z
1
=
x
1
+x
0
2
+
x
1
x
0
2
= x
1
y
2
z
2
=
x
2
+x
1
2

x
2
x
1
2
= x
1
y
2
+z
2
=
x
2
+x
1
2
+
x
2
x
1
2
= x
2
y
3
z
3
=
x
3
+x
2
2

x
3
x
2
2
= x
2
y
3
+z
3
=
x
3
+x
2
2
+
x
3
x
2
2
= x
3
.
.
.
.
.
.
So we can omit every other row in H, G and still produce

enough to be able to recover x
This is called downsampling.
We are also now in a position to truncate our matrix.

Indeed, if x = (x
0
, . . . , x
N
), then it is natural to truncate the
matrix and write:


Indeed, if x = (x
0
, . . . , x
N
matrix and write:


Indeed, if x = (x
0
, . . . , x
N
matrix and write:


Indeed, if x = (x
0
, . . . , x
N
matrix and write:
W
N
=
1
2
1
2
0 0 0 0 0 0
0 0
1
2
1
2
0 0 . . . 0 0
.
.
.
.
.
.
.
.
.
0 0 0 0 . . . 0 0
1
2
1
2
1
2
1
2
0 0 0 0 0 0
0 0
1
2
1
2
0 0 . . . 0 0
.
.
.
.
.
.
.
.
.
0 0 0 0 . . . 0 0
1
2
1
2

This matrix is easy to invert if we remember the formulas:
x
n
= y
n
+ z
n
and x
n1
= y
n
z
n
We have:
W
1
N
=
1 0 0
1 0 0
0 1 0
0 1 0
.
.
.
.
.
.
.
.
.
0 0 0
0 0 0
0 0 1
0 0 1
1 0 0
1 0 0
0 1 0
0 1 0
.
.
.
.
.
.
.
.
.
0 0 0
0 0 0
0 0 1
0 0 1

Note we are very close to having

W
N
an orthogonal matrix.
We have

W
T
N
=
1
2

W
1
N
.
If we multiply

W
N
by
2, we will obtain an orthogonal

matrix.
We have:

W
N
We have

W
T
N
=
1
2

W
1
N
.
If we multiply

W
N
by

matrix.
We have:

W
N
We have

W
T
N
=
1
2

W
1
N
.
If we multiply

W
N
by

matrix.
We have:

W
N
We have

W
T
N
=
1
2

W
1
N
.
If we multiply

W
N
by

matrix.
We have:

W
N
We have

W
T
N
=
1
2

W
1
N
.
If we multiply

W
N
by

matrix.
We have:
W
N
=
H
G
2
2
2
2
0 0 0 0
0 0
2
2
2
2
0 0
.
.
.
.
.
.
.
.
.
0 0 0 0 . . .
2
2
2
2
2
2
2
2
0 0 0 0
0 0
2
2
2
2
0 0
.
.
.
.
.
.
.
.
.
0 0 0 0 . . .
2
2
2
2

W
N
is called the Discrete Haar Wavelet Transform
The lter
h =
2
2
,
2
2
is called the Haar lter.
Note that H() =
2
2
+
2
2
e
i
satises H() = 0, but
H(0) =
2
2
. We will still consider this to be a lowpass lter-
the
2
2
resulted when we made the transform orthogonal.
W
N
The lter
h =
2
2
,
2
2
Note that H() =
2
2
+
2
2
e
i
H(0) =
2
2
the
2
2
W
N
The lter
h =
2
2
,
2
2
Note that H() =
2
2
+
2
2
e
i
H(0) =
2
2
the
2
2
W
N
The lter
h =
2
2
,
2
2
Note that H() =
2
2
+
2
2
e
i
H(0) =
2
2
the
2
2
The word wavelet comes from the more classical treatment

of the topic. Here, we work in L
2
(R) and downsampling is
basically a way to move between nested subspaces V
j
that
are generated by the translates and dilates of a single
scaling function.
If V
0
is the space of piecewise constants with possible
breaks at Z, then the characteristic function (t ) =
[0,1)
(t )
and its translates form an orthonormal basis for V
0
.

2
j
that
scaling function.
If V
0
[0,1)
(t )
0
.

2
j
that
scaling function.
If V
0
[0,1)
(t )
0
.
If V
1
breakpoints at
1
2
Z, then V
0
V
1
, and the functions
2(2t k)
form an orthornormal basis for V
1
.
Note that
(t ) =
2
2
(2t ) +
2
2
(2t 1)
is called a dilation equation.

We can get the Haar lter coefcients from the dilation

equation.
The word wavelet refers to the function (t ) that generates

a basis for the orthogonal complement of V
0
in V
1
.
In this case, the wavelet function is

(t ) =
1 0 t <
1
2
1
1
2
t < 1

equation.

0
in V
1
.

(t ) =
1 0 t <
1
2
1
1
2
t < 1

equation.

0
in V
1
.

(t ) =
1 0 t <
1
2
1
1
2
t < 1

equation.

0
in V
1
.

(t ) =
1 0 t <
1
2
1
1
2
t < 1
Note that (t ) V
1
and
(t ) = (t ) =
2
2
(2t )
2
2
(2t 1)
so that the highpass lter g =
2
2
,
2
2
can be read from this

dilation equation.
I opted to stay away from the classical approach to

wavelets.
It is beautiful theory, but too much for sophomores and

juniors.
I believe its better to give them a practical introduction to

Fourier series and convolution, and then derive the the
discrete wavelet transform by using a lowpass/highpass
lter pair and downsampling.

wavelets.

juniors.


wavelets.

juniors.


wavelets.

juniors.

Examples
Examples
Lets have a look at the Mathematica notebook
HaarTransforms1D.nb
for a bit more on Haar Transforms.
Implementing W
N
Implementing W
N
The natural inclination when coding the DHWT is to simply

write a loop and compute the lowpass portion and the
highpass portion in the same loop.
This bogs down in Mathematica and is also difcult to

generalize when we consider longer lters.
If we look at the lowpass portion of the transform, Hv, we

can see a better way to code things.
Implementing W
N
Implementing W
N



Implementing W
N
Implementing W
N



Implementing W
N
Implementing W
N



Implementing W
N
Implementing W
N
Consider Hv when v R
8
. We have
Hv =
2
2
v
1
+ v
2
v
3
+ v
4
v
5
+ v
6
v
7
+ v
8

Implementing W
N
Implementing W
N
If we rewrite this, we have
Hv =
2
2
v
1
+ v
2
v
3
+ v
4
v
5
+ v
6
v
7
+ v
8
v
1
v
2
v
3
v
4
v
5
v
6
v
7
v
8

2
2
2
2
= Vh
Implementing W
N
Implementing W
N
In a similar way we see that
Gv = Vg
So all we need to do to compute W
n
v is to create V, multiply it
with h and g, and join to the two blocks together!
Implementing W
N
Implementing W
N
Here is some Mathematica code to do it:
DHWT[v_]:=Module[{V,lp,hp,y},
V=Partition[v,2,2];
lp=V.{1,1};
hp=V.{1,-1};
y=Join[lp,hp];
Return[Sqrt[2]
*
y/2];
];
Implementing W
T
N
Implementing W
T
N
Writing the code for the inverse transform is a bit trickier.
Now the computation is

W
T
N
v =
H
T
G
T
Lets again look at a vector v R

8
and consider the
product W
T
8
v:
Implementing W
T
N
Implementing W
T
N

W
T
N
v =
H
T
G
T

8
and consider the
product W
T
8
v:
Implementing W
T
N
Implementing W
T
N

W
T
N
v =
H
T
G
T

8
and consider the
product W
T
8
v:
Implementing W
T
N
Implementing W
T
N

W
T
N
v =
H
T
G
T

8
and consider the
product W
T
8
v:
Implementing W
T
N
Implementing W
T
N
W
T
8
v =
2
2
1 0 0 0 1 0 0 0
1 0 0 0 1 0 0 0
0 1 0 0 0 1 0 0
0 1 0 0 0 1 0 0
0 0 1 0 0 0 1 0
0 0 1 0 0 0 1 0
0 0 0 1 0 0 0 1
0 0 0 1 0 0 0 1
v
1
v
2
v
3
v
4
v
5
v
6
v
7
v
8

Implementing W
T
N
Implementing W
T
N
or
W
T
8
v =
2
2
v
1
v
5
v
1
+ v
5
v
2
v
6
v
2
+ v
6
v
3
v
7
v
3
+ v
7
v
4
v
8
v
4
+ v
8

Implementing W
T
N
Implementing W
T
N
The matrix V takes a bit different shape this time.
Now V is
V =
v
1
v
5
v
2
v
6
v
3
v
7
v
4
v
8
We need to dot V with both h and g but then intertwine the

results.
Implementing W
T
N
Implementing W
T
N
Now V is
V =
v
1
v
5
v
2
v
6
v
3
v
7
v
4
v
8

results.
Implementing W
T
N
Implementing W
T
N
Now V is
V =
v
1
v
5
v
2
v
6
v
3
v
7
v
4
v
8

results.
Implementing W
T
N
Implementing W
T
N
Now V is
V =
v
1
v
5
v
2
v
6
v
3
v
7
v
4
v
8

results.
Implementing W
T
N
Implementing W
T
N
Lets return to the Mathematica notebook
HaarTransforms1D.nb
to see how to code the inverse.
2D Haar Transform
Lets now assume A is an N N image with N even.
How do we transform A?
If we compute W
N
A, we are simply applying the DHWT to
each column of A:
2D Haar Transform
If we compute W
N
each column of A:
2D Haar Transform
If we compute W
N
each column of A:
2D Haar Transform
If we compute W
N
each column of A:
2D Haar Transform
A W
N
A
2D Haar Transform
Weve processed the columns of A - what should we do to

process the rows of A as well?
Answer: Compute W
N
AW
T
N
.
2D Haar Transform

Answer: Compute W
N
AW
T
N
.
2D Haar Transform

Answer: Compute W
N
AW
T
N
.
2D Haar Transform
A W
N
AW
T
N
2D Haar Transform
If we look at W
N
AW
T
N
in block format, we can get a better
idea whats going on.
W
N
AW
T
N
=
H
G
H
G
T
=
HA
GA
H
T
G
T

=
HAH
T
HAG
T
GAH
T
GAG
T
B V
H D

2D Haar Transform
If we look at W
N
AW
T
N
W
N
AW
T
N
=
H
G
H
G
T
=
HA
GA
H
T
G
T

=
HAH
T
HAG
T
GAH
T
GAG
T
B V
H D

2D Haar Transform
If we look at W
N
AW
T
N
W
N
AW
T
N
=
H
G
H
G
T
=
HA
GA
H
T
G
T

=
HAH
T
HAG
T
GAH
T
GAG
T
B V
H D

2D Haar Transform
HAH
T
averages along the columns of A and then along the
rows of HA. This will produce an approximation (or blur) B
of A.
HAG
T
averages along the columns of A and then
differences along the rows of HA. This will produce vertical
differences V between B and A.
2D Haar Transform
HAH
T
of A.
HAG
T
2D Haar Transform
HAH
T
of A.
HAG
T
2D Haar Transform
GAH
T
differences along the columns of A and then
averages along the rows of GA. This will produce
horizontal differences H between B and A.
GAG
T
differences along the rows of GA. This will produce
diagonal differences V between B and A.
2D Haar Transform
GAH
T
GAG
T
2D Haar Transform
GAH
T
GAG
T
2D Haar Transform
To better understand these block forms, lets look at the
Mathematica notebook
HaarTransforms2D.nb
2D Haar Transform
Coding the 2D Haar transform is easy - we already have a

routine that will apply the DHWT to each column of A,
so we can easily write a routine to compute C = W

N
A.
Lets call this routine W.
Our goal is to compute B = W

N
AW
T
N
= CW
T
N
.
It turns out that writing code for CW

T
N
is a bit tedious, but if
we use some linear algebra . . .
2D Haar Transform


N
A.

N
AW
T
N
= CW
T
N
.

T
N
2D Haar Transform


N
A.

N
AW
T
N
= CW
T
N
.

T
N
2D Haar Transform


N
A.

N
AW
T
N
= CW
T
N
.

T
N
2D Haar Transform


N
A.

N
AW
T
N
= CW
T
N
.

T
N
2D Haar Transform
If we transpose both sides of B = CW

T
N
, we have
B
T
= W
N
C
T
So we can simply apply W to C

T
and transpose the result.
One student wasnt so sure about this . . .
Lets return to HaarTransforms2D.nb to write some

code for the 2D Haar Wavelet Transform.
2D Haar Transform

T
N
, we have
B
T
= W
N
C
T

T

2D Haar Transform

T
N
, we have
B
T
= W
N
C
T

T

2D Haar Transform

T
N
, we have
B
T
= W
N
C
T

T

2D Haar Transform

T
N
, we have
B
T
= W
N
C
T

T

2D Haar Transform
Iterating
Its time to explain the NumIterations directive you have

seen in the Mathematica notebooks.
We can motivate the idea by looking at the cumulative

energy of an image A and its wavelet transform.
2D Haar Transform
Iterating


2D Haar Transform
Iterating


2D Haar Transform
Iterating
Here is a 200 200 image and its transform:
A W
N
AW
T
N
2D Haar Transform
Iterating
Here are the cumulative energies for both A (red) and its
transform (brown):
2D Haar Transform
Iterating
To give you an idea, the largest 10000 elements in A make

up about 36.5% of the energy in A while the rst 10000
elements in the transform comprise about 99.5% of the
energy in the transform.
The wavelet transform is totally invertible, so if we were to

Huffman encode the transform, the bit stream should be
markedly smaller.
2D Haar Transform
Iterating


markedly smaller.
2D Haar Transform
Iterating


markedly smaller.
2D Haar Transform
Iterating
We can get even more concentration of the energy if we

iterate the wavelet transform. That is, after computing the
wavelet transform of A, we extract the blur and compute a
wavelet transform of it.
We could repeat this process p times if the dimensions of

A are divisible by 2
p
.
2D Haar Transform
Iterating


p
.
2D Haar Transform
Iterating


p
.
2D Haar Transform
Iterating
Now suppose we iterate 2 times:
2D Haar Transform
Iterating
or 3 times:
2D Haar Transform
Iterating
Here are the cumulative energy vectors for 1 iteration (brown),
2 iterations (blue), and 3 iterations (gray):
Teaching Ideas
In the Classroom
Teaching Ideas
The students really enjoy the material in this chapter. It is

quite straightforward and ties together everything new
weve done to date.
I have them look at the entropy of particular vectors when

processed by the Haar transform. This gives them some
idea of the potential for wavelet-based compression.
Teaching Ideas
In the Classroom
Teaching Ideas

weve done to date.

Teaching Ideas
In the Classroom
Teaching Ideas

weve done to date.

Computer Usage
In the Classroom
Computer Usage
As you might imagine, we do lots of coding in this chapter.
I let the students work in pairs and they write code for the
Haar transform and its inverse (1D and 2D) as well as
iterated versions.
They can get pretty frustrated with Mathematica at this

point - it is good to show them some simple debugging
techniques.
Computer Usage
In the Classroom
Computer Usage
iterated versions.

techniques.
Computer Usage
In the Classroom
Computer Usage
iterated versions.

techniques.
Computer Usage
In the Classroom
Computer Usage
iterated versions.

techniques.
Computer Usage
In the Classroom
Computer Usage
I let them use their own images/audio les (sometimes

dangerous).
To test their iterated 1D inverse, they must download an

audio clip from my website that has been transformed p
times, guess at p, and then apply their inverse to guess the
movie clip.
Computer Usage
In the Classroom
Computer Usage

dangerous).

movie clip.
Computer Usage
In the Classroom
Computer Usage

dangerous).

movie clip.
Todays Schedule
9:00-10:15 Lecture One: Why Wavelets?
10:30-11:45 Lecture Two: Digital Images, Measures, and
Huffman Codes
12:00-1:00 Lunch (Cafeteria)
1:30-2:45 Lecture Three: Fourier Series, Convolution and
Filters
3:00-4:15 Lecture Four: 1D and 2D Haar Transforms
5:30-6:30 Dinner (Cafeteria)

Discreet Haar Transform

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Discreet Haar Transform

Uploaded by

Copyright:

Available Formats

Todays Schedule Building the Haar Matrix Coding the Haar Transform 2D Haar Transform Iterating In the Classroom

Discrete Haar Wavelet Transforms

Consider again the lter h = (h

If we compute y = h x, we obtain the components

We could write down the convolution matrix

Consider again the lter h = (h

If we compute y = h x, we obtain the components

We could write down the convolution matrix

Consider again the lter h = (h

If we compute y = h x, we obtain the components

We could write down the convolution matrix

Consider again the lter h = (h

If we compute y = h x, we obtain the components

We could write down the convolution matrix

Wednesday, 7 June, 2006 Lecture 4

. . . but we cant invert the process.

What would we need to be able to invert the process?

We have averages of consecutive numbers - if we had the

The directed distance is exactly the sequence x convolved

. . . but we cant invert the process.

What would we need to be able to invert the process?

We have averages of consecutive numbers - if we had the

The directed distance is exactly the sequence x convolved

. . . but we cant invert the process.

What would we need to be able to invert the process?

We have averages of consecutive numbers - if we had the

The directed distance is exactly the sequence x convolved

. . . but we cant invert the process.

What would we need to be able to invert the process?

We have averages of consecutive numbers - if we had the

The directed distance is exactly the sequence x convolved

. . . but we cant invert the process.

What would we need to be able to invert the process?

We have averages of consecutive numbers - if we had the

The directed distance is exactly the sequence x convolved

Wednesday, 7 June, 2006 Lecture 4

Wednesday, 7 June, 2006 Lecture 4

So we can omit every other row in H, G and still produce

This is called downsampling.

We are also now in a position to truncate our matrix.

So we can omit every other row in H, G and still produce

This is called downsampling.

We are also now in a position to truncate our matrix.

So we can omit every other row in H, G and still produce

This is called downsampling.

We are also now in a position to truncate our matrix.

So we can omit every other row in H, G and still produce

This is called downsampling.

We are also now in a position to truncate our matrix.

Wednesday, 7 June, 2006 Lecture 4

Wednesday, 7 June, 2006 Lecture 4

Note we are very close to having

2, we will obtain an orthogonal

Note we are very close to having

2, we will obtain an orthogonal

Note we are very close to having

2, we will obtain an orthogonal

Note we are very close to having

2, we will obtain an orthogonal

Note we are very close to having

2, we will obtain an orthogonal

Wednesday, 7 June, 2006 Lecture 4

is called the Haar lter.