This action might not be possible to undo. Are you sure you want to continue?

**Examples. Verify that the properties of the inner product hold in each case.
**

1. 1 = ¹

n

and ¸x, y) := x

∗

y =

n

i=1

x

i

y

i

.

2. We can deﬁne an inner product on ¹

n

using a given invertible matrix, A, by,

¸x, y) := x

∗

A

∗

Ay.

3. 1 = ¹

mn

, the space of mn matrices. Given A, B ∈ ¹

mn

,

¸A, B) := trace

_

A

T

B

_

(or trace (A

∗

B) for (

mn

)

4. Complex valued functions: x(w), y(w) ∈ ((−∞, ∞).

¸x, y) :=

_

∞

−∞

x(w)y(w) dw.

Roy Smith: ECE 210a: 6.2

Inner product spaces

Inner product spaces

Deﬁned for a pair of elements of a vector space, x, y ∈ A,

¸x, y)

A

: A A −→ ¹ (or possibly ().

Deﬁning properties:

1. ¸x, x) ∈ ¹, ¸x, x) ≥ 0 and ¸x, x) = 0 ⇐⇒ x = 0.

2. ¸x, αy) = α¸x, y), for all scalars, α.

3. ¸x, y + z) = ¸x, y) +¸x, z).

4. ¸x, y) = ¸y, x). ( ¸y, x) denotes the complex conjugate).

If the vector space is clear we will drop the explicit subscript.

The pair, 1, and ¸•, •)

1

are known as an inner product space.

A complete inner product space is called a Hilbert space.

Roy Smith: ECE 210a: 6.1

Inner product spaces

Compatible norm

If we have an inner product space, we can deﬁne a compatible norm by,

|x| :=

_

¸x, x).

This is not the only possible norm, but compatibility with the inner product is required to

generalize ¹

3

intuition about distances and angles.

Notice that this norm looks a lot like a Euclidean norm (or “2-norm”).

In ¹

n

it is the Euclidean norm.

Norm properties:

1. |x| ≥ 0 and |x| = 0 ⇐⇒ x = 0 comes from the inner product properties.

2. |αx| = ¸αx, αx)

1/2

= (αα¸x, x))

1/2

=

_

[α[

2

¸x, x)

_

1/2

= [α[|x|.

3. Triangle inequality. This one is trickier ...

Roy Smith: ECE 210a: 6.4

Inner product spaces

Key idea:

Inner products convey the idea of an “angle” between vectors.

We will see that we can deﬁne such an angle by,

cos θ =

¸x, y)

|x||y|

.

This gives the idea of orthogonality; vectors can be perpendicular to one-another.

x ⊥ y ⇐⇒ ¸x, y) = 0 (i.e. θ = π/2).

This will allow us to generalize our geometric intuition in ¹

3

to higher dimensional spaces.

Roy Smith: ECE 210a: 6.3

Cauchy-Bunyakovskii-Schwarz inequality

Proof continued ...

From before,

¸y, y) −¸y, αx) ≥ 0.

This is equal to,

¸y, y) −¸y, αx) = ¸y, y) −α¸y, x) = ¸y, y) −

¸x, y)

|x|

2

¸y, x) =

|y|

2

|x|

2

−¸x, y)¸y, x)

|x|

2

≥ 0.

As the denominator is positive the numerator must also be positive and we get,

|y|

2

|x|

2

≥ ¸x, y)¸y, x).

As ¸x, y) = ¸y, x), the term, ¸x, y)¸y, x) = [¸x, y)[

2

. So now,

|y|

2

|x|

2

≥ [¸x, y)[

2

which, by taking square-roots, is [¸x, y)[ ≤ |y||x|.

H¨older inequality: generalization for p-norms.

If we have real-numbers, p > 1 and q > 1 such that,

1

p

+

1

q

= 1 then [x

∗

y[ ≤ |x|

p

|y|

q

.

Roy Smith: ECE 210a: 6.6

Cauchy-Bunyakovskii-Schwarz inequality

Cauchy-Bunyakovskii-Schwarz inequality

(the inequality previously known as Cauchy-Schwarz)

[¸x, y)[ ≤ |x||y|

Note that the norm in question is the one that’s compatible with the inner product.

This clearly makes the angle between x and y well deﬁned.

Proof:

Assume that x ,= 0 and choose α as,

α =

¸x, y)

|x|

2

which means that α|x|

2

= ¸x, y) or ¸x, αx −y) = 0.

Now look at |αx −y|

2

(which is ≥ 0).

|αx −y|

2

= ¸αx −y, αx −y) = α¸x, αx −y)

. ¸¸ .

= 0

−¸y, αx −y).

This means that,

−¸y, αx −y) ≥ 0, or equivalently, ¸y, y) −¸y, αx) ≥ 0.

Roy Smith: ECE 210a: 6.5

Parallelogram identity

Given a norm, can we always deﬁne an inner product?

No, not always.

Parallelogram identity

We can deﬁne an inner product if and only if the parallelogram identity is true.

|x + y|

2

+|x −y|

2

= 2(|x|

2

+|y|

2

).

y

x

x+y

x-y

The inner product can be deﬁned via,

¸x, y) :=

1

4

(|x + y|

2

−|x −y|

2

). (see Meyer for details)

Roy Smith: ECE 210a: 6.8

Compatible norms

Triangle inequality

The CBS inequality can be used to show that the triangle inequality holds for the norm

compatible with the inner product.

|x + y|

2

= ¸x + y, x + y)

= ¸x, x)+ ¸x, y) +¸y, x)

. ¸¸ .

+¸y, y)

≤ 2[¸x, y)[

≤ ¸x, x) + 2[¸x, y)[ +¸y, y)

≤ |x|

2

+ 2|x||y| +|y|

2

≤ (|x| +|y|)

2

,

and taking square-roots gives the triangle inequality.

So every inner product space is also a normed space.

Roy Smith: ECE 210a: 6.7

Orthogonality

Orthogonality

Consider a real inner product space (¸x, y) ∈ ¹)

x ⊥ y ⇐⇒ ¸x, y) = 0.

Pythagoras: Right angle triangles

x

x+y

y

x-y

θ=π/2

|x + y|

2

= |x −y|

2

= |x|

2

+|y|

2

So 0 = |x|

2

+|y|

2

−|x −y|

2

= ¸x, x) +¸y, y) −¸x −y, x −y)

= ¸x, x) +¸y, y) −¸x, x) −¸y, y) +¸x, y) +¸y, x) which implies that ¸x, y) = 0.

Roy Smith: ECE 210a: 6.10

Parallelogram identity

Inner products from other norms?

Can we form an inner product from |•|

1

or |•|

∞

?

No. The parallelogram identity doesn’t hold in either case.

So to retain the concept of an angle in ¹

n

or (

n

we need to use the |•|

2

(or something close

to it).

Elliptical norms

Suppose we are given a square, invertible matrix, A ∈ (

nn

.

Deﬁne the norm:

|x|

A

= |Ax|

2

, where |•|

2

is the usual Euclidean norm.

Can we deﬁne an inner product here?

Does the obvious deﬁnition satisfy the parallelogram identity?

Roy Smith: ECE 210a: 6.9

Orthogonormal bases

Orthogonal bases

A basis,

B = ¦ x

1

, x

2

, . . . , ¦ is orthogonal if ¸x

i

, x

j

) = 0 for all i ,= j.

The vectors, x

i

, are at right angles to one-another.

Orthonormal bases

A basis,

B = ¦ x

1

, x

2

, . . . , ¦ is orthonormal if ¸x

i

, x

j

) =

_

1 i = j

0 i ,= j

The vectors, x

i

, are at right angles to one-another and |x

i

| = 1.

Wherever possible we choose orthonormal bases for our vector spaces.

This allows us to extend our intuition in ¹

3

to higher dimensional spaces.

Roy Smith: ECE 210a: 6.12

Orthogonality

Angle between vectors

y

x

x-y

θ

|x|

2

+|y|

2

= |x −y|

2

+ 2 cos θ |x||y|.

Solving for cos θ gives the general angle formula,

cos θ =

|x|

2

+|y|

2

−|x −y|

2

2|x||y|

=

|x|

2

+|y|

2

−¸x −y, x −y)

2|x||y|

=

|x|

2

+|y|

2

−¸x, x) −¸y, y) +¸x, y) +¸y, x)

2|x||y|

=

2¸x, y)

2|x||y|

=

¸x, y)

|x||y|

The CBS inequality guarantees that this is in the interval [-1,1] and so θ is uniquely deﬁned.

Roy Smith: ECE 210a: 6.11

Fourier coeﬃcients

Fourier coeﬃcients and orthogonal bases

Suppose we have a basis,

B

1

= ¦ v

1

, v

2

, . . . , v

n

¦ span(B

1

) = 1.

We can express any vector, v ∈ 1 with respect to this basis by,

v =

n

i=1

α

i

v

i

with α

i

=

¸v

i

, v)

¸v

i

, v

i

)

. (If the basis is orthonormal, α

i

= ¸v

i

, v))

To see this,

¸v

j

, v) = ¸v

j

,

n

i=1

α

i

v

i

)

=

n

i=1

α

i

¸v

j

, v

i

)

= α

j

¸v

j

, v

j

) (all the other terms are zero)

So, α

j

=

¸v

j

, v)

¸v

j

, v

j

)

.

Roy Smith: ECE 210a: 6.14

Orthogonal projection

Orthogonal projection

Given two vectors, x and y in an inner product space, the orthogonal projection of y on

span¦x¦ is the vector,

αx where (αx −y) ⊥ x.

y

x

y - αx

αx

(αx −y) ⊥ x =⇒ ¸x, αx −y) = 0 =⇒ α¸x, x) −¸x, y) = 0

=⇒ α =

¸x, y)

¸x, x)

α is called the Fourier coeﬃcient of y with respect to x.

Roy Smith: ECE 210a: 6.13

Fourier coeﬃcients

Coordinate matrix representations of linear transformations (see 2.16 and 4.10)

Say we have an operator, T : 1 −→ J, with an orthogonal basis for each space,

B

1

= ¦ v

1

, v

2

, . . . , v

n

¦ and B

J

= ¦ w

1

, w

2

, . . . , w

n

¦

Recall that the matrix representation gives,

_

w

¸

BW

=

BW

[T ]

BV

_

v

¸

BV

=

_

[T (v

1

)]

BW

[T (v

2

)]

BW

. . . [T (v

n

)]

BW

¸ _

v

¸

BV

Examine the th column of the matrix. To express this in the B

J

basis we project it onto

each of the basis elements,

[T (v

j

)]

BW

=

_

¸

¸

¸

¸

_

¸w

1

, T (v

j

))

¸w

1

, w

1

)

.

.

.

¸w

m

, T (v

j

))

¸w

m

, w

m

)

_

¸

¸

¸

¸

_

so the ıth element is, T

ij

=

¸w

i

, T (v

j

))

¸w

i

, w

i

)

And so for an orthonormal basis: T =

_

_

¸w

1

, T (v

1

)) . . . ¸w

1

, T (v

n

))

.

.

.

.

.

.

¸w

m

, T (v

1

)) . . . ¸w

m

, T (v

n

))

_

_

.

Roy Smith: ECE 210a: 6.16

Fourier coeﬃcients

Fourier coeﬃcients and orthogonal bases

An ¹

2

example with an orthogonal (but not orthonormal) basis.

v

v

2

v

1

, v

v

2

, v

v

1

, v

1

v

2

, v

2

v

1

v

1

v

2

v =

¸v, v

1

)

¸v

1

, v

1

)

v

1

+

¸v, v

2

)

¸v

2

, v

2

)

v

2

.

Roy Smith: ECE 210a: 6.15

Sign up to vote on this title

UsefulNot useful- Math 235 Algebra
- Inner Product
- orthogonality
- Honors Advanced Calculus and Linear Algebra Hilbert 1
- Gram Schmidt
- Notes on Functional Analysis - Jon Johnsen
- Lecture 25
- bilinear-08
- Harvey Mudd - Gram-Schmidt Algorithm
- 06Chap2.pdf
- Orthonormal Basis
- ps1_2014
- Unit 5 Inner Product Spaces
- Fried Berg
- billly
- Reflections in a Euclidean Space
- Inner Product - Peter
- 10. Hilbert Spaces, Dirichlet's
- Lin Combos
- Homework 1
- True of False for Vector Spaces
- Midterm1 Solution
- LADE10 Linear transformations of R^2
- 2.4 Transition Matrices
- Test 1 Notes
- Hw 8 Solutions
- LA4
- section5-2LeonPartialSolutions
- math110su12notes_p24-34
- 3195 Study Vs
- Rhotrix Topological Space

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

We've moved you to where you read on your other device.

Get the full title to continue

Get the full title to continue reading from where you left off, or restart the preview.

scribd