2) x + y , z = x , z + y , z ,

H1 A 3) λx , y = λ x , y ,
H2
|| A*
4) x , x ≥ 0 with equality iff x = 0,
||
N(A) N(A*) where z is the complex conjugate of z.
⊕ ⊕
Every inner product on V induces a norm on V :
R(A*) R(A)
x = x, x .
dim(R(A*)) = dim(R(A))
An inner product space is a vector space with an inner
Figure 1. Fundamental relationship between an operator and its product.
adjoint. This diagram was originally introduced to the author by Example 5 (!n / " n ): The standard inner product on " n is
Prof. John T. Wen at Rensselaer Polytechnic Institute, who attributes defined as
it to the late George Zames.
n
6) ( α + β )x = αx + βx. x , y = yHx = ∑ xi yi ,
7) ( αβ )x = α (βx ). i =1

8) There exist scalars0 and1such that0x = 0and1x = x .

Example 1 (!n / " n ): The most common vector space is where y H denotes the complex conjugate transpose of y. IfW
n
! , the set of real n-vectors over the reals, or more generally is a positive definite Hermitian matrix, then another valid in-
" n , the set of complex n-vectors over the complex field. ner product on " n is
Example 2( l n ): The set of real (respectively, complex) in- n n
finite n-tuple sequences over the real (respectively, com- x , y = y H Wx = ∑ ∑ wij x j yi .
plex) field forms a vector space. i =1 j =1

Example 3 ( Ln ): The set of continuous functions

f : !n → ! forms a vector space. In fact, f is not required to Example 6 ( l n2 ): A valid inner product on infinite n-tuple
be continuous, only measurable [4]. This class of functions sequences is a simple extension of the inner product on " n .
includes piecewise continuous functions. Essentially, Ln in- Let x = ( x1 , x 2 , K ) and y = ( y1 , y 2 , K ) be infinite sequences
cludes the set of all physically realizable signals, in addition with xi , yi ∈ " n . Valid inner products include the following:
to a large number of nonphysical signals.
∞
The dimension of a vector space is defined through the
x, y = ∑ yiH xi
notion of a basis set. A set P = { p1 , p2 , K}, is called a Hamel l2
i =1
basis [5] for a vector space V = ( X , S ) if 1) the vectors in P ∞

are linearly independent, and 2) every vector in X can be x, y l2

= ∑ yiH Wxi , W − Hermitian
i =1
represented by a finite linear combination of elements from ∞

P. A fundamental theorem of functional analysis is that ev- x, y l2

= ∑ yiH xi wi , wi > 0.
ery vector space has a Hamel basis [5, p. 211]. The notion of
i =1 (1)
a Hamel basis is primarily used in analysis (see the proof of
The set of infinite n-tuple sequences such that x < ∞ is
Theorem 2). In applications, it is more convenient to work l2

with a complete basis, where infinite sums are allowed. An denoted as l n2 or simply as l 2 when n is understood.
example is the set of complex exponentials on L2 ( −∞ , ∞ ), Example 7 ( Ln2 ): The inner product for functions is again
which are a complete basis but not a Hamel basis (since any analogous to the inner product for l n and " n . In this case,
discontinuous function in L2 ( −∞ , ∞ ) cannot be represented the sum is replaced by an integral. The following are valid in-
by a finite linear combination of complex exponentials). ner products for functions:
Definition 2: The dimension of a vector space V is the car-
x, y =∫ y H ( t )x ( t ) dt
dinality of the smallest Hamel basis for V . L 2 ( Ω) Ω

Example 4: The dimension of !n and " n is n. The dimen- x, y L 2 ( Ω)

=∫ y H ( t )Wx ( t ) dt , W − Hermitian
Ω
sion of l n is ∞. The dimension of Ln is ∞.
An inner product from one vector space to another vec- x, y L 2 ( Ω)
= ∫ y H ( t )x ( t )w( t ) dt , w( t ) > 0 and measurable,
Ω
tor space is defined by the following properties. Let (2)
V = ( X , S ) be a vector space, let x , y , z be elements of X, and where the integral is a Lebesgue integral [4]. The set of mea-
let λ be an element of S; then ⋅, ⋅ : X × X → " is an inner surable functions on " n such that x L ( Ω) < ∞ is a valid vec-
2
product if tor space and is denoted as Ln2 (Ω ) or simply L2 when n and Ω
1) x , y = y , x , are understood.

70 IEEE Control Systems Magazine April 2002