Stat 7410 Outline - Review material

Useful Concepts in Linear Algebra

I. Vectors and Vector Spaces, linear independence, basis of a vector space,

dimension of a vector space, subspaces, space spanned by a collection of
vectors, LIN set of vectors
 Vector: Ordered n-tuple of real numbers
 a1 
 
 Notation: a   a2  , where ai  (real-valued elements) is a 3-dim
a 
 3
column vector. A row vector is denoted by a  (a1 a2 a3 )
 Vector Space: Vn  {vi  (vi1 vi 2 vin ), vij , i  1,2, , l}
  : n-dimensional Euclidean space
n

 ith elementary vector : ei  (0,0, ,1,0, 0)

II. Inner (dot) Product of two vectors x, y Vn :
x • y  xy   xi yi  yx  y  x
i
 Other notations: (x, y), x, y
 Kernel function:
k ( x, y)   ( x), ( y ) , where  :x  (x) F , the feature space.
 Length of a vector, angle between two vectors, orthogonal vectors,
Pythagoras’ theorem, Triangle inequality, Cauchy-Schwartz
inequality, orthogonal (orthonormal) basis.
 Gram-Schmidt process for forming an o.n.b., Extension of an o.n.b.
for a subspace V to an o.n.b. for the space Vn .
 Coordinates of a vector in terms of an o.n.b.

III. Matrices (definition, sums and products, transpose, trace, determinant, non-
singular matrix, inverse)
 Linear space of a matrix, Null space of a matrix
 Space spanned by columns of a matrix A, rank of a matrix

IV. Characteristic roots and vectors (eigen–values and eigen-vectors) of A and

their properties, Spectral decomposition of a real symmetric matrix A.
 V. Singular Value Decomposition (SVD) of a matrix A.

VI. Vector norm, matrix norm

VII. Partitioned matrices, Symmetric and Idempotent matrices

VIII. Linear forms, Quadratic forms, Positive semi-definite (p.s.d) ) (n.n.d.),

Positive Definite.

IX. Simultaneous diagonalization of two matrices

X. Systems of Linear Equations (Solution space), generalized inverse of a matrix,

computation of a g-inverse, Moore-Penrose inverse of a matrix.