Analisis Peubah Ganda

Pertemuan II
MATRIX ALGEBRA
DICTIONARY.COM
Definitions
Matrices vs Vector
• Away from the definition, a matrix is simply a rectangular
way of storing data.
• A matrix A is a system of numbers with n rows and p
columns:

• A vector is a matrix where one dimension is equal to size

one.
– Column vector: A column vector is a matrix of size r × 1.
– Row vector: A row vector is a matrix of size 1 × c.
• Vectors allow for geometric representations of matrices.
Special Matrices
Special Matrices
Matrices/Vector Algebraic Operation
• As mentioned in the definition at
the beginning of class, algebra is
simply a set of math that defines
basic operations.
– Identity
– Zero
– Addition
– Subtraction
– Multiplication
– Division
• Matrix algebra is simply the use of
these operations with matrices.
VECTOR
 Vector as Geometric Representation
• An array x of n real
numbers x1, x2,..., xn is
called a vector, and it is
written as

• A vector x can be
represented geometrically
as a directed line in n
dimensions with
component x1 along the
first axis, x2 along the
second axis,..., and xn along
the nth axis.
Vector Length
• A vector has both direction and length.
• The length of a vector emanating from the origin is given by
the Pythagorean formula:
Vector Angle

• The inner (or dot) product of two vectors x and y is the sum of
element-by-element multiplication:
x′y = x1y1 + x2y2 + . . . + xkyk
• Then,
Vector Projections
• The projection (or shadow) of a vector x on a vector y is

• The length of the projection is

• Through such projections, a set of linear independent vectors

can be created from any set of vectors.
• One process used to create such vectors is through the Gram-
Schmidt Process.
Linear Dependencies
• A set of vectors are said to be linearly dependent if a1, a2, . . . ,
ak exist, and:
➢ a1x1 + a2x2 + . . . + akxk = 0.
➢ a1, a2, . . . , ak are not all zero.
• Such linear dependencies occur when a linear combination is
added to the vector set.
• Matrices comprised of a set of linearly dependent vectors are
singular.
• A set of linearly independent vectors forms what is called a
basis for the vector space.
• Any vector in the vector space can then be expressed as a
linear combination of the basis vectors.
Orthogonal Vectors
• We say that 2 vectors are orthogonal if they are perpendicular
to each other. i.e. the dot product of the two vectors is zero.
• We say that a set of vectors {v1, v2, …, vn} are mutually
orthogonal if every pair of vectors is orthogonal. i.e.
• vi . vj = 0 , for all i ≠ j
Orthonormal Vectors
• A set of vectors S is orthonormal if every vector in S has
magnitude 1 and the set of vectors are mutually orthogonal.
• If a’a = 1, the vector a is said to be normalized (has magnitude
1)
• The vector a can always be normalized by dividing by its
length. Thus

is normalized
MATRICE
Matrix Properties
• The following are some algebraic properties of
matrices:
o (A + B) + C = A + (B + C) - Associative
o A + B = B + A - Commutative
o c(A + B) = cA + cB - Distributive
o (c + d)A = cA + dA
o (A + B)′ = A′ + B′
o (cd)A = c(dA)
o (cA)′ = cA′
Matrix Properties
• The following are more algebraic properties of matrices:
o c(AB) = (cA)B
o A(BC) = (AB)C
o A(B + C) = AB + AC
o (B + C)A = BA + CA
o (AB)′ = B′A′
o For xj such that Axj is defined:
Matrix Determinant
• A square matrix can be characterized by a scalar value
called a determinant.
detA = |A|
• Much like the matrix inverse, calculation of the
determinant is very complicated and tedious, and is best
left to computers.
• What can be learned from determinants is if a matrix is
singular:
➔ If the square matrix A is singular, its determinant is 0
➔ If A is nonsingular, its determinant is nonzero
• If A and B are square and the same size, the determinant
of the product is the product of the determinants:
|AB| = |A||B|.
Eigenvalues and Eigenvectors
• A square matrix can be decomposed into a set of
eigenvalues and eigenvectors.
Ax = λx

• λ is called an eigenvalue of A, and x is an eigenvector

of A corresponding to λ. To find λ and x, we write
(2.104) as
(A − λI)x = 0.

• Ordinarily, we normalize x so that it has length unity;

that is, 1 = x'x. It is convenient to denote normalized
eigenvectors by e
POSITIVE DEFINITE MATRICES

• The symmetric square matrix A is said to be

positive definite if x’Ax > 0 for all possible vectors
x (except x = 0).
• Similarly, A is positive semidefinite if x’Ax ≥ 0 for
all x ≠ 0.
• The diagonal elements aii of a positive definite
matrix are positive.
• If A is positive definite, its determinant is positive.
• The eigenvalues and eigenvectors of positive definite
and positive semidefinite matrices have the following
properties:
1. The eigenvalues of a positive definite matrix are
all positive.
2. The eigenvalues of a positive semidefinite matrix
are positive or zero, with the number of positive
eigenvalues equal to the rank of the matrix.
Orthogonal Matrix
• A matrix C = (c1, c2, . . . , cp) whose columns are
normalized and mutually orthogonal is called an
orthogonal matrix.

• Since the elements of C’C are products of columns of

C, which have the properties ci’ci = 1 for all i and ci’cj
= 0 for all i ≠ j, we have
C’C = CC’ = I.
Spectral Decompositions
• Imagine that a matrix A is of size k × k.
• A then has:
– k eigenvalues: i, i = 1, . . . , k.
– k eigenvectors: ei, i = 1, . . . , k (each of size k × 1.
• A can be expressed by:

• This expression is called the Spectral Decomposition,

where A is decomposed into k parts.
• One can find A−1 by taking 1/λ in the spectral
decomposition.
Random Vector and Random Matrices

• A random vector is a vector whose elements are random

variables.
• A random rnatrix is a matrix whose elements are random
variables.
• The expected value of a random matrix (or vector) is the
matrix (vector) consisting of the expected values of each of its
elements.
Mean Vector and Covariances Matrice

• The means and covariances of the p x 1 random vector X can

be set out as matrices. The expected value of each element is
contained in the vector of means μ = E{X), and the p variances
σii and the p(p — 1)/2 distinct covariances σik (i < k) are
contained in the symmetric variance-covariance matrix ∑ =
E(X - μ)(X - μ)'. Specifically,
GEOMETRI SAMPEL
Data Representation

𝑥11 𝑥12 ⋯ 𝑥1𝑝

𝑥21 𝑥22 ⋯𝑥2𝑝
x’ = [X1 X2 … Xp] 𝑿=
⋮ ⋮ ⋱ ⋮
𝑥𝑛1 𝑥𝑛2 ⋯𝑥𝑛𝑝

Random vectors of Sample Matrix of p

p random random variables and n
variables observations

Each row of X represents a multivariate observation. The sample

then consists of n measurements, each of which has p
components.
 The data can be plotted in two
different ways.

• The row vector x’j, representing the jth ob-

servation, contains the coordinates of a
point.
• The scatter plot of n points in p-dimensional
space provides information on the locations
and variability of the points.

• considering the data as p vectors

in n-dimensional space.
• we depict yi,..., yp as vectors
rather than points, as in the p-
dimensional scatter plot.
Example
If the points are regarded as
solid spheres, the sample mean
vector 𝑥,ҧ is the center of
balance.
Geometrical Interpretation of the Sample
Define the n x 1 vector 1 = (1,1,..., 1). The vector 1 forms equal angles
with each of the n coordinate axes, so the vector (1/√n) has unit length
in the equal-angle direction. Then Consider the vector y’i = (x1, x2, ...,
xn).
1. The projection of a column yi of the data matrix X onto the equal
angular vector 1 is the vector 𝑥ҧ𝑖 𝟏. The vector 𝑥ҧ𝑖 𝟏 has length
𝑛 𝑥ҧ1 . Therefore, the i-th sample mean, 𝑥ҧ𝑖 , is related to the
length of the projection of yi on 1.

2. The information comprising Sn is obtained from the deviation

vectors 𝑑𝑖 = 𝑦𝑖 − 𝑥ҧ𝑖 𝟏 . The square of the length of di is nsii, and
the (inner) product between di and dk is nsik .

3. The sample correlation rik is the cosine of the angle between di

and dk.
Illustration for p = 3 and n = 3
where 𝑥ҧ𝑖 𝟏 is the projection of yi
on the unit vector (1/√n)1

where 𝑥ҧ𝑖 𝟏 is perpendicular to yi

— 𝑥ҧ𝑖 𝟏
The elements of di are the deviations of the measurements on the
i-th variable from their sample mean.

Now consider the squared lengths of the

deviation vectors.

we see that the squared length is proportional

to the variance of the measurements on the i-
th variable. Equivalently, the length is
proportional to the standard deviation.
For any two deviation vectors di and dk

Let θik denote the angle formed by the vectors di and dk. From
the length of projection formula, we get

Thus,
So, the relation implies …

The correlation matrix can be obtained from the covariance matrix,

and vice versa. Define

Then
RANDOM SAMPLES AND THE EXPECTED
VALUES OF THE SAMPLE MEAN AND
COVARIANCE MATRIX
Random Sample
Suppose, the data have not yet been observed, but we intend to
collect n sets of measurements on p variables. Before the
measurements are made, their values cannot, in general, be
predicted exactly. Consequently, we treat them as
random variables.

Each set of measurements Xj

on p variables is a random
vector,

If the row vectors X’1, X’2, … , X’n above represent independent observations
from a common joint distribution with density function f(x) = f(x1, x2, … , xp).
then X1, X2, … , X„ are said to form a random sample from f(x).
Sampling Distribution of 𝑋ത and Sn
Let X1, X2,..., Xn be a random sample from a joint distribution that has
mean vector μ and covariance matrix ∑.
Generalized Variance
The overall sample covariance matrix gives a picture of the
covariation between each variable in the sample.

where

Sometimes it is desirable to assign a single numerical value for the

variation expressed by S.
The generalized sample variance provides one way of writing the
information on all variances and covariances as a single number.

Generalized Sample Variance

is computed by |S| (the
determinant of the sample
covariance matrix).
Geometrical Interpretation of Generalized Variance
Consider the area generated within the plane by two deviation
vectors 𝒅𝟏 = 𝒚𝟏 − 𝒙 ഥ𝟏 𝟏 and 𝒅𝟐 = 𝒚𝟐 − 𝒙
ഥ𝟐 𝟏- Let Ldi be the length of
di and Ldj, the length of d2.
Meanwhile,

Then,
Generalized variance also has interpretations in the p-
space scatter plot representation of the data.

To begin, imagine a
multidimensional
cube (an ellipsoid)
that represents the
distribution of sample
matrix X.
Example
Deviation

Matrix S
LINEAR COMBINATIONS OF
VARIABLES
Let a1, a2, . . . , ap be constants and consider the linear combination
of the elements of the vector y,

where a = (a1, a2, . . . , ap). If the same coefficient vector a is applied

to each yi in a sample, we have

The sample mean of z can be found either by averaging the n

values z1 = ay1, z2 = ay2, . . . , zn = ayn

Sample
Variance
Responsi
1. Is the following matrix positive definite?

2. Check that

is an orthogonal matrix.