Professional Documents
Culture Documents
Linalg PDF
Linalg PDF
Review of vectors
wn
By default, vectors are column vectors
2
Linear independence
A vector w is linearly dependent on v1 , v2 if
w = α1 v1 + α2 v2 for some α1 , α2 ∈ R
Example:
2 1 0
6 = 2 1 + 4 1
4 0 1
Other names: linear combination, linear span
3
Basis
Every w ∈ Rn is a linear combination of the linearly independent set
1 0 0 1 0 0
0 1
0 0 1 0
B = . , . , . . . . w = w1 . +w2 . + . . . + wn .
.. .. ..
.. .. ..
0 0 1 0 0 1
|{z} |{z} |{z}
e1 e2 en
Basis ≡ any linearly independent set that spans the entire space
Any basis for Rn has exactly n elements
4
Norms
A function ρ(v) of a vector v is called a norm if
ρ(v) ≥ 0 and ρ(v) = 0 implies v = 0
ρ(αv) = |α| ρ(v) for all α ∈ R
ρ(v1 + v2 ) ≤ ρ(v1 ) + ρ(v2 ) (triangle inequality)
ρ generalizes the notion of “length”
Examples:
qP
n
`2 norm: kxk2 = i=1
|xi |2 ... usual length
Pn
`1 norm: kxk1 = i=1
|xi |
`∞ norm: kxk∞ = max1≤i≤n |x|i
P 1p
n
`p norm, 1 ≤ p < ∞: kxkp = i=1
|x|pi
5
Inner product
The inner-product or dot-product of two vector v, w ∈ Rn is defined as
n
X
v·w= vi wi
i=1
√
The `2 norm kvk2 = v·v
The angle θ between two vectors v and w is given by
v·w
cos(θ) =
kvk2 kwk2
6
Financial Engineering and Risk Management
Review of matrices
2
Matrix Operations: Transpose
Transpose: A ∈ Rm×d
>
a11 a12 . . . a1d a11 a21 ... am1
a21 a22 . . . a2d a12 a22 ... ad2
A> = . .. = .. .. ∈ R
d×m
.. .. .. ..
.. . . . . . . .
am1 am2 . . . amd a1d a2d ... amd
Example:
"
2 1
#
2 3 7
A= : 2 × 3 matrix ... A> = 3 6 : 3 × 2 matrix
1 6 5
7 5
" #
2
v = 6 : column vector ... v> = 2
6 4 : row vector
4
3
Matrix Operations: Multiplication
Multiplication: A ∈ Rm×d , B ∈ Rd×p then C = AB ∈ Rm×p
b1j
b2j
cij = ai1 ai2 . . . aid .
..
bdj
Examples:
"2#
2 3 7 2(2) + 3(6) + 7(4) 50
6 = =
1 6 5 1(2) + 6(6) + 5(4) 58
4
s s >
1
p 1 1 1
`2 norm:
2 2
−2
= 1 + (−2) =
1 −2 =
−2 −2 −2
2
4
Linear functions
A function f : Rd 7→ Rm is linear if
Examples
x1
" #
3
f (x) : R 7→ R: f (x) = 2 3 4 x2 = 2x1 + 3x2 + 4x3
x3
"x1 #
3 2 2 3 4 2x1 + 3x2 + 4x3
f (x) : R →
7 R : f (x) = x2 =
1 0 2 x1 + 2x3
x3
5
Rank of a matrix
column rank of A ∈ Rm×d = number of linearly independent columns
range(A) = {y : y = Ax for some x}
column rank of A = size of basis for range(A)
column rank of A = m ⇒ range(A) = Rm
Example:
1 2 3 1
A= , rank = 1, range(A) = λ :λ∈R
2 4 6 2
A−1 A = AA−1 = I
6
Financial Engineering and Risk Management
Review of linear optimization
Hedge an obligation X ∈ Rm
Have to pay Xi if state i occurs
Buy/short sell θ = (θ1 , . . . , θd )> shares to cover obligation
2
Hedging problem (contd)
Position θ ∈ Rd purchased at time t = 0
θj = number of shares of asset j purchased, j = 1, . . . , d
Pd
Cost of the position θ = j=1
pj θj = p> θ
θ1
d
θ2 X
y = S1 S2 . . . Sd . =
.. θj S j
j=1
θd
Payoff y hedges X if y ≥ X.
3
Hedging problem (contd)
Optimization problem:
Pd
min pj θj (≡ p> θ)
Pj=1
d
subject to j=1 Sij θj ≥ Xi , i = 1, . . . , m (≡ Sθ ≥ X)
4
Linear programming duality
Linear program
P = minx c> x
subject to Ax ≥ b
Dual linear program
D = maxu b> u
subject to A> u = c
u≥0
Theorem.
Weak Duality: P ≥ D
Bound: x feasible for P, u feasible for D, c> x ≥ P ≥ D ≥ b> u
Strong Duality: Suppose P or D finite. Then P = D.
Dual of the dual is the primal (original) problem
5
More duality results
Here is another primal-dual pair
minx c> x = maxu b> u
subject to Ax = b subject to A> u = c
P = min{c> x : Ax ≥ b}
≥ min{c> x − u> (Ax − b) : Ax ≥ b} for all u ≥ 0
≥ b> u + min{(c − A> u)> x : x ∈ Rn }
>
b u A> u = c
=
−∞ otherwise
≥ max{b> u : A> u = c}
6
Financial Engineering and Risk Management
Review of nonlinear optimization
Hessian ∇2 f (x) =
.. .. .. ..
positive semidefinite
. . . .
∂2f ∂2f ∂2f
∂xn ∂x1 ∂xn ∂x2
... ∂xn2
2
Unconstrained nonlinear optimization
Optimization problem
Gradient
" #
− 94
2x1 + 3x2
∇f (x) = =0 ⇒ x = 0,
3x1 + 3x22 3
2
2 3
Hessian at x: H =
3 6x2
2 3
x = 0: H = . Not positive definite. Not local minimum.
3 0
− 94 2 3
x= 3 : H= . Positive semidefinite. Local minimum
2
3 9
3
Lagrangian method
Constrained optimization problem
maxx∈R2 2 ln(1 + x1 ) + 4 ln(1 + x2 ),
s.t. x1 + x2 = 12
4
Portfolio Selection
Optimization problem
maxx µ> x − λx> Vx
s.t. 1> x = 1
Constraints make the problem hard!
Lagrangian function
L(x, v) = µ> x − λx> Vx − v(1> x − 1)