Matrix Factorization and

its applications
By Zachary
16th Nov, 2010

Outline
Expression

power of matrix
Various matrix factorization
methods
Application of matrix
factorization

What can matrix

represent?
System

of equations
User rating matrix
Image
Matrix structure in graph theory
Adjacent matrix
Distance matrix

Different matrix factorization

methods
LU

decomposition
Singular Value
Decomposition(SVD)
Probabilistic Matrix
Factorization(PMF)
Non-negative Matrix
Factorization(NMF)

Application of matrix
factorization
LU

decomposition

Solving system of equations

SVD

decomposition

Low rank matrix approximation

Pseudo-inverse

Application of matrix
factorization
PMF

Recommendation system
NMF

Learning the parts of objects

PMF
Consider

a typical
recommendation problem
Given a n by m matrix R with some
entries unknown
n rows represent n users
m columns represent m movies
EntryRij represent the ith users rating on
the jth movie

We are interested in the unknown

entries possible values
i.e. Predict users ratings

PMF
We

can model the problem as

R=UV
U (k by n) is the latent feature matrix
for users
How much the user likes action movie?
How much the user likes comedy movie?

V (k by m) is the latent feature matrix

for movies
To what extent is the movie an action movie?
To what extent is the movie a comedy
movie?

PMF
If

we can learn U and V from

existing ratings, then we can
compute unknown entries by
multiplying these two matrices.
Lets consider a probabilistic
approach.

PMF

PMF
We

want to maximize

Equivalent

Can

to minimizing

be solved using steepest

descent method

Extension to PMF
We

can augment the model as

long as we have additional data
matrix that share comment latent
feature matrix

NMF
Consider

the following problem

M = 2429 facial images

Each image of size n = 19 by 19 = 361
Matrix V = n by m is the original dataset
We want to approximate V by two lower
rank matrix W (n by 49) and H (49 by m)
V ~ WH
Constraints
All entries of W and H are non-negative

NMF
How

well can W and H

approximate V
How can we interpret the result

NMF
Assumption

Maximize logarithm likelihood and

we get the objective function

Criticize of NMF
NMF

doesnt always
give parts based result
Sparseness constraints

For

more information,
refer to Non-negative matrix
factorization with sparseness constrains

Questions?
Thank

you