Singular Value Decomposition: A case study for

removing background from videos

1. Introduction

1.1 What is SVD?

 SVD deals with decomposing a matrix into a product of 3

matrices as shown:

If the dimensions of A are m x n:

 U is an m x m matrix of Left Singular Vectors

 S is an m x n rectangular diagonal matrix of Singular
Values arranged in decreasing order
 V is an n x n matrix of Right Singular Vectors

1.2 Applications of the SVD:

1) Solving homogeneous linear equations

2) Range, null space and rank
3) Low-rank matrix approximation
4) Image Compression
5) Image Recovery
6) Eigen faces
7) Spectral Clustering
8) Background Removal from Videos
 2. SVD based - removal of background
from videos

 We have all seen the music videos, marriage videos, TV

commercials and programs manage to have an eye
catching background behind the actors and people.

 They hire professionals to do editing and they do the

work manually using editors. But still this is very
expensive and tedious.

 We live in 21st century where all the things are done

automatically so why put that much manual effort?

 As being the emerging trend an Artificial intelligence and

Machine learning, we can apply machine learning to
accomplish this within few minutes by applying the SVD
concept.
So, let’s distinguish the background of a video from its
foreground:

 The background of a video is essentially static – it does

not see a lot of movement. All the movement is seen in
the foreground. This is the property that we exploit to
separate the background from the foreground.

 Here are the steps we can follow for implementing this

approach:

 Create matrix M from video – This is done by sampling

image snapshots from the video at regular intervals,
flattening these image matrices to arrays, and storing
them as the columns of matrix M.

 We get the following plot for matrix M:

Here is a frame of the video after removing the background:

We have discussed a very useful application of SVD so far.

But how does the math behind SVD actually work? Let’s
understand these points in the next section.
 3. Why is SVD used in Dimensionality
Reduction?

 You might be wondering why we should go through

with this seemingly painstaking decomposition. The
reason can be understood by an alternate
representation of the decomposition. See the
figure below:

The decomposition allows us to express our original matrix

as a linear combination of low-rank matrices.

In a practical application, you will observe that only the first

few, say k, singular values are large. The rest of the singular
values approach zero. As a result, terms except the first few
can be ignored without losing much of the information. See
how the matrices are truncated in the figure below:
 Summary
 Using SVD, we are able to represent our large matrix A
by 3 smaller matrices U, S and V
 This is helpful in large computations
 We can obtain a k-rank approximation of A. To do this,
select the first k singular values and truncate the 3
matrices accordingly