Open in app Get started

Published in Towards Data Science

Vihar Kurama Follow

May 7, 2018 · 5 min read · Listen

Save

Linear Algebra for Deep Learning

The Math behind every deep learning program.

Deep Learning is a subdomain of machine learning, concerned with the

algorithm which imitates the function and structure of the brain called the
artificial neural network.

Linear algebra is a form of continuous rather than discrete mathematics,

many computer scientists have little experience with it. A good understanding
of linear algebra is essential for understanding and working with many
machine learning algorithms, especially deep learning algorithms.

5.5K 8
:
 Why Math?
Linear algebra, probability and calculus are the ‘languages’ in which machine
learning is formulated. Learning these topics will contribute a deeper
understanding of the underlying algorithmic mechanics and allow
development of new algorithms.

When confined to smaller levels, everything is math behind deep learning. So

it is essential to understand basic linear algebra before getting started with
deep learning and programming it.
:
 src

The core data structures behind Deep-Learning are Scalars, Vectors, Matrices
and Tensors. Programmatically, let’s solve all the basic linear algebra problems
using these.

Scalars
Scalars are single numbers and are an example of a 0th-order tensor. The
notation x ∈ ℝ states that x is a scalar belonging to a set of real-values numbers,
ℝ.

There are different sets of numbers of interest in deep learning. ℕ represents

the set of positive integers (1,2,3,…). ℤ designates the integers, which combine
positive, negative and zero values. ℚ represents the set of rational numbers
that may be expressed as a fraction of two integers.

Few built-in scalar types are int, float, complex, bytes, Unicode in Python. In
In NumPy a python library, there are 24 new fundamental data types to
describe different types of scalars. For information regarding datatypes refer
documentation here.

Defining Scalars and Few Operations in Python:

:
 The following code snippet explains few arithmetic operations on Scalars.

<class 'int'>
<class 'float'>
12.5
-2.5
37.5
:
 0.6666666666666666

The following code snippet checks if the given variable is scalar or not.

True
:
 False
True

Vectors
Vectors are ordered arrays of single numbers and are an example of 1st-order
tensor. Vectors are fragments of objects known as vector spaces. A vector
space can be considered of as the entire collection of all possible vectors of a
particular length (or dimension). The three-dimensional real-valued vector
space, denoted by ℝ^3 is often used to represent our real-world notion of
three-dimensional space mathematically.

To identify the necessary component of a vector explicitly, the ith scalar

element of a vector is written as x[i].

In deep learning vectors usually represent feature vectors, with their

original components defining how relevant a particular feature is.
Such elements could include the related importance of the intensity of a set of
pixels in a two-dimensional image or historical price values for a cross-section
of financial instruments.

Defining Vectors and Few Operations in Python:

:
 <class 'list'>
[1, 2, 3, 4, 5, 6]
[5 7 9]
<class 'numpy.ndarray'>
[-3 6 -3]

Matrices
Matrices are rectangular arrays consisting of numbers and are an example of
2nd-order tensors. If m and n are positive integers, that is m, n ∈ ℕ then the
m×n matrix contains m*n numbers, with m rows and n columns.

The full m×n matrix can be written as:

:
 It is often useful to abbreviate the full matrix component display into the
following expression:

In Python, We use numpy library which helps us in creating ndimensional

arrays. Which are basically matrices, we use matrix method and pass in the
lists and thereby defining a matrix.

$python

>>> import numpy as np

>>> x = np.matrix([[1,2],[2,3]])
>>> x
matrix([[1, 2],
[2, 3]])

>>> a = x.mean(0)
>>> a
matrix([[1.5, 2.5]])
>>> # Finding the mean with 1 with the matrix x.
>>> z = x.mean(1)
>>> z
matrix([[ 1.5],
[ 2.5]])
:
 >>> z.shape
(2, 1)
>>> y = x - z
matrix([[-0.5, 0.5],
[-0.5, 0.5]])
>>> print(type(z))
<class 'numpy.matrixlib.defmatrix.matrix'>

Defining Matrices and Few Operations in Python:

Matrix Addition
Matrices can be added to scalars, vectors and other matrices. Each of these
operations has a precise definition. These techniques are used frequently in
machine learning and deep learning so it is worth familiarising yourself with
them.
:
 Matrix-Matrix Addition
C = A + B (Shape of A and B should be equal)

The methods shape return the shape of the matrix, and add takes in two
arguments and returns the sum of those matrices. If the shape of the matrices
is not same it throws an error saying, addition not possible.
:
 Matrix-Scalar Addition
Adds the given scalar to all the elements in the given matrix.
:
 Matrix Scalar Multiplication
Multiplies the given scalar to all the elements in the given matrix.
:
 Matrix Multiplication
A of shape (m x n) and B of shape (n x p) multiplied gives C of shape (m x p)

src
:
 Matrix Transpose
With transposition you can convert a row vector to a column vector and vice
versa:

A=[aij]mxn

AT=[aji]n×m
:
 Tensors
The more general entity of a tensor encapsulates the scalar, vector and the
matrix. It is sometimes necessary — both in the physical sciences and machine
learning — to make use of tensors with order that exceeds two.
:
 src

We use Python libraries like tensorflow or PyTorch in order to declare tensors,

rather than nesting matrices.

To define a simple tensor in PyTorch

:
 Few Arithmetic Operations on Tensors in Python
:
 For more documentation regarding tensors and PyTorch click here.

Important Links

To get started with deep learning in python:

Deep Learning with Python

The human brain imitation.
towardsdatascience.com
:
 Introduction To Machine Learning
Machine Learning is an idea to learn from examples and
experience, without being explicitly programmed. Instead of…
towardsdatascience.com

Closing Notes
Thanks for reading. If you found this story helpful, please click the below to
spread the love.

Special Thanks to Samhita Alla for her contributions towards the article.
:
 Sign up for The Variable
By Towards Data Science

Every Thursday, the Variable delivers the very best of Towards Data Science: from hands-on tutorials and
cutting-edge research to original features you don't want to miss. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review
our Privacy Policy for more information about our privacy practices.

Get this newsletter

About Help Terms Privacy

Get the Medium app

: