Professional Documents
Culture Documents
Welcome
Machine Learning
Andrew Ng
Andrew Ng
SPAM
Andrew Ng
Machine Learning
- Grew out of work in AI
- New capability for computers
Examples:
- Database mining
Large datasets from growth of automation/web.
E.g., Web click data, medical records, biology, engineering
- Applications can’t program by hand.
E.g., Autonomous helicopter, handwriting recognition, most of
Natural Language Processing (NLP), Computer Vision.
Andrew Ng
Machine Learning
- Grew out of work in AI
- New capability for computers
Examples:
- Database mining
Large datasets from growth of automation/web.
E.g., Web click data, medical records, biology, engineering
- Applications can’t program by hand.
E.g., Autonomous helicopter, handwriting recognition, most of
Natural Language Processing (NLP), Computer Vision.
Andrew Ng
Machine Learning
- Grew out of work in AI
- New capability for computers
Examples:
- Database mining
Large datasets from growth of automation/web.
E.g., Web click data, medical records, biology, engineering
- Applications can’t program by hand.
E.g., Autonomous helicopter, handwriting recognition, most of
Natural Language Processing (NLP), Computer Vision.
Andrew Ng
Machine Learning
- Grew out of work in AI
- New capability for computers
Examples:
- Database mining
Large datasets from growth of automation/web.
E.g., Web click data, medical records, biology, engineering
- Applications can’t program by hand.
E.g., Autonomous helicopter, handwriting recognition, most of
Natural Language Processing (NLP), Computer Vision.
- Self-customizing programs
E.g., Amazon, Netflix product recommendations
Andrew Ng
Machine Learning
- Grew out of work in AI
- New capability for computers
Examples:
- Database mining
Large datasets from growth of automation/web.
E.g., Web click data, medical records, biology, engineering
- Applications can’t program by hand.
E.g., Autonomous helicopter, handwriting recognition, most of
Natural Language Processing (NLP), Computer Vision.
- Self-customizing programs
E.g., Amazon, Netflix product recommendations
- Understanding human learning (brain, real AI).
Andrew Ng
Andrew Ng
Introduction
What is machine
learning
Machine Learning
Andrew Ng
Machine Learning definition
Andrew Ng
Machine Learning definition
• Arthur Samuel (1959). Machine Learning: Field of
study that gives computers the ability to learn
without being explicitly programmed.
Andrew Ng
Machine Learning definition
• Arthur Samuel (1959). Machine Learning: Field of
study that gives computers the ability to learn
without being explicitly programmed.
Andrew Ng
Machine Learning definition
• Arthur Samuel (1959). Machine Learning: Field of
study that gives computers the ability to learn
without being explicitly programmed.
• Tom Mitchell (1998) Well-posed Learning
Problem: A computer program is said to learn
from experience E with respect to some task T
and some performance measure P, if its
performance on T, as measured by P, improves
with experience E.
Andrew Ng
“A computer program is said to learn from experience E with respect to
some task T and some performance measure P, if its performance on T,
as measured by P, improves with experience E.”
Suppose your email program watches which emails you do or do
not mark as spam, and based on that learns how to better filter
spam. What is the task T in this setting?
Andrew Ng
Housing price prediction.
400
300
Price ($)
200
in 1000’s
100
0
0 500 1000 1500 2000 2500
Size in feet2
Tumor Size
Andrew Ng
- Clump Thickness
- Uniformity of Cell Size
Age - Uniformity of Cell Shape
…
Tumor Size
Andrew Ng
You’re running a company, and you want to develop learning algorithms to address each
of two problems.
Problem 1: You have a large inventory of identical items. You want to predict how many
of these items will sell over the next 3 months.
Problem 2: You’d like software to examine individual customer accounts, and for each
account decide if it has been hacked/compromised.
Andrew Ng
Supervised Learning
x2
x1
Andrew Ng
Unsupervised Learning
x2
x1
Andrew Ng
Andrew Ng
Andrew Ng
Genes
Individuals
Individuals
Speaker #1 Microphone #1
Speaker #2 Microphone #2
Andrew Ng
Microphone #1: Output #1:
[W,s,v] = svd((repmat(sum(x.*x,1),size(x,1),1).*x)*x');
Given a set of news articles found on the web, group them into
set of articles about the same story.
Given a database of customer data, automatically discover market
segments and group customers into different market segments.
Given a dataset of patients diagnosed as either having diabetes or
not, learn to classify new patients as having diabetes or not.
Andrew Ng
Linear
regression
with
one
variable
Model
representa6on
Machine
Learning
Andrew
Ng
500
Housing
Prices
400
(Portland,
OR)
300
Price
200
(in
1000s
100
of
dollars)
0
0
500
1000
1500
2000
2500
3000
Size
(feet2)
Supervised
Learning
Regression
Problem
Andrew
Ng
Training
Set
How
do
we
represent
h
?
Learning Algorithm
Andrew
Ng
Linear
regression
with
one
variable
Cost
func6on
Machine Learning
Andrew
Ng
Size
in
feet2
(x)
Price
($)
in
1000's
(y)
Training
Set
2104
460
1416
232
1534
315
852
178
…
…
Hypothesis:
‘s:
Parameters
How
to
choose
‘s
?
Andrew
Ng
3
3
3
2 2 2
1 1 1
0
0
0
0
1
2
3
0
1
2
3
0
1
2
3
Andrew
Ng
y
x
Andrew
Ng
Linear
regression
with
one
variable
Cost
func6on
intui6on
I
Machine
Learning
Andrew
Ng
Simplified
Hypothesis:
Parameters:
Cost Func6on:
Goal:
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameter
)
3 3
2
2
y
1
1
0
0
0
1
2
3
-‐0.5
0
0.5
1
1.5
2
2.5
x
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameter
)
3 3
2
2
y
1
1
0
0
0
1
2
3
-‐0.5
0
0.5
1
1.5
2
2.5
x
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameter
)
3 3
2
2
y
1
1
0
0
0
1
2
3
-‐0.5
0
0.5
1
1.5
2
2.5
x
Andrew
Ng
Linear
regression
with
one
variable
Cost
func6on
intui6on
II
Machine
Learning
Andrew
Ng
Hypothesis:
Parameters:
Cost Func6on:
Goal:
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
500
400
Price
($)
300
in
1000’s
200
100
0
0
1000
2000
3000
Size
in
feet2
(x)
Andrew
Ng
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
Linear
regression
with
one
variable
Gradient
Machine
Learning
descent
Andrew
Ng
Have
some
func6on
Want
Outline:
• Start
with
some
• Keep
changing
to
reduce
un6l
we
hopefully
end
up
at
a
minimum
Andrew
Ng
J(θ0,θ1)
θ1
θ0
Andrew
Ng
J(θ0,θ1)
θ1
θ0
Andrew
Ng
Gradient
descent
algorithm
Andrew
Ng
Linear
regression
with
one
variable
Gradient
descent
intui6on
Machine
Learning
Andrew
Ng
Gradient
descent
algorithm
Andrew
Ng
Andrew
Ng
If
α
is
too
small,
gradient
descent
can
be
slow.
Andrew
Ng
at
local
op6ma
Andrew
Ng
Gradient
descent
can
converge
to
a
local
minimum,
even
with
the
learning
rate
α
fixed.
Andrew
Ng
Gradient
descent
algorithm
Linear
Regression
Model
Andrew
Ng
Andrew
Ng
Gradient
descent
algorithm
update
and
simultaneously
Andrew
Ng
J(θ0,θ1)
θ1
θ0
Andrew
Ng
J(θ0,θ1)
θ1
θ0
Andrew
Ng
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
(for
fixed
,
this
is
a
func6on
of
x)
(func6on
of
the
parameters
)
Andrew
Ng
“Batch”
Gradient
Descent
Andrew Ng