Learning from observations

 Inductive Learning - learning from examples

Machine Learning
 Learning & Adaptation

• ”Modification of a behavioral tendency by expertise.”

(Webster 1984)
• ”A learning machine, broadly defined is any device whose
actions are influenced by past experiences.” (Nilsson 1965)
• ”Any change in a system that allows it to perform better
the second time on repetition of the same task or on another
task drawn from the same population.” (Simon 1983)
• ”An improvement in information processing ability that results
from information processing activity.” (Tanimoto 1990)

2
Machine Learning

 Machine learning involves automatic

procedures that learn a task from a series of
examples

 Most convenient source of examples is data

3
Learning

Definition:
A computer program is said to learn from
experience E with respect to some class of
tasks T and performance measure P, if its
performance at tasks in T, as measured by
P, improves with experience.

4
Machine Learning Models
 Classification
 Regression
 Clustering
 Time series analysis
 Association Analysis
 Sequence Discovery
 ….

5
 Classification example

New data

Train set Test set

Loan
Learning system Model Yes/No
6
Machine Learning Methods
 Artificial Neural Networks
 Decision Trees
 Instance Based Methods (CBR, k-NN)
 Bayesian Networks
 Evolutionary Strategies
 Support Vector Machines
 ..

7
 Classification example
Features: height, weight

x x
o
o x x
o x x
Height o o
o x
o x x o x
o o oo x
x
x
x - weight-lifters
x
o - ballet dancers

Weight

8
 Classification example - Simple Model
Features: height, weight

Decision boundary

x x
o
o x x
o x x
Height o o
o x
o x x o x
o o oo x
x
x
x - weight-lifters
x
o - ballet dancers

Weight

9
 Classification example - Complex model
Features: height, weight

Complex Decision boundary

x x
o
o x x
o x x
Height o o
o x
o x x o x
o o oo x
x
x
x - weight-lifters
x
o - ballet dancers

Weight
Note: A simple decision boundary is better than a complex
one - It GENERALIZES better. 10
Learning Paradigms
 Supervised learning - with teacher
 inputs and correct outputs are provided by the teacher
 Reinforced learning - with reward or punishment
 an action is evaluated
 Unsupervised learning - with no teacher
 no hint about correct output is given

11