Introduction to ML

1. What is Machine Learning

Machine Learning is the study of algorithms that improve their performance P at some task T with experience E.
A well-defined learning task is given by <P, T, E>.

2. When do we use Machine Learning?

ML is used when:
1. Human expertise does not exist (navigating on Mars)
2. Humans can’t explain their expertise (speech recognition)
3. Models must be customized (personalized medicine)
4. Models are based on huge amounts of data (genomics)

3. Some examples of tasks that are best solved by using a learning algorithm
1. Recognizing patterns:
– Facial identities or facial expressions
– Handwritten or spoken words
– Medical images
2. Generating patterns:
– Generating images or motion sequences
3. Recognizing anomalies:
– Unusual credit card transactions
– Unusual patterns of sensor readings in a nuclear power plant
4. Prediction:
– Future stock prices or currency exchange rates

4. Sample Applications of Machine Learning

• Web search
• Space exploration
• Robotics
• Information extraction
• Social networks
• Debugging software

5. Defining the Learning Task

Improve on task T, with respect to performance metric P, based on experience E.
T: Playing checkers
P: Percentage of games won against an arbitrary opponent
E: Playing practice games against itself
T: Recognizing hand-written words
P: Percentage of words correctly classified
E: Database of human-labeled images of handwritten words
T: Driving on four-lane highways using vision sensors
P: Average distance traveled before a human-judged error
E: A sequence of images and steering commands recorded while observing a human driver.
T: Categorize email messages as spam or legitimate.
P: Percentage of email messages correctly classified.
E: Database of emails, some with human-given labels

Page - 1 of 4
 Introduction to ML

6. Machine Learning in Automatic Speech Recognition

ML used to predict of phone states from the sound spectrogram

7. Types of Machine Learning Algorithm

1. Supervised (inductive) learning – Given: training data + desired outputs (labels)

These methods rely on labeled training data sets to learn a function between input variables (X) and output variables
(Y). The most common types include:
a. Classification methods, which predict the output of a given data sample when the output variable is
categorical, for instance dead or alive, sick or healthy.
b. Regression methods, which predict output variables that are real values, such as the age of a person or the
amount of snowfall.
c. Ensemble methods, which combine predictions from weaker algorithmic output to predict new output.
2. Unsupervised learning – Given: training data (without desired outputs)

These methods use only input variables (X), not output variables, and rely on unlabeled training data sets to map the
underlying structure of the data. Common examples include:
a. Association methods, which uncover probability of items in a collection, as in market-basket analysis.
b. Clustering methods, which group samples of objects based on similarity.
Independent component analysis – separate a combined signal into its original sources.

3. Semi-supervised learning – Given: training data + a few desired outputs

Semi-supervised learning is an approach to machine learning that combines a small amount of labeled data with a large
amount of unlabeled data during training. In this learning method, training data are given with a few desired output.
A common example of an application of semi-supervised learning is a text document classifier.
Page - 2 of 4
 Introduction to ML

4. Reinforcement learning – Rewards from sequence of actions

These methods allow the user or other designated agent to decide the best next action, based on the current state and
learned behaviors that maximize the rewards. This approach is often used in robotics.
Examples: Credit assignment problem, Robot in a maze etc.
The agent environment interface in Reinforcement Learning:

5. Inverse Reinforcement Learning – Learn policy from user demonstrations.

In Reinforcement Learning (RL), agents are trained on a reward and punishment mechanism. The agent is rewarded
for correct moves and punished for the wrong ones. In doing so, the agent tries to minimize wrong moves and maximize
the right ones.
Example: Stanford Autonomous Helicopter

8. Designing a Learning System

 Choose the training experience
 Choose exactly what is to be learned i.e. the target function
 Choose how to represent the target function
 Choose a learning algorithm to infer the target function from the experience

9. Training vs. Test Distribution

We generally assume that the training and test examples are independently drawn from the same overall distribution of data.
We call this “i.i.d” which stands for “independent and identically distributed”
 If examples are not independent, requires collective classification
 If test distribution is different, requires transfer learning

10. ML in a Nutshell
Machine learning is essentially a subfield of artificial intelligence (AI). In a nutshell, the goal of machine learning is to learn
from data and make accurate outcome predictions, without being explicitly programmed. Tens of thousands of machine learning
algorithms where innovating Hundreds new every year.
Every ML algorithm has three components:
1. Representation
2. Optimization
3. Evaluation

Page - 3 of 4
 Introduction to ML

11. Various Function Representations

1. Numerical functions 3. Instance-based functions
 Linear regression  Nearest-neighbor
 Neural networks  Case-based
 Support vector machines 4. Probabilistic Graphical Models
2. Symbolic functions  Naïve Bayes
 Decision trees  Bayesian networks
 Rules in propositional logic  Hidden-Markov Models (HMMs)
 Rules in first-order predicate logic  Probabilistic Context Free Grammars
(PCFGs)
 Markov networks

12. Various Search/Optimization Algorithms.

1. Gradient descent 3. Divide and Conquer
 Perceptron  Decision tree induction
 Backpropagation  Rule learning
4. Evolutionary Computation
2. Dynamic Programming
 Genetic Algorithms (GAs)
 HMM Learning
 Genetic Programming (GP)
 PCFG Learning
 Neuro-evolution

13. Various Evaluation Algorithms

 Accuracy  Cost / Utility
 Precision and recall  Margin
 Squared error  Entropy
 Likelihood  K-L divergence etc.
 Posterior probability

14. ML in Practice

15. Lessons Learned about Learning

 Learning can be viewed as using direct or indirect experience to approximate a chosen target function.
 Function approximation can be viewed as a search through a space of hypotheses (representations of functions) for
one that best fits a set of training data.
 Different learning methods assume different hypothesis spaces (representation languages) and/or employ different
search techniques.

Page - 4 of 4