Chapter Five

What is Learning?

• In the psychology literature, learning is considered one of the keys to human intelligence.
• what learning is? Learning is:
– Memorizing something
– Knowing facts through observation and exploration
– Improving motor and/or cognitive skills through practice
• The idea behind learning is that percepts should not only be used for acting now, but also
for improving the agent’s ability to act in the future.
– Learning is essential for unknown environments, i.e. when designer lacks omniscience. It
enables to organize new knowledge into general, effective representations
– Learning modifies the agent's decision making mechanisms to improve performance
2
 Cont…
• Learning can be described as normally a relatively permanent change that occurs in behavior as a result of
experience.
• Learning occurs in various regimes, for example: -
• It is possible to learn to open a lock as a result of trial and error.
• It is possible to learn how to use a word processor as a result of following particular instructions.
• There’s no decisive definition of learning, but here are some that do justice: -
• “Learning is any process by which a system improves performance from experience.” Herbert Simon
• “Learning is constructing or modifying representations of what is being experienced.” Ryszard Michalski
• “Learning is making useful changes in our minds.” Marvin Minsky
• The idea behind learning is that percepts should be used not only for acting, but also for improving the
agent's ability to act in the future.
• Learning takes place as the agent observes its interactions with the world and its own decision-making
processes.
3
 Learning
• An agent is learning if it improves its performance on future tasks
after making observations about the world.
• 3 main reasons why would we want an agent to learn:
1. The designers cannot anticipate all possible situations that the
agent might find itself in.
2. The designers cannot anticipate all changes over time
3. Sometimes human programmers have no idea how to program
a solution themselves.
• Therefore, Instead of trying to hard code all the knowledge, it
makes sense to learn it. 4
 Forms of learning
• Any component of an agent can be improved by learning from
data. The improvements, and the techniques used to make them,
depend on four major factors:
– Which component is to be improved.
– What prior knowledge the agent already has.
– What representation is used for the data and the component.
– What feedback is available to learn from.

5
 The Basic Learning Model

• A computer program is said to learn from experience E with respect to some class of tasks
T and performance measure P,
– if its performance at tasks T, as measured by P, improves with experience E.
• Learning agents consist of four main components:
– learning element -- the part of the agent responsible for improving its performance
– performance element -- the part that chooses the actions to take
– critic – provides feedback for the learning element how the agent is doing with respect
to a performance standard
– problem generator -- suggests actions that could lead to new, informative experiences
(suboptimal from the point of view of the performance element, but designed to
improve that element)
6
Learning Agents

7
8
9
 Agent component to be improved.
– A direct mapping from conditions on the current state to actions
– A means to infer relevant properties of the world from the percept
sequence
– Information about the way the world evolves and about the results of
possible actions the agent can take
– Utility information indicating the desirably of world states
– Action-value information indicating the desirably of actions
– Goals that describe classes of states whose achievement maximizes the
agent’s utility

10
 Example:
• An agent training to be come a taxi driver.
– Every time the instructor shouts “Brake!” the agent might learn a
condition– action rule for when to brake (component 1); the agent
also learns every time the instructor does not shout.
– By seeing many camera images that it is told contain buses, it can
learn to recognize them (2).
– By trying actions and observing the results—for example, braking
hard on a wet road—it can learn the effects of its actions (3).
– Then, when it receives no tip from passengers who have been
thoroughly shaken up during the trip, it can learn a useful11
 Learning Techniques
Rote Learning
• In this kind of learning there is no prior knowledge.
• When a computer stores a piece of data, it is performing an elementary form of learning.
• Examples of correct behavior are stored and when a new situation arises it is matched with the learnt
examples.
• The values are stored so that they are not re-computed later. One of the earliest game-playing programs
is Checkers Program (Samuel, 1963). This program learned to play checkers well enough to beat its
creator/designer.
Deductive Learning
• Deductive learning works on existing facts and knowledge and deduces new knowledge from the old.
• This is best illustrated in the following example: -
• Example: Assume, A = B and B = C, then we can deduce with much confidence that C = A.

12
• Deductive learning does not generate “new” knowledge at all, it simply memorizes the logical
consequences of what is known already.
• This implies that virtually all mathematical research would not be classified as learning “new”
things.
• However, regardless of whether this is termed as new knowledge or not, it certainly makes the
reasoning system more efficient.
Inductive Learning (Learning from Observations)
• Inductive learning takes examples and generalizes rather than starting with existing knowledge.
• For example, having seen many cats, all of which have tails, one might conclude that all cats have
tails.
• There is scope of error in inductive reasoning, but still it is a useful technique that has been used as
the basis of several successful systems.

13
14
Cont…
• Inductive learning: system tries to induce a “general
rule” from a set of observed instances.
• Supervised learning: learning algorithm is given the
correct value of the function for particular inputs,
and changes its representation of the function to try
to match the information provided by the feedback.

15
 Learning: Symbol-Based
• Our world is a world of symbols.
• We use symbolic interpretations to understand the world around us.
• For instance, if we saw a ship and were to tell a friend about its size, we will not say that we saw a
254 meters long ship, instead we’d say that we saw a “huge” ship about the size of “Our Building”.
• And our friend would understand the relationship between the size of the ship and its hugeness with
the analogies of the symbolic information associated with the two words used: “huge” and “Our
Building”.
• Similarly, the techniques we are to learn now use symbols to represent knowledge and information.
Let us consider a small example to help us see where we’re headed.
• Example: What if we were to learn the concept of a GOOD STUDENT.
• We would need to define, first of all some attributes of a student, on the basis of which we could tell
apart the good student from the average.

16
• Then we would require some examples of good students and average students.
• To keep the problem simple, we can label all the students who are “not good” (Average, Below Average,
Satisfactory, Bad) as NOT GOOD STUDENT.
• Let’s say we choose two attributes to define a student, Grade and Class Participation. Both attributes can
have either of the two values, High or Low.
• Now from the above Student concept, the learner program might look like: -
• Student (GOOD STUDENT): Grade (High)  Class Participation (High)
• Student (GOOD STUDENT): Grade (High)  Class Participation (Low)
• Student (NOT GOOD STUDENT): Grade (Low)  Class Participation (High)
• Student (NOT GOOD STUDENT): Grade (Low)  Class Participation (Low)

17
• As you can see the system is composed of symbolic information, based on
which the learner can even generalize that a student is a GOOD STUDENT if
his/her grade is High, even if the class participation is Low.
• Student (GOOD STUDENT): Grade (High)  Class Participation (?)
• This is the final rule that the learner has learnt from the enumerated
examples.
• Here the “?” means that the attribute class participation can have any value,
as long as the grade is High.

18
 Machine Learning(ML)
• The goal of machine learning is to build computer systems that can learn from their experience and adapt to
their environments.
• Obviously, learning is an important aspect or component of intelligence.
Why Machine Learning?
• One response to the idea of AI is to say that computers cannot think because they only do what their
programmers tell them to do.
• However, it is not always easy to tell what a particular program will do, but given the same inputs and
conditions it will always produce the same outputs.
• If the program gets something right once, it will always get it right and if it makes a mistake once it will always
make the same mistake every time it runs.
• Also, humans are able to notice similarities between things and therefore can generate new ideas about the
world we live in.
• So, machine learning is a prerequisite for any mature program of artificial intelligence.

19
 Three Phases of Machine Learning
 Machine Learning follows three phases, these are: -
Phase I: Training
• A training set of examples of correct behavior is analyzed and some representation of the newly learnt
knowledge is stored.
• This is often some form of rules.
Phase II: Validation
• The rules are checked and if necessary, additional training is given. Sometimes additional test data are
used.
• But instead of using a human to validate the rules, some other automatic knowledge-based component
may be used.
• The role of tester is often called the critic.
Phase III: Application
• The rules are used in responding to some new situations.
20
AI vs ML vs DL
• AI- a technique which enables a machine to mimic
human behavior.
• ML-A subset of AI technique which use statistical
methods to enable machine to improve with experience.
• Deep learning (DL) is a particular kind of machine
learning that is inspired by the functionality of our brain
cells called neurons which led to the concept of artificial
neural network.
21
 Learning from Observations
• Supervised Learning – learn a function from a set of training
examples which are pre-classified feature vectors.
– Data – instantiations of some or all of the random variables describing the domain; they are evidence
– Hypotheses – probabilistic theories of how the domain works

feature vector class

(shape,color) (circle, green) ?
(square, red) I
(square, blue) I
(triangle, blue) ?
(circle, red) II
(circle blue) II
(triangle, red) I Given a previously unseen feature vector, what is
(triangle, green) I
(ellipse, blue) II
the rule that tells us if it is in class I or class II?
(ellipse, red) II
22
 Learning from Observations
• Unsupervised Learning – No classes are given. The idea is to find
patterns in the data. This generally involves clustering.

• Reinforcement Learning – learn from feedback after a decision is made.

– the agent must learn from reinforcement (reward or punishment)
23
 Learning Probabilistic Models
• Agents can handle uncertainty by using the methods of
probability and decision theory, but first they must learn their
probabilistic theories of the world from experience.
• Probabilistic models are statistical models

24
 Bayesian Learning
• Bayesian learning simply calculates the probability of the
hypothesis and it makes predictions on that basis.
• That is, the predictions are made by using all the hypotheses
• The probability of each hypothesis is obtained by Bayes’ rule.

25
 Bayes’ Rule
• This simple equation underlies most modern AI systems for probabilistic inference.

P(X | h) P(h)
P(h | X) = -----------------
P(X) Often assumed
constant and
left out.
• h is the hypothesis (such as the class).
• X is the feature vector to be classified.
• P(X | h) is the prior probability that this feature vector occurs, given that h is true.
• P(h) is the prior probability of hypothesis h.
• P(X) = the prior probability of the feature vector X. 26
27
Bayes’ Rule for Machine Learning

• Allows us to reason from evidence to

hypotheses
• Another way of thinking about Bayes’
rule:

28
Basics of Bayesian Learning

• Goal: find the best hypothesis from some space of hypotheses, given the observed data
(evidence) .

• Define best to be: most probable hypothesis in

• In order to do that, we need to assume a probability distribution over the class .

• In addition, we need to know something about the relation between the data observed and the
hypotheses (E.g., a coin problem.)

– As we will see, we will be Bayesian about other things, e.g., the parameters of the model

29
Basics of Bayesian Learning

• - the prior probability of a hypothesis

Reflects background knowledge; before data is observed. If no information -
uniform distribution.

• - The probability that this sample of the Data is observed. (No knowledge of the
hypothesis)

• The probability of observing the sample , given that hypothesis is the target

• The posterior probability of . The probability that is the target, given that has
been observed.
30
Bayes Theorem

• increases with and with

• decreases with

31
 Learning Scenario

• The learner considers a set of candidate hypotheses (models), and

attempts to find the most probable one , given the observed data.

• Such maximally probable hypothesis is called maximum a posteriori

hypothesis (MAP); Bayes theorem is used to compute it:

32
Learning Scenario (2)

• We may assume that a priori, hypotheses are equally

probable:

• We get the Maximum Likelihood hypothesis:

• Here we just look for the hypothesis that best explains the data
33
34
35
Choosing Hypothesis

36
37
• Compare posterior probability, probability of no-cancer with positive report is large compared to probability
of cancer with positive report, so hMAP=lcancer. i.e. the new patient with positive result is not having cancer.

38
• Compare posterior probability, probability of no-cancer with negative
report is large compared to probability of cancer with negative report, so
hMAP=lcancer. i.e. the new patient with positive result is not having
cancer.
39
Neural Net Learning
• Motivated by studies of the brain.

• A network of “artificial neurons” that learns a function.

• Doesn’t have clear decision rules like decision trees, but highly
successful in many different applications. (e.g. face detection)
• Knowledge is represented in numeric form

40
Biological Neuron

• Dendrites brings the input signals from other neurons

• Cell body gets the input signals from all dendrites and aggregates them. It then decides
whether to send output signal through Axon or not
• Axon carries the impulse generated by cell to other neurons
• Axon is connected to dendrites of other neurons through synapse
McCulloch-pitts Model of Neuron

• NET = X1W1+X2W2+....+XnWn
• f (NET)= Out

For simple threshold function

f (NET)= Out = 1 if NET >=T

=0 if NET <
T
 Activation functions
• Activation functions are mathematical equations that determine the output of a neural
network.
 Architectures of NN
What do we mean by architecture of NN?
• Way in which neurons are connected to together
Feed Forward NN Recurrent NN Symmetrically
connected NN
Feed-forward example
Perceptron
• The perceptron(or single-layer perceptron) is the simplest model of a
neuron that illustrates how a neural network works.
• The perceptron is a machine learning algorithm developed in 1957 by
Frank Rosenblatt and first implemented in IBM 704.

46
How the Perceptron Works
• Example:
– The perceptron has three inputs x1, x2 and x3 and one output.

• Since the output of the perceptron could be either 0 or 1, this perceptron is an example
of binary classifier.
The Formula
Let’s write out the formula that joins the inputs and the weights together to produce the output
Output = w1x1 + w2x2 + w3x3

47
EnD

48