IA03

Path finding algorithms Optimisation-based algorithms Machine-learning based algorithms Rule-based algorithms
1 Path finding algorithms
2 Optimisation-based algorithms
3 Machine-learning based algorithms
4 Rule-based algorithms
L3 - Computer science University of Mohamed Khider Biskra

Chapter 2: Techniques of AI 1 / 35


Introduction
Types of machine learning
Classification: Decision tree
Regression: Linear regression
Neural Networks


Introduction
Neural Networks

Introduction
• Machine learning is a type of artificial intelligence that

involves training computer algorithms to make predictions
or decisions based on patterns found in data.
• It is about teaching computers to learn from examples,
rather than being explicitly programmed.

Apple or an orange?
• Let’s say we want to build a program that can automatically

recognize whether a fruit is an apple or an orange based on
some caracteristics (color, shape, weight, etc.).
Figure 1: Classification of fruits.

Apple or an orange: classical programming
• One way to do this is to write explicit rules for how to

identify each type of fruit.
• For example, we could write code that looks for certain colors
or shapes that are specific to apples or oranges.
• This approach can be very difficult and time consuming, and
it may not work well if the fruits are not perfectly shaped or
colored (different shapes and weights for each fruit!).
• We could use a machine learning algorithm to learn from
examples of apples and oranges and then use that
knowledge to recognize new fruits.

Apple or an orange: Machine learning
1 We would first gather a large dataset of images of apples and

oranges.
2 Use this dataset to train a machine learning algorithm:
recognize patterns in the that are specific to each type of fruit.
3 Test the trained model on new images of fruits that it has
never seen before: use the patterns learned during training to
make a prediction about whether the fruit is an apple or an
orange.

Multiplication tables
• Let’s say you want to build a model that can predict the
result of any multiplication table.
Nb1 nb2 Result
2 2 4
2 3 6
2 4 8
2 5 10
2 7 14
2 8 16
3 1 3
3 3 9
3 4 12
3 6 18
3 7 21
3 9 27
..... .... ...
3 5 ?
Table 1: Predicting multiplication results.

Multiplication tables
1 Gather a large dataset of multiplication table and their

corresponding answers: with different numbers and their
corresponding answers.
2 Select a machine learning method for training data. It would
learn to recognize patterns in the data and learn how to
make predictions based on those patterns.
3 Predict the answer for any new multiplication: input the
two numbers and the algorithm would use the patterns it
learned from the training data to make a prediction of the
expected answer.

Multiplication
Figure 2: Prediction using machine learning.


Introduction
Neural Networks

Based on the nature of data and working mechanism
• There are three main types of machine learning: supervised

learning, unsupervised learning, and reinforcement learning.
• Supervised learning involves training a model on labeled
data, where the desired outcome is already known.
• Unsupervised learning involves finding patterns in unlabeled
data, where the desired outcome is not known.
• Reinforcement learning involves training a model to make
decisions based on feedback received from the environment.

Based on the desired outcomes
• Machine learning algorithms can be classified as regression or

classification algorithms.
• Regression algorithms are used to predict a continuous value,
such as the price of a house.
• Classification algorithms are used to predict a categorical
value, such as whether an email is spam or not.


Introduction
Neural Networks

Decision Tree
• Determine whether it is possible to play golf or not given

the weather conditions? (Figure 2)
Table 2: Weather dataset.

Outlook Temperature Humidity Windy Play
sunny hot high FALSE no
sunny hot high TRUE no
overcast hot high FALSE yes
rainy mild high FALSE yes
rainy cool normal FALSE yes
rainy cool normal TRUE no
overcast cool normal TRUE yes
sunny mild high FALSE no
sunny cool normal FALSE yes
rainy mild normal FALSE yes
sunny mild normal TRUE yes
overcast mild high TRUE yes
overcast hot normal FALSE yes
rainy mild high TRUE no

Decision Tree
• Given several characteristics, the decision starts with one of
these characteristics and moves to another, and so on until
arriving at the final decision (the label of the corresponding
class).
Figure 3: Decision Tree

Decision tree
• This sequence is represented in the form of a tree, where the

nodes are the characteristics, and the branches are the
different values of each characteristic.
• Each leaf represents the decision of belonging to a class of
data that satisfies all the tests along the path from the root to
that leaf, while traversing all the nodes along the way.

Algorithm ID3
• This algorithm uses the entropy function and information

gain to decide which is the best feature to choose as the new
node in the tree.
X
H(S) = − P(Ci )log2 (P(Ci )) (1)
ci∈C
• The gain is given by equation 2.
X
IG (S, fj ) = H(S) − P(fj v )H(fj v ) (2)
fj v ∈fj

Decision tree
• We calculate the probability of each class (Yes/No):

• P(Play=Yes) = 9/14
• P(Play=No) = 5/14
• We calculate the entropy of the data:
• H(S) = - P(Play=yes) * log2(P(Play=Yes)) - P(Play=no) *
log2(P(Play=no))
• H(S) = - 9/14 * log2(9/14) - 5/14 * log2(9/14)
• H(S) = 0.41 + 0.53 = 0.94

Decision Tree
• We calculate the information gain for each attribut:
Table 3: Frequences of the attribut outlook.

Outlook play (yes) play (No)
Sunny 2 3
Overcast 4 0
rainny 3 2

Attribut Outlook
• We calculate the probability of each set:

• P(S Sunny) = 5/14
• P(S Overcast) = 4/14
• P(S Rainny) = 5/14
• We calculate the entropy according to each value of the
attribut:
• H(S Sunny) = - 2/5 * log2(2/5) - 3/5 * log2(3/5) = 0.971
• P(S Overcast) = - 4/4 * log2(4/4) - 0/4 * log2(0/4) = 0
• P(S Rainny) = - 3/5 * log2(3/5) - 2/5 * log2(2/5) = 0.971

Attribut Outlook
• The information gain of the attribut ”Outlook” is given by:

• IG(S, Outlook) = H(S) - P(S= Sunny) * H(S Sunny) -
P(S Overcast) * H(S Overcast) - P(S Rainny) * H(S Rainny)
• IG(S, Outlook) = 0.94 - 5/14 * 0.971 - 4/14 * 0 - 5/14 *
0.971
• IG(S, Outlook) = 0.247

Information Gain: all attributs
• We calculate the information gain for all other attributs (see

Table 4).
Table 4: Les valeurs de gain pour chaque attribut.

outlook temperature humidity windy
IG 0.247 0.029 0.152 0.048
• The best attribut is ”Outlook” because its information gain

is the greater.
• Since the entropy of the Outlook attribute for the value
”Overcast” is 0, this set contains samples of the same class
(Yes). Therefore, this set forms a leaf.

The final decision tree
• The final decision tree is given by:
Figure 4: Final decision tree


Introduction
Neural Networks

Linear regression
• Suppose we have a dataset of the height and weight of 10

individuals:
Table 5: Height and weight dataset.

Height (in) Weight (lbs)
63 127
65 140
68 157
70 165
71 170
72 175
73 180
75 190
77 200
80 220

Linear regression
Figure 5: Graphic representation of dataset.

Linear regression
• Find the line Weight = β0 + β1 ∗ Height. β0 , β1 are unkown?
Figure 6: Graphic representation of dataset.

Finding Bi
• The equation of the line is given by:

ŷ = β1 ∗ x + β0 (3)
• To find the values of β0 and β1 , we have to find minimise the

error function E = ni=1 (Yi − Ŷi )2 .
P
• Ŷi is the predicted value.

• Yi is the desired value.
Xn n
X
arg min (Yi − Ŷi )2 = arg min (Yi − β1 ∗ x − β0 )2 (4)
β0 ,β1 β0 ,β1
i=1 i=1

Finding the regression line
• The values of β0 and β1 are given by:

Pn
xi yi − ȳ xi
β1 = Pi=1n 2
(5)
i=1 xi − xi x̄
β0 = ȳ − β1 x̄ (6)

Fit line
• Weight = −208.24 + 5.38 ∗ Height
Figure 7: Fit line.

Make prediction
• For a given new height, we can find the weight using the
previous equation.
• For example, if height = 66 than the weight is
Weight = −208.24 + 5.38 ∗ 66 = 146.84

Chapter 2: Techniques of AI
Figure 8: Making prediction for a new example. 33 / 35

Introduction
Neural Networks



IA03

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

IA03

Uploaded by

Copyright:

Available Formats

Path finding algorithms Optimisation-based algorithms Machine-learning based algorithms Rule-based algorithms

1 Path finding algorithms

3 Machine-learning based algorithms

L3 - Computer science University of Mohamed Khider Biskra

1 Path finding algorithms

3 Machine-learning based algorithms

L3 - Computer science University of Mohamed Khider Biskra

1 Path finding algorithms

3 Machine-learning based algorithms

L3 - Computer science University of Mohamed Khider Biskra

1 Path finding algorithms

3 Machine-learning based algorithms

L3 - Computer science University of Mohamed Khider Biskra

• Machine learning is a type of artificial intelligence that

L3 - Computer science University of Mohamed Khider Biskra

• Let’s say we want to build a program that can automatically

Figure 1: Classification of fruits.

L3 - Computer science University of Mohamed Khider Biskra

Apple or an orange: classical programming

• One way to do this is to write explicit rules for how to

L3 - Computer science University of Mohamed Khider Biskra

Apple or an orange: Machine learning

1 We would first gather a large dataset of images of apples and

L3 - Computer science University of Mohamed Khider Biskra

Table 1: Predicting multiplication results.

1 Gather a large dataset of multiplication table and their

L3 - Computer science University of Mohamed Khider Biskra

Figure 2: Prediction using machine learning.

L3 - Computer science University of Mohamed Khider Biskra

1 Path finding algorithms

3 Machine-learning based algorithms

L3 - Computer science University of Mohamed Khider Biskra

Based on the nature of data and working mechanism

• There are three main types of machine learning: supervised

L3 - Computer science University of Mohamed Khider Biskra

Based on the desired outcomes

• Machine learning algorithms can be classified as regression or

L3 - Computer science University of Mohamed Khider Biskra

1 Path finding algorithms

3 Machine-learning based algorithms

L3 - Computer science University of Mohamed Khider Biskra

• Determine whether it is possible to play golf or not given

Table 2: Weather dataset.

L3 - Computer science University of Mohamed Khider Biskra

Figure 3: Decision Tree

• This sequence is represented in the form of a tree, where the

L3 - Computer science University of Mohamed Khider Biskra

• This algorithm uses the entropy function and information

• The gain is given by equation 2.

L3 - Computer science University of Mohamed Khider Biskra

• We calculate the probability of each class (Yes/No):

L3 - Computer science University of Mohamed Khider Biskra

• We calculate the information gain for each attribut:

Table 3: Frequences of the attribut outlook.

L3 - Computer science University of Mohamed Khider Biskra

• We calculate the probability of each set:

L3 - Computer science University of Mohamed Khider Biskra

• The information gain of the attribut ”Outlook” is given by:

L3 - Computer science University of Mohamed Khider Biskra

Information Gain: all attributs

• We calculate the information gain for all other attributs (see

Table 4: Les valeurs de gain pour chaque attribut.

• The best attribut is ”Outlook” because its information gain