3/9/2022

What is AI Technique?

In the real world, the knowledge has some unwelcomed properties −

• Its volume is huge, next to unimaginable.
• It is not well-organized or well-formatted.
• It keeps changing constantly.

AI Technique is a manner to organize and use the knowledge

efficiently in such a way that −
• It should be perceivable by the people who provide it.
• It should be easily modifiable to correct errors.
• It should be useful in many situations though it is incomplete or
inaccurate.

AI techniques elevate the speed of execution of the complex program

it is equipped with.

68

68

AI Techniques
• The techniques are concerned with how we represent, manipulate and
reason with knowledge in order to solve problems.
Example
• Techniques, not all "intelligent" but used to behave as intelligent
- Describe and match, Constraint satisfaction
- Tree Searching
-Rule based systems
• Biology-inspired AI techniques are currently popular
-Neural Networks- Genetic Algorithms
- Reinforcement learning

69

69

1
 3/9/2022

Describe and match

▪ Model is a description of a system’s behavior.
▪ Finite state model :consists of a set of states, a set of input
events and the relations between them. Given a current state
and an input event you can determine the next current state of
the model.
▪ Computation model is a finite state machine. It includes of a
set of states,a set of start states,an input
alphabet,and a transitionfunction which
maps input symbols and current states to a next state.
▪ Representation of computational systeminclude start and
end state descriptions and a set of possible transition rules
that might be applied. Problem is to find the appropriate
transition rules.

70

70

Describe and match

▪ Transition relation:If a pair of states (S, S') is such that one
move takes the system from S to S‘, then the transition relation is
represented by S => S’.
• State-transition systemis called deterministic if every state has
at most one successor; it is called non-deterministic if at least
one state has more than one successor.
• Examples of some possible transitions between states are
shown for theTowers of Hanoi puzzle.

71

71

2
 3/9/2022

Puzzle : Towers of Hanoi with only 2 disks

Towers of Hanoi with ndiscs can be solved in 2 n-1 steps.

72

72

Puzzle : Towers of Hanoi with only 2 disks

73
73

3
 3/9/2022

If we have 2 disks:
•First we move the smaller one (top) disk to aux peg
•Then we move the larger one (bottom) disk to
destination peg
•And finally, we move the smaller one from aux to
destination peg.

74
74

Towers of Hanoi with more than 2 discs

We divide the stack of disks in two parts. The largest disk
(nth disk) is in one part and all other (n-1) disks are in
second part.
Our ultimate aim is to move disk n from source to
destination and then put all other (n-1) disks onto it. Now
we can imagine to apply the same in recursive way for all
given set of disks.
So steps to follow are:
Step 1−Move n-1 disks from source to aux
Step 2−Move nth disk from source to dest
Step 3−Move n-1 disks from aux to dest
Exercise: Draw the possible state transitions in the towers of Hanoi
with 3 discs.
75
75

4
 3/9/2022

Try this at: http://www.mathsisfun.com/games/towerofhanoi.html

76

77

5
 3/9/2022

78

Constraint Satisfaction Techniques

▪ A factored representation for each state: a set of variables, each of which
has a value. A problem is solved when each variable has a value that
satisfies all the constrains on the variable.

▪ Constraint is a logical relation among variables. e.g. “circle is inside the

square” –The constraints relate objects without precisely specifying their
positions; moving any one, the relation is still maintained.

▪ Constraint satisfaction is a process of finding a solution to a set

of constraints – the constraints express allowed values for variables and
finding solution is evaluation of these variables that satisfies all
constraints.

79

79

6
 3/9/2022

▪ A constrain satisfaction problem consist of three components:

-a finite set of variables,X= {X 1 , …,Xn}.
-a set of domains,D= {D 1, …,Dn}, one for each variable.
-Cis a finite set of constraintsC 1 , C2, …, Cn. Each constraint Ci
limits the values that variables can take, e.g.,X 1 ≠X 2
▪A state is an assignment of values to some or all variables.
▪ Consistent assignment: assignment does not violate the
constraints.
▪ An assignment is complete when every variable is
mentioned.
▪A solution to a CSP is a complete assignment that satisfies
all constraints.
80

80

Example: Map-Coloring

• Variables: WA, NT, Q, NSW, V, SA, T

• Domains: Di = {red,green,blue}
• Constraints : adjacent regions must have different colors
–e.g., WA ≠ NT
• So (WA,NT) must be in {(red,green),(red,blue),(green,red), …}

81

7
 3/9/2022

Solutions are complete and consistent assignments,

–e.g., WA = red, NT = green, Q = red, NSW = green,
V = red, SA = blue, T = green

82

Rule-Based Systems (RBSs)

• A production rule consists of two parts: condition
(antecedent) part and conclusion (action, consequent)
part,
• i.e: IF <conditions> THEN <actions>
• When all conditions of a rule are satisfied the rule is
triggered.
• Example
IF Gauge is OKAND [TEMPERATURE] > 120
THEN Cooling system is in the state of
overheating

83

83

8
 3/9/2022

•A RBS consists of:

–Working memory (facts memory)
–Production rules memory (Rule base)
–Inference engine (Interpreter), it cycles through
three steps:
• match facts against rules
• select a rule
• execute the rule

84

84

85

9
 3/9/2022

Working Memory

• Contains facts about the world observed or derived from

a rule;
stored as a triplet < object, attribute, values >
• e.g. < car, color, red > : “The color of my car is red”.
• Contains temporary knowledge about problem-
solving session.
• Can be modified by the rules.

86

86

Rule Base
• RB contains rules; each rule is a step in a problem solving.
• Rules are domain knowledge and modified only from outside.
• Rule syntax is IF <condition> THEN <action>
e.g. IF <(temperature, over, 20> THEN <add (ocean, swimable, yes)>
• If the conditions are matched to the working memory and if fulfilled
then rule may be fired.
• RB actions are :
- “Add” fact(s) to WM;
-“Remove” fact(s) from WM;
-“Modify” fact(s) in WM;

87

87

10
 3/9/2022

Interpreter
▪ It is the domain independent reasoning mechanism for RBS.
▪ It selects rule from Rule Base and applies by performing action.
▪ It operates on a cycle:
Retrieval - Finds the rules that matches the current
WM;
Refinement - Prunes, reorders and resolves conflicts;
Execution - Executes the actions of the rules in the
Conflict Set, then applies the rule by performing action.

88

88

Production System Model

89

11
 3/9/2022

Popular Search Algorithms

Searching is the universal technique of problem solving in AI. There are

some single-player games such as tile games, Sudoku, crossword, etc. The
search algorithms help you to search for a particular position in such games.

Single Agent Path finding Problems

The games such as 3X3 eight-tile, 4X4 fifteen-tile, and 5X5 twenty four tile puzzles
are single-agent-path-finding challenges. They consist of a matrix of tiles with a
blank tile. The player is required to arrange the tiles by sliding a tile either
vertically or horizontally into a blank space with the aim of accomplishing some
objective.
The other examples of single agent pathfinding problems are Travelling Salesman
Problem, Rubik’s Cube, and Theorem Proving.

90

90

Search Terminology

• Problem Space−It is the environment in which the search takes place.

(A set of states and set of operators to change those states)
• Problem Instance−It is Initial state + Goal state.
• Problem Space Graph−It represents problem state. States are shown
by nodes and operators are shown by edges.
• Depth of a problem−Length of a shortest path or shortest sequence of
operators from Initial State to goal state.
• Space Complexity−The maximum number of nodes that are stored in
memory.
• Time Complexity−The maximum number of nodes that are created.
• Admissibility‫ا لمقبوليه‬−A property of an algorithm to always find an
optimal solution.
• Branching Factor−The average number of child nodes in the problem
space graph.
• Depth−Length of the shortest path from initial state to goal state.

91

91

12
 3/9/2022

Tree Searching
▪ Many problems (e.g. goal reduction, constraint networks) can be described in the form of a
search tree. A solution to the problem is obtained by finding a path through this tree.
▪ A search through the entire tree, until a satisfactory path is found, is called exhaustive
search‫ش امله‬.
▪ They are most simple, as they do not need any domain-specific knowledge. They work fine
with small number of possible states.
▪ Requirements−
▪ State description
▪ A set of valid operators
▪ Initial state
▪ Goal state description
• Exhaustive search (or uninformed search or Brute-Force Search Strategies)
–Breadth-first search: Use a queue to implement
• Search nodes each level at a time
–Depth-first search: Use a stack to implement
• Search nodes path at a time

92

92

• A tree search starts at the root

and explores nodes from there,
A
looking for a goal node (a node
that satisfies certain conditions,
depending on the problem)
B C
• For some problems, any goal node
is acceptable (N or J); for other
D E F G
problems, you want a minimum-
depth goal node, that is, a goal
H I J K node nearest the root (only J)

L M N O P Q
Goal nodes
93

93

13
 3/9/2022

Depth-first searching
• A depth-first search (DFS)
explores a path all the way to a
A leaf before backtracking and
exploring another path
• For example, after searching
B C A, then B, then D, the
search backtracks and tries
D E F G another path from B
• Node are explored in the order
A BDEHLMNIOP CF
H I J K
GJKQ
• N will be found before J
L M N O P Q

94

Breadth-first searching

• A breadth-first search (BFS)

explores nodes nearest the
A root before exploring nodes
further away
B C • For example, after searching A,
then B, then C, the search
D E F G proceeds with D, E, F, G
• Node are explored in
the order A B C D E F G H I
H I J K
JKLMNOPQ
• J will be found before N
L M N O P Q

95

14
 3/9/2022

Depth- vs. breadth-first searching

• When a breadth-first search succeeds, it finds

a minimum-depth (nearest the root) goal node
• When a depth-first search succeeds, the found
goal node is not necessarily minimum depth
• For a large tree, breadth-first search
memory requirements may be excessive
• For a large tree, a depth-first search may take an
excessively long time to find even a very nearby goal
node

96

Backtracking
▪ Backtracking is a technique used to solve problems
with a large search space, by systematically trying
and eliminating possibilities.
▪ similar to the brute force approach where it tries all
of the solutions but the only difference is that it
eliminates/avoids the partial candidate solutions as
soon as it finds that that path cannot lead to a
solution.
▪ A standard example of backtracking would be going
through a maze. At some point in a maze, you might
have two options of which direction to go.

97

97

15
 3/9/2022

▪ One strategy would be to try

going through Portion A of the
maze.
Portion B
● If you get stuck before you
find your way out, then you

Portion A
"backtrack"to the junction.

▪ At this point in time you know

that Portion A will NOTlead you
out of the maze,
● so you then start searching in
Portion B

98

98

• Clearly, at a single junction you

could have even more than 2
choices.

• The backtracking strategy says

to try each choice, one after the
other,
–if you ever get stuck, C
B
"backtrack"to the junction and
try the next choice. A

• If you try all choices and never

found a way out, then there IS
no solution to the maze. 99

99

16
 3/9/2022

In the above mentioned tree, what path leads to the

word “BACK” (first 4 letters of backtracking)?
100

100

101

101

17
 3/9/2022

Following are the observations based on the above

steps,

▪ Every time when we hitNO, we went back to parent

of that node and checked other child nodes/paths –
this is back tracking
▪ Avoided all of the sub paths for the node whenever
we hit non match (NO). This enabled us to avoid
unnecessary paths exploration(2nd node C – at level 2
and others)

102

102

Following is the generalized back tracking

algorithm in pseudo code,

103

103

18
 3/9/2022

Example: N-Queens puzzle

104

104

Example: 4-queens problem

For this, we are given 4 x 4 chessboard and we need to
place 4 queens in non-attacking places on this board.
Following are the only two configurations which exist for
this 4 queen’s problem.

http://www.hbmeyer.de/backtrack/achtdamen/nQueens.htm
https://sadakurapati.wordpress.com/category/game/

105

19
 3/9/2022

Solution
▪ In chess, queens can move all
the way down any row, column
or diagonal (so long as
no pieces are in the way). 1 2 3 4
–Due to the first two 1
restrictions, it's clear that each 2
row and column of the board
will have exactly one queen. 3
4
• We can divide this problem
into a sub problem of placing
single queen on the
chess board.

106

106

▪ The backtracking strategy is as follows:

1) Place a queen on the first available square in row
1/column 1.

2) Move onto the next row/column, placing a queen on

the first available square there (that doesn't conflict
with the previously placed queens).
3) Continue in this fashion until either:
a) you have solved the problem, or
b) you get stuck.
–When you get stuck, remove the queens that got you
there, until you get to a row/column where there is
another valid square to try.

107

107

20
 3/9/2022

X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}

[4-Queens slides copied from B.J. Dorr]

108

X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}

109

21
 3/9/2022

X1 X2
1 2 3 {1,2,3,4} { , ,3,4}
4
1
2
3
4 X3 X4
{ ,2, ,4} { ,2,3, }

110

X1 X2
1 2 3 {1,2,3,4} { , ,3,4}
4
1
2
3
4 X3 X4
{ ,2, ,4} { ,2,3, }

111

22
 3/9/2022

X1 X2
1 2 3 {1,2,3,4} { , ,3,4}
4
1
2
3
4 X3 X4
{ , , , } { ,2, , }

112

X1 X2
1 2 3 {1,2,3,4} { , , ,4}
4
1
2
3
4 X3 X4
{ ,2, ,4} { ,2,3, }

113

23
 3/9/2022

X1 X2
1 2 3 {1,2,3,4} { , , ,4}
4
1
2
3
4 X3 X4
{ ,2, , } { , ,3, }

114

X1 X2
1 2 3 {1,2,3,4} { , , ,4}
4
1
2
3
4 X3 X4
{ ,2, , } { , ,3, }

115

24
 3/9/2022

X1 X2
1 2 3 {1,2,3,4} { , , ,4}
4
1
2
3
4 X3 X4
{ ,2, , } { , , , }

116

X1 X2
1 2 3 { ,2,3,4} {1,2,3,4}
4
1
2
3
4 X3 X4
{1,2,3,4} {1,2,3,4}

117

25
 3/9/2022

X1 X2
1 2 3 { ,2,3,4} { , , ,4}
4
1
2
3
4 X3 X4
{1, ,3, } {1, ,3,4}

118

X1 X2
1 2 3 { ,2,3,4} { , , ,4}
4
1
2
3
4 X3 X4
{1, ,3, } {1, ,3,4}

119

26
 3/9/2022

X1 X2
1 2 3 { ,2,3,4} { , , ,4}
4
1
2
3
4 X3 X4
{1, , , } {1, ,3, }

120

X1 X2
1 2 3 { ,2,3,4} { , , ,4}
4
1
2
3
4 X3 X4
{1, , , } {1, ,3, }

121

27
 3/9/2022

X1 X2
1 2 3 { ,2,3,4} { , , ,4}
4
1
2
3
4 X3 X4
{1, , , } { , ,3, }

122

X1 X2
1 2 3 { ,2,3,4} { , , ,4}
4
1
2
3
4 X3 X4
{1, , , } { , ,3, }

Exercise: Get the second solution.

Ans: [3, 1, 4, 2]

123

28
 3/9/2022

Informed (Heuristic) Search Strategies

To solve large problems with large number of possible states, problem-specific

knowledge needs to be added to increase the efficiency of search algorithms.

Heuristic Evaluation Functions

They calculate the cost of optimal path between two states. A heuristic
function for sliding-tiles games is computed by counting number of moves that
each tile makes from its goal state and adding these number of moves for all tiles.

Pure Heuristic Search

It expands nodes in the order of their heuristic values. It creates two lists, a closed
list for the already expanded nodes and an open list for the created but
unexpanded nodes.
In each iteration, a node with a minimum heuristic value is expanded, all its
child nodes are created and placed in the closed list. Then, the heuristic function
is applied to the child nodes and they are placed in the open list according to
their heuristic value. The shorter paths are saved and the longer ones are
disposed.

124

124

Informed (Heuristic) Search Strategies

A * Search
It is best-known form of Best First search. It avoids expanding paths that
are already expensive, but expands most promising paths first.
f(n) = g(n) + h(n), where
• g(n) the cost (so far) to reach the node
• h(n) estimated cost to get from the node to the goal
• f(n) estimated total cost of path through n to goal. It is implemented using priority
queue by increasing f(n).

Greedy Best First Search

It expands the node that is estimated to be closest to goal. It expands nodes
based on f(n) = h(n). It is implemented using priority queue.
Disadvantage−It can get stuck in loops. It is not optimal.

125

125

29
 3/9/2022

Local Search Algorithms

They start from a prospective solution and then move to a neighboring
solution. They can return a valid solution even if it is interrupted at any
time before they end.

Hill-Climbing Search
It is an iterative algorithm ‫ خوارزمية ت كرارية‬that starts with an arbitrary‫ت عسفي‬
solution to a problem and attempts to find a better solution by changing a
single element of the solution incrementally‫ت دريجيا‬. If the change produces
a better solution, an incremental change is taken as a new solution. This
process is repeated until there are no further improvements.

function Hill-Climbing (problem), returns a state that is a local maximum.

Disadvantage− This algorithm is neither complete, nor optimal.

inputs: problem, a problem
local variables: current, a node
neighbor, a node
current <-Make_Node(Initial-
State[problem]) loop
do neighbor <- a highest_valued successor
of current
if Value[neighbor] ≤ Value[current] then
return State[current]
current <- neighbor 126
end

126

Local Search Algorithms

Local Beam Search

In this algorithm, it holds k number of states at any given time.

At the start, these states are generated randomly. The
successors of these k states are computed with the help of
objective function. If any of these successors is the maximum
value of the objective function, then the algorithm stops.

Otherwise the (initial k states and k number of successors of the

states = 2k) states are placed in a pool. The pool is then sorted
numerically. The highest k states are selected as new initial
states. This process continues until a maximum value is reached.

function BeamSearch( problem, k), returns a solution state.

start with k randomly generated states
loop
generate all successors of all k states
if any of the states = solution, then return the state
else select the k best successors
end
12
7

127

30
 3/9/2022

Local Search Algorithms

Simulated Annealing‫محاكاة ا لتلدين‬

Annealing is the process of heating and cooling a metal to change its internal structure
for modifying its physical properties. When the metal cools, its new structure is seized,
and the metal retains its newly obtained properties. In simulated annealing process, the
temperature is kept variable.
We initially set the temperature high and then allow it to ‘cool' slowly as the algorithm
proceeds. When the temperature is high, the algorithm is allowed to accept worse
solutions with high frequency.

Start
• Initialize k = 0; L = integer number of variables;
• From i →j, search the performance difference Δ.
• If Δ <= 0 then accept else if exp(-Δ/T(k)) > random(0,1) then accept;
• Repeat steps 1 and 2 for L(k) steps.
• k = k + 1;

Repeat steps 1 through 4 till the criteria is met.

End

128

128

Local Search Algorithms

Travelling Salesman Problem

In this algorithm, the objective is to find a low-cost tour that starts from a
city, visits all cities en-route exactly once and ends at the same starting city.
Start
Find out all (n -1)! Possible solutions, where n is the total number of cities.
Determine the minimum cost by finding out the cost of each of these (n -1)! solutions.
Finally, keep the one with the minimum cost.
end

129

129

31
 3/9/2022

Fuzzy Logic Systems

• An important technology dealing with vague, imprecise and uncertain knowledge
and data.
• It is concerned with the use of fuzzy values that capture the meaning of
words, human reasoning and decision making.
• The values of the linguistics variable are words rather than
numbers. e.g.,
IF speed is high THEN stopping_distance is long
IF speed is slow THEN stopping_distance is short

CERTAINLY YES
POSSIBLY YES
CANNOT SAY
POSSIBLY NO
CERTAINLY NO

130

130

Fuzzy Logic Systems

Fuzzy Logic Systems Architecture
It has four main parts as shown −
• Fuzzification Module−It transforms the system inputs, which are crisp
numbers, into fuzzy sets. It splits the input signal into five steps such as−
• Knowledge Base−It stores IF-THEN rules provided by experts.
• Inference Engine−It simulates the human reasoning process by making
fuzzy inference on the inputs and IF-THEN rules.
• Defuzzification Module−It transforms the fuzzy set obtained by the
inference engine into a crisp value.

LPx is La rge Positive

MPx is M edium Positive

Sx is Sm all

MNx is M edium Negative

LNx is La rge Negative

131

131

32
 3/9/2022

Fuzzy Logic Systems

The membership functions work on fuzzy sets of variables.

Membership Function

Membership functions allow you to quantify linguistic term and represent a fuzzy
set graphically. A membership functionfor a fuzzy set Aon the universe of
discourse X is defined as μA:X →[0,1].

Here, each element of Xis mapped to a value between 0 and 1. It is called

membership valueor degree of membership. It quantifies the degree of
membership of the element in Xto the fuzzy set A.
• x axis represents the universe of discourse.
• y axis represents the degrees of membership in the [0, 1] interval.

There can be multiple membership functions applicable to fuzzify a numerical

value. Simple membership functions are used as use of complex functions does not
add more precision in the output.

132

132

Fuzzy Logic Systems

All membership functions for LP, MP, S, MN,and LNare shown as below
−

The triangular membership function shapes are most common among various
other membership function shapes such as trapezoidal, singleton, and Gaussian.
Here, the input to 5-level fuzzifier varies from -10 volts to +10 volts. Hence
the corresponding output also changes.
133

133

33
 3/9/2022

Example of a Fuzzy Logic System

Let us consider an air conditioning system

with 5-level fuzzy logic system. This system
adjusts the temperature of air conditioner by
comparing the room temperature and the
target temperature value.

Algorithm
• Define linguistic Variables and terms
(start)
• Construct membership functions for
them. (start)
• Construct knowledge base of rules
(start)
• Convert crisp data into fuzzy data sets
using membership functions.
(fuzzification)
• Evaluate rules in the rule base.
(Inference Engine)
• Combine results from each rule.
(Inference Engine)
• Convert output data into non-fuzzy
values. (defuzzification)

134

134

Example of a Fuzzy Logic System

Development
Step 1−Define linguistic variables and terms
Linguistic variables are input and output variables in the form of simple words
or sentences. For room temperature, cold, warm, hot, etc., are linguistic terms.
Temperature (t) = {very-cold, cold, warm, very-warm, hot}
Every member of this set is a linguistic term and it can cover some portion
of overall temperature values.

Step 2−Construct membership functions for them

The membership functions of temperature variable are as shown −

135

135

34
 3/9/2022

Example of a Fuzzy Logic System

Step3−Construct knowledge base rules
Create a matrix of room temperature values versus target temperature values that an
air conditioning system is expected to provide.

RoomTemp.
/Target Very_Cold Cold Warm Hot Very_Hot

Very_Cold No_Change Heat Heat Heat Heat

Cold Cool No_Change Heat Heat Heat

Warm Cool Cool No_Change Heat Heat

Hot Cool Cool Cool No_Change Heat

Very_Hot Cool Cool Cool Cool No_Change

Build a set of rules into the knowledge base in the form of IF-THEN-ELSE structures.

Sr. No. Condition Action

1 IF temperature=(Cold OR Very_Cold) AND target=Warm THEN Heat

2 IF temperature=(Hot OR Very_Hot) AND target=Warm THEN Cool

3 IF (temperature=Warm) AND (target=Warm) THEN 136 No_Change

136

Example of a Fuzzy Logic System

Step 4−Obtain fuzzy value

Fuzzy set operations perform evaluation of rules. The operations used for OR and
AND are Max and Min respectively. Combine all results of evaluation to form a final
result. This result is a fuzzy value.

Step 5−Perform defuzzification

Defuzzification is then performed according to membership function for output
variable.

137

137

35
 3/9/2022

Fuzzy Logic Systems

Application Areas of Fuzzy Logic

The key application areas of fuzzy logic are as given −
Automotive Systems
• Automatic Gearboxes
• Four-Wheel Steering
• Vehicle environment control
Consumer Electronic Goods
• Hi-Fi Systems
• Photocopiers
• Still and Video Cameras
• Television
Domestic Goods
• Microwave Ovens
• Refrigerators
• Toasters
• Vacuum Cleaners
• Washing Machines
Environment Control
• Air Conditioners/Dryers/Heaters
• Humidifiers

138

138

Fuzzy Logic Systems

Advantages of FLSs
• Mathematical concepts within fuzzy reasoning are very simple.
• You can modify a FLS by just adding or deleting rules due to flexibility of
fuzzy logic.
• Fuzzy logic Systems can take imprecise, distorted, noisy input information.
• FLSs are easy to construct and understand.
• Fuzzy logic is a solution to complex problems in all fields of life, including
medicine, as it resembles human reasoning and decision making.

Disadvantages of FLSs
• There is no systematic approach to fuzzy system designing.
• They are understandable only when simple.
• They are suitable for the problems which do not need high accuracy.

139

139

36
 3/9/2022

Neural Networks

Yet another research area in AI, neural networks, is inspired from the natural
neural network of human nervous system.

What are Artificial Neural Networks (ANNs)?

The inventor of the first neurocomputer, Dr. Robert Hecht-Nielsen, defines a neural
network as −

“...a computing system made up of a number of simple, highly interconnected processing

elements, which process information by their dynamic state response to external inputs.”

140

140

Neural Networks
Basic Structure of ANNs
The idea of ANNs is based on the belief that working of human brain by making
the right connections, can be imitated using silicon and wires as living neuronsand
dendrites.

The human brain is composed of 86 billion nerve cells called neurons.They are
connected to other thousand cells by Axons.Stimuli from external environment or
inputs from sensory organs are accepted by dendrites. These inputs create electric
impulses, which quickly travel through the neural network. A neuron can then send
the message to other neuron to handle the issue or does not send it forward.

141

141

37
 3/9/2022

Neural Networks

ANNs are composed of multiple nodes, which imitate biological neuronsof human
brain. The neurons are connected by links and they interact with each other. The
nodes can take input data and perform simple operations on the data. The result
of these operations is passed to other neurons. The output at each node is called
its activationor node value.

Each link is associated with weight.ANNs are capable of learning, which takes
place by altering weight values. The following illustration shows a simple ANN
−

142

142

Neural Networks

Types of Artificial Neural Networks

There are two Artificial Neural Network topologies − FeedForwardand Feedback.

FeedForward ANN
In this ANN, the information flow is unidirectional. A unit sends information to other
unit from which it does not receive any information. There are no feedback loops.
They are used in pattern generation/recognition/classification. They have fixed
inputs and outputs.

143

143

38
 3/9/2022

Neural Networks

FeedBack ANN
Here, feedback loops are allowed. They are used in content
addressable memories.

Working of ANNs
In the topology diagrams shown, each arrow
represents a connection between two neurons and
indicates the pathway for the flow of information.
Each connection has a weight, an integer number
that controls the signal between the two neurons.
If the network generates a “good or desired”
output, there is no need to adjust the weights.
However, if the network generates a “poor or
undesired” output or an error, then the system
alters the weights in order to improve subsequent
results.

144

144

Neural Networks
Machine Learning in ANNs
ANNs are capable of learning and they need to be trained. There are several learning
strategies −
• Supervised Learning−It involves a teacher that is scholar than the ANN itself.
For example, the teacher feeds some example data about which the teacher
already knows the answers. For
example, pattern recognizing. The ANN comes up with guesses while
recognizing. Then the teacher provides the ANN with the answers. The network
then compares it guesses with the teacher’s “correct” answers and makes
adjustments according to errors.
• Unsupervised Learning−It is required when there is no example data set with
known answers. For example, searching for a hidden pattern. In this case,
clustering i.e. dividing a set of elements into groups according to some unknown
pattern is carried out based on the existing data sets present.
• Reinforcement Learning−This strategy built on observation. The ANN makes
a decision by observing its environment. If the observation is negative, the
network adjusts its weights to be able to make a different required decision the
next time.

Back Propagation Algorithm

It is the training or learning algorithm. It learns by example. If you submit to the algorithm
the example of what you want the network to do, it changes the network’s weights so that
it can produce desired output for a particular input on finishing the training.
Back Propagation networks are ideal for simple Pattern Recognition and Mapping Tasks.
145

145

39
 3/9/2022

Reinforcement Learning (RL)

▪ In the case of the agent acts on its environment, it receives some
evaluation of its action (reinforcement), but is not told of which
action is the correct one to achieve its goal.

Task
- Learn how to behave successfully to achieve a goal
while interacting with an external environment
- Learn via experiences!

Examples
Game playing: player knows whether it win or lose, but not know
how to move at each step
Control: a traffic system can measure the delay of cars, but
not
know how to decrease it.

146

146

Neural Networks
Applications of Neural Networks
They can perform tasks that are easy for a human but difficult for a machine −
• Aerospace−Autopilot aircrafts, aircraft fault detection.
• Automotive−Automobile guidance systems.
• Military−Weapon orientation and steering, target tracking, object discrimination, facial
recognition, signal/image identification.
• Electronics−Code sequence prediction, IC chip layout, chip failure analysis, machine vision,
voice synthesis.
• Financial−Real estate appraisal, loan advisor, mortgage screening, corporate bond rating,
portfolio trading program, corporate financial analysis, currency value prediction, document
readers, credit application evaluators.
• Industrial−Manufacturing process control, product design and analysis, quality inspection
systems, welding quality analysis, paper quality prediction, chemical product design analysis,
dynamic modeling of chemical process systems, machine maintenance analysis, project bidding,
planning, and management.
• Medical−Cancer cell analysis, EEG and ECG analysis, prosthetic design, transplant time
optimizer.
• Speech−Speech recognition, speech classification, text to speech conversion.
• Telecommunications−Image and data compression, automated information services, real-time
spoken language translation.
• Transportation−Truck Brake system diagnosis, vehicle scheduling, routing systems.
• Software−Pattern Recognition in facial recognition, optical character recognition, etc.
• Time Series Prediction−ANNs are used to make predictions on stocks and natural calamities.
• Signal Processing−Neural networks can be trained to process an audio signal and filter it
appropriately in the hearing aids.
• Control−ANNs are often used to make steering decisions of physical vehicles.
• Anomaly Detection−As ANNs are expert at recognizing patterns, they can also be trained to
generate an output when something unusual occurs that misfits the pattern.
147

147

40
 3/9/2022

Genetic Algorithms (GA)

GAs are part of evolutionary computing‫ا لحوسبة ا لتطورية‬.
▪.
▪Genetic algorithms are categorized as global search heuristics.
▪ Genetic algorithms are implemented as a computer
simulation, where techniques are inspired by evolutionary
biology.
▪Mechanics of biological evolution:
- Every organismhas a set of rules, describing how that
organism is built, and encoded in the genes of an
organism.
- The genes are connected together into long strings
called
chromosomes (Blueprint). 148

148

Genetic Algorithms (GA)

▪ Mechanics of biological evolution:
- Each gene represents a specific trait (feature) of the organism
and has several different settings, e.g. setting for a hair color
gene may be black or brown.
- The genes and their settings are
referred as an organism's genotype.
- When two organisms mate they share their genes. The
resultant offspringmay end up having half the genes from
one parent and half from the other. This process is calledcross
over.
- A gene may be mutatedand expressed in the organism as
a completely new trait.

149

149

41
 3/9/2022

Chromosome, Genes and

Genomes

150

150

Genetic Algorithm Steps

151

151

42
 3/9/2022

Genetic Algorithm Steps

Genetic Algorithms does unsupervised learning –

the right answer is not known beforehand.
152

152

Regression in AI

Regression‫ ا النحدار‬analysis is a statistical method to

model the relationship between a dependent
(target) and independent (predictor) variables with
one or more independent variables. More
specifically, Regression analysis helps us to
understand how the value of the dependent
variable is changing corresponding to an
independent variable when other independent
variables are held fixed. It predicts continuous/real
values such astemperature, age, salary, price,etc.

153

43
 3/9/2022

• Example:Suppose there is a marketing company A, who does various

advertisement every year and get sales on that. The below list shows the
advertisement made by the company in the last 5 years and the corresponding
sales:

• Now, the company wants to do the advertisement of $200 in the year 2019and
wants to know the prediction about the sales for this year. So to solve such type of
prediction problems in machine learning, we need regression analysis.

154

Terminologies Related to the

Regression Analysis:
• Dependent Variable:The main factor in Regression analysis which we want
to predict or understand is called the dependent variable. It is also
calledtarget variable.
• Independent Variable:The factors which affect the dependent variables or which
are used to predict the values of the dependent variables are called independent
variable, also called as apredictor.
• Outliers:Outlier is an observation which contains either very low value or very high
value in comparison to other observed values. An outlier may hamper‫ ت ع وق‬the
result, so it should be avoided.
• Multicollinearity:If the independent variables are highly correlated with each
other than other variables, then such condition is called Multicollinearity. It should
not be present in the dataset, because it creates problem while ranking the most
affecting variable.
• Underfitting and Overfitting:If our algorithm works well with the training dataset
but not well with test dataset, then such problem is called Overfitting. And if our
algorithm does not perform well even with training dataset, then such problem is
called underfitting.

155

44
 3/9/2022

Types of Regression

156

Linear Regression:
• Linear regression is a statistical regression method which is
used for predictive analysis.
• It is one of the very simple and easy algorithms which works
on regression and shows the relationship between the
continuous variables.
• It is used for solving the regression problem in machine learning.
• Linear regression shows the linear relationship between the
independent variable (X-axis) and the dependent variable (Y-axis),
hence called linear regression.
• If there is only one input variable (x), then such linear regression
is calledsimple linear regression. And if there is more than one
input variable, then such linear regression is calledmultiple linear
regression.
• The relationship between variables in the linear regression
model can be explained using the below image. Here we are
predicting the salary of an employee on the basis ofthe year of
experience.

157

45
 3/9/2022

Linear Regression:

158

Logistic Regression:

• Logistic regression is another supervised learning algorithm

which is used to solve the classification problems.
Inclassification problems, we have dependent variables in a
binary or discrete format such as 0 or 1.
• Logistic regression algorithm works with the categorical
variable such as 0 or 1, Yes or No, True or False, Spam or not
spam, etc.
• It is a predictive analysis algorithm which works on the concept
of probability.
• Logistic regression is a type of regression, but it is different
from the linear regression algorithm in the term how they are
used.
• Logistic regression usessigmoid functionor logistic function
which is a complex cost function. This sigmoid function is used to
model the data in logistic regression. The function can be
represented as:

159

46
 3/9/2022

Logistic Regression:

• It uses the concept of threshold levels, values above the threshold level are
rounded up to 1, and values below the threshold level are rounded up to 0.
• There are three types of logistic regression:
–Binary(0/1, pass/fail)
–Multi(cats, dogs, lions)
–Ordinal(low, medium, high)

160

Polynomial Regression:

• Polynomial Regression is a type of regression which

models thenon-linear datasetusing a linear model.
• It is similar to multiple linear regression, but it fits a
non- linear curve between the value of x and
corresponding conditional values of y.
• Suppose there is a dataset which consists of datapoints
which are present in a non-linear fashion, so for such case,
linear regression will not best fit to those datapoints. To
cover such datapoints, we need Polynomial regression.
• In Polynomial regression, the original features are
transformed into polynomial features of given degree and
then modeled using a linear model.Which means the
datapoints are best fitted using a polynomial line.

161

47
 3/9/2022

Polynomial Regression:

• The equation for polynomial regression also derived from linear regression
equation that means Linear regression equation Y= b0+ b1x, is transformed
into Polynomial regression equation Y= b0+b1x+ b2x2+ b3x3+.....+ bnxn.
• Here Y is thepredicted/target output, b 0, b1,... bn are the
regression coefficients. x is ourindependent/input variable.
• The model is still linear as the coefficients are still linear with quadratic

162

Support Vector Regression:

• Support Vector Machine is a supervised learning algorithm which can
be used for regression as well as classification problems. So if we use it
for regression problems, then it is termed as Support Vector Regression.
• Support Vector Regression is a regression algorithm which works for
continuous variables. Below are some keywords which are used inSupport
Vector Regression:
• Kernel:It is a function used to map a lower-dimensional data into
higher dimensional data.
• Hyperplane:In general SVM, it is a separation line between two classes,
but in SVR, it is a line which helps to predict the continuous variables and
cover most of the datapoints.
• Boundary line:Boundary lines are the two lines apart from
hyperplane, which creates a margin for datapoints.
• Support vectors:Support vectors are the datapoints which are nearest
to the hyperplane and opposite class.

163

48
 3/9/2022

Support Vector Regression:

• In SVR, we always try to determine a hyperplane with a maximum margin, so that maximum number
of datapoints are covered in that margin.The main goal of SVR is to consider the maximum
datapoints within the boundary lines and the hyperplane (best-fit line) must contain a maximum
number of datapoints. Consider the below image:

• Here, the blue line is called hyperplane, and the other two lines are known as boundary lines.

164

Decision Tree Regression:

• Decision Tree is a supervised learning algorithm which can be

used for solving both classification and regression problems.
• It can solve problems for both categorical and numerical data
• Decision Tree regression builds a tree-like structure in which each
internal node represents the "test" for an attribute, each branch
represent the result of the test, and each leaf node represents the
final decision or result.
• A decision tree is constructed starting from the root node/parent
node (dataset), which splits into left and right child nodes (subsets
of dataset). These child nodes are further divided into their children
node, and themselves become the parent node of those nodes.
Consider the below image:

165

49
 3/9/2022

Decision Tree Regression:

• This image showing the example of Decision Tee regression,

here, the model is trying to predict the choice of a person
between Sports cars or Luxury car.

166

Random Forest Regression:

• Random forest is one of the most powerful
supervised learning algorithms which is capable
of performing regression as well as classification
tasks.
• The Random Forest regression is an ensemble
learning method which combines multiple
decision trees and predicts the final output based
on the average of each tree output. The combined
decision trees are called as base models, and it
can be represented more formally as:
• g(x)= f 0(x)+ f1(x)+ f2(x)+....

167

50
 3/9/2022

Random Forest Regression:

• Random forest uses Bagging or Bootstrap Aggregation technique of
ensemble learning in which aggregated decision tree runs in parallel
and do not interact with each other.
• With the help of Random Forest regression, we can prevent
Overfitting in the model by creating random subsets of the dataset.

168

Ridge Regression:
• Ridge regression is one of the most robust versions of linear regression in
which a small amount of bias is introduced so that we can get better long
term predictions.
• The amount of bias added to the model is known asRidge Regression
penalty. We can compute this penalty term by multiplying with the lambda to
the squared weight of each individual features.
• The equation for ridge regression will be:

• A general linear or polynomial regression will fail if there is high collinearity

between the independent variables, so to solve such problems, Ridge
regression can be used.
• Ridge regression is a regularization technique, which is used to reduce the
complexity of the model. It is also called asL2 regularization.
• It helps to solve the problems if we have more parameters than samples.

169

51
 3/9/2022

Lasso Regression:

• Lasso regression is another regularization technique

to reduce the complexity of the model.
• It is similar to the Ridge Regression except that
penalty term contains only the absolute weights
instead of a square of weights.
• Since it takes absolute values, hence, it can shrink the
slope to 0, whereas Ridge Regression can only shrink it
near to 0.
• It is also called asL1 regularization. The equation
for Lasso regression will be:
•

170

52