Search Terminology

 Problem Space − It is the environment in which the search takes place. (A set of states and
set of operators to change those states)
 Problem Instance − It is Initial state + Goal state.
 Problem Space Graph − It represents problem state. States are shown by nodes and
operators are shown by edges.
 Depth of a problem − Length of a shortest path or shortest sequence of operators from
Initial State to goal state.
 Space Complexity − The maximum number of nodes that are stored in memory.
 Time Complexity − The maximum number of nodes that are created.
 Admissibility − A property of an algorithm to always find an optimal solution.
 Branching Factor − The average number of child nodes in the problem space graph.
 Depth − Length of the shortest path from initial state to goal state.

Breadth-First Search
It starts from the root node, explores the neighboring nodes first and moves towards
the next level neighbors. It generates one tree at a time until the solution is found. It
can be implemented using FIFO queue data structure. This method provides shortest
path to the solution.
If branching factor (average number of child nodes for a given node) = b and depth =
d, then number of nodes at level d = bd.
The total no of nodes created in worst case is b + b 2 + b3 + … + bd.
Disadvantage − Since each level of nodes is saved for creating next one, it consumes
a lot of memory space. Space requirement to store nodes is exponential.
Its complexity depends on the number of nodes. It can check duplicate nodes.
Depth-First Search
It is implemented in recursion with LIFO stack data structure. It creates the same set of
nodes as Breadth-First method, only in the different order.
As the nodes on the single path are stored in each iteration from root to leaf node, the
space requirement to store nodes is linear. With branching factor b and depth as m, the
storage space is bm.
Disadvantage − This algorithm may not terminate and go on infinitely on one path.
The solution to this issue is to choose a cut-off depth. If the ideal cut-off is d, and if
chosen cut-off is lesser than d, then this algorithm may fail. If chosen cut-off is more
than d, then execution time increases.
Its complexity depends on the number of paths. It cannot check duplicate nodes.

Informed (Heuristic) Search Strategies

To solve large problems with large number of possible states, problem-specific
knowledge needs to be added to increase the efficiency of search algorithms.
Heuristic Evaluation Functions
They calculate the cost of optimal path between two states. A heuristic function for
sliding-tiles games is computed by counting number of moves that each tile makes
from its goal state and adding these number of moves for all tiles.
Pure Heuristic Search
It expands nodes in the order of their heuristic values. It creates two lists, a closed list
for the already expanded nodes and an open list for the created but unexpanded
nodes.
In each iteration, a node with a minimum heuristic value is expanded, all its child nodes
are created and placed in the closed list. Then, the heuristic function is applied to the
child nodes and they are placed in the open list according to their heuristic value. The
shorter paths are saved and the longer ones are disposed.
A * Search
It is best-known form of Best First search. It avoids expanding paths that are already
expensive, but expands most promising paths first.
f(n) = g(n) + h(n), where

 g(n) the cost (so far) to reach the node

 h(n) estimated cost to get from the node to the goal
 f(n) estimated total cost of path through n to goal. It is implemented using priority queue by
increasing f(n).

Greedy Best First Search

It expands the node that is estimated to be closest to goal. It expands nodes based on
f(n) = h(n). It is implemented using priority queue.
Disadvantage − It can get stuck in loops. It is not optimal.

Local Search Algorithms

They start from a prospective solution and then move to a neighboring solution. They
can return a valid solution even if it is interrupted at any time before they end.
Hill-Climbing Search
It is an iterative algorithm that starts with an arbitrary solution to a problem and
attempts to find a better solution by changing a single element of the solution
incrementally. If the change produces a better solution, an incremental change is taken
as a new solution. This process is repeated until there are no further improvements.
function Hill-Climbing (problem), returns a state that is a local maximum.
inputs: problem, a problem
local variables: current, a node
neighbor, a node
current <-Make_Node(Initial-State[problem])
loop
do neighbor <- a highest_valued successor of current
if Value[neighbor] ≤ Value[current] then
return State[current]
current <- neighbor

end
Disadvantage − This algorithm is neither complete, nor optimal.
Local Beam Search
In this algorithm, it holds k number of states at any given time. At the start, these states
are generated randomly. The successors of these k states are computed with the help
of objective function. If any of these successors is the maximum value of the objective
function, then the algorithm stops.
Otherwise the (initial k states and k number of successors of the states = 2k) states are
placed in a pool. The pool is then sorted numerically. The highest k states are selected
as new initial states. This process continues until a maximum value is reached.
function BeamSearch( problem, k), returns a solution state.
start with k randomly generated states
loop
generate all successors of all k states
if any of the states = solution, then return the state
else select the k best successors
end

Simulated Annealing
Annealing is the process of heating and cooling a metal to change its internal structure
for modifying its physical properties. When the metal cools, its new structure is seized,
and the metal retains its newly obtained properties. In simulated annealing process, the
temperature is kept variable.
We initially set the temperature high and then allow it to ‘cool' slowly as the algorithm
proceeds. When the temperature is high, the algorithm is allowed to accept worse
solutions with high frequency.
Start

 Initialize k = 0; L = integer number of variables;

 From i → j, search the performance difference Δ.
 If Δ <= 0 then accept else if exp(-Δ/T(k)) > random(0,1) then accept;
 Repeat steps 1 and 2 for L(k) steps.
 k = k + 1;
Repeat steps 1 through 4 till the criteria is met.
End
What is Knowledge Representation?
Knowledge Representation in AI describes the representation of knowledge.
Basically, it is a study of how the beliefs, intentions, and judgments of an intelligent
agent can be expressed suitably for automated reasoning. One of the primary purposes
of Knowledge Representation includes modeling intelligent behavior for an agent.

Knowledge Representation and Reasoning (KR, KRR) represents information from the
real world for a computer to understand and then utilize this knowledge to
solve complex real-life problems like communicating with human beings in natural
language. Knowledge representation in AI is not just about storing data in a database, it
allows a machine to learn from that knowledge and behave intelligently like a human
being.

The different kinds of knowledge that need to be represented in AI include:

 Objects
 Events
 Performance
 Facts
 Meta-Knowledge
 Knowledge-base

Now that you know about Knowledge representation in AI, let’s move on and know
about the different types of Knowledge.

Different Types of Knowledge

There are 5 types of Knowledge such as:

  Declarative Knowledge – It includes concepts, facts, and objects and

expressed in a declarative sentence.

 Structural Knowledge – It is a basic problem-solving knowledge that describes

the relationship between concepts and objects.

 Procedural Knowledge – This is responsible for knowing how to do something

and includes rules, strategies, procedures, etc.

 Meta Knowledge – Meta Knowledge defines knowledge about other types of

Knowledge.

 Heuristic Knowledge – This represents some expert knowledge in the field or

subject.
These are the important types of Knowledge Representation in AI. Now, let’s have a
look at the cycle of knowledge representation and how it works.

Cycle of Knowledge Representation in AI

Artificial Intelligent Systems usually consist of various components to display their
intelligent behavior. Some of these components include:

 Perception
 Learning
 Knowledge Representation & Reasoning
 Planning
 Execution

Here is an example to show the different components of the system and how it works:
Example

The above diagram shows the interaction of an AI system with the real world and
the components involved in showing intelligence.

 The Perception component retrieves data or information from the environment. with

the help of this component, you can retrieve data from the environment, find out the
source of noises and check if the AI was damaged by anything. Also, it defines how to
respond when any sense has been detected.
 Then, there is the Learning Component that learns from the captured data by the
perception component. The goal is to build computers that can be taught instead of
programming them. Learning focuses on the process of self-improvement. In order to
learn new things, the system requires knowledge acquisition, inference, acquisition of
heuristics, faster searches, etc.
 The main component in the cycle is Knowledge Representation and Reasoning which
shows the human-like intelligence in the machines. Knowledge representation is all
about understanding intelligence. Instead of trying to understand or build brains from the
bottom up, its goal is to understand and build intelligent behavior from the top-down and
focus on what an agent needs to know in order to behave intelligently. Also, it defines
how automated reasoning procedures can make this knowledge available as needed.
 The Planning and Execution components depend on the analysis of knowledge
representation and reasoning. Here, planning includes giving an initial state, finding their
preconditions and effects, and a sequence of actions to achieve a state in which a
particular goal holds. Now once the planning is completed, the final stage is the
execution of the entire process.

So, these are the different components of the cycle of Knowledge Representation in AI.
Now, let’s understand the relationship between knowledge and intelligence.
 

Predicate logic:

Predicate logic has a long pedigree. Its roots go back at least as far as Aristotle,
although in its current form it was developed starting in the late nineteenth century.
Associated with it are techniques for the analysis of many conceptual structures in our
common thought. Because these analytical techniques are well-understood, and
because it is relatively easy to express the formulae of predicate logic in AI languages
such as LISP or POP-11, it has been a very popular knowledge representation
symbolism within AI. Predicate logic also embodies a set of systematic procedures
for proving that certain formulae can or cannot be logically derived from others and
such logical inference procedures have been used as the backbone for problem-

Natural Language Processing

Natural Language Processing (NLP) refers to AI method of communicating with an
intelligent systems using a natural language such as English.
Processing of Natural Language is required when you want an intelligent system like
robot to perform as per your instructions, when you want to hear decision from a
dialogue based clinical expert system, etc.
The field of NLP involves making computers to perform useful tasks with the natural
languages humans use. The input and output of an NLP system can be −

 Speech
 Written Text

Components of NLP
There are two components of NLP as given −
Natural Language Understanding (NLU)
Understanding involves the following tasks −

 Mapping the given input in natural language into useful representations.

 Analyzing different aspects of the language.

Natural Language Generation (NLG)

It is the process of producing meaningful phrases and sentences in the form of natural
language from some internal representation.
It involves −
 Text planning − It includes retrieving the relevant content from knowledge base.
 Sentence planning − It includes choosing required words, forming meaningful phrases,
setting tone of the sentence.
 Text Realization − It is mapping sentence plan into sentence structure.
The NLU is harder than NLG.

Difficulties in NLU
NL has an extremely rich form and structure.
It is very ambiguous. There can be different levels of ambiguity −
 Lexical ambiguity − It is at very primitive level such as word-level.
 For example, treating the word “board” as noun or verb?
 Syntax Level ambiguity − A sentence can be parsed in different ways.
 For example, “He lifted the beetle with red cap.” − Did he use cap to lift the beetle or he lifted
a beetle that had red cap?
 Referential ambiguity − Referring to something using pronouns. For example, Rima went
to Gauri. She said, “I am tired.” − Exactly who is tired?
 One input can mean different meanings.
 Many inputs can mean the same thing.

NLP Terminology
 Phonology − It is study of organizing sound systematically.
 Morphology − It is a study of construction of words from primitive meaningful units.
 Morpheme − It is primitive unit of meaning in a language.
 Syntax − It refers to arranging words to make a sentence. It also involves determining the
structural role of words in the sentence and in phrases.
 Semantics − It is concerned with the meaning of words and how to combine words into
meaningful phrases and sentences.
 Pragmatics − It deals with using and understanding sentences in different situations and
how the interpretation of the sentence is affected.
 Discourse − It deals with how the immediately preceding sentence can affect the
interpretation of the next sentence.
 World Knowledge − It includes the general knowledge about the world.

Steps in NLP
There are general five steps −
 Lexical Analysis − It involves identifying and analyzing the structure of words. Lexicon of a
language means the collection of words and phrases in a language. Lexical analysis is
dividing the whole chunk of txt into paragraphs, sentences, and words.
 Syntactic Analysis (Parsing) − It involves analysis of words in the sentence for grammar
and arranging words in a manner that shows the relationship among the words. The
sentence such as “The school goes to boy” is rejected by English syntactic analyzer.

 Semantic Analysis − It draws the exact meaning or the dictionary meaning from the text.
The text is checked for meaningfulness. It is done by mapping syntactic structures and
objects in the task domain. The semantic analyzer disregards sentence such as “hot ice-
cream”.
 Discourse Integration − The meaning of any sentence depends upon the meaning of the
sentence just before it. In addition, it also brings about the meaning of immediately
succeeding sentence.
  Pragmatic Analysis − During this, what was said is re-interpreted on what it actually meant.
It involves deriving those aspects of language which require real world knowledge.

Implementation Aspects of Syntactic Analysis

There are a number of algorithms researchers have developed for syntactic analysis,
but we consider only the following simple methods −

 Context-Free Grammar
 Top-Down Parser
Let us see them in detail −
Context-Free Grammar
It is the grammar that consists rules with a single symbol on the left-hand side of the
rewrite rules. Let us create grammar to parse a sentence −
“The bird pecks the grains”
Articles (DET) − a | an | the
Nouns − bird | birds | grain | grains
Noun Phrase (NP) − Article + Noun | Article + Adjective + Noun
= DET N | DET ADJ N
Verbs − pecks | pecking | pecked
Verb Phrase (VP) − NP V | V NP
Adjectives (ADJ) − beautiful | small | chirping
The parse tree breaks down the sentence into structured parts so that the computer
can easily understand and process it. In order for the parsing algorithm to construct this
parse tree, a set of rewrite rules, which describe what tree structures are legal, need to
be constructed.
These rules say that a certain symbol may be expanded in the tree by a sequence of
other symbols. According to first order logic rule, if there are two strings Noun Phrase
(NP) and Verb Phrase (VP), then the string combined by NP followed by VP is a
sentence. The rewrite rules for the sentence are as follows −
S → NP VP
NP → DET N | DET ADJ N
VP → V NP
Lexocon −
DET → a | the
ADJ → beautiful | perching
N → bird | birds | grain | grains
V → peck | pecks | pecking
The parse tree can be created as shown −

Now consider the above rewrite rules. Since V can be replaced by both, "peck" or
"pecks", sentences such as "The bird peck the grains" can be wrongly permitted. i. e.
the subject-verb agreement error is approved as correct.
Merit − The simplest style of grammar, therefore widely used one.
Demerits −
 They are not highly precise. For example, “The grains peck the bird”, is a syntactically
correct according to parser, but even if it makes no sense, parser takes it as a correct
sentence.
 To bring out high precision, multiple sets of grammar need to be prepared. It may require a
completely different sets of rules for parsing singular and plural variations, passive
sentences, etc., which can lead to creation of huge set of rules that are unmanageable.
Top-Down Parser
Here, the parser starts with the S symbol and attempts to rewrite it into a sequence
of terminal symbols that matches the classes of the words in the input sentence until it
consists entirely of terminal symbols.
These are then checked with the input sentence to see if it matched. If not, the process
is started over again with a different set of rules. This is repeated until a specific rule is
found which describes the structure of the sentence.
Merit − It is simple to implement.
Demerits −

 It is inefficient, as the search process has to be repeated if an error occurs.

 Slow speed of working.