Master of Science in Artificial Intelligence, 2020-2022

Knowledge
Representation
and Reasoning
University Politehnica of Bucharest
Department of Computer Science
Fall 2020

Adina Magda Florea

 Lecture 1 & 2
Lecture outline
▪ Course goals
▪ Grading
▪ Textbooks and readings
▪ Syllabus
▪ Why KR?
▪ KR&R Challenges
▪ Models of KR
▪ First order logic

2
 Course goals
▪ Provide an overview of existing representational
frameworks developed within AI, their key
concepts and inference methods.
▪ Acquiring skills in representing knowledge
▪ Understanding the principles behind different
knowledge representation techniques
▪ Being able to read and understand research
literature in the area of KR&R
▪ Being able to complete a project in this research
area
3
 Grading
▪ Course grades
Mid-term exam 10%
Final exam 30%
Homeworks 25%
Laboratory 25%
Course tests 10%
▪ Requirements: min 7 lab attendances, min 50% of term
activity (mid-term ex, homeworks, lab)
▪ Academic Honesty Policy
It will be considered an honor code violation to give or
use someone else's code or written answers, either for
the assignments or exam tests. If such a case occurs,
we will take action accordingly. 4
 Textbooks and Readings
▪ Textbooks
• Artificial Intelligence: A Modern Approach (3rd Edition)
by Stuart Russell and Peter Norvig Prentice Hall, 2010
http://aima.cs.berkeley.edu/
• Probabilistic Graphical Models. Principles and Techniques
by Daphne Koller and Nir Friedman, MIT Press, 2009
https://mitpress.mit.edu/books/probabilistic-graphical-models

• Deep Learning
by Ian Goodfellow and Yoshua Bengio and Aaron Courville, MIT
Press, 2016
https://www.deeplearningbook.org/

▪ Readings
• Reading materials will be assigned to you.
• You are expected to do the readings before the class 5
 Syllabus
1. General knowledge representation issues

2. Symbolic representations
• First order predicate logic revisited
• Description logics
• Knowledge representation for the Semantic Web
3. Probabilistic graphical models
Autonomous PGM
• Bayesian networks
• Markov models
• Hidden Markov models
6
 Syllabus
Controlled PGM
• Markov Decision Processes
• Partially Observable Markov Decision Processes
4. Nonsysmbolic representation formalisms
• Deep feedforward networks
• Convolutional networks
• Recurrent networks
5. Explainable AI
• Class Debate
7
 Lecture 1
Introduction

8
 1. Why KR?
What is knowledge?
▪ We understand by "knowledge" all kinds of
facts about the world.
▪ Knowledge is necessary for intelligent
behavior (human beings, robots).
▪ In this course we consider representation of
knowledge and how we can use it in making
intelligent artifacts (based on software,
hardware or both).
9
 2. KR&R Challenges
▪ Challenges of KR&R:
• representation of
commonsense knowledge
• the ability of a knowledge-
based system to achieve
computational efficiency
for different types of
inferences
• the ability to represent and
manipulate uncertain
knowledge and information.
10
 3. What is KR?
Randall Davis, Howard Shrobe, Peter Szolovits, MIT

▪ A knowledge representation is most

fundamentally a surrogate, a substitute
for the thing itself, used to enable an entity
to determine consequences by reasoning
about the world.
▪ It is a set of ontological commitments,
i.e., an answer to the question: In what
terms should I think about the world?
11
 What is KR?
Simply put:
▪ If A represents B
then A stands for B and is usually more
easily accessible than B.

12
 4. What is Reasoning?
▪ Reasoning is the use of representations
(symbolic or nonsymbolic) of some facts in
order to derive new ones.
▪ Inference – a form of reasoning

13
 5. Models of KRR
Symbolic models
▪ Logical models
• First order logic
• Higher order logics
• Description logics
▪ Rule based models
• Rule based systems (OPS5, MYCIN, CLIPS)
• Sematic rule systems (RuleML, SWRL)
• Rules based languages for event processing
▪ Object oriented (Structured) models
• Semantic networks, Frames
• Ontologies 14
 Models of KRR
Probabilistic models
• Probability theory
• Bayesian networks
• Markov networks
• Hidden Markov Models
• Markov Decision Process
• Partially Observable Markov Decision
Process

15
 Models of KRR
Non-symbolic models
▪ Neural networks, deep neural networks
▪ Reinforcement models
▪ Genetic algorithms
▪ Fuzzy logic and systems (stands in the
middle between s and non-s)

16
 6. Logical models
▪ The abstract study of propositions and
sentences, and of deductive arguments
▪ Use a symbolic notation to express such
structures clearly and unambiguously and
to enable manipulations and tests
of validity to be more easily applied.

17
 knowledge propositional first-order

Paul is a man a man(Paul)

Bill is a man b man(Bill)

men are mortal c (x) (man(x) 

First order logic mortal(x))

knowledge first-order second-order

smaller is ( x) (( y) (( z) transitive(<)

((<(x,y)  <(y,z) 
transitive <(x,z)))))
part-of is ( x) (( y) (( z) transitive(part-of)
((part-of(x,y) 
transitive part-of(y,z) 
part-of(x,z)))))
R is transitive iff not expressible ( R) ((transitive(R) 
whenever R(x,y) and (see however pseudo- ( x) (( y) (( z)
R(y,z) hold, R(x,z) second order) ((R(x,y)  R(y,z) 
holds too R(x,z)))))))
18
Higher order logic
 PrL, FOPL Linear model
Extend PrL, PL

Symbolic logic models

Situation calculus
Adds states, actions
Symbol level
Modal logic
Modal operators Structured models
Knowledge level
Description Logics
Subsumption relationships Temporal logic Dynamic logic
Not directly based on PL Modal operators for time Modal operators
Linear time for actions
Branching time

Logics of knowledge CTL logic

and belief Branching time
Modal operators B and K BDI logic
and action
Adds agents, B, D, I
19
 Examples
Propositional Logic A  B → C
First Order Predicate Logic
P(a)
( x)(P(x) → Q(x))
Q(a)

Situation calculus - describes change in FOPL

Function Result(Action,State) = NewState
▪ Ex: At((0,0), S0)  Free(0,1)  Exit(east) → At((0,1),
Result(move_east,S0))
▪ Try to prove the goal At((0,3), _) and determines
actions that lead to it
20
 Examples
Description logics
Deals with concepts and relationships among these
concepts
Woman ≡ Person Female
Mother ≡ Woman hasChild.Person
MotherWithManyChildren ≡ Mother  3 hasChild

21
 Examples
Modal logic – uses modal operators
▪  - “possibly true”;  - “necessarily true”
▪  a a = “robots will be able to cry”
▪  b b = “sun will rise tomorrow”
▪  P ≡ ~~P P = “it will rain today”
▪ It is possible that it will rain today if and only if it
is not necessary that it will not rain today

Temporal logic
Xp - p is true in the next moment – next
p U q - p is true until q becomes true – until 22
 Examples
Logics of knowledge and belief
▪ FOPL augmented with two modal operators K
and B
K(a,T) - a knows T
B(a,T) - a believes T

Distribution axiom:
K(a, T)  K(a, T → V) → K(a, V)

"The agent ought to be able to reason with its

knowledge"
23
 7. Formal logic
▪ All propositions/statements are represented as
formulas which have a precise syntax and a
semantics according to the logic in question.
▪ Logical system = Formal language +
semantics
▪ We need to define syntax, semantics and how to
obtain logical consequences
▪ Entailment = the fact that a set of statements
have as a logical consequence another
statement 24
 7.1 Logical consequence (entailment)

A. Model centered approach to logical

consequence
▪ Models are abstract mathematical structures that
provide possible interpretations for each of the
objects in a formal language.
▪ Given a model for a language - define what it is
for a sentence in that language to be true
(according to that model) or not.

KB |=L 
25
 Logical consequence (entailment)

B. Proof centered approach to logical

consequence: the validity of a reasoning
process (argument) amounts to finding a
proof of the conclusions from the
premises.

▪ KB |- L 

26
 7.2 Model centered approach
▪ Based on interpretations of a sentence / formula
▪ Model of a sentence – the interpretation for which the
sentence is true
▪ Sentences: valid, invalid (contradiction), satisfiable
▪ Define entailment or logical consequence
▪ A formula F is a logical consequence of a set of
formulas P1,…Pn iff F is true in all interpretations in
which P1,…Pn are true.
▪ P1,… Pn |= L F
▪ T Formula F is a logical consequence of a set of
formulas P1,…Pn iff P1,…Pn →F is valid.
▪ T Formula F is a logical consequence of a set of
formulas P1,…Pn iff P1…  Pn  ~F is inconsistent. 27
 7.3 Proof centered approach

▪ Theorem, deduction

▪ Formal system S =< A , F , A ,  >

▪ Inference rule R 
R
R  F  F , y =  y1 ,..., y n  ⎯⎯→ x, x, y i  F , i = 1, n
n

▪ Premise set  = {y1 , ... , yn } E 0 =   A

E1 = E 0 U{x| y  E 0n , y  x}
n 1
E 2 = E1 U{x| y  E1n , y  x} E i ( i  0)
n 1
28
 Proof centered approach

▪ Consequence of  - the elements of Ei

▪ If E 0 =   A then x  E i is deductible from 

 |⎯S x

▪ Theorems - the elements of Ei

E 0 = A (  = )

▪ Demonstration |⎯ S x - x is provable
(start only from A)
29
 7.4 Properties of logical systems
Important properties of logical systems:
▪ Consistency - no theorem of the system contradicts
another.
▪ Soundness - the system's rules of proof will never
allow a false inference from a true premise. If a system
is sound and its axioms are true then its theorems are also guaranteed to
be true.

▪ Completeness - there are no true sentences in the

system that cannot, at least in principle, be proved
in the system.
▪ Some logical systems do not have all three properties. Kurt Godel's
incompleteness theorems show that no standard formal system of
arithmetic can be consistent and complete. 30
 Properties of logical systems
▪ A formal logical system L is complete iff
 |= L  implies  |⎯ 
(i.e., all valid sentences are provable)
▪ A fromal logical system L is sound iff
 |⎯  implies  |= L 
(i.e., no invalid sentence is provable)

31
 8. Propositional logic - Syntax
S := <Sentence>
<Sentence> := <AtomicSentence> |
<ComplexSentence>
<AtomicSentence> := "TRUE" | "FALSE" |
"P" | "Q" | "S"
<ComplexSentence> := "(" <Sentence> ")" |
<Sentence> <Connective> <Sentence> |
"NOT" <Sentence>
<Connective> := "NOT" | "AND" | "OR" | "IMPLIES" |
"EQUIVALENT"
32
 Propositional logic - Semantics
▪ The meaning or semantics of a sentence
determines its interpretation.
▪ Given the truth values of all symbols in a sentence,
it can be “evaluated” to determine its truth value
(True or False) based on the rules of truth tables
▪ A model of a formula is an assignment of truth
values to propositional symbols in the sentence
which makes the truth value of the sentence True.

33
9. Predicate logic - Syntax

34
 Predicate logic - Semantics
▪ Universe (aka Domain) : Set U
▪ Variables – values in U
▪ Function symbols – (total) functions over U
▪ Terms – values in U
▪ Predicate symbols – relations over U
▪ Sentences (formulas) – have Boolean truth
values
▪ Interpretation of a sentence – linear
interpretation 35
 10. Proving conclusions
▪ If M is a model of A then M |= A
Suppose we want to prove H |= G (entailment).
▪ Equivalently, we can prove that
• F = H → G is valid.
▪ Equivalently, we can prove that
• ~F, i.e., H ~G is unsatisfiable (contradiction)

▪ This principle of “refutation theorem proving”

is the basis of (almost) all automated theorem
proving methods.
36
 How can we do it?
▪ Use model centered approaches
• Truth tables
• Semantic tableaux
▪ Use proof centered approaches
• Combination of different inference rules
• One powerful rule of inference –
resolution
37
 11. Semantic tableaux
▪ Use refutation theorem proving (proof by
contradiction)
▪ Build semantic tableaux
▪ Use tableaux rules to show that:
• Either each possible interpretation leads to a
contradiction – proof of contradiction
Or
• There is an interpretation which satisfies the
formula (has a model) – no proof of
contradiction
38
 Semantic tableaux

() rule for F = A  B

() rule for F = A  B
(and other disjunctions)
(and other conjunctions)

39
 Semantic tableaux

() rule for F = x: A(X,...) () rules for F = x: A(X,...)
substitute a -variable X substitute a -variable X with
with a
an arbitrary term t new constant c

40
 Semantic tableaux – all 4 rules

• () rule for F = A  B

• () rule for F = A  B
• () rule for F = x: A(X,...)
– substitute a -variable X with an
arbitrary term t
• () rules for F = x: A(X,...)
– substitute a -variable X with a
c new new constant c
t arbitrary

▪ A branch is closed if it contains complementary

formulas
▪ A tableaux is closed if every branch is closed 41
 Example in propositional logic
(1) aᴧw→p
(2) i˅a
(3) ¬w → m
(4) e → ¬i ᴧ ¬m
(5) e

42
 Semantic tableaux
Theorem (Soundness, Completeness of
Tableaux calculus):
Let A1,..., Ak and F be (propositional or first-order)
logic sentences.

Then the following are equivalent:

1. A1, ..., Ak |= F
2. A1  ...  Ak  F is unsatisfiable (inconsistent)
3. There is a closed tableaux for {A1, ..., Ak ,  F}

43
 12. Resolution
▪ Use refutation theorem proving
▪ Use resolution inference rule for proving
contradiction
▪ Transform sentences in normal forms, i.e.
clauses
▪ Any sentence in FOPL (without equality) can be
transformed in a set of clauses
▪ Any set of sentences S can be represented in
clausal/normal form (S’)
▪ S is inconsistent iff the equivalent S’ in
normal form is inconsistent 44
 Resolution
▪ Resolution works only on expressions in clausal
form
▪ A literal is either an atomic sentence or a
negation of an atomic sentence
▪ A clause is a set of literals and represents a
disjunction of the literals in the set
▪ A clause set is a set of clauses and represents
a conjunction of the clauses in the set

45
 Conversion to clausal form
In the first step (Implications out), we eliminate all occurrences of the ⇒, ⇐, and ⇔
operators by substituting equivalent sentences involving only the ∧, ∨, and ¬ operators.

φ⇒ψ → ¬φ ∨ ψ
φ⇐ψ → φ ∨ ¬ψ
φ⇔ψ → (¬φ ∨ ψ) ∧ (φ ∨ ¬ψ)

In the second step (Negations in), negations are distributed over other logical operators and
quantifiers until each such operator applies to a single atomic sentence.

¬¬φ →φ
¬(φ ∧ ψ) → ¬φ ∨ ¬ψ
¬(φ ∨ ψ) → ¬φ ∧ ¬ψ
¬∀ν.φ → ∃ν.¬φ
¬∃ν.φ → ∀ν.¬φ
46
 Conversion to clausal form
In the third step (Standardize variables), we rename variables so that each quantifier has a
unique variable, i.e. the same variable is not quantified more than once within the same
sentence. The following transformation is an example.

∀x.(p(x) ⇒ ∃x.q(x)) → ∀x.(p(x) ⇒ ∃y.q(y))

In the fourth step (Existentials out), we eliminate all existential quantifiers. The method
for doing this is a little complicated, and we describe it in two stages.
If an existential quantifier does not occur within the scope of a universal quantifier, we
simply drop the quantifier and replace all occurrences of the quantified variable by a new
constant; i.e., one that does not occur anywhere else in our database. The constant used to
replace the existential variable in this case is called a Skolem constant. The following
example assumes that a is not used anywhere else.

∃x.p(x) → p(a)

47
 Conversion to clausal form
If an existential quantifier is within the scope of any universal quantifiers, there is the
possibility that the value of the existential variable depends on the values of the
associated universal variables. Consequently, we cannot replace the existential variable
with a constant. Instead, the general rule is to drop the existential quantifier and to
replace the associated variable by a term formed from a new function symbol applied to
the variables associated with the enclosing universal quantifiers. Any function defined
in this way is called a Skolem function. The following example illustrates this
transformation. It assumes that f is not used anywhere else.

∀x.(p(x) ∧ ∃z.q(x, y, z)) → ∀x.(p(x) ∧ q(x, y, f(x, y)))

In the fifth step (Alls out), we drop all universal quantifiers. Because the remaining
variables at this point are universally quantified, this does not introduce any
ambiguities.
∀x.(p(x) ∧ q(x, y, f(x, y))) → p(x) ∧ q(x, y, f(x, y))

48
 Conversion to clausal form
In the sixth step (Disjunctions in), we put the expression into conjunctive normal form,
i.e. a conjunction of disjunctions of literals. This can be accomplished by repeated use
of the following rules.
φ ∨ (ψ ∧ χ) → (φ ∨ ψ) ∧ (φ ∨ χ)
(φ ∧ ψ) ∨ χ → (φ ∨ χ) ∧ (ψ ∨ χ)
φ ∨ (φ1 ∨ ... ∨ φn) → φ ∨ φ1 ∨ ... ∨ φn
(φ1 ∨ ... ∨ φn) ∨ φ → φ1 ∨ ... ∨ φn ∨ φ
φ ∧ (φ1 ∧ ... ∧ φn) → φ ∧ φ1 ∧ ... ∧ φn
(φ1 ∧ ... ∧ φn) ∧ φ → φ1 ∧ ... ∧ φn ∧ φ
In the seventh step (Operators out), we eliminate operators by separating any conjunctions
into its conjuncts and writing each disjunction as a separate clause.

φ1 ∧ ... ∧ φn → φ1
→ ...
→ φn
φ1 ∨ ... ∨ φn → {φ1, ... , φn} 49
 Powerful inference rule - Resolution

Resolution
▪ Binary resolution
▪ Factorization
▪ General resolution

50
 Unification
A substitution is a finite mapping of variables to terms.
{x←a, y←f(b), z←v}
A substitution σ is a unifier for an expression φ and an expression ψ if and only if φσ=ψσ,
i.e. the result of applying σ to φ is the same as the result of applying σ to ψ. If two
expressions have a unifier, they are said to be unifiable. Otherwise, they are nonunifiable.
The expressions p(x, y) and p(a,v) have a unifier, e.g. {x←a, y←b, v←b} and are,
therefore, unifiable. The results of applying this substitution to the two expressions are
shown below.
p(x, y){x←a, y←b, v←b}=p(a, b)
p(a, v){x←a, y←b, v←b}=p(a, b)
We say that a substitution σ is as general as or more general than a substitution τ if and
only if there is another substitution δ such that σδ=τ. For example, the substitution
{x←a, y←v} is more general than {x←a, y←f(c), v←f(c)} since there is a substitution
{v←f(c)} that, when applied to the former, gives the latter.
{x←a, y←v}{v←f(c)}={x←a, y←f(c), v←f(c)}
In resolution, we are interested only in unifiers with maximum generality. A most
general unifier, or mgu, σ of two expressions has the property that it as general as or
more general than any other unifier. 51
 Binary resolution
Given a clause with a literal φ and a second clause with a literal ¬ψ such that φ and
ψ have a most general unifier σ, we can derive a conclusion by applying σ to the
clause consisting of the remaining literals from the two original clauses.

{φ1, ... , φ, ... , φm}

{ψ1, ... , ¬ψ, ... , ψn}
{φ1, ... , φm, ψ1, ..., ψn}σ
where σ=mgu(φ, ψ)
Consider the example shown below. The first clause contains the positive literal p(a,y)
and the second clause contains a negative occurrence of p(x, f(x)). The substitution
{x←a, y←f(a)} is a most general unifier of these two expressions. Consequently, we
can collect the remaining literals r (y) and q(g(x)) into a clause and apply the
substitution to produce a conclusion.
{p(a, y), r(y)}
{¬p(x, f(x)), q(g(x))}
52
{r(f(a)), q(g(a))}
 Binary resolution
Unfortunately, this simple version of the Resolution Principle is not quite good enough.
Consider the two clauses shown below. Given the meaning of these two clauses, it should
be possible to resolve them to produce the empty clause. However, the two atomic
sentences do not unify. The variable x must be bound to a and b at the same time.
{p(a, x)}
{¬p(x, b)}
Fortunately, this problem can easily be fixed by extending the Resolution Principle
slightly as shown below. Before trying to resolve two clauses, we select one of the clauses
and rename any variables the clause has in common with the other clause.
{φ1, ... , φ, ... , φm}
{ψ1, ... , ¬ψ, ... , ψn}

{φ1τ, ... , φmτ, ψ1, ... , ψn}σ

where τ is a variable renaming on {φ1, ... , φ, ... , φm}
where σ=mgu(φτ, ψ)
53
 General resolution
Renaming solves this problem. Unfortunately, we are still not quite done. There is one
more technicality that must be addressed to finish the story. As stated, even with the
extension mentioned before, the rule is not quite good enough. Given the clauses shown
below, we should be able to infer the empty clause {}; however, this is not possible with
the preceding definition. The clauses can be resolved in various ways, but the result is
never the empty clause.

{p(x), p(y)}
{¬p(u), ¬p(v)}

The good news is that we can solve this additional problem with one last modification to
our definition of the Resolution Principle. If a subset of the literals in a clause Φ has a
most general unifier γ, then the clause Φ' obtained by applying γ to Φ is called a factor of
Φ. For example, the literals p(x) and p(f(y)) have a most general unifier {x←f(y)}, so the
clause {p(f(y)), r(f(y), y)} is a factor of {p(x), p(f(y)),r(x, y)}. Obviously, any clause is a
trivial factor of itself.

54
 Resolution principle
Using the notion of factors, we can give a complete definition for the Resolution
Principle. Suppose that Φ and Ψ are two clauses. If there is a literal φ in some factor of
Φ and a literal ¬ψ in some factor of Ψ, then we say that the two clauses Φ and
Ψ resolve and that the new clause ((Φ'-{φ})∪(Ψ'-{¬ψ}))σ is a resolvent of the two
clauses.
Φ
Ψ
((Φ'-{φ})∪(Ψ'-{¬ψ}))σ
where τ is a variable renaming on Φ
where Φ' is a factor of Φτ and φ∈Φ'
where Ψ' is a factor of Ψ and ¬ψ∈Ψ'
where σ=mgu(φ, ψ)
Using this enhanced definition of resolution, we can solve the problem mentioned
above. Once again, consider the premises {p(x), p(y)} and {¬p(u), ¬p(v)}. The first
premise has the factor {p(x)}, and the second has the factor {¬p(u)}, and these two
55
factors resolve to the empty clause in a single step.
 Soundness of Resolution

▪ Theorem. Resolution is sound. That is, all

derived formulas are entailed by the given
ones.

John Alan Robinson

1930-2016

Inventor of resolution
inference rule

56
 Completeness of Resolution

▪ Theorem: Resolution is refutationally

complete.
▪ A set of clauses S is unsatisfiable iff there
is a deduction of the empty clause from S.

57
 Completeness of Resolution

To prove we need semantic trees

▪ Complete semantic tree - any path from
the root to a leaf contains all elements of
the Herbrand base either in positive or
negative form
S = P  Q R Herbrand base of S is
P ~P P AQ = {P,~P,~Q
Q, R}
~Q,~R
Q ~Q Q ~Q Q R P ~P R
~R

R R R R R Q R R 58
~R ~R ~R ~R ~R ~Q ~R ~R
 Completeness of Resolution
S = {P( x), Q( f ( x))}
Herbrand base of S is

A = {P(a),Q(a), P(f (a)),Q(f (a)), P(f (f (a))),Q(f (f (a))),...}

P(a) ~P(a)

Q(f(a))
Q(f(a))
~Q(f(a))
~Q(f(a))

P(f(a)) ~P(f(a))

59
 Completeness of Resolution
Failure node – node falsifies a clause of S,
while parent does not falsify any clause in S
Closed semantics tree – all paths end in failure nodes

S = {P, Q  R , ~ P  ~ Q, ~ P  ~ R}
A = {P, Q, R}
P ~P P ~P

Q ~Q Q ~Q Q ~Q

R R R R R
~R ~R ~R ~R ~R
60
 Completeness of Resolution

S = {P, ~ P  Q, ~ P ~ Q} P ~P
T
Inference node – Q ~Q Q ~Q
all children are failure nodes
complete semantic tree

(1)
S  {~ P} inference node
(2) P ~P (3)
T'
Q ~Q P
(4) (5)
~P V ~ Q ~P V Q failure node
failure node
61
closed semantic tree
 Completeness of Resolution

S = {P, ~ P  Q, ~ P ~ Q} (1)
inference node
(2) P ~P (3)
T'
Q ~Q P
(4) (5)
~P V ~ Q ~P V Q failure node
failure node
closed semantic tree
inference node
(1)
P ~P S  {~ P} T"' (1)
T" (2) (3)
S  {~ P} 
~P P
Any set of clauses that contains the empty clause has a closed semantic tree
62
with one failure node, the empty clause.
 Completeness of Resolution

S = {P( x) , ~ P( x)  Q( f ( x) ) , ~ Q( f ( a ) )}
A = {P(a),Q(a), P(f (a)),Q(f (a)), P(f (f (a))),Q(f (f (a))),...}

P(a) ~P(a)
Failure nodes
Q(f(a)) ~Q(f(a))

Inference nodes
63
 Completeness of Resolution
▪ These observations can be used to prove the
completeness of resolution.
▪ We build a semantic tree for a set of clauses that
are unsatisfiable and we gradually reduce the
tree to one node, corresponding to the empty
clause.
▪ In the same time, we add to the initial set of
clauses the obtained resolvents, thus getting the
proof by resolution.

64
 Resolution strategies
Pure Literal Elimination
▪ A literal occurring in a clause set is pure if and only if it has no instance
that is complementary to an instance of another literal in the clause set. A
clause that contains a pure literal is useless for the purposes of refutation,
since the literal can never be resolved away. Consequently, we can safely
remove such a clause. Removing clauses with pure literals defines a
deletion strategy known as pure-literal elimination.

65
 Resolution strategies
Tautology Elimination
▪ A tautology is a clause containing a pair of complementary literals. For
example, the clause {p(f(a)), ¬p(f(a))} is a tautology. The clause
{p(x), q(y), ¬q(y ), r(z)} also is a tautology, even though it contains
additional literals.
▪ As it turns out, the presence of tautologies in a set of clauses has no effect
on that set's satisfiability. A satisfiable set of clauses remains satisfiable, no
matter what tautologies we add. An unsatisfiable set of clauses remains
unsatisfiable, even if we remove all tautologies. Therefore, we can remove
tautologies from a database, because we need never use them in subsequent
inferences. The corresponding deletion strategy is called tautology
elimination.
▪ Note that the literals in a clause must be exact complements for tautology
elimination to apply. We cannot remove non-identical literals, just because
they are complements under unification. For example, the clauses
{¬p(a), p(x)}, {p(a)}, and {¬p(b)} are unsatisfiable. However, if we were 66
to remove the first clause, the remaining clauses would be satisfiable.
 Resolution strategies
Subsumption Elimination
▪ In subsumption elimination, the deletion criterion depends on a relationship
between two clauses in a database. A clause Φ subsumes a clause Ψ if and
only if there exists a substitution σ such that Φσ ⊆ Ψ. For example,
{p(x), q(y)} subsumes {p(a), q(v), r(w)}, since there is a substitution
{x←a, y←v} that makes the former clause a subset of the latter.
▪ If one member in a set of clauses subsumes another member, then the set
remaining after eliminating the subsumed clause is satisfiable if and only if
the original set is satisfiable. Therefore, subsumed clauses can be
eliminated. Since the resolution process itself can produce tautologies and
subsuming clauses, we need to check for tautologies and subsumptions as
we perform resolutions.

67
 Resolution strategies
Unit Resolution
▪ A unit resolvent is one in which at least one of the parent clauses is a unit clause,
i.e. one containing a single literal. A unit derivation is one in which all derived
clauses are unit resolvents. A unit refutation is a unit derivation of the empty
clause.
▪ Inference procedures based on unit resolution are easy to implement and are usually
quite efficient. It is worth noting that, whenever a clause is resolved with a unit
clause, the conclusion has fewer literals than the parent does. This helps to focus
the search toward producing the empty clause and thereby improves efficiency.
▪ Unfortunately, inference procedures based on unit resolution generally are not
complete. For example, the clauses {p, q}, {¬p, q}, {p ,¬q}, and {¬p, ¬q} are
inconsistent. Using general resolution, it is easy to derive the empty clause.
However, unit resolution fails in this case, since none of the initial clauses contains
just one literal.
▪ On the other hand, if we restrict our attention to Horn clauses (i.e. clauses with at
most one positive literal), the situation is much better. In fact, it can be shown that
there is a unit refutation of a set of Horn clauses if and only if it is unsatisfiable.
68
 Resolution strategies
Input Resolution
▪ An input resolvent is one in which at least one of the two parent clauses is a
member of the initial (i.e., input) database. An input deduction is one in
which all derived clauses are input resolvents. An input refutation is an
input deduction of the empty clause.
▪ It can be shown that unit resolution and input resolution are equivalent in
inferential power in that there is a unit refutation from a set of sentences
whenever there is an input refutation and vice versa.
▪ One consequence of this fact is that input resolution is complete for Horn
clauses but incomplete in general. The unsatisfiable set of clauses {p, q},
{¬p, q}, {p ,¬q}, and {¬p, ¬q} provides an example of a deduction on
which input resolution fails. An input refutation must (in particular) have
one of the parents of {} be a member of the initial database. However, to
produce the empty clause in this case, we must resolve either two single
literal clauses or two clauses having single-literal factors. None of the
members of the base set meet either of these criteria, so there cannot be an69
input refutation for this set.
 Resolution strategies
Linear Resolution
▪ Linear resolution (also called ancestry-filtered resolution) is a slight
generalization of input resolution. A linear resolvent is one in which at
least one of the parents is either in the initial database or is an ancestor of
the other parent. A linear deduction is one in which each derived clause is a
linear resolvent. A linear refutation is a linear deduction of the empty
clause.
▪ Linear resolution takes its name from the linear shape of the proofs it
generates. A linear deduction starts with a clause in the initial database
(called the top clause) and produces a linear chain of resolution. Each
resolvent after the first one is obtained from the last resolvent (called
the near parent) and some other clause (called the far parent). In linear
resolution, the far parent must either be in the initial database or be an
ancestor of the near parent.

70
 Resolution strategies
Linear Resolution - cont
▪ Much of the redundancy in unconstrained resolution derives from the
resolution of intermediate conclusions with other intermediate conclusions.
The advantage of linear resolution is that it avoids many useless inferences
by focusing deduction at each point on the ancestors of each clause and on
the elements of the initial database.
▪ Linear resolution is known to be refutation complete. Furthermore, it is not
necessary to try every clause in the initial database as top clause. It can be
shown that, if a set of clauses Δ is satisfiable and Δ∪{Φ} is unsatisfiable,
then there is a linear refutation with Φ as top clause. So, if we know that a
particular set of clauses is consistent, we need not attempt refutations with
the elements of that set as top clauses.

71
 Resolution strategies
Set of Support Resolution
▪ If we examine resolution traces, we notice that many conclusions come
from resolutions between clauses contained in a portion of the database that
we know to be satisfiable. For example, in many cases, the set of premises
is satisfiable, yet many of the conclusions are obtained by resolving
premises with other premises rather than the negated conclusion As it turns
out, we can eliminate these resolutions without affecting the refutation
completeness of resolution.
▪ A subset Γ of a set Δ is called a set of support for Δ if and only if Δ-Γ is
satisfiable. Given a set of clauses Δ with set of support Γ, a set of support
resolvent is one in which at least one parent is selected from Γ or is a
descendant of Γ. A set of support deduction is one in which each derived
clause is a set of support resolvent. A set of support refutation is a set of
support deduction of the empty clause.

72
 Resolution strategies
Set of Support Resolution - cont
▪ If we examine resolution traces, we notice that many conclusions come
from resolutions between clauses contained in a portion of the database that
we know to be satisfiable. For example, in many cases, the set of premises
is satisfiable, yet many of the conclusions are obtained by resolving
premises with other premises rather than the negated conclusion As it turns
out, we can eliminate these resolutions without affecting the refutation
completeness of resolution.
▪ A subset Γ of a set Δ is called a set of support for Δ if and only if Δ-Γ is
satisfiable. Given a set of clauses Δ with set of support Γ, a set of support
resolvent is one in which at least one parent is selected from Γ or is a
descendant of Γ. A set of support deduction is one in which each derived
clause is a set of support resolvent. A set of support refutation is a set of
support deduction of the empty clause.

73
 13. Automated Theorem Proving
▪ ATP or Automated Reasoning
▪ The International Joint Conference on Automated
Reasoning (IJCAR) is the premier international joint
conference on all topics in automated reasoning
▪ The CADE ATP System Competition
evaluates the performance of sound, fully automatic, ATP systems. The evaluation
is in terms of:
• the number of problems solved,
• the number of problems solved with a solution output, and
• the average runtime for problems solved;
in the context of:
▪ a bounded number of eligible problems, chosen from the TPTP Problem
Library, and
• specified time limits on solution attempts. 74
 Thousands of Problems for Theorem Provers

▪ http://www.tptp.org/ 75
 ATP
▪ There are several ATP systems
▪ One of the most popular is Prover 9

76
 ATP
Metis is an automatic theorem prover for first order logic with equality.
Coded in Standard ML (SML) with an emphasis on keeping the code as
simple as possible.
Compiled using MLton to give respectable performance on
standard benchmarks.
Reads problems in the standard .tptp file format of the TPTP problem set.

ACL2 ("A Computational Logic for Applicative Common Lisp") is a software system
consisting of a programming language, an extensible theory in a first-order logic,
and an automated theorem prover.
ACL2 is designed to support automated reasoning in inductive logical theories,
mostly for the purpose of software and hardware verification.
The input language and implementation of ACL2 are written in Common Lisp.

77
 14. Prover9

▪ Prover9 is an automated theorem prover for

first-order and equational logic
▪ Prover9 is the successor of the Otter prover.
▪ http://www.cs.unm.edu/~mccune/prover9/

78
 Prover9
▪ If you pass the exam and win the lottery you
are happy
▪ If you learn or if you are lucky you pass the
exam
▪ Mike does not learn
▪ Mike is lucky
▪ If you are lucky you win the lottery

▪ Mike is happy
79
 Prover9
▪ pass(x) & wins(x) -> happy(x).
▪ learn(x) | lucky(x) -> pass(x).
▪ - learn(mike).
▪ lucky(mike).
▪ lucky(x) -> wins(x).

▪ happy(mike).

80
1 pass(x) & wins(x) -> happy(x) # label(non_clause). [assumption].
2 learn(x) | lucky(x) -> pass(x) # label(non_clause). [assumption].
3 lucky(x) -> wins(x) # label(non_clause). [assumption].
4 happy(mike) # label(non_clause) # label(goal). [goal].
6 -pass(x) | -wins(x) | happy(x). [clausify(1)].
7 -lucky(x) | pass(x). [clausify(2)].
8 -lucky(x) | wins(x). [clausify(3)].
9 lucky(mike). [assumption].
10 -lucky(x) | -wins(x) | happy(x). [resolve(7,b,6,a)].
11 -wins(mike) | happy(mike). [resolve(10,a,9,a)].
12 -happy(mike). [deny(4)].
13 -wins(mike). [resolve(11,b,12,a)].
14 wins(mike). [resolve(8,a,9,a)].
15 $F. [resolve(13,a,14,a)].

81
 Another example with Factorization

▪ Another example that Resolution is not

complete without factorization

82
 Russell's antinomy
▪ According to naive set theory, any definable collection is
a set.
▪ Let R be the set of all sets that are not members of
themselves.
▪ If R is not a member of itself, then its definition says that it
must contain itself, and if it contains itself, then it contradicts
its own definition as the set of all sets that are not members
of themselves.
▪ This contradiction is Russell's paradox or antinomy
▪ Symbolically:
▪ Let R = {x | x  x}, then R  R  R  R

83
 Factorization – Russell's antinomy
A barber shaves men if and only if they do not shave
themselves. Should the barber shave himself or not?
(A1) ~Shaves(x,x)  Shaves(barber,x)
This is obviously
(A2) Shaves(barber,y)  ~Shaves (y,y) a contradiction
Can we prove it?
(C1) Shaves(x,x)  Shaves (barber,x)
(C2) ~Shaves (barber,y)  ~Shaves (y,y)

(Res1) Shaves(barber,barber)  ~Shaves (barber,barber)

(Res2) ~Shaves (barber,y)  Shaves (barber,y)
Only with resolution we can not prove contradiction
(FC1): Shaves (barber,barber)
(FC2): ~Shaves (barber,barber)
84
However we can do it if we use factorization
 Factorization – Russell's antinomy
Prover 9
-Shaves(x,x) -> Shaves(barber,x).
Shaves(barber,y) -> - Shaves (y,y).

1 -Shaves(x,x) -> Shaves(barber,x) #

label(non_clause). [assumption].
2 Shaves(barber,x) -> -Shaves(x,x) #
label(non_clause). [assumption].
3 Shaves(x,x) | Shaves(barber,x). [clausify(1)].
4 -Shaves(barber,x) | -Shaves(x,x). [clausify(2)].
5 Shaves(barber,barber). [factor(3,a,b)].
6 $F. [factor(4,a,b),unit(a,5)].
85
 Readings
▪ From AIMA
http://aima.cs.berkeley.edu/newchap07.pdf
▪ http://aima.eecs.berkeley.edu/slides-
pdf/chapter07.pdf
▪ http://aima.eecs.berkeley.edu/slides-
pdf/chapter08.pdf
▪ Acknowledgement. The content of the slides in
small font are taken from
http://intrologic.stanford.edu/notes/chapter_12.ht
ml
86