Knowledge Engineering

(IT4362)

Quang Nhat Nguyen

(quang.nguyennhat@hust.edu.vn)

Hanoi University of Science and Technology

School of Information and Communication Technology
Academic year 2021-2022
Content
◼ Introduction
◼ First-order logic
◼ Knowledge representation
◼ Logic programming
◼ Expert systems
◼ Uncertain reasoning
◼ Knowledge discovery by Machine learning
◼ Knowledge discovery by Data mining

Knowledge Engineering 2
Expert systems – Introduction
◼ Expert Systems (ESs) are computer programs that try to
replicate knowledge and skills of human experts in some area,
and then solve problems in this area (as the way human experts
would)
◼ ESs achieves a high level of performance in task areas that, for
non-expert persons, require years of special education and
training
◼ ESs take their roots in Cognitive Science – the study of human
mind using combination of AI and Psychology
◼ ESs were the first successful applications of AI in solving real–
world problems in medicine, chemistry, finance, etc.
◼ In business, ESs help many companies to save millions $

Knowledge Engineering 3
Brief history of ESs
◼ 1943: Post, E. L. proved that any computable problem can be
solved using a set of IF–THEN rules
◼ 1961: General Problem Solver (GPS), by A. Newell and H.
Simon, that could solve many different problems using only a
small set of rules
◼ 1969: DENDRAL (Feigenbaum, Buchanan, Lederberg) was the
first system that showed the importance of domain–specific
knowledge (expertise)
◼ 1970s: MYCIN (Buchanan & Shortliffe) medical diagnosis
system introduced the use of certainty factors
◼ 1982: R1 (a.k.a. XCON) by McDermott was the first commercial
ES (by 1986 it was saving DEC $40 millions p.a.)

Knowledge Engineering 4
Expertise and Expert
◼ Expertise is a specialized type of knowledge and skill that
experts have in some domain
◼ Expertise must be extracted and made explicit so that it can be
encoded in an ES
◼ Expertise requires long time to learn (e.g., it takes 6 years to
become a doctor)
◼ Expertise is easily recalled to solve problems in the domain (of
the expertise)
◼ Expertise enables to find good (high-quality) solutions
◼ A domain expert
❑ A person who has deep knowledge and strong practical experience
in the domain – who can do things that other people cannot
Knowledge Engineering 5
Characteristics of an ES
◼ High quality performance (result/solution)
❑ An ES is built to perform at a human expert level in a narrow,
specialized domain
◼ The speed of reaching a solution
❑ A (very) accurate decision or diagnosis may not be useful if it is too
late to apply
◼ ESs apply heuristics to guide the reasoning, and thus reduce
the search area for a solution
◼ An important feature of an ES is its explanation capability
❑ Enable the ES to review its own reasoning and explain its decisions
◼ ESs employ symbolic reasoning when solving a problem
❑ Symbols are used to represent different types of knowledge, such
as facts, concepts and rules
Knowledge Engineering 6
Structure of an ES (1)

(http://www.se.cuhk.edu.
hk/~seg7450/)

Knowledge Engineering 7
Structure of an ES (2)
◼ Development environment
❑ The part of an ES that is used by the builders (of the ES)
❑ Include the knowledge base, the inference engine, knowledge
acquisition, and improving reasoning capability
❑ The knowledge engineer and the expert are considered part of this
environment

◼ Consultation environment
❑ The part of an ES that is used by a non-expert to obtain expert
knowledge and advice
❑ Includes the workplace, inference engine, explanation facility,
recommended action, and user interface

Knowledge Engineering 8
Structure of an ES (3)
◼ Three major components in an ES are:
❑ Knowledge base
❑ Inference engine
❑ User interface

◼ An ES may also contain:

❑ Knowledge acquisition subsystem
❑ Blackboard (workplace)
❑ Explanation subsystem (justifier)
❑ Knowledge refining subsystem

Knowledge Engineering 9
Structure of an ES (4)
◼ Knowledge acquisition
❑ The extraction and formulation of knowledge derived from various
sources, especially from experts

◼ Knowledge base
❑ The assembly of all the information and knowledge about a specific
field of interest
❑ Contains the domain knowledge useful for problem solving
❑ E.g., In a rule-based ES, the knowledge base contains a set of rules

◼ Inference engine
❑ The part of an expert system that actually performs the reasoning
function

Knowledge Engineering 10
Structure of an ES (5)
◼ User interfaces
❑ To interact with users
❑ Accept input commands from the user
❑ Display the results generated by other parts of the systems
◼ Blackboard (workplace)
❑ An area of working memory set aside for the description of the
current problem and for recording intermediate results in an ES
◼ Explanation subsystem (justifier)
❑ The component of an ES that can explain the system’s reasoning
and justify its conclusions
◼ Knowledge-refining subsystem
❑ A subsystem that has the ability to analyze its own performance,
learn, and improve itself for future consultations
Knowledge Engineering 11
Main application areas of ESs (1)
◼ Interpretation: Drawing high-level conclusions
(explanations) based on observed data
◼ Configuration (Design): Assembling proper components
of a system in the proper way
◼ Diagnosis: Determining the cause of malfunctions,
disease, etc. based on observed evidence
◼ Repair (Remedy): Prescribing and implementing
remedies for a problem
◼ Decision support: Helping (supporting) the users in their
decision making (e.g., selection decision)

Knowledge Engineering 12
Main application areas of ESs (2)
◼ Prediction (Prognosis): Projecting probable outcomes of
a given situation
◼ Planning: Proposing a series of actions to achieve a goal
◼ Monitoring: Comparing observed behavior to the
expected behavior to judge performance
◼ Instruction: Intelligent teaching so that a student can ask
Why, How, and What-If questions – just as if a human was
teaching

Knowledge Engineering 13
Examples of ESs
◼ Chemical and biological ESs
❑ CRYSALIS, DENDRAL, TQMSTUNE, CLONER, MOLGEN, SECS , SPEX
◼ Electronic ESs
❑ ACE, IN-ATE, NDS, EURISKO, PALLADIO, REDESIGN, CADHELP,
SOPHIE
◼ Medical ESs
❑ PUFF, VM, ABEL, AI/COAG, AI/RHEUM, CADUCEUS, ANNA, BLUE BOX,
MYCIN, CONCOCIN, ATTENDING, GUIDON
◼ Industrial/Engineering ESs
❑ REACTOR, DELTA, STEAMER
◼ Geological ESs
❑ DIPMETTER, LITHO, MUD, PROSPECTOR
◼ Computer ESs
❑ PTRANS, BDS, XCON, XSEL, XSITE, YES/MVS, TIMM
Knowledge Engineering 14
DENDRAL
◼ DENDRAL (Feigenbaum et al, 1969) was an ES that used rules
to infer molecular structure from spectral information
◼ The challenge was that the number of possible molecules was
so large that it was impossible to check all of them using simple
rules (weak method)
❑ A weak method is the one that can be applied to a broad range of
problems
◼ The researchers consulted experts in chemistry and added
several more specific rules to their program. The number of
combinations the program had to test was reduced dramatically
◼ DENDRAL demonstrated the importance of the domain–specific
knowledge

Knowledge Engineering 15
MYCIN
◼ Designed as a consultant for physicians
◼ An interactive ES that diagnoses certain infectious
diseases, prescribes anti-microbial therapy, and can
explain its reasoning in detail
◼ In a controlled test, its performance equaled that of
specialists
◼ MYCIN was given the ability to explain both its line of
reasoning and its knowledge
◼ Although MYCIN was never used routinely by physicians,
it has substantially influenced other artificial intelligence
(AI) research

Knowledge Engineering 16
Advantages of ESs
◼ Increased productivity – find solutions much faster than
(non-expert) humans
◼ Availability of expertise (human experts can be at one
place at a time) – the expertise can be used in many
places, in many situations
◼ Can be used in dangerous environments (e.g. in space)
◼ In ESs, knowledge is separated from its processing (the
knowledge base and the inference engine are split up)
❑ Allow either of them to be updated separately

◼ Explanation capability

Knowledge Engineering 17
Disadvantages of ESs
◼ Difficulty in engineering, especially acquiring the expertise
◼ Mistrust (i.e., being suspected) by the users
◼ Effective only in specific areas (areas of expertise)
❑ Developed to solve problems in a specific domain

◼ In general, ESs do not have an ability to learn from the

experience
❑ An ES cannot automatically modify its knowledge base
❑ The knowledge engineer is still responsible for revising and
maintaining the system

Knowledge Engineering 18
Certainty factor – Introduction
◼ Sources of uncertainty in rules
❑ The uncertainties of individual rules
❑ Uncertainty with the likelihood of the consequent
❑ Another source of uncertainty is the combining of the evidence
◼ Should the evidence be combined as (E1 AND E2 AND E3) or as
(E1 AND E2 OR E3)?

◼ The goal of the knowledge engineer is to minimize or

eliminate these uncertainties
◼ Handling uncertainty in rules by using certainty factor
(a.k.a. confidence factor)

Knowledge Engineering 19
Belief and disbelief
◼ The fundamental problem:
❑ P(H | E) implies a cause-and-effect relationship between E and H

◼ Certainty factors – representing uncertainty

❑ Ordinary probability
◼ Associated with the frequency of reproducible events

❑ Probability representing the degree of confirmation

◼ It confirms a hypothesis based on some evidence (the degree
of likelihood of a belief)

Knowledge Engineering 20
Measures of belief and disbelief (1)
◼ In MYCIN, the degree of confirmation was originally
defined as the certainty factor
❑ The difference between belief and disbelief:
CF(H,E) = MB(H,E) - MD(H,E); where
◼ CF: the certainty factor in the hypothesis H due to evidence E

◼ MB: the measure of increased belief in H due to E

◼ MD: the measure of increased disbelief in H due to E

◼ Certainty factor
❑ A way of combining belief and disbelief into a single number
❑ Can be used to rank hypotheses in order of importance

Knowledge Engineering 21
Measures of belief and disbelief (2)
◼ The measures of belief and disbelief were defined in terms
of probabilities by:
 1
 max{P( H | E ), P( H )} − P( H ) if P( H ) = 1
MB( H , E ) =  ;
 otherwise
1 − P( H )

 1
 P( H ) − min{P( H | E ), P( H )} if P( H ) = 0
MD( H , E ) =  ;
 otherwise
P( H )

Knowledge Engineering 22
Characteristics of MB, MD, CF
◼ Ranges
❑ 0  MB  1
❑ 0  MD  1
❑ -1  CF  1
◼ Certain true hypothesis: P(H|E) = 1
❑ MB = 1
❑ MD = 0
❑ CF = 1
◼ Certain false hypothesis: P(H’|E) = 1
❑ MB = 0
❑ MD = 1
❑ CF = -1
◼ Lack of evidence: P(H|E) = P(H)
❑ MB = 0
❑ MD = 0
❑ CF = 0
Knowledge Engineering 23
Measures of belief and disbelief (3)
◼ The certainty factor, CF, indicates the net belief in a hypothesis
based on some evidence
❑ Positive CF: The evidence supports the hypothesis since MB > MD
❑ CF=1: The evidence definitely proves the hypothesis
❑ CF=0: Either of two possibilities
◼ CF = MB - MD = 0 could mean that both MB and MD are 0 (i.e.,
there is no evidence)
◼ The second possibility is that MB = MD and both are nonzero
(i.e., the belief is equal to the disbelief)
❑ Negative CF: The evidence favors the negation of the hypothesis
since MB < MD (i.e., there is more reason to disbelieve a
hypothesis than to believe it)
◼ With certainty factors there are no constraints (importance) on the
individual values of MB and MD. Only the difference is important
Knowledge Engineering 24
Measures of belief and disbelief (4)
◼ Certainty factors allow an expert to express a belief
without committing a value to the disbelief
CF(H,E) + CF(H',E) = 0

◼ For the example of the student graduating if an A mark

(grade) is given in the course
❑ CF(H,E) = 0.70 and CF (H’, E) = - 0.70
❑ I am 70 percent certain that I will graduate if I get an A
mark in this course
❑ I am -70 percent certain that I will not graduate if I get
an A mark in this course

Knowledge Engineering 25
Combining CFs within a rule
◼ For conjunctive premises:
❑ CF(P1 and P2) = MIN(CF(P1), CF(P2))

◼ For disjunctive premises:

❑ CF(P1 or P2) = MAX(CF(P1), CF(P2))

◼ Example:
(P1 and P2) or P3  R1(0.7) and R2(0.3)
❑ CF(P1(0.6) and P2(0.4)) = MIN(0.6, 0.4) = 0.4
❑ CF((0.4) or P3(0.2)) = MAX(0.4, 0.2) = 0.4

❑ CF for R1 = 0.7 × 0.4 = 0.28

❑ CF for R2 = 0.3 × 0.4 = 0.12

Knowledge Engineering 26
Combining CFs from different rules
◼ Assume that two rules support the same result R
◼ Let R1 represent the result R from one rule and R2 the
result R from another rule
◼ One of the following situations applies:
❑ If CF(R1) and CF(R2) are both positive, then:

CF(R) = CF(R1) + CF(R2) − CF(R1) × CF(R2)

❑ If CF(R1) and CF(R2) are both negative, then:
CF(R) = CF(R1) + CF(R2) + CF(R1) × CF(R2)
❑ Otherwise:
CF ( R1 ) + CF ( R2 )
CF ( R ) =
1 − MIN{| CF ( R1 ) |, | CF ( R1 ) |}
Knowledge Engineering 27
Problem: CFs and Probabilities
◼ Suppose that
P(d1) = 0.8; P(d1 | e) = 0.9
P(d2) = 0.2; P(d2 | e) = 0.8
◼ The increase in belief in d1 is:
P(d1 | e) − P(d1 ) 0.9 − 0.8
CF(d1,e) = MB(d1,e) – 0 = = = 0.5
1 − P(d1 ) 1 − 0.8
◼ The increase in belief in d2 is:
P(d 2 | e) − P(d 2 ) 0.8 − 0.2
CF(d2,e) = MB(d2,e) – 0 = = = 0.75
1 − P(d 2 ) 1 − 0.2
◼ Thus CF(d1,e) < CF(d2,e) even though P(d1|e) > P(d2|e)
❑ CF values could be the opposite of conditional probabilities!
❑ Since one purpose of CF is to rank hypotheses in terms of likely hypothesis,
it is a contradiction for a hypothesis to have a higher conditional probability
P(H|E) and yet have a lower certainty factor CF(H,E)
Knowledge Engineering 28