Concept Learning

(Classification)
Lecture 5: Machine Learning U is the set of all possible objects.

Concept C, where C ⊂ U.

Learning concept C means that given any object X, one

can recognise whether X ∈ C, i.e. whether or not object X
ACSC 368 matches concept description C.

The definition of a match depends on the learning system

Harris Papadopoulos and the language used.

Concept Learning (cont.) The Concept of an Arch

Consequently, the problem of concept learning can be
defined as follows:
Given the training set S of examples, find a formula F
expressed in the concept description such that for all X:
• If X is a positive example in S then X matches F.
• If X is a negative example in S then X does not match F.
Hence we want
X matches F ⇒ X ∈ C
In a ‘blocks world’ the following attributes are expected to
be relevant:
• Parts in the example
• Shape
• Which parts support which other parts
• Which parts touch
We can the describe an arch using a relational or structural
description, using relations from the set:
{has_part, isa, supports, touch}
(Alternative descriptions are possible, e.g. functional)

Learning the Concept ‘Arch’ Hence we take the initial concept:

Using Semantic Nets C1

has_part has_part
p3 has_part
E1 – positive example
p1 p2 supports supports
p1 p3 p2
E1
has_part has_part isa
has_part
isa isa
supports supports rectangle
p1 p3 p2

isa We don’t know:

isa isa • If we need all these relations
rectangle • If we need any other relations

1
 p3 We need to discriminate against E2
E2 – negative example
p1 p2 Concept becomes:

E2 C2
has_part has_part has_part has_part
has_part has_part

p1
supports
p3
supports
p2 p1 supports p3 supports p
2

isa isa
isa isa isa isa
rectangle rectangle

touch must_not_touch

p3 We need to discriminate against E3

E3 – negative example
p1 p2 Concept becomes:

C3
E3 has_part has_part
has_part has_part has_part
has_part
p1 supports p3 must_support p
2
p1 supports p3 p2
isa
isa isa isa
isa isa rectangle
rectangle
must_not_touch

p3 We need to discriminate against E4

E4 – negative example
p1 p2 Concept becomes:

C4
E4 has_part has_part
has_part has_part has_part
has_part
must_support must_support
p1 p3 p2
supports
p1 p3 p2
isa
isa isa isa
isa isa rectangle
rectangle
must_not_touch

2
 p3 We need to generalise to include E5
E5 – positive example
p1 p2 Concept becomes:

E5 C5
has_part has_part has_part has_part
has_part has_part

supports supports must_support must_support p

p1 p3 p2 p1 p3 2

isa isa
isa isa isa stable_poly isa
triangle
rectangle
rectangle
must_not_touch

p3
Will C5 recognise: as an arch? Procedure for Learning
p1 p2
To learn a concept C from a given sequence of examples
p3 E1, E2, …, En (where E1 must be a positive example of C),
How about: do:
p1 p2 1. Adopt E1 as the initial hypothesis concept C1.
2. Process all the remaining examples comparing each one
p3 with the current concept and modifying the concept
How about: accordingly. Discriminate against negative examples,
p1 p2 and generalise from positive examples.

How many examples will we need to learn from until we

recognise any reasonable arch?

Discrimination Rules
Generalisation Rule
(Specialisation)
If negative example and If positive example and
example contains relation R which is not in current Corresponding objects in example and concept can be
concept generalised according to taxonomic hierarchy (with isa
then relation)
forbid R in concept (add must_not_R) then
replace object in concept by generalisation
If negative example and
example does not contain relation R which is present in
current concept
then
require R in concept (add must_R)

3
 Example Taxonomic Hierarchy Prolog Representation of a Concept

polygon We represent the concept learned so far by a data structure

of the form:
isa isa
concept([p1, p2, p3], MustRels, Relations, MustNotRels).
convex_poly concave_poly
where
isa isa MustRels is a list of relations which must hold for a
positive example
stable_poly unstable_poly
MustNotRels is a list of relations which must not hold for
isa isa a positive example
isa isa
Relations is a list of relations which my become Musts or
circle may become unnecessary
triangle rectangle trapezium

Example Concept Representation Prolog Representation of Examples

For example, the concept C3 would be represented as: We represent an example by a data structure of the form:
concept([p1, p2, p3], example((Type, object(PartsList, RelationsList))).
[supports(p2, p3)],
e.g. we would represent E2 as follows:
[supports(p1, p3),
isa(p1, rectangle), example((neg, object([A, B, C],
isa(p2, rectangle), [supports(A, C), supports(B, C),
isa(p3, rectangle)], isa(A, rectangle), isa(B, rectangle),
[touch(p1, p2)]). isa(C, rectangle), touch(A, B)]))).

Another Example Difference

The positive example E5 would be represented as follows:
example((pos, object([A, B, C],
[supports(A, C), supports(B, C),
isa(A, rectangle), isa(B, rectangle),
isa(C, triangle)]))).
We represent the difference between an example and the
current concept by a data structure of the form:
diff(Missing, Extras).
where
Missing is a list of relations which are in the relations of
the concept, but not in the relations of the example.
Extras is a list of relations which are in the relations of the
example, but not in the relations of the concept.

4
Example Difference Representations
For example, the difference between the example E2 and
the concept C1 would be represented as:
diff([], [touch(A, B)]).
and the difference between the example E3 and the concept
C2 would be represented as:
diff([supports(B, C)], []).