Previous Example

# Attribute Class
Outlook Temperature Humidity Windy Play
1 sunny hot high no N
2 sunny hot high yes N
3 overcast hot high no P
4 rainy moderate high no P
5 rainy cold normal no P
6 rainy cold normal yes N
7 overcast cold normal yes P
8 sunny moderate high no N
9 sunny cold normal no P
10 rainy moderate normal no P
11 sunny moderate normal yes P
12 overcast moderate high yes P
13 overcast hot normal no P
14 rainy moderate high yes N
A discrete feature can appear only once (or not appear at all)
along the unique path from the root to a leaf.
Can we test on Humidity with a threshold of 95%?
Yes, but with a different feature
5F

x2<5
• The depth of a node is the number of edges from the node
to the tree's root node.
A root node will have a depth of 0.
• The height of a node is the number of edges on the longest
path from the node to a leaf.
A leaf node will have a height of 0
• The height of a tree would be the height of its root node,
or equivalently, the depth of its deepest node.
 Simple Tree
Outlook

sunny rainy
overcast

Humidity P Windy

high normal yes no

N P N P

The size of a decision tree is the number of nodes in the tree.

 Complicated Tree
Temperature

hot
cold moderate

Outlook Outlook Windy

sunny rainy sunny rainy yes no

overcast overcast

P P Windy Windy P Humidity N Humidity

yes no yes no high normal high normal

N P P N Windy P Outlook P

yes no sunny rainy

overcast
N P N P null
Disadvantage of Information Gain: Tends to prefer splits that result in large
number of partitions, each being small but pure.
Consider a collection of n instances that are completely separated by an
attribute A. The the SplitInformation value will be log2(n).
If attribute B that splits the same n instances exactly in half will have
split information of 1
If A and B produce same information gain, then clearly B will score
according to Gain Ratio
Split Information