Instructions: Enter a, c, and e

Confusion Matrix

Condition X

"+"

0.2 a

"-"

0.8 b

Individual Probabilities
P("+")
a
p("-")
b
p(Test POS)
c
p(Test NEG)
d
p(Test POS, "+")
e
p(Test NEG, "+")
f
p(Test "POS, "-")
g
p(Test "NEG", "-")
h

Name
Incidence of Condition "+"
Incidence of "Condition "-"
Classification Incidence "PO
Classification Incidence "NE
True Positives
False Negatives
False Positives
True Negatives

Probability Distributions
P(X)
P(Y)
p(X,Y)
P(X)p(Y)

p(a,b)
p(c,d)
p(e,f,g,h)
p(ac,ad,bc,bd)

Definition of Independence P(X,Y) = P(X)p(Y)

X, Y Independent or Dependent?

Dependent
Conditional Probabilities
p(Test POS | "+")
p(Test NEG | "+")
p(Test POS | "-")
p(Test NEG | "-")

e/a
f/a
g/b
h/b

0.50
0.50
0.25
0.75

p("+" | Test POS)

p( "-" | Test POS)
p("+" | Test NEG)
p("-" | Test NEG)

e/c
g/c
f/d
h/d

0.33
0.67
0.14
0.86

Note that "Relative Entropy" of p and q, written D(p||q

Defined as the summation of all p(i)*log(p(i)/qIi)
is also called "Kullback-Leibler Divergence" (or "KL Div

Test Classification Y
"Positive"
"Negative"
0.3 c
0.7 d
0.1 e

0.1 f

0.2 g

0.6 h

ce of Condition "+"
ce of "Condition "-"
cation Incidence "POS"
cation Incidence "NEG"

H(X)
0.7219

H(Y)

egatives

0.8813

gatives
Name
Probability of the Condition
Probability of the Classification
Joint Distribution of X and Y
Product Distribution of X and Y
= P(X)p(Y)

H(X,Y)
1.5710
0.1
0.06

Name
True Positive Rate
False Negative Rate
False Positive Rate
True Negative Rate

Mutual Information I(X:Y) = R

= e*log(e/ac)
0.0323

H(Y|X)
0.8490

Positive Predictive Value (PPV)

1- PPV
1- NPV
Negative Predictive Value (NPV)

and q, written D(p||q)

p(i)*log(p(i)/qIi)
ergence" (or "KL Divergence" for short)

H(X|Y)
0.6897

INFORMATION METRICS
= a*log(1/a)
+ b*log(1/b)
0.4644
0.2575

= c*log(1/c)
+ d*log(1/d)
0.5211
0.3602

= e*log(1/e)
+ f*log(1/f)
+ g*Log(1/g)
0.3322
0.3322
0.4644
e
ac

0.10 f
0.14 ad

formation I(X:Y) = Relative Entropy of Joint and Product Distributions --- D(p(X
0.0736965594 + f*log(f/ad)
-0.0485426827

(a *H(e/a, f/a))
+
0.2000
1.0000

(c *H(e/c, g/c)
+
0.3000
0.9183

I(X;Y) =
H(X)
- H(X|Y)
0.0323
0.7219
0.6897
I(X;Y) =
H(Y)
- H(Y|X)
0.0323
0.8813
0.8490
I(X;Y) =
H(X)
+ H(Y)
0.0323
0.7219

- H(X,Y)
0.8813
1.5710

+ h*log(1/h)
0.4422
0.20 g
0.24 bc

0.60 h
0.56 bd

roduct Distributions --- D(p(X,Y||p(X)p(Y))

+ g*log(g/bc) -0.05261 + h*log(h/bd)

(b *H(g/b, h/b)
0.8000
0.8113

(d *H(f/d, h/d)
0.7000
0.5917

0.059721

Copyright Daniel Egger/ At

gger/ Attribution 4.0 International (

ational (CC BY 4.0)