P. 1
Stprtool Pattern Recognition

Stprtool Pattern Recognition

|Views: 224|Likes:
Published by John Soldera

More info:

Published by: John Soldera on Nov 29, 2011
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

06/21/2013

pdf

text

original

The classified object is described by the vector of observations x ∈ X ⊆ Rn

and a

binary hidden state y ∈ {1,2}. The class conditional distributions pX|Y(x|y), y ∈ {1,2}

are known to be multi-variate Gaussian distributions. The parameters (µ1,Σ1) and

(µ2,Σ2) of these class distributions are unknown. However, it is known that the

parameters (µ1,Σ1) belong to a certain finite set of parameters {(µi

,Σi

):i ∈ I1}.

Similarly (µ2,Σ2) belong to a finite set {(µi

,Σi

):i ∈ I2}. Let q:X ⊆ Rn

→ {1,2}

17

−1.5

−1

−0.5

0

0.5

1

−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Figure 2.4: Linear classifier based on the Fisher Linear Discriminant.

be a binary linear classifier (2.2) with discriminant function f(x) = w ·x + b. The

probability of misclassification is defined as

Err(w,b) = max

i∈I1∪I2

ε(w,b,µi

,Σi

) ,

where ε(w,b,µi

,Σi

) is probability that the Gaussian random vector x with mean

vector µi

and the covariance matrix Σi

satisfies q(x) = 1 for i ∈ I2 or q(x) = 2 for

i ∈ I1. In other words, it is the probability that the vector x will be misclassified by

the linear rule q.

The Generalized Anderson’s task (GAT) is to find the parameters (w∗,b∗) of the

linear classifier

q(x) =

1 , for f(x) = w∗ ·x + b∗ ≥ 0 ,
2 , for f(x) = w∗ ·x + b∗ < 0 ,

such that the error Err(w∗,b∗) is minimal

(w∗,b∗) = argmin

w,b

Err(w,b) = argmin

w,b

max

i∈I1∪I2

ε(w,b,µi

,Σi

) . (2.14)

The original Anderson’s task is a special case of (2.14) when |I1| = 1 and |I2| = 1.

The probability ε(w,b,µi

,Σi

) is proportional to the reciprocal of the Mahalanobis

distance ri

between the (µi

,Σi

) and the nearest vector of the separating hyperplane

H = {x ∈Rn

: w·x + b = 0}, i.e.,

ri

= min

x∈H

(µi

x)·(Σi

)1

(µi

x) = w·µi

+ b

w·Σi

w

.

18

The exact relation between the probability ε(w,b,µi

,Σi

) and the corresponding Ma-

halanobis distance ri

is given by the integral

ε(w,b,µi

,Σi

) =

ri

1
2π

e−1

2t2

dt .

(2.15)

The optimization problem (2.14) can be equivalently rewritten as

(w∗,b∗) = argmax

w,b

F(w,b) = argmax

w,b

min

i∈I1∪I2

w·µi

+ b

w·Σi

w

.

which is more suitable for optimization. The objective function F(w,b) is proven to

be convex in the region where the probability of misclassification Err(w,b) is less than

0.5. However, the objective function F(w,b) is not differentiable.

The STPRtool contains implementations of the algorithm solving the original An-

derson’s task as well as implementations of three different approaches to solve the

Generalized Anderson’s task which are described bellow. An interactive demo on the

algorithms solving the Generalized Anderson’s task is implemented in demo anderson.

References: The original Anderson’s task was published in [1]. A detailed description

of the Generalized Anderson’s task and all the methods implemented in the STPRtool

is given in book [26].

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->