P. 1
Stprtool Pattern Recognition

# Stprtool Pattern Recognition

|Views: 224|Likes:

### Availability:

See more
See less

06/21/2013

pdf

text

original

The classiﬁed object is described by the vector of observations x ∈ X ⊆ Rn

and a

binary hidden state y ∈ {1,2}. The class conditional distributions pX|Y(x|y), y ∈ {1,2}

are known to be multi-variate Gaussian distributions. The parameters (µ1,Σ1) and

(µ2,Σ2) of these class distributions are unknown. However, it is known that the

parameters (µ1,Σ1) belong to a certain ﬁnite set of parameters {(µi

,Σi

):i ∈ I1}.

Similarly (µ2,Σ2) belong to a ﬁnite set {(µi

,Σi

):i ∈ I2}. Let q:X ⊆ Rn

→ {1,2}

17

−1.5

−1

−0.5

0

0.5

1

−0.2

0

0.2

0.4

0.6

0.8

1

1.2

Figure 2.4: Linear classiﬁer based on the Fisher Linear Discriminant.

be a binary linear classiﬁer (2.2) with discriminant function f(x) = w ·x + b. The

probability of misclassiﬁcation is deﬁned as

Err(w,b) = max

i∈I1∪I2

ε(w,b,µi

,Σi

) ,

where ε(w,b,µi

,Σi

) is probability that the Gaussian random vector x with mean

vector µi

and the covariance matrix Σi

satisﬁes q(x) = 1 for i ∈ I2 or q(x) = 2 for

i ∈ I1. In other words, it is the probability that the vector x will be misclassiﬁed by

the linear rule q.

The Generalized Anderson’s task (GAT) is to ﬁnd the parameters (w∗,b∗) of the

linear classiﬁer

q(x) =

1 , for f(x) = w∗ ·x + b∗ ≥ 0 ,
2 , for f(x) = w∗ ·x + b∗ < 0 ,

such that the error Err(w∗,b∗) is minimal

(w∗,b∗) = argmin

w,b

Err(w,b) = argmin

w,b

max

i∈I1∪I2

ε(w,b,µi

,Σi

) . (2.14)

The original Anderson’s task is a special case of (2.14) when |I1| = 1 and |I2| = 1.

The probability ε(w,b,µi

,Σi

) is proportional to the reciprocal of the Mahalanobis

distance ri

between the (µi

,Σi

) and the nearest vector of the separating hyperplane

H = {x ∈Rn

: w·x + b = 0}, i.e.,

ri

= min

x∈H

(µi

x)·(Σi

)1

(µi

x) = w·µi

+ b

w·Σi

w

.

18

The exact relation between the probability ε(w,b,µi

,Σi

) and the corresponding Ma-

halanobis distance ri

is given by the integral

ε(w,b,µi

,Σi

) =

ri

1
2π

e−1

2t2

dt .

(2.15)

The optimization problem (2.14) can be equivalently rewritten as

(w∗,b∗) = argmax

w,b

F(w,b) = argmax

w,b

min

i∈I1∪I2

w·µi

+ b

w·Σi

w

.

which is more suitable for optimization. The objective function F(w,b) is proven to

be convex in the region where the probability of misclassiﬁcation Err(w,b) is less than

0.5. However, the objective function F(w,b) is not diﬀerentiable.

The STPRtool contains implementations of the algorithm solving the original An-

derson’s task as well as implementations of three diﬀerent approaches to solve the

Generalized Anderson’s task which are described bellow. An interactive demo on the

algorithms solving the Generalized Anderson’s task is implemented in demo anderson.

References: The original Anderson’s task was published in [1]. A detailed description

of the Generalized Anderson’s task and all the methods implemented in the STPRtool

is given in book [26].

scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->