Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword or section
Like this
18Activity
P. 1
Machine Learning, Neural Networks and Statistical Classification

Machine Learning, Neural Networks and Statistical Classification

Ratings: (0)|Views: 373|Likes:
Published by sumonedu

More info:

Published by: sumonedu on Jul 11, 2009
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

07/03/2013

pdf

text

original

 
Machine Learning, Neural and StatisticalClassification
Editors: D. Michie, D.J. Spiegelhalter, C.C. TaylorFebruary 17, 1994
 
Contents
1 Introduction 1
1.1 INTRODUCTION
 ¡ ¡ ¢¡¡£ ¤ £ ¡ ¢¡¡¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
11.2 CLASSIFICATION
 ¡ ¢ ¡ ¡ £¤£ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £¡¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
11.3 PERSPECTIVES ON CLASSIFICATION
¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
21.3.1 Statistical approaches
 ¤ £¡¢ ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢ ¡¡¢ ¡ ¡ ¡ ¢ 
21.3.2 Machine learning
¡£ ¤ £ ¡ ¢¡¡¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
21.3.3 Neural networks
 ¡ ¡ £ ¤ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
31.3.4 Conclusions
 ¡ ¢ ¡ ¡ £¤£ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £¡¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
31.4 THE STATLOG PROJECT
 £¤£ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £¡¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
41.4.1 Quality control
¢ ¡ ¡ £ ¤ £¡¢ ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢¡¡¢ ¡ ¡ ¡ ¢ 
41.4.2 Caution in the interpretations of comparisons
 £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
41.5 THE STRUCTURE OF THIS VOLUME
¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
5
2 Classication 6
2.1 DEFINITION OF CLASSIFICATION
 ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
62.1.1 Rationale
 ¢ ¡ ¡ ¢ ¡ ¡ £ ¤ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
62.1.2 Issues
 ¡ ¢¡¡¢ ¡ ¡ £ ¤ £¡¢ ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢¡¡¢ ¡ ¡ ¡ ¢ 
72.1.3 Class definitions
 ¡ ¡ £ ¤ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
82.1.4 Accuracy
 ¢ ¡ ¡ ¢ ¡ ¡ £ ¤ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
82.2 EXAMPLES OF CLASSIFIERS
 £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢¡ ¡ ¢ ¡ ¡ ¡ ¢ 
82.2.1 Fisher’s linear discriminants
 ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £¡¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
92.2.2 Decision tree and Rule-based methods
¡¡¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
92.2.3 k-Nearest-Neighbour
 £¤£ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £¡¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
102.3 CHOICE OF VARIABLES
¡£ ¤ £ ¡ ¢¡¡¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
112.3.1 Transformations and combinations of variables
 ¡ ¢¡¡¢ ¡ ¡ ¡ ¢ 
112.4 CLASSIFICATION OF CLASSIFICATION PROCEDURES
 ¡ ¢ ¡ ¡ ¡ ¢ 
122.4.1 Extensions to linear discrimination
¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
122.4.2 Decision trees and Rule-based methods
 ¡ ¢ ¡ £ ¡ ¢¡¡¢ ¡ ¡ ¡ ¢ 
12
 
ii [Ch. 0
2.4.3 Density estimates
¡£ ¤ £ ¡ ¢¡¡¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
122.5 A GENERAL STRUCTURE FOR CLASSIFICATION PROBLEMS
 ¡ ¢ 
122.5.1 Prior probabilities and the Default rule
¡¡¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
132.5.2 Separating classes
¡£ ¤ £ ¡ ¢¡¡¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
132.5.3 Misclassification costs
 ¤ £¡¢ ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢ ¡¡¢ ¡ ¡ ¡ ¢ 
132.6 BAYESRULE GIVEN DATA
¥ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢¡ ¡ ¢ ¡ ¡ ¡ ¢ 
142.6.1 Bayes rule in statistics
 ¤ £¡¢ ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢ ¡¡¢ ¡ ¡ ¡ ¢ 
152.7 REFERENCE TEXTS
¢ ¡ ¡ £ ¤ £¡¢ ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢¡¡¢ ¡ ¡ ¡ ¢ 
16
3 Classical Statistical Methods 17
3.1 INTRODUCTION
 ¡ ¡ ¢¡¡£ ¤ £ ¡ ¢¡¡¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
173.2 LINEAR DISCRIMINANTS
 £¤£ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £¡¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
173.2.1 Linear discriminants by least squares
¡¡¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
183.2.2 Special case of two classes
 ¡ ¢¡¡¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
203.2.3 Linear discriminants by maximum likelihood
 £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
203.2.4 More than two classes
 ¤ £¡¢ ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢ ¡¡¢ ¡ ¡ ¡ ¢ 
213.3 QUADRATIC DISCRIMINANT
 £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢¡ ¡ ¢ ¡ ¡ ¡ ¢ 
223.3.1 Quadratic discriminant - programming details
 £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
223.3.2 Regularisation and smoothed estimates
 ¡ ¢ ¡ £ ¡ ¢¡¡¢ ¡ ¡ ¡ ¢ 
233.3.3 Choice of regularisation parameters
 ¢ ¡ ¡ ¢ ¡ £¡¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
233.4 LOGISTIC DISCRIMINANT
 ¤ £¡¢ ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢ ¡¡¢ ¡ ¡ ¡ ¢ 
243.4.1 Logistic discriminant - programming details
 ¡ £¡¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
253.5 BAYESRULES
 ¢ ¡ ¡ ¢ ¡ ¡ £ ¤ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
273.6 EXAMPLE
 £¡¡¢ ¡ ¡ ¢¡ ¡ £¤ £ ¡ ¢¡¡¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
273.6.1 Linear discriminant
 £¤£ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £¡¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
273.6.2 Logistic discriminant
 £¤£ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £¡¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
273.6.3 Quadratic discriminant
 ¤ £¡¢ ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢ ¡¡¢ ¡ ¡ ¡ ¢ 
27
4 Modern Statistical Techniques 29
4.1 INTRODUCTION
 ¡ ¡ ¢¡¡£ ¤ £ ¡ ¢¡¡¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
294.2 DENSITY ESTIMATION
¡£ ¤ £ ¡ ¢¡¡¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
304.2.1 Example
 ¢ ¡ ¡ ¢ ¡ ¡ £ ¤ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
334.3
¦
-NEAREST NEIGHBOUR
 ¤ £¡¢ ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢ ¡¡¢ ¡ ¡ ¡ ¢ 
354.3.1 Example
 ¢ ¡ ¡ ¢ ¡ ¡ £ ¤ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
364.4 PROJECTION PURSUIT CLASSIFICATION
¡¡¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
374.4.1 Example
 ¢ ¡ ¡ ¢ ¡ ¡ £ ¤ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
394.5 NAIVE BAYES
 ¢ ¡ ¡ ¢ ¡ ¡ £ ¤ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
404.6 CAUSAL NETWORKS
 ¡ ¡ £ ¤ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
414.6.1 Example
 ¢ ¡ ¡ ¢ ¡ ¡ £ ¤ £ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £ ¡ ¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
454.7 OTHER RECENT APPROACHES
 ¡ ¢¡¡¡¢ ¡ ¡ ¢¡£ ¡ ¢ ¡ ¡ ¢¡¡¡¢ 
464.7.1 ACE
 ¡ ¡ ¢ ¡ ¡ ¢ ¡ ¡ £ ¤£ ¡ ¢ ¡ ¡ ¡ ¢ ¡ ¡ ¢ ¡ £¡¢ ¡ ¡ ¢ ¡ ¡ ¡ ¢ 
464.7.2 MARS
 ¡ ¢¡¡¢ ¡ ¡ £ ¤ £¡¢ ¡ ¡ ¡ ¢¡¡¢ ¡ £ ¡ ¢¡¡¢ ¡ ¡ ¡ ¢ 
47

Activity (18)

You've already reviewed this. Edit your review.
1 hundred reads
1 thousand reads
jameel11 liked this
bhaktamelt liked this
iky77 liked this
kemalihsan_17959 liked this
Cezara Pastrav liked this
cuadra7 liked this
lopez_fj9228 liked this
kkaragul liked this

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->