Introduction to Classification and Prediction

• Classification
• Dividing up objects so that each is assigned into one of a number of (mutually exhaustive and
exclusive)categories know as Class
• Ex. People who are closely resemble, slightly resemble, not resemble some scene committing a
crime.
• predicts categorical class labels (discrete or nominal)
• classifies data (constructs a model) based on the training set and the values
(class labels) in a classifying attribute and uses it in classifying new data
• Prediction
• models continuous-valued functions, i.e., predicts unknown or missing
values
• Typical applications
• Credit approval
• Target marketing
• Medical diagnosis
• Fraud detection
7/10/22 Data Mining: Concepts and Techniques 1
 Classification—A Two-Step Process
• Model construction: describing a set of predetermined classes
• Each tuple/sample is assumed to belong to a predefined class, as
determined by the class label attribute
• The set of tuples used for model construction is training set
• The model is represented as classification rules, decision trees, or
mathematical formulae
• Model usage: for classifying future or unknown objects
• Estimate accuracy of the model
• The known label of test sample is compared with the classified
result from the model
• Accuracy rate is the percentage of test set samples that are
correctly classified by the model
• Test set is independent of training set, otherwise over-fitting will
occur
• If the accuracy is acceptable, use the model to classify data tuples
whose class labels are not known
7/10/22 Data Mining: Concepts and Techniques 2
 Day Wind Rain Status
Weekday No Yes Late
Weekend Yes No Late
Weekday No No Late
Weekday Yes No Late
 Process (1): Model Construction

Classification
Algorithms
Training
Data

Day Wind Rain Status Classifier

Weekday No No On time (Model)
Weekend No No Late
Weekday No No On time
Weekday Yes Yes Late
IF Day = ‘Weekend’
Weekday Yes No Late
OR Wind= ‘Yes’ OR Rain=‘Yes’
THEN Status = ‘Late’
7/10/22 Data Mining: Concepts and Techniques 4
 Process (2): Using the Model in Prediction
Day Wind Rain Status
Weekday No Yes Late
Weekend Yes No Late
Weekday No No Late
Weekday Yes No Late Classifier

Testing
Data Unseen Data

(Weekday,No,No)
Day Wind Rain
Weekday No Yes Status?
Weekend Yes No
Weekday
Weekday
No
Yes
No
No
Late?
7/10/22 Data Mining: Concepts and Techniques 5
 Supervised vs. Unsupervised Learning

• Supervised learning (classification)

• Supervision: The training data (observations,
measurements, etc.) are accompanied by labels indicating
the class of the observations
• New data is classified based on the training set
• Unsupervised learning (clustering)
• The class labels of training data is unknown
• Given a set of measurements, observations, etc. with the
aim of establishing the existence of classes or clusters in
the data
7/10/22 Data Mining: Concepts and Techniques 6
 Issues regarding Classification and Prediction
• Issues: Data Preparation
• Data cleaning
• Preprocess data in order to reduce noise and handle missing
values
• Relevance analysis (feature selection)
• Remove the irrelevant or redundant attributes
• Data transformation
• Generalize and/or normalize data

7/10/22 Data Mining: Concepts and Techniques 7

 Issues regarding classification and prediction
• Issues: Evaluating Classification Methods
• Accuracy
• classifier accuracy: predicting class label
• predictor accuracy: guessing value of predicted attributes
• Speed
• time to construct the model (training time)
• time to use the model (classification/prediction time)
• Robustness: handling noise and missing values
• Scalability: efficiency in disk-resident databases
• Interpretability
• understanding and insight provided by the model
• Other measures, e.g., goodness of rules, such as decision tree size
7/10/22 or compactness of classification rules
Data Mining: Concepts and Techniques 8
 Distance Measures
• Distance between two instances(b/w two points in n dimentinal
space)
• Three requirements of any distance measure
• Dist(A,A)=0
• Symmetry condition: Dist(A,B)=Dist(B,A)
• Triangular inequality: The shortest distance between any two points is a
straight line-Dist(A,B)<=Dist(A,Z)+Dist(Z,B)
B

A Z
 Distance Measures
• Popular distance measures
• Euclidean Ditance:
• In 2D, training set (a1,a2) unseen
instance (b1,b2) the length
(distance) is:
• ED between 2 points(a1,…,an),
(b1,…,bn) in nD space
• Manhattan Distance(City Block
Distance):
• The CBD between two points (a1,a2)
and (b1,b2) is () +()
• Maximum dimension distance:
• The largest absolute distance
between any pair of corresponding
attribute values
• Ex.(6.2, −7.1, −5.0, 18.3, −3.1, 8.9)
and (8.3,12.4,−4.1, 19.7, −6.2, 12.4)
is: 12.4 − (−7.1) = 19.5.