Discriminant Analysis

Introduction

• Discriminant Analysis is a technique for analyzing data when the dependent variable
is categorical (say a dependent variable called loyalty having two groups- brand
switchers and brand loyal coded as 0 and 1 or a firm: which may be classified as a
business group or stand-alone) and the independent variables are metric.

• By categorical is meant any non- metric data (nominal or ordinal) while by metric is
meant interval or ratio scale data.

• There are several purposes for Discriminant Analysis

– To classify cases/ objects (individuals, firms, respondents) into groups using a
discriminant prediction equation.
– To assess the relative importance of the independent variables in classifying the
dependent variable.
– To determine the percent of variance in the dependent variable explained by the
independents.
 KitchenAid Survey Results for the Evaluation*
of a New Consumer Product
Purchase Intention Subject X1 X2 X3
Number Durability Performance Style

Group 1
Would purchase 1 8 9 6
2 6 7 5
3 10 6 3
4 9 4 4
5 4 8 2
Group Mean 7.4 6.8 4.0
Group 2
Would not purchase 6 5 4 7
7 3 7 2
8 4 5 5
9 2 4 3
10 2 2 2
Group Mean 3.2 4.4 3.8

Difference between group means 4.2 2.4 0.2

*Evaluations made on a 0 (very poor) to 10 (excellent) rating scale.

 Discriminant Function

Discriminating variables: These are the independent variables, also called predictors.

Criterion variable: This is the dependent variable, also called the grouping variable.

Discriminant function: A discriminant function, is a latent variable which is created as a linear

combination of discriminating (independent) variables, such that:
L = b1x1 + b2x2 + ... + bnxn + c,
where
- L is the discriminant score,
- b's are discriminant coefficients,
- x's are discriminating variables, and
- c is a constant
This is analogous to multiple regression, but the b's are discriminant coefficients or weights
which maximize the distance between the means of the criterion (dependent) variable.

There will always be (n-1) discriminant function for n group discriminant analysis. For
example if a dependent variable has three groups- switchers, undecided and loyal, we will have
two discriminant function. Also for three groups for each respondent we will have two
discriminant scores since we have two discriminant function.
 Assumptions of Discriminant Analysis
Assumption-
1) Multivariate Normality of the Independent Variables

2) Equality of the covariance matrices of the independent variables

across the groups of the dependent variable (Box’s M test should be
insignificant). This test is however influenced by sample size.

3) No Multi-collinearity

4) All the relationships are linear

5) Absence of Outliers.
 Differences Between Multiple Linear Regression, Discriminant Analysis

Multiple Linear Regression Discriminant Analysis

The independent variable can be continuous or The independent variable should be continuous
categorical

The dependent variable is always continuous The dependent variable is always categorical

All the five assumptions of regression are to be

met All the five assumptions of regression are to be met

Only one group no question of unequal group Groups primarily should be of equal sizes
sizes

Objective is to reduce Type -II error Objective is to reduce Type -II error
Continuous Variables- interval / ratio variables
Categorical Variables- nominal, ordinal variable
 Similarities and Differences between ANOVA, Regression,
and Discriminant Analysis

ANOVA REGRESSION DISCRIMINANT ANALYSIS

Similarities
• Number of One One One
dependent
variables
• Number of
independent Multiple Multiple Multiple
variables

Differences
• Nature of the
dependent Metric Metric Categorical
variables
• Nature of the
independent Categorical Metric Metric
variables
 Concept of Centroid
Discriminant score, also called the DA score, is the value resulting from applying
a discriminant function formula to the data for a given case. It is a Z score and is a
standardized value, which is obtained for every object of the study.

The discriminant scores for an object could be calculated as follows-

unstandardized independent (discriminant) variable coefficients * value of the
variable + constant term.

By averaging the discriminant scores for all the individuals within a
particular group, we arrive at a group mean or centroid. When there
are two groups we have two centroids and when there are three groups
we have three centroids.

The centroids indicate the most typical location of any member from a
particular group, and a comparison of the centroids shows how far apart
the groups are in-terms of the discriminant function.
Discriminant Score (L) = -1.505 + .137 Avg tenure + .043 Size of the
Firms. The discriminant score for each employee can be calculated by
putting the values for the average Tenure and Size of the firms for each
respondent.
 Steps of Discriminant Analysis (DA)

1)Wilks' lambda is used to test if the discriminant model as a whole is significant.

• Wilk’s lambda is the ratio of within group sums of squares to total sum of squares. In this
example about 91.4 % of the variance is not explained by group differences.

• A lambda of 1 occurs when observed group means are equal while a small lambda indicates
that the group means appear to differ.

• Wilk’s Lambda is used to test the null hypothesis that the means of all of the discriminating
variables are equal across groups of the dependent variable. If the means of the independent
variables are equal for all groups, the means will not be a useful basis for predicting the
group to which the case belongs, and hence there is no relationship between the
discriminating variables and the dependent variable.

2) If the chi-square test corresponding to the Wilk’s Lambda shows significance, then the
individual independent (discriminating) variables are assessed to see which differ
significantly in mean by group and these are used to classify the dependent variable.
• Eigenvalue, also called the characteristic root of each discriminant function, reflects the
ratio of importance of the dimensions (factors) which classify cases of the
dependent variable.

• For two-group DA, there is one discriminant function and one eigenvalue, which
accounts for 100% of the explained variance.

• Canonical correlation, R*, is a measure of the association between the groups formed
by the dependent and the given discriminant function.

• When R* is zero, there is no relation between the groups and the function. When the
canonical correlation is large, there is a high correlation between the discriminant
functions and the groups. Squared canonical correlation, Rc2, is the percent of variation
in the dependent discriminated by the set of independents in DA or MDA.

• Unstandardized discriminant coefficients are used in the formula for making the

classifications in DA, much as b coefficients are used in regression in making
predictions. The constant plus the sum of products of the unstandardized coefficients
with the observations yields the discriminant scores.
 Standardized discriminant coefficients, also termed the standardized canonical
discriminant function coefficients, are used to compare the relative importance of the
independent variables, much as beta weights are used in regression.

 (Model) Wilks' lambda- A significant lambda means one can reject the null hypothesis that
the groups have the same mean discriminant function scores and conclude the
model is discriminating. 

 (Variable) Wilks' lambda also can be used to test which independents contribute

significantly to the discriminant function. The smaller the variable Wilks' lambda for an
independent variable, the more that variable contributes to the discriminant function.
 Sample Size

The sample size must be large enough to:

– have at least one more observation per group than the
number of independent variables, but striving for at
least 20 cases per group.

– have a large enough sample to divide it into an

estimation and holdout sample, each meeting the
above requirements.
 Assessing Predictive Accuracy
• The classification matrix and hit ratio replace R2 as the measure of model fit.

The classification table, also called a classification matrix or prediction matrix is

used to assess the performance of DA. This is simply a table in which the rows are the
observed categories of the dependent and the columns are the predicted categories of
the dependent.

Percentage Correctly Classified / Hit Ratio – percentage of objects (individuals,

firms, respondents) correctly classified by the discriminant function. Hit ratio =
number of objects in the diagonal / total number of objects.
– t test to determine the level of significance of the classification accuracy:

– The hit ratio must be compared not to zero but to the percent that would
have been correctly classified by chance alone. For two-group discriminant
analysis with a 50-50 split in the dependent variable, the expected percent
is 50%.

– C equal = 1/ number of groups. (when the groups are of equal size)

– Proportional Chance Criterion: