Discriminant functions is a linear combinations of the predictor

(discriminant functions) or independent variables, which will best

discriminate between the categories (Dependent variable).
DISCRIMINANT
It is a predictive model used for classifying objects in a particular group.
ANALYSIS

It is a technique for analyzing data when the criterion or dependent

variable is categorical and the predictor or independent variables are
interval in nature.
 Discriminant Analysis
Examination of whether significant
differences exist among the groups.
Determination of which predictor variables
contribute to most of the intergroup
differences.
Discard variables which contribute little for
group distinction (insignificant variable).
Evaluation of the accuracy of classification.
When the criterion variable has two categories, it is known
as two-group discriminant analysis, with three or more
categories are involved, the technique is referred to as
multiple discriminant analysis.
In the two-group case, you can derive only one discriminant
function. In multiple discriminant analysis, with G groups
and k predictors, it is possible to estimate up to the smaller
of G - 1, or k, discriminant functions.
The first function has the highest ratio of between-groups to
within-groups sum of squares. The second function,
uncorrelated with the first, has the second highest ratio, and
so on. Not all the functions may be statistically significant.
 Discriminant Model Specification

The discriminant analysis model involves linear combinations of the following form:
D = b0 + b1X1 + b2X2 + b3X3 + . . . + bkXk
Where:
D = discriminant score
b 's = discriminant coefficient or weight
X 's = predictor or independent variable
The coefficients, or weights (b), are estimated so that the groups differ as much as
possible on the values of the discriminant function.
This occurs when the ratio of between-group sum of squares to within-group sum of
squares for the discriminant scores is at a maximum.
 Conducting Discriminant Analysis (1)

Membership should be mutually exclusive – You can’t be successful and unsuccessful at the same time.
Absence of outliers
Independent variables can’t be highly correlated - may lead to multicollinearity/ ambiguous measure.
In case of sample size – the smallest group must exceed the number of independent variables.
The null hypothesis that, in the population, the means of all discriminant functions in all groups are equal.
The interpretation of the discriminant weights, or coefficients, is similar to that in multiple regression
analysis.
Independent variables can be included in the model simultaneously or sequentially.

Note: If your data does not comply above stated assumptions – you may consider Logistic Regression.
Discriminant Function Analysis

Nutshell: This model is similar to regression analysis but uses maximum likelihood technique to
classify a object or case in a particular group on the basis of specified cut-off score.
This model establishes and ascertains the relationship between a group of independent variables and
one categorical variable.
Dependent variable: Categorical, can have more than two categories and the coding must be integers.
Independent variables: Continuous
One discriminant function for 2 group discriminant analysis and 2 discriminant function for 3 group
discriminant analysis and so on.
The first function usually is the most powerful differentiating dimension. The subsequent functions
may represent additional dimensions for differentiation.
 Conducting Discriminant Analysis (2)

Identify the objectives, the criterion variable, and the independent variables. The predictor variables should
be selected based on a theoretical model or previous research, or the experience of the researcher.

One part of the sample, called the estimation or analysis sample, is used for estimation and the other part,
called the holdout or validation sample, is reserved for validation.

Often the distribution of the number of cases in the analysis and validation samples follows the distribution
in the total sample.
Assumptions and Significance

Assumptions:
In this model, the predictor variables should have a multivariate normal distribution.
Highly sensitive to outliers.
Test of Significance:
Null Hypothesis: Means of the groups on the discriminant function – the centroids are equal.
Centroids are the mean discriminant score for each group.
Wilk’s λ is used to test for significant differences between groups and varies in between 0 to 1.
Wilk’s λ also tells us the variance of the dependent variable that is not explained by the discriminant
function
Statistics Associated with Discriminant Analysis (1 of 4)

Canonical correlation. Canonical correlation measures the extent of association between the discriminant
scores and the groups. It is a measure of association between the single discriminant function and the set of
dummy variables that define the group membership.

Centroid. The centroid is the mean values for the discriminant scores for a particular group. There are as
many centroids as there are groups, as there is one for each group. The means for a group on all the
functions are the group centroids.

Classification matrix. Sometimes also called confusion or prediction matrix, the classification matrix
contains the number of correctly classified and misclassified cases.
Statistics Associated with Discriminant Analysis (2 of 4)

Unstandardized Discriminant function coefficients. The discriminant function coefficients are the
multipliers of variables, when the variables are in the original units of measurement.
Standardized discriminant function coefficients. Used as the multipliers when the variables have been
standardized to a mean of 0 and a variance of 1.
Discriminant scores. The unstandardized coefficients are multiplied by the values of the variables. These
products are summed and added to the constant term to obtain the discriminant scores.
Eigenvalue. For each discriminant function, the Eigenvalue is the ratio of between-group to within-group
sums of squares. Large Eigenvalues imply superior functions.
Statistics Associated with Discriminant Analysis (3 of 4)

F values and their significance. These are calculated from a one-way ANOVA, with the grouping
variable serving as the categorical independent variable. Each predictor, in turn, serves as the metric
dependent variable in the ANOVA.
Group means and group standard deviations. These are computed for each predictor for each group.
Pooled within-group correlation matrix. The pooled within-group correlation matrix is computed by
averaging the separate covariance matrices for all the groups.
Statistics Associated with Discriminant Analysis (4 of 4)

Structure correlations. Also referred to as discriminant loadings, the structure correlations represent the
simple correlations between the predictors and the discriminant function.

Total correlation matrix. If the cases are treated as if they were from a single sample and the
correlations computed, a total correlation matrix is obtained.

Wilks'λ. Sometimes also called the U statistic, Wilks' λ for each predictor is the ratio of the within-
group sum of squares to the total sum of squares. Its value varies between 0 and 1. Large values of λ
(near 1) indicate that group means do not seem to be different. Small values of λ (near 0) indicate that
the group means seem to be different.
Results of Two-Group Discriminant Analysis

SPSS Output – Descriptive Statistics

Group Means blank blank blank blank blank
Visit Income Travel Vacation Hsize Age
1 60.52000 5.40000 5.80000 4.33333 53.73333
2 41.91333 4.33333 4.06667 2.80000 50.13333
Total 51.21667 4.86667 4.93333 3.56667 51.93333
Group Standard Deviations blank blank blank blank blank
1 9.83065 1.91982 1.82052 1.23443 8.77062
2 7.55115 1.95180 2.05171 0.94112 8.27101
Total 12.79523 1.97804 2.09981 1.33089 8.57395
Pooled Within-Groups Correlation Matrix Income Travel Vacation Hsize Age
INCOME 1.00000 blank blank blank blank
TRAVEL 0.19745 1.00000 blank blank blank
VACATION 0.09148 0.08434 1.00000 blank blank
HSIZE 0.08887 −0.01681 0.07046 1.00000 blank
AGE −0.01431 −0.19709 0.01742 −0.04301 1.00000
Results of Two-Group Discriminant Analysis

Wilks λ (U Statistic) and univariate F ratio with 1 and 28 degrees of freedom

Variable Wilks’ λ F Significance

INCOME 0.45310 33.80 0.0000
TRAVEL 0.92479 2.277 0.1425
VACATION 0.82377 5.990 0.0209
HSIZE 0.65672 14.64 0.0007
AGE 0.95441 1.338 0.2572

Canonical Discriminant Functions

Percent Cumulati Canonical After
Functio Eigenvalu of ve Correlati Functio Wilks’ Chi-
n e Variance Percent on n λ blan
k
Square df Sig.
blank blank blank blank blank : 0 0.3589 26.130 5 0.000
1
1* *Marks the1.7862
1 canonical discriminant functions remaining in the analysis. blank
100.00 100.00 0.8007 : blank blank blank blank
Summary of Canonical Discriminant Functions

Percent of Cumulative Canonical Eigen value describes amount of variance in the dependent variable
Function Eigenvalue Variance Percent Correlation explained by that function. The larger the variance is better. In this
case, the Eigen value is 1.7862 describes 100% variance.
1* 1.7862 100.00 100.00 0.8007 In this case dependent has two categories, there is only one
discriminant function.
After Wilks’ λ Chi-Square df Sig. Canonical correlation measures the association between the
Function discriminant function and the dependent variable. You have to take
0 0.3589 26.130 5 0.0001 the square value of canonical correlation (0.8007) to explain the
percentage of variance explained in the dependent variable.

Wilks λ is a measures how effectively each function separates cases

into groups. Smaller value of Wilks λ indicates greater
discriminatory ability of the function.
Results of Two-Group Discriminant Analysis

blank Func 1
Standard Canonical Discriminant Function Coefficients INCOME 0.74301
This standardized discriminant function coefficients serve the TRAVEL 0.09611
same purpose as β weights in multiple regression. Here, the VACATION 0.23329
coefficients indicate the relative importance of the independent
HSIZE 0.46911
variables in predicting the dependent. Coefficients with large
value indicated greater discriminating ability. AGE 0.20922

Structure Matrix blank Func 1

Pooled within-groups correlations between discriminating variables and INCOME 0.82202
canonical discriminant functions (variables ordered by size of correlation HSIZE 0.54096
within function). This matrix is similar to factor loading in the factor VACATION 0.34607
analysis. Since we have only two groups that is the reason we have only one
TRAVEL 0.21337
function.
AGE 0.16354
Results of Two-Group Discriminant Analysis

Structure Matrix
Discriminant function is a latent variable which has been
created due to the linear combination of independent
variables. blank Func 1
INCOME 0.82202
Statisticians suggest that loading of less than .30 may be
removed from the model. Although it is the prerogative of HSIZE 0.54096
the researcher to do what is the best fit for the model. VACATION 0.34607
TRAVEL 0.21337
Discriminating variables are independent variables. AGE 0.16354

This table shows the Pearson correlations between

independent variables and standardized canonical
discriminant functions.
 Results of Two-Group Discriminant Analysis

blank Func 1
Unstandardized Canonical Discriminant Function Coefficients INCOME 0.8476710E-01
This table contains the unstandardized discriminant function TRAVEL 0.4964455E-01
coefficients. These coefficients should be inserted in the equation VACATION 0.1202813
just like we put the value of coefficients in regression. These values HSIZE 0.4273893
are in other words used in the linear equation to classify cases or
AGE 0.2454380E-01
objects.
(constant) −7.975476
D = -7.975 + 0.847 (income) +0.12(importance attached to
vacation) + 0.427(household size).
Group Func 1
Canonical Discriminant Functions Evaluated at Group Means 1 1.29118
(Group Centroids). + value = likely to visit, - value otherwise
2 −1.29118