ANOVA

Analysis of variance (ANOVA) is an analysis tool used in statistics that splits an observed aggregate
variability found inside a data set into two parts: systematic factors and random factors. The
systematic factors have a statistical influence on the given data set, while the random factors do not.
Analysts use the ANOVA test to determine the influence that independent variables have on the
dependent variable in a regression study.

The t- and z-test methods developed in the 20th century were used for statistical analysis until 1918,
when Ronald Fisher created the analysis of variance method. ANOVA is also called the Fisher
analysis of variance, and it is the extension of the t- and z-tests. The term became well-known in
1925, after appearing in Fisher's book, "Statistical Methods for Research workers.” It was employed
in experimental psychology and later expanded to subjects that were more complex.

What Does the Analysis of Variance Reveal?

The ANOVA test is the initial step in analyzing factors that affect a given data set. Once the test is
finished, an analyst performs additional testing on the methodical factors that measurably contribute
to the data set's inconsistency. The analyst utilizes the ANOVA test results in an f-test to generate
additional data that aligns with the proposed regression models.

The ANOVA test allows a comparison of more than two groups at the same time to determine
whether a relationship exists between them. The result of the ANOVA formula, the F statistic (also
called the F-ratio), allows for the analysis of multiple groups of data to determine the variability
between samples and within samples.

If no real difference exists between the tested groups, which is called the null hypothesis, the result
of the ANOVA's F-ratio statistic will be close to 1. The distribution of all possible values of the F
statistic is the F-distribution. This is actually a group of distribution functions, with two
characteristic numbers, called the numerator degrees of freedom and the denominator degrees of
freedom.

F-Statistics

 ANOVA measures two sources of variation in the data and compares their relative sizes.

 Variation BETWEEN groups:

For each data value look at the difference between its group mean and the overall mean.

 Variation WITHIN groups:

For each data value we look at the difference between that value and the mean of its group.

 The ANOVA F- statistic is a ratio of the between Group Variation divided by the within
Group Variation:

Assumptions of ANOVA
The assumptions of the ANOVA test are the same as the general assumptions for any parametric test:

1. Independence of observations: the data were collected using statistically-valid methods, and
there are no hidden relationships among observations. If your data fail to meet this
assumption because you have a confounding variable that you need to control for statistically,
use an ANOVA with blocking variables.
2. Normally-distributed response variable: The values of the dependent variable follow
a normal distribution.
3. Homogeneity of variance: The variation within each group being compared is similar for
every group. If the variances are different among the groups, then ANOVA probably isn’t the
right fit for the data.

ANOVA Real Life Example

A grocery chain wants to know if three different types of advertisements affect mean sales
differently. They use each type of advertisement at 10 different stores for one month and
measure total sales for each store at the end of the month.

To see if there is a statistically significant difference in mean sales between these three types
of advertisements, researchers can conduct a one-way ANOVA, using “type of
advertisement” as the factor and “sales” as the response variable.

If the overall p-value of the ANOVA is lower than our significance level, then we can
conclude that there is a statistically significant difference in mean sales between the three
types of advertisements. We can then conduct post hoc tests to determine exactly which types
of advertisements lead to significantly different results.

CLASSIFICATION OF ANOVA

 The Analysis of variance is classified into two ways:

1. One-way Classification

2. Two-way classification

One-Way ANOVA

A one-way ANOVA evaluates the impact of a sole factor on a sole response variable. It determines
whether all the samples are the same. The one-way ANOVA is used to determine whether there are
any statistically significant differences between the means of three or more independent (unrelated)
groups. For example, we might want to know if three different studying techniques lead to different
mean exam scores. To see if there is a statistically significant difference in mean exam scores, we can
conduct a one-way ANOVA.
 Limitations of the One Way ANOVA:

A one way ANOVA will tell you that at least two groups were different from each other. But it won’t
tell you which groups were different. If your test returns a significant f-statistic, you may need to run
an ad hoc test (like the Least Significant Difference test) to tell you exactly which groups had
a difference in means.

Two Way ANOVA:

A Two Way ANOVA is an extension of the One Way ANOVA. With a One Way, you have
one independent variable affecting a dependent variable. With a Two Way ANOVA, there are two
independents. Use a two way ANOVA when you have one measurement variable (i.e. a quantitative
variable) and two nominal variables. In other words, if your experiment has a quantitative outcome
and you have two categorical explanatory variables, a two way ANOVA is appropriate.
For example, you might want to find out if there is an interaction between income and gender for
anxiety level at job interviews. The anxiety level is the outcome, or the variable that can be measured.
Gender and Income are the two categorical variables. These categorical variables are also the
independent variables, which are called factors in a Two Way ANOVA.
The factors can be split into levels. In the above example, income level could be split into three levels:
low, middle and high income. Gender could be split into three levels: male, female, and transgender.
Treatment groups are all possible combinations of the factors. In this example there would be 3 x 3 =
9 treatment groups.

Assumptions for Two Way ANOVA:

 The population must be close to a normal distribution.
 Samples must be independent.
 Population variances must be equal (i.e. homoscedastic).
 Groups must have equal sample sizes.

MANOVA
 
When we have multiple or more than two independent variables, we use MANOVA. The main
purpose of the MANOVA test is to find out the effect on dependent/response variables against a
change in the IV. 
 
It answers the following questions:
 
 Does the change in the independent variable significantly affect the dependent variable? 
 What are interactions among the dependent variables?
 What are interactions between independent variables?
 
MANOVA is advantageous as compared to ANOVA because it allows you to test multiple dependent
variables and protects from Type I errors where we ignore a true null hypothesis. 

Example

Consider this example:

Suppose the National Transportation Safety Board (NTSB) wants to examine the safety of compact
cars, midsize cars, and full-size cars. It collects a sample of three for each of the treatments (cars
types). Using the hypothetical data provided below, test whether the mean pressure applied to the
driver’s head during a crash test is equal for each types of car. Use α = 5%.

Table ANOVA.1

Compact cars Midsize cars Full-size cars

http://www.drbrambedkarcollege.ac.in/sites/default/files/Anova.pdf