FURTHER READING P VALUE

Six Sigma Academy Amsterdam, 2021 ©

Like so many statistical tests, the Kolmogrov Smirnov test too, relies on hypotheses. Hypotheses are
just statements which can be either rejected or not rejected.

In the case of the Kolmogrov Smirnov test, there are two hypotheses, H0 (the so-called null
hypothesis) and H1 (the alternative hypothesis).

• H0: The data is normally distributed.

• H1: The data is not normally distributed.

We can calculate the likelihood of these 2 hypothesis. If we would add the likelihoods up, they have
to add up to 100%. For instance, if the chance that H0 is true is 8%, then by definition the chance
that H1 would be true, would be 92%. After all, it is either H0 or H1 that is valid. There is no third
option. Imagine this, if there is 8% chance that you win the lottery, there must be a 92% chance that
you do not win it, right? Those two options too, have to add up to 100%.

The p value is actually the statistical measure of likelihood that H0 is true. It is a %, but expressed as
a decimal by dividing the % value by 100. For example, p = 0.1 means 10%, p = 0.01 means 1% and p
= 0.001 means 0.1%.

As was mentioned, the p value is the statistical measure of likelihood that H0 is true. It is common in
statistical practice, to only reject the null hypothesis H0, unless there is overwhelming evidence
against it. What is overwhelming evidence? There is no universally agreed threshold, but most
professionals use the following decision rule:

Reject H0 if there is less than 5% chance that it is true. In other words, reject H0 when p< 0.05. Some
go one step further and only reject H0 if there is less than 1% chance that it is true. In other words,
they reject H0 when p< 0.01.

Why 5%? Why not 6% or 4%? Why 1%? Why not 1.1% or 0.9%? That is a good and valid question.
The answer is that it is simply convention. It is simply that these thresholds have become the norm
amongst those who do statistical analysis. As such, they are largely arbitrary values.

In the case of the Kolmogrov-Smirnov test, we would reject H0 (that states that the data is normally
distributed), if p < 0.05. In other words, we would reject H0 and choose H1 (that states that the data
is not normally distributed) when there is less than 5% chance that the data is normally distributed.
We stress again the arbitrariness of this 5% value. We also once again stress that it is convention to
follow that threshold.