Two types of error

When making a decision to reject or not reject H0, there are two types of error we wish to
avoid:

Type I error: rejecting H0 when it is true.

Type II error: not rejecting H0 when it is false (rejecting H1 when it is true).

Balancing the risks of these two types of error is the basis for calculating the dividing line
between accepting or rejecting H0. It is analogous to a court decision, where a defendant is
on trial. The null hypothesis is innocent (presumption of innocence). The two errors are:

Type I: convicting an innocent person

Type II: letting a guilty person go free.

If the jury is convinced ‘beyond all reasonable doubt’ that the defendant is guilty then he is
convicted. The ‘beyond all reasonable doubt’ criterion is there to avoid type I errors if
possible, even though it might lead to the occasional type II error. Twelve jurors decide on
the case and have to reach a unanimous decision. If they disagree, the judge will sometimes
accept a majority (10-2) verdict. This presumably raises the risk of a type I error while
lowering the risk of a type II error.

The punishment can influence the balance of error probabilities. In civil cases (e.g. a libel
action) where no criminal activity is involved, the jury decides ‘on the balance of
probabilities’. The defendant, if convicted, would not have a criminal record, nor go to jail,
so the court is more willing to make a type I error and reduce the probability of a type II
error.

Similarly, in a criminal case, if the punishment is the death penalty the jury may be less
willing to convict since the consequences of a type I error are catastrophic...

We can summarise the decision making situation as follows:

True situation
Decision H0 true H0 false
Accept H0 Correct decision Type II error
Reject H0 Type I error Correct decision

Type I and Type II errors in hypothesis testing

Suppose we test the hypothesis

H0:  = 100 versus

H1:  < 100

and let us suppose that x ~ N(, 2/n). The distributions of x under H0 and H1 are shown
below:
 H1 H0

I II
_
xD 100 x

Rejection region Non-rejection region

If the decision line is set at x D then areas I and II show the probabilities of making type I and
type II errors respectively. (Area I gives the area under H0 to the left of x D, which would
imply falsely rejecting H0.) Moving the line left (right) means reducing (increasing) the
probability of making a type I error and reducing (increasing) the probability of a type II
error. The optimal position of x D comes from balancing the two error probabilities (and the
costs of the two types of error).

The optimal position of x D is difficult (if not impossible) to establish, since the precise
position of the distribution under H1 is unknown. Hence we compromise by setting the
probability of a type I error equal to 5% (usually). Hence any value of the test statistic falling
into the 5% tail of the Normal distribution rejects the null hypothesis.