T-test

The t test is one type of inferential statistics. It is used to determine whether there is a
significant difference between the means of two groups. With all inferential statistics, we
assume the dependent variable fits a normal distribution.
When you perform a t-test, you're usually trying to find evidence of a significant difference
between population means (2-sample t) or between the population mean and a
hypothesized value (1-sample t). The t-value measures the size of the difference relative to the
variation in your sample data.

Definition

A t-test’s statistical significance indicates whether or not the

difference between two groups’ averages most likely reflects a “real”
difference in the population from which the groups were sampled.

Example

Let’s say you’re interested in whether the average New Yorker spends
more than the average Kansan per month on movies.

You ask a sample of 3 people from each state about their movie
spending. You might observe a difference in those averages (like $14
for the average Kansan and $18 for the average New Yorker). But that
difference is not statistically significant; it could easily just be random
luck of which 3 people you randomly sampled that makes one group
appear to spend more money than the other. If instead you ask 300
New Yorkers and 300 Kansans and still see a big difference, that
difference is less likely to be caused by the sample being
unrepresentative.
Note that if you asked 300,000 New Yorkers and 300,000 Kansans, the
result would likely be statistically significant even if the difference
between the group was only a penny. The t-test’s effect
size complements its statistical significance, describing the magnitude
of the difference, whether or not the difference is statistically
significant.

Definition

A statistically significant t-test result is one in which a difference

between two groups is unlikely to have occurred because the sample
happened to be atypical. Statistical significance is determined by the
size of the difference between the group averages, the sample size,
and the standard deviations of the groups. For practical purposes
statistical significance suggests that the two larger populations from
which we sample are “actually” different.