Comparing Two Groups:

Between-Groups Designs (Part 2)

Dadang Sudana
(Source: Hatch & Farhadi, 1988; Hatch & lazaraton, 1991)
CASE 2 COMPARISONS

• Case 2 studies are much more common in applied linguistics research than are Case 1.
• We often want to compare the performance of two groups. Perhaps the scores are of an
experimental group vs. a control group, Spanish vs. Korean ESL students, or field
dependent vs. field independent learners.
• Let's assume that you have two bilingual classrooms. In one classroom, instruction is
conducted in Spanish in the morning by one teacher and in English in the afternoon by a
second teacher.
• In the second classroom, two teachers take turns instructing children throughout the day.
One uses Spanish and the other uses English.
• At the end of the year, the children are tested in each language. The x for the first class is
82.7; the x for the second is 78.1 (and the distribution shows that x and s.d. are
appropriate measures of central tendency and variability or dispersion for these data).
• The question is whether the test performance of the children in the two classrooms
differed significantly. By ‘eyeballing' the means, it appears that the first class 'did better'
than the second.
• This procedure will differ from that described for a Case I t-test because instead of placing
a mean in a distribution of Xs, we want to look at a distribution of differences between Xs.
• The basic problem is finding this sampling distribution. Once again, we will use sample
statistics to estimate population parameters.
• Using our statistics, we will estimate the differences that we would get if we found and
tested another two classes and compared their means, then found another two classes and
tested them to compare their means, and so on until we felt we had tested the population.
• We would find the difference between each pair of classes on the test and plot these in a
sampling distribution. Then we would place our difference in means in that distribution
and decide whether the difference "belongs" or "is typical" of that distribution.
• If it is, we will not be able to reject the null hypothesis because the difference IS normal
(not extreme) for the distribution of differences.
SAMPLING DISTRIBUTION OF DIFFERENCES
BETWEEN MEANS
• Whenever we want to compare the means of two different groups, we can visualize the
procedure where we collect data on the test from two classes, another two classes, and
another two until we have enough differences between means to represent the
population.
• We compute the differences between the means for each set of two classes. Then we
construct a frequency distribution of all these differences which will be called a sampling
distribution of differences between means.
• The distribution--if it includes many, many differences between means-- should have the
following characteristics:
1. The distribution is normally distributed.
2. It has a mean of zero.
3. It has a standard deviation called the standard error of differences between
means.
• The distribution will be bell-shaped, and we will need a "ruler" to discover the place of
the difference between our two means.
• We will use one which measures the standard error of difference between means. This
should ring a bell.
• This is the third time around for finding the place of our data in a distribution in exactly
the same way.
• To review, to find an individual score in a normal distribution, we use a z-score formula:
• To place a sample mean in a distribution of means, We used a t-test formula that said:

• Now, in comparing the difference between two means, the formula is:

• Now, since we believe that the difference between the two population means will be zero (because
they are from the same population), we can immediately simplify this formula by deleting the
second part of the numerator.
• Let's apply this formula now to some data. Lazaraton (1985) planned to evaluate a set of
authentic language materials used in a beginning-level ESL class.
• There were two sections of beginning ESL, one of which would form the experimental
group and the other the control group.
• While students were placed in the class on the basis of a placement test, it was important
to equate the listening skills of the two classes prior to giving the instruction.
• Here is a table showing the scores of the two groups:

ESLPE Listening Scores & Total Scores

Two Classes
Group n Mean s t
value df p

LISTENING SCORE
Control 19 11.7 4.0
- - -
Ex per. 19 10.5 4.7
- - -

TOTAL ESL SCORE

Control 19 66.9 7.7
- - -
Ex per. 20 63.4 8.4
- - -
LET'S FILL OUT THE CHART FOR THIS EXAMPLE.

• Research hypothesis? There is no effect of group on listening comprehension (i.e., there

is no difference in the means of the experimental and control groups) Significance
level? .05
• 1- or 2-tailed? 2-tailed
• Design
• Dependent variable(s)? Listening comprehension
• Measurement? Scores (interval)
• Independent variable(s)? Group
• Measurement? Nominal (experimental vs. control)
• Independent or repeated-measures? Independent
• Other features? Intact groups
• Statistical procedure? t-test
• Box diagram:

X
• In statistics books. you may find the null hypothesis stated in the following way for the t-test:

Ho= The two samples arc from the same population; the difference between the two sample
means which represent population means is zero ( - = 0).
• The null hypothesis says that we expect that any difference found between the two sample g1uup~
falls well within the normal differences found for any two means in the population. To reject the
null hypothesis, we must show that the difference falls in the extreme left or right tail of the
distribution.
• Here is the formula:
• The denominator is the standard error of differences between means. The subscripts in the
numerator just identify one mean as coming from the experimental group and the other from the
control group.
• The numerator is the easy part. The difference between the two means (10.5 - 11.7) is -1.2. The
question is whether that difference is significant. To find out, we must place this difference in a
sampling distribution and discover how far it is from the central point of that distribution. We
measure the distance with a "ruler"--in this case the standard error of differences between means.
• The denominator is the ruler. You remember from our discussions of Case l studies that we will use
sample statistics (in this case, the standard deviations from the mean of the two groups) to estimate
the mean of the population.
• We know that in making this estimate, we need to correct the "ruler" for the size of the classes (the
n's of the two groups).
• The formula for the 'ruler" is:

• Since we have the s and the n for each group, we call fill in this information as follows:
• Calculation:

• At this point, we will place the observed value of t into the sampling distribution of
differences between means for the appropriate degrees of freedom.
• Since there were 19 Ss in each group for this test, each group has 18 df. We add these
together to get the df for the study. 18 + 18 = 36 df. Another way of saying the same
thing is df = n 1 + n1 - 2 .
• We turn to the t-test Table in the Appendix and look at the intersection of 36 df and .05.
It's not there, so we choose the 30 df row instead (to be more conservative). The t critical
needed for rejection of the null hypothesis is 2.042.
• We cannot reject the null hypothesis. This information may be presented in the simple
statement: t = .845, df ~ 36, p = n.s. The abbreviation n.s. stands for a "non-significant
difference.“ If there is sufficient space, a table such as the following may be used to give
the same information.
ESLPE Listening Scores
Two Classes

• Group n Mean s t value df p

• LISTENING SCORE
• Control 19 11.7 4.0 0.845 36 n.s.
• Ex per. 19 10.5 4.7
• The conclusion we can draw about the differences between these two groups before the
treatment began is that their listening scores did not differ; the groups are of
approximately the same listening ability.
• (We cannot say, however, that they are significantly similar since we have only tested the
null hypothesis of no difference.)
• t-test procedures are extremely useful and powerful statistical tests for comparing two
means (assuming X is the best measure of central tendency for the data).
• They can be used when the means are independent (i.e., between-groups) and when they
are paired (i.e., within pairs).
• As with all tests, care must be taken to observe the requirements for the test. Prior to
applying the procedure, be sure to check each of these. And, after applying the procedure,
interpret the findings with care and common sense.
• Kerjakan Latihan pada halaman 120 – 121, Hatch & Farhady, 1982.