1.

Alan suffers from depressive complaints and visits his general practitioner, where he is asked
to complete a depression screening questionnaire. Alan’s norm score turns out to be 11. The
cut-off score for referring someone to specialized care is a norm score of 10 or higher.
The questionnaire has the following characteristics: M = 6, SD = 2, coefficient alpha is .84.
Should the general practitioner refer Alan, taking the 95% confidence interval into account?

Yes, because the interval ranges from 10 – 12.

SEM = 2* √ (1-.84) = 2 * 0.4 = 0.8

1.96 * 0.8 = 1.568

11 – 1.568 = 9.432
11 + 1.568 = 12.568

2. Glenda has been referred to treat her high level of antisocial behaviours. Before treatment she
completed a questionnaire about callous and unemotional traits, the CU-traits questionnaire.
Her norm score was 17.
After treatment, she again completed the CU traits questionnaire, this time her norm score was
11.
The mean of the CU-traits questionnaire is 8, the variance is 16, the reliability is .79.
Has the training resulted in an actual improvement in her antisocial behaviours, taking into
account the 95% confidence interval?

Yes, because the interval does not include 0.

Difference between scores: 17 – 11 = 6

Then you have 2 options to calculate the interval:

1. SEdiff = SDx * √ (2 – rxx1 – rxx2)

2. SEdiff = √ (SEM12 + SEM22)

The first one is the most direct one and is thus preferrable.
Reliability rxx1= rxx2 , because you are using the same instrument twice, and is .79
The SD is unknown, but the variance is known. Given that SD is the square root of
variance, you can calculate the SD by taking the square root of 16, thus SD is 4.

Completing formula gives:

SEdiff = 4 * √ (2 – .79 - .79) = 4 * 0.648 = 2.592

Confidence interval: we want is 95%, thus 1.96 * SEdiff:

1.96 * 2.592 = 5.081

Calculating just the lower limit (because that is where the 0 is):
6 – 5.0809 = 0.91
Not below 0, so no 0 in interval.

The other option, if you want to do more calculations (but rather not of course):
SEdiff = √ (SEM12 + SEM22)
 You need the SEM in this case, for which you need the following formula:
SEM = SD * √ (1 – rxx).
The SEM is the same assessments, so for both spots in the formula for SEdiff.

Reliability, thus rxx, is .79.

SD is 4.

Completing the formula gives:

SEM = SD * √ (1 – rxx) = 4 * 0.4583 = 1.833

SEdiff = √ (SEM12 + SEM22) = √ (1.8332 + 1.8332) = 2.592

Confidence interval we need is the 95%, thus 1.96 * SEdiff:

1.96 * 2.592 = 5.081

Calculating just the lower limit (because that is where the 0 is):
6 – 5.0809 = 0.91
Not smaller than 0, so no 0 in interval.

3. Macy completes an extra training at Athena. On the practice exam before her training (M =
7.7, SD = 1.1) she scores a 5.2, on the practice exam after training (M = 7.6, SD = 1.1) she
scores a 5.6. The reliability of the first practice exam is .97, the reliability of the second
practice exam is .94.
Did her grade improve due to the training?

No, because the interval includes 0.

Difference score:
5.6 – 5.2 = 0.4

SEdiff difference score:

SEdiff = SD * √ (2-.97-.94) = 1.1* 0.3 = 0.33

95%-CI, because that is typical for the difference scores and then especially the lower
limit since we are interested in the 0;
Score – 1.96 * SEdiff
0.4 - 1.96 * 0.33 = - 0.24568

4. Alexander completes a study advice test. He scores a 31 on the topic ‘Organizing’. The cut-off
value for a positive advice is 30 or higher. The test has the following characteristics: M = 16,
variance = 9, coefficient alpha is .77.
Would you strongly, definitely advise him to follow a study that involves ‘Organizing’, taking
the 90% confidence interval into account?

No, because the interval ranges from 29 – 33, so you should be more careful in our
conclusion and should suggest it, rather than strongly advise it.

SD is the root of the variance, so 3.

SEM = 3 * √ (1-.77) = 3 * 0.4796 = 1.4387
 1.65 * 1.4387 = 2.374
31 – 2.374 = 28.6261
31 + 2.374 = 33.374

5. We develop a short questionnaire to measure ‘enjoying lectures’. The questionnaire contains

the items (1) “I like the chairs in the lecture hall’, (2) “I like to get up early in the morning”,
and (3) “The teacher is funny”. The questionnaire has a total test variance of 6.4. The
variances of the separate items are 1.8, 2.3, and 1.4, respectively. The reliability is .51.

If we remove item 1, the total test variance decreases to 6.0.

If we remove item 2, the total test variance decreases to 5.7.
If we remove item 3, the total test variance decreases to 6.2

Deleting which item results in the largest increase in reliability?

Item 1:
The reliability becomes 0.77 (N = 2, sum item variances is 3.7, variance total score is
6.0)
Item 2:
The reliability becomes 0.88 (N = 2, sum item variances is 3.2, variance total score is
5.7)
Item 3:
The reliability becomes 0.68 (N = 2, sum item variances is 4.1, variance total score is
6.2)

Thus, it is best to delete item 2.

6. A version of the CITO language assessment is known to have a standard deviation in

measurement error of 5,1 in the general population. Moreover, the standard deviation in true
score is 22,6. Can you calculate the reliability? If so, what is it?
Yes, you have enough information to determine the reliability.

Use the first formula on your sheet.

Reliability = variance true score / (variance true score + variance measurement errors)

Variance true score:

SD is provided, transform that to variance; thus 22.6 * 22.6 = 510.76.
Variance measurement errors:
SD is provided, transform that to variance; thus 5.1 * 5.1 = 26.01

Completing the formula then gives 510.76 / (510.76+26.01) = 0.95. Thus, r = 0.95