Chapter 04 Reliability

Name:
Class: Date:
Chapter 04: Reliability
1. When talking about errors in terms of psychological testing, we are referring to the fact that:
a. someone got an answer incorrect.
b. there is always some inaccuracy in the measurement.
c. the test was inappropriate for that particular group.
d. the score is too subjective to be accurate.
ANSWER: b
2. The basic theory of reliability was first worked out by
a. Karl Pearson.
b. Charles Spearman.
c. Julian Stanley.
d. Lee Cronbach.
ANSWER: b
3. The work of Charles Spearman combined what two measurement concepts?
a. mean and variance
b. sample statistics and population parameters
c. sampling error and correlation
d. reliability and validity
ANSWER: c
4. Who developed methods for evaluating sources of error in behavioral research?
a. Edward Thorndike
b. Kuder and Richardson
c. Charles Spearman
d. Cronbach
ANSWER: d
5. According to classical test theory, errors of measurement are
a. always overestimates of true score.
b. always underestimates of true score.
c. random.
d. constant.
ANSWER: c
Copyright Cengage Learning. Powered by Cognero. Page 1

Name: Class: Date:
6. If we repeatedly administered the same test to the same individual, the standard deviation of the person's score would
be the
a. standard error of the mean.
b. variance.
c. reliability of the test.
d. standard error of measurement.
ANSWER: d
7. Because classic test theory assumes a person's true score is the same over time, repeating the same test over and
over gives a distribution of scores that reflect what?
a. systematic error
b. random error
c. reliability
d. internal consistency
ANSWER: b
8. Classical Test Theory assumes that
a. errors are systematic.
b. errors are random.
c. true scores cannot be estimated.
d. the length of a test has no bearing on its reliability.
ANSWER: b
9. Classical Test Theory assumes
a. the length of a test has no bearing on its reliability.
b. measurement errors occur systematically.
c. it is not possible to estimate true scores.
d. the distribution of random errors is the same for every respondent.
ANSWER: d
10. Theoretically, if Susie repeatedly took the 6th grade achievement test, you would be able to find her true score by
finding the ____ of the distribution of her scores.
a. mean
b. standard deviation
c. variance
d. standard error of measurement
ANSWER: a
Name: Class: Date:
11. If you have three clocks in your house, and every clock is 10 minutes fast, this is an example of
a. systematic error.
b. random error.
c. measurement error.
d. a rubber yardstick.
ANSWER: a
12. We can get an idea of how much measurement error is present in a score through the
a. true score.
b. observed score.
c. standard error of the mean.
d. standard error of measurement.
ANSWER: d
13. What is Cronbach known for?
a. Developing measures to evaluate sources of error
b. Creating the basics of multivariate analysis
c. Developed the basics of contemporary measurement theory
d. Distinguished between objective and subjective measures
ANSWER: a
14. Which of the following is an important distinction between systematic errors and random errors?
a. Random errors are more likely than systematic errors to cause errors in conclusions.
b. Systematic errors occur only in objective measures and random errors occur only in subjective measures.
c. Random errors can be eliminated by careful wording of test items.
d. Systematic errors are extremely rare among psychological tests.
ANSWER: a
15. Assuming the “rubber yardstick” shrinks and expands at random, what can be said about the distribution of scores
from the rubber yardstick?
a. It will have a mean of zero (0).
b. It will be normal.
c. It will have a standard error of zero (0).
d. It will be skewed.
ANSWER: b

Name: Class: Date:
16. What is Spearman known for?
a. Working out the basics of reliability theory
b. Developing the notion of sampling error
c. Creating methods for measuring error
d. Developing multivariate analysis
ANSWER: a
17. Repeated use of the same test typically results in different scores. How does classical test theory account for this?
a. poor test validity
b. systematic variability
c. random error
d. inattention
ANSWER: c
18. Theoretically, reliability is
a. the correlation of the observed test score with the true score.
b. the square root of the ratio of true to the observed score.
c. the ratio of true to the observed score squared.
d. not possible to define.
ANSWER: a
19. When creating a test, one generally uses a subset of items to represent a larger construct. This is known as
a. a population parameter.
b. a domain sampling.
c. a sampling error.
d. descriptive statistics.
ANSWER: b
20. An observed score is composed of
a. the residual and the true score.
b. the criterion and the predictor.
c. the measurement error and the predictor.
d. the true score and the measurement error.
ANSWER: d

Name: Class: Date:
21. In the domain sampling model, the error that is being considered is the error caused by
a. choosing the wrong domain.
b. systematic error.
c. using a limited sample of items.
d. random error.
ANSWER: c
22. The problems created by using a limited number of items to represent a larger and more complicated construct are
explicitly considered in the ____ model.
a. multivariate
b. random sampling
c. domain sampling
ANSWER: c
23. Professor Pine constructed five different short history tests by randomly drawing questions from the huge pool of
all possible questions about the current material. He has created
a. randomly parallel tests.
b. a large sample size.
c. systematic errors.
d. attenuation effects.
ANSWER: a
24. Dr. Smith is trying to determine the reliability of a new personality test. Two randomly parallel tests, A and B, have a
correlation of .81. What is the estimated reliability of the new personality test?
a. .81
b. -.9
c. .9
d. .81/t
ANSWER: c

Name: Class: Date:
25. How does the domain sampling model conceptualize reliability?
a. The absolute value of the difference between the standard error of measurement and the variance
b. The ratio of variance of the observed scores on the short version of a test and the variance of the long-run
true scores
c. The sum of squares of the difference between the observed and true scores
d. The ratio of the number of sample items to the number of domain items, multiplied by the mean of the sample
distribution
ANSWER: b
26. Why might different random samples of domain items yield different estimates of the true score?
a. sampling error
b. poor reliability
c. respondent error
d. item bias
ANSWER: a
27. Tests designed according to item response theory
a. are no longer considered useful.
b. can only be used with non-objective material
c. yield more reliable results with fewer items
d. provide low-tech methods for field use.
ANSWER: c
28. As opposed to reliability based on the classical test theory, ____ focuses on the range of item difficulty that is useful
in assessing an individual's ability.
a. domain sampling
b. internal consistency
c. coefficient alpha
d. item response theory
ANSWER: d
29. If a researcher is attempting to assess the reliability of a measure of depression, the method of choice would be
a. internal consistency.
b. time sampling.
c. the test-retest method.
d. more than one of these.
ANSWER: a
Name: Class: Date:
30. A reliability coefficient of .60 suggests that
a. 64% of the variance on the test is error.
b. 40% of the variance on the test is error.
c. 78% of the variance on the test is error.
d. the test can be used for clinical purposes but not for research.
ANSWER: b
31. Federal government guidelines require that a test be
a. standardized for use among all U.S. sub-populations.
b. factor analyzed before it can be used to make employment decisions.
c. reliable before it can be used to make employment decisions.
d. reliable above the .90 level.
ANSWER: c
32. Upon repeated applications of the same test, performance on the second application may be affected by previous
experience on the test. This is known as
a. attenuation.
b. a carryover effect.
c. shrinkage.
d. selected recall.
ANSWER: b
33. The difference between David's two typing tests, one at the beginning of the semester and one at the end, reflects the
fact that he typed quite a few term papers during the semester. This reflects
a. attenuation.
b. random error.
c. practice effects.
d. domain sampling.
ANSWER: c
34. Dr. Janine developed two equivalent forms of a test and administered them both, in counter-balanced order, to a
group of people on the same day in order to assess reliability. What is this called?
a. test- retest
b. parallel forms
c. split-half
d. KR 20
ANSWER: b
Name: Class: Date:
35. Sources of error associated with time sampling are measured using
a. the test-retest method.
b. the split half method.
c. KR 20.
d. the alpha method.
ANSWER: a
36. Suppose you were trying to estimate the reliability of a whole test on the basis of the correlation between scores on
the two halves of the test. In order to correct for using scores based on the halves, you might use the
a. KR 20.
b. alpha method.
c. Spearman-Brown formula.
d. split half method.
ANSWER: c
37. The Spearman Brown formula corrects for deflated reliability due to
a. half-length tests.
b. small sample size.
c. systematic error.
d. poor test item construction.
ANSWER: a
38. The method for estimating the internal consistency of a test that simultaneously considers all possible ways of splitting
the items is the
a. Spearman Brown formula.
b. Kuder-Richardson formula.
c. Cronbach's alpha.
d. the odd-even method.
ANSWER: b
39. A split-half correlation, KR 20, and coefficient alpha are all used to evaluate
a. standard errors of measurement.
b. internal consistency.
c. variance.
d. validity.
ANSWER: b

Name: Class: Date:
40. Which of the following would tend to provide the most conservative estimate of split-half reliability?
a. the Phillips method
b. the Spearman-Brown formula
d. the odd-even reliability coefficient
ANSWER: c
41. The difference between KR 20 and coefficient alpha is
a. KR 20 can be used to evaluate time sampling problems while alpha cannot.
b. Alpha can be used to evaluate time sampling problems while KR 20 cannot.
c. KR 20 can only be used for items scored right or wrong but Alpha can be used for items in any format.
d. Alpha can only be used for items scored right or wrong but KR 20 can be used for items in any format.
ANSWER: c
42. Difference scores are created by
a. subtracting one test score from another.
b. subtracting the true score from a predicted score.
c. eliminating error from true scores.
d. giving a test to two different individuals.
ANSWER: a
43. The reliability of a difference score is
a. equal to the reliability of the most reliable of the two measures.
b. equal to the reliability of the least reliable of the two measures.
c. the average reliability of the two measures.
d. expected to be lower than the reliability of either of the two measures.
ANSWER: d
44. Measures of test-retest reliability are sometimes considered inappropriate for the evaluation of health status because
a. health status tests should not given at multiple points in time.
b. variations in health status may be related to true changes over time rather than measurement error.
c. there is no domain of health status.
d. health status is too complicated to measure.
ANSWER: b

Name: Class: Date:
45. What is the impact of carryover effects on test-retest reliability?
a. Test-retest reliability is not influenced by carryover effects.
b. Carryover effects result in an overestimation of reliability.
c. Carryover effects result in an underestimation of reliability.
d. Test-retest reliability increases carryover effects.
ANSWER: b
46. Which of the following is true of the parallel forms method?
a. It is the most often used method for estimating reliability.
b. It provides one of the most rigorous methods for estimating reliability.
c. It is largely ineffective with psychological tests.
d. Sophisticated computer programs have made it unnecessary.
ANSWER: b
47. Jennifer read a report in which the agreement between raters of children's aggressive behavior was .50, indicating
a. the raters agreed at chance levels.
b. agreement was poor.
c. agreement was excellent.
d. agreement was moderate.
ANSWER: d
48. Which of the following is a problem in evaluating the agreement between observers in behavioral studies?
a. The observers are usually not trained.
b. The behaviors being studied are usually not directly observable.
c. There will always be some agreement by chance.
d. There is no method for evaluating the agreement between observers.
ANSWER: c
49. The kappa statistic is used to
a. assess the level of agreement among several observers.
b. estimate the correlation between a continuous variable and an artificially dichotomous variable.
c. estimate the percentage of disagreement between observers.
d. estimate the validity of behavioral observation.
ANSWER: a

Name: Class: Date:
50. The preferred method for assessing the level of agreement between observers is the
a. kappa statistic
b. Spearman coefficient
d. rank-order statistic
ANSWER: a
51. Which of the following is a source of measurement error?
a. respondent sampling
b. scorer sampling
c. internal consistency
d. external consistency
ANSWER: c
52. Which of the following is used to estimate the number of items that should be added to a test to achieve a specified
reliability?
a. KR 20
b. coefficient alpha
c. Spearman-Brown prophecy formula
d. split-half technique
ANSWER: c
53. Correction for attenuation is used
a. to estimate the validity of a test.
b. to correct for tests that are short.
c. to correct for tests that are long.
d. to estimate the true correlation between variables that have been measured with error.
ANSWER: d
54. In order to determine the unidimensionality of a test, you can use
a. factor analysis.
b. split half reliability.
c. parallel forms assessment.
d. the Spearman-Brown prophecy formula.
ANSWER: a

Name: Class: Date:
55. Items are probably measuring the same thing when the correlation between an item and the total score
a. is high.
b. is low.
c. approaches 0.
d. is negative.
ANSWER: a
56. If the same test, given at different points in time to the same test takers, yields different scores, then the method
typically used to assess this source of error is
a. test-retest.
b. alternate forms/parallel forms.
c. split-half.
d. KR 20.
ANSWER: a
57. Approximately what value must a reliability coefficient have for most purposes in basic research?
a. .90
b. .50
c. .70
d. .30
ANSWER: c
58. The standard error of measurement allows us to
a. estimate the degree to which a test provides inaccurate readings.
b. have an acceptable margin of error.
c. determine the source of error.
d. avoid any measurement error.
ANSWER: a
59. Standard errors of measurement are used to
a. determine whether an observed score is the "true" score.
b. determine the standard deviation of the scores.
c. calculate the exact true score.
d. create confidence intervals around specific observed test scores.
ANSWER: d

Name: Class: Date:
60. Test constructors can improve test reliability by
a. increasing the number of items.
b. decreasing the number of items.
c. retaining items that have the most face validity.
d. reducing the item to total correlation.
ANSWER: a
61. The formula used to estimate how long a test must be to achieve a desired level of reliability is
a. kappa
b. prophecy
c. Spearman
d. Thorndike
ANSWER: b
62. The prophecy formula is used to
a. predict expected values.
b. estimate how long a test must be to achieve a desired level of reliability.
c. estimate how long a test must be to achieve a desired level of validity.
d. calculate variability.
ANSWER: b
63. Tests will be most reliable if they are
a. multidimensional.
b. unidimensional.
c. brief.
d. criterion-referenced.
ANSWER: b
64. What is the most useful indicator of reliability for the interpretation of individual scores?
a. split-half variance
b. item sampling
c. test-retest
ANSWER: d

Name: Class: Date:
65. Reliability theory combines De Moivre's concept of sampling error with Pearson’s concept of _____________ in the
context of measurement.
a. coefficient alpha
b. internal consistency
c. product moment correlation
d. domain sampling
ANSWER: c
66. Classical test theory assumes that
a. there are no errors in measurement.
b. each person has a true score.
c. observed scores almost always reflect true ability.
d. errors of measurement are systematic.
ANSWER: b
67. The reliability coefficient is
a. the mean of the observed scores.
b. the variance of the observed scores.
c. the ratio of the mean of the true scores on a test to the mean of the observed scores.
d. the ratio of the variance of the true scores on a test to the variance of the observed scores.
ANSWER: d
68. For which of these constructs is it most appropriate to measure test-retest reliability?
a. IQ
b. Depression
c. Literacy
d. Blood pressure
ANSWER: a
69. Carryover effects only affect reliability when changes over time are
a. large.
b. systematic.
c. random.
d. due to practice effects.
ANSWER: c

Name: Class: Date:
70. Interrater reliability is of concern in
a. personality testing.
b. behavioral observation studies.
c. factor analysis.
d. parallel forms assessment.
ANSWER: b
71. The intercorrelations among items within the same test is referred to as
a. interrater reliability.
b. discriminability.
c. standard errors of measurement.
d. internal consistency.
ANSWER: d
72. In the domain sampling model, the reliability of a test increases as
a. the number of items increases.
b. the number of items decreases.
c. the number of test administrations increases.
d. the number of test administrations decreases.
ANSWER: a
73. Classical Test Theory is based on certain assumptions. Discuss these basic assumptions and the theory behind them,
and then address the challenges to any of these assumptions.
ANSWER: Answer not provided.
74. There are several methods to estimate reliability. Compare and contrast the different methods of reliability
discussed in this chapter, stressing the importance of coefficient alpha.
75. Discuss the challenges to the use of difference scores.
76. Describe some of the advantages and disadvantages associated with behavioral observation techniques. Provide
examples.
77. Briefly discuss each of the APA’s standards for reliability.
Name: Class: Date:
78. Describe the reasons for the large movement from Classical Test Theory to Item Response Theory.
79. Explain how someone might decide how reliable is “reliable enough” for a measure. What settings might warrant
more stringent criteria for reliability, and why?

Chapter 04 Reliability

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 04 Reliability

Uploaded by

Copyright:

Available Formats

Name:

Copyright Cengage Learning. Powered by Cognero. Page 1

Copyright Cengage Learning. Powered by Cognero. Page 3

Copyright Cengage Learning. Powered by Cognero. Page 4

Copyright Cengage Learning. Powered by Cognero. Page 5

Copyright Cengage Learning. Powered by Cognero. Page 8

Copyright Cengage Learning. Powered by Cognero. Page 9

Copyright Cengage Learning. Powered by Cognero. Page 10

Copyright Cengage Learning. Powered by Cognero. Page 11

Copyright Cengage Learning. Powered by Cognero. Page 12

Copyright Cengage Learning. Powered by Cognero. Page 13

Copyright Cengage Learning. Powered by Cognero. Page 14

Copyright Cengage Learning. Powered by Cognero. Page 16

You might also like