Chapter 06 Writing and Evaluating Test Items

Name:
Class: Date:
Chapter 06: Writing and Evaluating Test Items
1. The tendency for test takers to agree on most of the items is called a(n)
a. guessing threshold.
b. acquiescence response set.
c. item difficulty.
d. the miss rate.
ANSWER: b
2. This test item is an example of a
a. polytomous format.
b. dichotomous format.
c. Likert format.
d. category format.
ANSWER: a
3. Distractors that are obviously incorrect
a. lower the reliability of the test.
b. increase the reliability of the test.
c. have no impact on the reliability of the test.
d. reduce the likelihood of correct guessing.
ANSWER: a
4. True-false examinations use
a. a dichotomous format.
b. a polytomous format.
c. a Likert format.
d. a category format.
ANSWER: a
5. In multiple choice examinations, incorrect alternatives are called
a. flags.
b. non-categories.
c. distractors.
d. miss rates.
ANSWER: c
Copyright Cengage Learning. Powered by Cognero. Page 1

Name: Class: Date:
6. In order to correct for guessing
a. a correction formula can be used.
b. distractors should be eliminated.
c. the number of items should be increased.
d. distractors should be increased.
ANSWER: a
7. The expected level of chance performance, for a 200-item multiple-choice exam with four choice alternatives, is
a. 25 correct.
b. 50 correct.
c. 75 correct.
d. 100 correct.
ANSWER: b
8. Suppose that you are taking a multiple choice test where there is no correction for guessing. If you aren't sure of the
answer,
a. only guess if you have some confidence you are correct.
b. you should always guess on a speed test.
c. you should always guess.
d. you should never guess.
ANSWER: c
9. What describes the chances that a low-ability test taker will obtain each score?
a. acquiescence response set
b. the miss rate
c. guessing threshold
d. the moments method
ANSWER: c
10. What format do some personality tests use because it requires an absolute judgment?
a. multiple-choice
b. Likert
c. dichotomous
d. category
ANSWER: c

Name: Class: Date:
11. Which item format can best be factor analyzed to find which ones group together?
a. multiple-choice
b. Likert
c. dichotomous
d. forced-choice
ANSWER: b
12. Suppose you got 75 items correct on a 100-item, six alternative, multiple-choice exam. What would your score be
after we corrected for guessing?
a. 50
b. 57
c. 63
d. 70
ANSWER: d
13. Under what circumstance is it NOT to your advantage to guess on a multiple-choice exam?
a. when you are making a "wild guess" and a correction formula is being used
b. in any test situation where you are making a "wild guess"
c. when you can rule out one or more of the alternatives as being incorrect
d. when the guessing threshold is low
ANSWER: a
14. The difference between Likert scales and category formats is that
a. category formats are used only in health settings.
b. category formats tends to be dichotomous while Likert scales tends to be polytomous.
c. category formats tend to have a smaller number of choices.
d. Likert scales tend to have a smaller number of choices.
ANSWER: d
15. A test format that is typically used for attitude measurement is the
a. checklist format.
b. dichotomous format.
c. category format.
d. Likert format.
ANSWER: d

Name: Class: Date:
16. When distractors are likely to be selected as alternative responses on multiple-choice tests,
a. validity is increased.
b. item reliability is increased.
c. item reliability is decreased.
d. guessing is reduced.
ANSWER: b
17. One method for measuring chronic pain asks the respondent to group statements according to how accurately they
describe his/her discomfort. This would be an example of the
a. Q-sort format.
b. checklist format.
c. Likert format.
d. category format.
ANSWER: a
18. The following is an item from an attitude scale:
Physical punishment is essential in order to control children.
Strongly disagree
Disagree
Neither agree or disagree
Agree
Strongly agree
This item is in the
a. category format.
b. Likert format.
c. dichotomous format.
d. polytomous format.
ANSWER: b
19. One problem with the use of category rating scales is that
a. many respondents are confused by dichotomous formats.
b. responses are sometimes influenced by the context in which objects are rated.
c. rating scales must be at least 100 points in order to be meaningfully interpreted.
d. category rating scale data do not have ordinal scale property.
ANSWER: b

Name: Class: Date:
20. Describing the chances that low-ability test takers will obtain each score is called the
a. dichotomous format.
b. polytomous format.
c. guessing threshold.
d. 50% threshold.
ANSWER: c
21. When teachers are initially told that the students they will be teaching are either not very imaginative or are very
imaginative, ratings using an adjective checklist will tend to reflect this original assessment. This is an example of
a. the effect of context.
b. visual analogue.
c. low sample size.
d. forced choice effect.
ANSWER: a
22. Which method involves scoring that is very time consuming?
a. dichotomous format.
b. visual analogue scale.
c. Likert scale.
d. multiple-choice format.
ANSWER: b
23. Which testing method is popular for measuring self-rated health?
a. q-sort technique
b. visual analogue scale
c. checklists
d. category formats
ANSWER: b
24. Which of the following item writing recommendations has research support?
a. All answer options should be plausible.
b. Items should cover important concepts and objectives.
c. All parts of an item or exercise should appear on the same page.
d. There should be an equal number of true and false statements.
ANSWER: c

Name: Class: Date:
25. Which of the following is a disadvantage of true-false tests?
a. They are typically only useful with simple information.
b. They encourage memorization without understanding.
c. They are difficult to administer.
d. They encourage rapid responding.
ANSWER: b
26. Which type of item tends to lose reliability and become obsolete over time?
a. factual items
b. skill-based items
c. items based on abstract concepts
d. simple items
ANSWER: c
27. What is the impact of adding distractors on polytomous item reliability?
a. The number of distractors is inversely related to item reliability.
b. Large numbers of distractors can greatly increase reliability.
c. Adding distractors may not increase reliability if the distractors are implausible.
d. Reliability is optimized when there are 8 to 10 distractors.
ANSWER: c
28. Which of the following increases the likelihood that students will guess when they are not sure of the correct response
on a multiple choice item?
a. when they expect a low grade
b. when the items are easy
c. when the course is a required course
d. when they dislike the subject
ANSWER: a
29. How do Likert format tests differ from tests made of dichotomous and polytomous items?
a. Likert format tests require far fewer items to achieve reliability and validity.
b. Likert format items quantify characteristics rather than classifying responses as correct or incorrect.
c. Likert format tests cannot be validated whereas dichotomous and polytomous item tests can be validated.
d. Likert format items require higher literacy levels than do dichotomous and polytomous items.
ANSWER: b

Name: Class: Date:
30. Why have checklists fallen out of favor?
a. They are simplistic.
b. They are prone to error.
c. They are difficult to write well.
d. They cannot be validated.
ANSWER: b
31. As the proportion of people who get an item on a test correct increases, the measure of item difficulty
a. decreases.
b. remains the same.
c. increases.
d. approaches chance.
ANSWER: c
32. A multiple-choice test with five options has a chance performance level of
a. .50.
b. .25.
c. .20.
d. .10.
ANSWER: c
33. When Lupe argued that one of the questions on the five-alternative test was unfairly difficult, the teacher simply
replied by saying that the item .60.difficulty was optimal at
a. .50.
b. .60.
c. .625.
d. .70.
ANSWER: b
34. The optimal item difficulty of a six-alternative test is
a. .50.
b. .585.
c. .60.
d. .625.
ANSWER: b

Name: Class: Date:
35. If the five applicants for the chief financial officer position of ABC Company are highly qualified, the company should
use a test that
a. has easier items.
b. discriminates 20% of the time.
c. contains mostly difficult items.
d. contains items ranging in difficulty from .30 to .70.
ANSWER: c
36. The optimum level of item difficulty for a five-alternative multiple choice item is
a. .50.
b. .60.
c. .70.
d. .80.
ANSWER: b
37. For most tests, the maximum amount of information about differences between individuals can be obtained from items
in the difficulty range of
a. .30 to .70.
b. .40 to .80.
c. between .55 and .85.
d. above .90.
ANSWER: a
38. If 50% of the individuals taking a particular test get a certain item correct, the difficulty (or easiness) level of that item
would be
a. .05.
b. .25.
c. .50.
d. .10.
ANSWER: c
39. In the extreme group method of item analysis,
a. point-biserial correlations are used.
b. data from some test-takers are not used in the analysis.
c. only the performance of those who scored extremely well is studied.
d. distractors are eliminated.
ANSWER: b

Name: Class: Date:
40. The method of item analysis which looks at the correlation between performance on an item (correct or incorrect)
and total test score is
a. the extreme group method.
b. the tetrachoric method.
c. the point-biserial method.
d. the item characteristic curve method.
ANSWER: c
41. Exhibit 6-1
Refer to Exhibit 6-1. Which item discriminates at various levels of performance?
a. item a
b. item b
c. item c
d. item d
e. item e
ANSWER: e

Name: Class: Date:
42. Exhibit 6-1
Refer to Exhibit 6-1. Which item discriminates well at low levels of performance but not at high levels?
a. item a
b. item b
c. item c
d. item d
e. item e
ANSWER: c

Name: Class: Date:
43. Exhibit 6-1
Refer to Exhibit 6-1. Which item is unrelated to total test score performance?
a. item a
b. item b
c. item c
d. item d
e. item e
ANSWER: a

Name: Class: Date:
44. Exhibit 6-1
Refer to Exhibit 6-1. Which item is inversely related to performance on the test?
a. item a
b. item b
c. item c
d. item d
e. item e
ANSWER: b

Name: Class: Date:
45. Exhibit 6-1
Refer to Exhibit 6-1. In constructing a test, we would most likely eliminate
a. item a
b. item b
c. item c
d. item d
e. item e
ANSWER: d
46. When 100% of the test-takers get an item correct, the item will have a
a. low difficulty index (0%).
b. high discriminability index.
c. discriminability index of approximately .5.
d. very low discriminability index.
ANSWER: d
47. The proportion of test takers that get a "good" item correct increases as a function of the
a. item characteristic curve.
b. total test score.
c. validity of the test.
d. item difficulty.
ANSWER: b

Name: Class: Date:
48. Dr. H likes to start off his tests with a few easier items in order to boost the confidence of the test takers. This is an
example of
a. human factors.
b. the psychometric properties of the test.
c. optimum item difficulty.
d. item difficulty.
ANSWER: a
49. An employment test attempted to find out if individuals who scored high on specific items that assessed an individual's
ability to work well in a team related strongly to the test as a whole. The purpose of the study was to evaluate
a. human factors.
b. optimum item difficulty.
c. item discriminability.
d. categories.
ANSWER: c
50. When test items are evaluated against total test score, we use a(n)
a. internal criterion.
b. external criterion.
c. multivariate analysis.
d. criterion referenced test.
ANSWER: a
51. In item analysis, the internal criteria against which items are evaluated refers to the
a. discrimination index.
b. total test score.
c. criterion.
d. predictor.
ANSWER: b
52. In order to evaluate a criterion referenced test, the test was administered to a group of students who had studied a
learning unit and to another group who had not studied the learning unit. For each item on the test, the criterion for
mastery would be
a. the point-biserial correlation.
b. below the antimode.
c. above the antimode.
d. the validity coefficient.
ANSWER: c

Name: Class: Date:
53. Proponents of criterion-referenced tests have criticized item analysis procedures because they
a. cannot be used for criterion-referenced tests.
b. have statistical flaws.
c. do not provide information about the type of errors that students make.
d. have no relevance for educational tests.
ANSWER: c
54. The approach to test construction in which the item characteristic curve for each individual item is analyzed is called
a. prophecy theory.
b. classical test theory.
c. item response theory.
d. item analysis theory.
ANSWER: c
55. The average of a series of item characteristic curves is known as
a. the average characteristic curve.
b. the standard error of the item characteristic.
c. a test characteristic curve.
d. the variance ratio curve.
ANSWER: c
56. One of the major advantages of tests developed using item response theory is that they
a. can be easily adapted for computer administration.
b. are longer.
c. are easier to administer.
d. can be developed with little effort.
ANSWER: a
57. The extreme group method and the point biserial method are both used to estimate
a. reliability.
b. validity.
c. discriminability.
d. difficulty.
ANSWER: c

Name: Class: Date:
58. The least frequent score in a frequency polygon is the
a. negative discriminator.
b. discrimination point.
c. antimode.
d. criterion.
ANSWER: c
59. In experimental psychology, the proportion of the top third of the class that correctly answered the last question of the
final was .93 while .89 of the bottom third of the class answered correctly. The professor should decide not to include
this question in the next final because the discrimination index indicates
a. negative discrimination.
b. chance level performance.
c. that students were incorrectly prepared.
d. that the item does not discriminate well.
ANSWER: d
60. Professor Plum created class intervals from the test scores for his class. He made a line graph using these intervals
on the X-axis and the proportion of students who answered a particular question correctly on the Y-axis. The result is
a. a discrimination index.
b. a correlation index.
c. an item characteristic curve.
d. a histogram.
ANSWER: c
61. An item characteristic curve that rises gradually and then turns down for people at the highest levels of performance
a. is likely to occur when students are making a wild guess.
b. can happen when 'none of the above' is one of the multiple choice options.
c. turns down at a point referred to as the antimode.
d. indicates an item with a high level of difficulty.
ANSWER: b
62. In order to choose questions for a final version of a test, the examiners created a graph with difficulty on one axis and
discriminability on the other. The examiners should use the questions that
a. all above the .50-point on the discriminability axis.
b. fall below the .50 point on the discriminability axis.
c. fall between .30 and .70 on difficulty and above .30 on discriminability.
d. fall above the .50 point on discriminability and difficulty.
ANSWER: c

Name: Class: Date:
63. The proportion of responses that are expected to be correct for each level of ability can be represented by
a. an item characteristic curve.
b. a discrimination index.
c. a test characteristic curve.
d. the optimum difficulty.
ANSWER: c
64. Peaked conventional tests present items
a. from a wide range of difficulty levels.
b. at optimum difficulty levels.
c. at levels appropriate for the test taker.
d. mostly at or near average difficulty levels.
ANSWER: d
65. Which type of test is especially helpful for evaluating progress in individualized programs of instruction?
a. peaked conventional
b. criterion-referenced
c. rectangular-referenced
d. dichotomous
ANSWER: b
66. Your text presents empirical evidence that indicates school children tend to repeat the same kind of errors when given
problems of a particular type. This highlights the
a. importance of ranking students.
b. need to provide corrective feedback.
c. concept of 'teaching to the test.'
d. ability of IRT procedures to initiate guidance.
ANSWER: b
67. The difference in proportion of correct responses for each item between the top third of the class and the bottom third
of the class is an example of a(n)
a. point biserial correlation.
b. discrimination index.
c. item difficulty.
d. guessing threshold.
ANSWER: b

Name: Class: Date:
68. Item analysis is different from the classical method of reliability because
a. a smaller number of items leads to higher reliability in item analysis.
b. item analysis ignores total score.
c. item analysis describes results for individual items, whereas reliability describes results for the scale as a whole.
d. item analysis generally considers the validity of the test.
ANSWER: c
69. In most situations, a good test should contain items
a. from a wide range of difficulty levels.
b. at optimum difficulty levels.
c. at levels appropriate for the test taker.
d. mostly at our near average difficulty levels.
ANSWER: a
70. Why should tests include items from a variety of difficulty levels?
a. Students are less likely to guess when there is a wide range of item difficulty.
b. Good tests encourage students to do their best and a range of difficulty helps less confident students.
c. Good tests discriminate at a variety of difficulty levels.
d. Students like to have some easy items because they are more likely to respond correctly by chance.
ANSWER: c
71. Which of the following methods is used in the analysis of item discriminability?
a. test-retest
b. extreme group
c. characteristic curves
d. factor analysis
ANSWER: b
72. Which of the following is true of the characteristic curve for a “good” test item?
a. It is normally distributed.
b. It is bimodal and positively skewed.
c. It has a gradual, positive slope.
d. It is negatively accelerated.
ANSWER: c

Name: Class: Date:
73. What is the first step in developing criterion-referenced tests?
a. Getting expert agreement about how to construct the test
b. Clearly specifying the objectives in precise statements about what is to be learned
c. Identifying the types of people who are likely to take the test and either score very well or very poorly
d. Creating items with a wide range of difficulty and administering them to experts in the field
ANSWER: b
74. Which of the following is NOT one of DeVellis (2016)’s guidelines for writing test items?
a. Consider mixing positively and negatively worded items.
b. Define clearly what you want to measure.
c. Consider using “double-barreled” items that convey two or more ideas at the same time.
d. Generate an item pool.
ANSWER: c
75. Items that retain their reliability over time tend to be those that focus on
a. skills.
b. abstract concepts.
c. easier concepts.
d. foundational knowledge.
ANSWER: a
76. The most common sort of dichotomous format is
a. multiple choice.
b. Q-sort.
c. Likert.
d. true-false.
ANSWER: d
77. To be reliable, a true-false test must
a. use simple items.
b. contain many items.
c. have a high guessing threshold.
d. contain few items.
ANSWER: b

Name: Class: Date:
78. How many distractors does this item contain?
a. 0
b. 1
c. 3
d. 4
ANSWER: c
79. For essay items, the reliability of the scoring procedure should be assessed by determining the association(s) between
a. the item score and score on the overall test.
b. the test score and scores on tests measuring similar constructs.
c. each item on the test.
d. two scores provided by independent scorers.
ANSWER: d
80. The optimal difficulty level for items is usually about halfway between 100% of the respondents getting the item
correct and
a. a 50/50 chance for choosing a correct response.
b. the level of success expected by chance alone.
c. the highest possible variance in correct responses for the item set.
d. zero respondents getting the item correct.
ANSWER: b
81. Which of the following is NOT true of item response theory?
a. It assigns values to specific items.
b. Tests based on this theory are easily adaptable for computer administration.
c. It defines a respondent's score by the total number of items correct.
d. It provides information about item functioning.
ANSWER: c
82. Develop several test items and describe methods for analyzing the appropriateness or inappropriateness of their
inclusion on a test. (Hint: It may be helpful to actually "administer" these items to a group of friends.)
ANSWER: Answer not provided.
83. Criterion-referenced tests offer some advantages over tests that may be graded more subjectively and they are now
quite prolific in school systems across the country. However, they have some specific problems. Discuss these
advantages and disadvantages (feel free to integrate information from previous chapters). How would you improve
this situation?
84. Identify and discuss the limitations of item analysis.
Name: Class: Date:
85. What factors should you consider when determining the optimal number of distractors for a multiple choice item?
86. Describe the conditions under which it is advisable to guess on a multiple choice test.

Chapter 06 Writing and Evaluating Test Items

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 06 Writing and Evaluating Test Items

Uploaded by

Copyright:

Available Formats

Name:

Copyright Cengage Learning. Powered by Cognero. Page 1

Copyright Cengage Learning. Powered by Cognero. Page 2

Copyright Cengage Learning. Powered by Cognero. Page 3

Copyright Cengage Learning. Powered by Cognero. Page 4

Copyright Cengage Learning. Powered by Cognero. Page 5

Copyright Cengage Learning. Powered by Cognero. Page 6

Copyright Cengage Learning. Powered by Cognero. Page 7

Copyright Cengage Learning. Powered by Cognero. Page 8

Copyright Cengage Learning. Powered by Cognero. Page 9

Copyright Cengage Learning. Powered by Cognero. Page 10

Copyright Cengage Learning. Powered by Cognero. Page 11

Copyright Cengage Learning. Powered by Cognero. Page 12

Copyright Cengage Learning. Powered by Cognero. Page 13

Copyright Cengage Learning. Powered by Cognero. Page 14

Copyright Cengage Learning. Powered by Cognero. Page 15

Copyright Cengage Learning. Powered by Cognero. Page 16

Copyright Cengage Learning. Powered by Cognero. Page 17

Copyright Cengage Learning. Powered by Cognero. Page 18

Copyright Cengage Learning. Powered by Cognero. Page 19

Copyright Cengage Learning. Powered by Cognero. Page 21

You might also like