POWERPOINT Educ 29: IDENTIFYING QUALILTIES DESIRED IN ESSAY ITEMS Desirability of Using Brief-Response versus Extended-Response Questions.

An essay question qualifies as a brief-response item if students can read and fully answer he item in tem minutes. A question requiring a longer response is considered an extendedresponse item. If purpose of using an extended-response question is to have students demonstrate a greater number of skills. The extended-response can generally be broken into shorter questions that, individually, can be answered within ten minutes. (Not all extended extended-response essay question can be subdivided.)

Criteria for Evaluating Essay Items: 1. Does the item measure the specified skill? An item should be selected or constructed only after the skills to be measured have been identified. Example. Here is an objective, and two essay items intended to measure that objective. Which item (A or B) provides the best measure of this objective? Concept: given descriptions of space vehicles, state with explanation whether or not they are satellites of Earth. A. the mission of the Hubble Space Telescope is to photograph faraway objects from its orbit above the Earth. Is the Hubble Space Telescope a satellite of Earth? Explain your reasoning. B. What determines whether a space vehicle is an Earth satellite? Explain your reasoning. 2. Is the level of reading skills required by this item below the students ability? As with other formats, measures of reading ability and achievement will be confounded unless the level reading skill required to understand the test item is below that of students taking the test. 3. Will all or almost all students answer this item in less than ten minutes? As discussed earlier, extended-response items should be avoided within the context of classroom test. 4. Will the scoring plan result in different readers assigning similar scored to a given students response? Consistency in scoring is essential with all item formats. The need for consistency is more obvious for objectively scored items such as multiple choices. For instance, if two people grade the same answers to a multiple choice exam but derive quite different scores, the scoring process is likely to be judged inadequately.

3 Characteristics to Be Included in Scoring Plan of each Essay Item: 1. The total number of points the item is worth. 2. The specific attributes to be evaluated in students responses for each attribute. 3. The criteria for awarding points including partial credits. The points associated with each essay item should be proportional to the relative importance of the skills being tested. Second, the attributes to be evaluated must be specified. Concept: given a description of a storm involving strong winds, state explanation whether or not it is a hurricane. Exercise: A storm cold front approached Florida and severe storm resulted. As he front approached, sustained winds from the southwest increased to 50 miles an hour with some gusts higher than 80. As the front passed the wind switched to the northwest and slowly diminished. Was this storm a hurricane? Explain your conclusion, briefly describing how each piece of information presented above was used in drawing your conclusion. Attributes to Be Scored 1. Classifies the storms as not a hurricane. 2. Identifies absence of a closed circulation around a calm center. 3. Identifies absence of sustained winds above 75 miles per hour. 4. Identifies three characteristics of this weather system that are not defining characteristics of a hurricane.