Selected- Response,

Short-Answer, and
Essay Items
Assessing Deep Understanding and
Reasoning
 Overview:
DEEP UNDERSTANDING
•Nature of
REASONING SKILLS •rich
•Bloom’s Taxonomy •Sophisticated
•Revision of Bloom’s •Enduring
Taxonomy •Six Facets
•Ennis’s critical thinking
•Quellmalz and Hoskyn’s
reasoning skills SHORT-
ANSWER ITEMS

ASSESSING DEEP
UNDERSTANDING
AND REASONGING
ESSAY ITEMS
SELECT-RESPONSE •Restricted-
ITEMS Response
•Binary-choice •Extended-response
•Constructing
•Multiple-choice
•Scoring
•Interpretive exercises •Holistic
•Analytic
What is Deep Understanding?
Knowledge Simple Deep
Understanding Understanding
Fragmented Comprehend
Ritualize Apply Penetrating
Fragile Rudimentary explanations Elegant
Literal Think about Sophisticated explanation
Superficial Interpret Justify
Surface Illustrate Compare and Contrast
Temporary Describe Construct
Recall Expert
Recognize Critical Thinking
Novice Reasoning
Inflexible Grasp Structure
Formulaic Rethinking
Revising
Reflective
Enduring
Infer
 What is Reasoning?
• Something students do with their knowledge, a kind of cognitive or mental
operation that employs their understanding to some end
• Involves some kind of mental manipulation of knowledge

SIX FACETS OF
UNDERSTANDING
•Explanation
•Interpretation
•Application
•Perspective
•Empathy
•Self-knowledge
Bloom’s Taxonomy of the Cognitive
Domain
 Revision of Bloom’s Taxonomy
• Analyze
– Differentiating
– Organizing
– Attributing
• Evaluate
– Checking
– Critiquing
• Create
– Generating
– Planning
– Producing
 Ennis’s Taxonomy of Critical Thinking Dispositions and
Abilities
CRITICAL THINKING – reasonable reflective thinking that is
focused on deciding what to believe or do (Ennis, 1989)
- the act of thinking critically must include a decision or
judgement about a belief, action, or answer

Quellmalz And Hoskyn’s Framework For Reasoning

Strategies
4 COMMON ELEMENTS THAT MAKE SENSE FOR TEACHING
AND ASSESSMENT:
1. Analysis
2. Comparison
3. Inference and Interpretation
4. Evaluation
Assessing Deep Understanding and Reasoning
APPLICATION OF QUELLMALZ ANS HOSKYN’S FRAMWORK AND REASONING STRATEGIES
Literacy
NARRATIVE
Story, plot and character
elements; setting; style
PERSUASIVE ISSUE
Position, reasons, evidence,
conclusion
EXPOSITORY
Main idea, support and
elaboration, organization and
coherence style
Narrative elements, themes,
points of view, evidence,
accuracy, organization
Themes, motivation, mood,
bias, predict cause and effect
Significance, coherence, clarity,
style, believability
Social Science
Elements of an event, features
of a culture, features of a
historical period
Leaders, cultures, political
systems, ideologies, time
periods, accounts of an event
Causes and influences, product
future effects, infer
consequences
Significance of contributions,
practicality, credibility of
arguments, alternative
interpretation
Science
Components of a process,
features of animate and
inanimate objects, evolution of
species
Regions, climates, scientific
processes, energy sources,
habitats, ecosystems
Test hypotheses, draw
conclusion, infer consequences,
link cause and effect,
interdependencies
Soundness of scientific
procedures, credibility of
conclusions, significance of
findings, feasibility, impact
Critical Thinking Steps and Abilities
• Clarify Problem
– Identify or formulate a question; put the problem in context; ask questions or
seek information to clarify the problem.
• Gather Information
– Distinguish verifiable facts from value claims; determine the credibility of a
source; distinguish relevant from irrelevant information, claims, or reasons;
detect bias.
• Make inferences
– Recognize logical inconsistencies in deductive reasoning; recognize
unwarranted claims or generalizations from inductive reasoning.
• Conduct advance clarification
– Identify unstated assumptions; identify ambiguous or illogical arguments;
determine the strength of an argument; detect fallacy labels such as straw
person, name-calling, and non sequitur; detect inconsistencies; detect
stereotypes; consider alternative judgements; distinguish cause and effect
from relationships.
• Make a judgement
– Decide on an answer, solution, or course of action.
 Short-Answer and Selected-Response Items
• Short Answer
– Comparing
– Deductive reasoning • Interpretive Exercises
– Credibility of a source Suggestions:
– Analysis/prediction 1. Identify The Reasoning Skills To
– Investigating Be Assessed Before Selection Or
– Analysis Development Of The Interpretive
• Binary-Choice Exercise.
2. Keep Introductory Material As
• Multiple Choice Brief As Possible
– Distinguish fact from opinion 3. Select Similar But New
– Identifying assumptions Introductory Material.
– Recognizing bias 4. Construct Several Test Items For
– Comparison Each Exercise
– Analysis 5. Construct Items To Assess Deep
– Synthesis Understanding And Reasoning
– Hypothesizing
– Problem solving
– Critical Thinking
– Predicting
 Suggestions to help ensure interpretive
exercise quality:
1. Identify the Reasoning Skills to Be Assessed to
before Selection or Development of the Interpretive
Exercise.
2. Keep Introductory Material as Brief as Possible.
3. Select Similar but New Introductory Material.
4. Construct Several Test Items for Each Exercise
5. Construct Items to Assess Deep Understanding and
Reasoning.
1. Drawing inferences, analyzing perspectives
2. Recognizing relevance of information
3. Analysis, inference error analysis
 Bar Average Task Scoring Criteria
• LEVEL 4 -Explanation, work or drawing on the graph shows a correct and
complete understanding of the concept of average in the context for the
problem. The strategy used to obtain the correct answer is appropriate and is
implemented completely and correctly.
• LEVEL 3 - Explanation, work or drawing on the graph shows a correct and
complete understanding of the concept of average in the context for the
problem; however, the implementations of the strategy contain a minor error
or omission.
• LEVEL 2 - Explanation, work or drawing on the graph shows a correct and
complete understanding of the concept of average in the context for the
problem, but the use of a strategy to obtain the is somewhat incomplete,
unclear, or incorrect.
• LEVEL 1 – Problem is revealed from the beginning, strategy used is unclear,
incorrect of no strategy is apparent.
• LEVEL 0 – No understanding of the concept of average in the context of the
problem is evident.
 ESSAY ITEMS
• Advantages and Disadvantages
– Advantages:
– Easy assessment of deep understanding, complex thinking and
reasoning skills
– Allows you to evaluate student’s ability to communicate their
reasoning
– Less time-consuming to construct
– Disadvantages:
– Reading and scoring answers is very time-consuming
– Scoring essays is notoriously unreliable
– Many factors can affect the teacher’s professional judgements
– Essay items do not provide very good sampling of content
knowledge
• Constructing Essay Items
1. Construct the item to elicit skills identified in the
learning target.
2. Write items so that the students clearly understand
the specific task

• Scoring Essays
1. Outline what constitutes a good or acceptable
answers as a scoring key.
2. Select an appropriate scoring method.
3. Clarify the role of writing mechanics.
4. Use systematic process in scoring many essays at the
same time.
5. If possible, keep the identity of the student
anonymous.
 Example of Essay Holistic Scoring Guides
Item: Compare and contrast the first and second Iraq wars. Show how
they were similar and how they were different along geographic,
political and natural resources.
Level of Performance Description

Exceptional (5) Thorough and detailed understanding of both wars.

Excellent (4) Complete understanding of both wars.

Very Good (3) Mostly complete understanding of both wars.

Acceptable (2) Incomplete understanding of one or both wars.

Poor (1) Incomplete understanding of both wars.

 Example of Essay Analytic Scoring Guide
ITEM: Compare and contrast the first and second Iraq wars. Show how they were
similar and how they were different along geographic, political and natural
dimensions
Facet Inadequate Adequate Very Good Excellent Points
1 2 3 4
Understands Clearly does not Demonstrates Demonstrates Demonstrates
both wars understand minimal complete complete
understanding understanding understanding of
of most aspects all aspects

Similarities Does not address Shows one Shows two Shows at least
correct correct thee correct
similarity similarities similarities
Differences Does not address Show one Show two Include at least
correct correct three correct
difference differences differences
Inclusion of Fails to include Include one Includes two Includes at least
dimensions one correct correct correct three correct
dimension dimension dimensions dimensions

TOTAL POINTS