Activities: Writing Clicker Workshops

University of Colorado Science

Questions
Education Initiative
http://colorado.edu/sei
ddddddddd

Writing Questions Courtesy Science Education Initiative at the University of Colorado;

http://colorado.edu/sei
Overview Materials
These activities give participants a chance to try writing and 1. Write question based on
revising clicker questions, with feedback from peers or group pedagogical goal: Handout,
leader. below, including Question Cycle,
and Question Writing Tips. Useful
to have small whiteboards or
Objectives butcher paper and markers so
that questions can be written
The goal is for participants to learn the features of good
large enough to share.
clicker questions, and get a chance to practice these skills 2. Write question based on learning
where they can get formative feedback. Participants should goal: Handout, below, including
be able to identify the cognitive level, or depth of a Question Writing Tips. Useful to
question, as well as its pedagogical goal. have small whiteboards or butcher
paper and markers so that
questions can be written large
Activities enough to share.
1. Write a question based on a pedagogical goal 3. Revise and improve existing
question: Handout, below.
Participants choose a pedagogical goal from the Question 4. Rate and Swap: 5-10 mins.
Cycle handout (see below); e.g., predict outcome. They Handout, below, including one of
draft a clicker question related to their content area, Blooms Taxonomy handouts.
Useful to have small whiteboards
including some plausible distractors, to fulfill this
or butcher paper and markers so
pedagogical goal. This activity occurs before discussing
that questions can be written
features of effective questions; this draft question is large enough to share.
intended to serve as a touchstone during the rest of the
workshop, as participants shop for ideas to improve it and
later revise the question based on what they have learned. Time
Then, after discussing features of good clicker questions, 1. Write question based on
participants work on revising their question and share with a pedagogical goal: 5-10 minutes
neighbor. to write, 10 mins to revise.
2. Write question based on learning
Note that this only works well in workshops focused on goal: 5-10 minutes to write, 10
question-writing; otherwise instructors dont want to take the mins to revise
time for this activity. 3. Revise and improve existing
question: 5-10 mins
Variation #1: Instead, do this activity after a discussion of 4. Rate and Swap: 5-10 mins.
features of effective questions, and have participants then
swap with a neighbor.

Variation #2: Have participants brainstorm a question in

groups of 2-3, and then swap with another group to revise
each others questions.

Variation #3: Draft a question on a topic of their choice, and

then determine the pedagogical goal of that question after
the fact.

Writing Questions Courtesy Science Education Initiative at the University of Colorado;

http://colorado.edu/sei
Activities: Writing Questions
ddddddddd

2. Write a question based on a content learning goal

Same as above, but participants choose a learning goal

related to their content area. Here are some example
learning goals in the sciences:

Biology: Recognize the components of a cell and

describe why each is necessary for the function of a
cell
Physics: Identify the different ways that light can
interact with an object (i.e., transmitted, absorbed,
reflected).
Chemistry: Explain trends in boiling points in terms
of intermolecular interactions
Earth science: Understand the formation of the
three major types of rocks (igneous, sedimentary, and
metamorphic) and the processes by which they form,
relating them by the rock cycle.
Math: Solve a system of linear equations in two
variables using algebra or graphing.

3. Try to revise and improve an existing question

It can be difficult in a short workshop for participants to

really engage with the process of writing and revising their
own question; examining an existing question can be an
easier strategy. This is challenging with a multidisciplinary
audience, however. You might give each disciplinary group
their own question (from the Gallery Walk, for example).

4. Rate and swap

Participants are given a handout of Blooms Taxonomy levels

and verbs. Using the question that they wrote, they rate it
on Blooms Taxonomy, swap with a neighbor to come to
consensus on its Blooms level, and then try to write a
question at a higher level of Blooms using the verbs on the
Blooms Taxonomy sheet. Note: It is often difficult to
identify a single Blooms level for a question, typically it will
span two categories.

Writing Questions Courtesy Science Education Initiative at the University of Colorado;

http://colorado.edu/sei
Variation: Have participants rate the Blooms level of a pre-
written question, and attempt to Bloomify it up.

Writing Questions Courtesy Science Education Initiative at the University of Colorado;

http://colorado.edu/sei
 Write a Draft Question: Pedagogical
Goal
To Do
Think about a topic you will be teaching next week (or that you recently
taught). On your own, write a draft question that addresses one of the
pedagogical goals from the Question Cycle. Include 3-4 plausible distractors.

This doesnt need to be perfect we will work on revising it later in the

workshop.

If you finish, swap with your neighbor and discuss.

Notes

Writing Questions Courtesy Science Education Initiative at the University of Colorado;

http://colorado.edu/sei
 Write a Draft Question: Content Goal
To
Do

Look at the learning goals below. On your own, write a draft question that
addresses this learning goal. Include 3-4 plausible distractors.
Biology: Recognize the components of a cell and describe why each is
necessary for the function of a cell
Physics: Identify the different ways that light can interact with an object
(i.e., transmitted, absorbed, reflected).
Chemistry: Explain trends in boiling points in terms of intermolecular
interactions
Earth science: Understand the formation of the three major types of
rocks (igneous, sedimentary, and metamorphic) and the processes by
which they form, relating them by the rock cycle.
Math: Solve a system of linear equations in two variables using algebra
or graphing.
Then, consider the pedagogical goals from the Question Cycle: What is the
pedagogical goal of this question?

This doesnt need to be perfect we will work on revising it later in the

workshop.

If you finish, swap with your neighbor and discuss.

Writing Questions Courtesy Science Education Initiative at the University of Colorado;

http://colorado.edu/sei
Notes

Writing Questions Courtesy Science Education Initiative at the University of Colorado;

http://colorado.edu/sei
 Rate & Swap
To
Do

Use the Blooms Taxonomy handout (below) to rate the Blooms level of the
question that you wrote. (You can indicate that it belongs to more than one
level).

Swap your question with a neighbor. Do you agree on the Blooms level of
your question?

Try to Bloomify up the level of your question rewrite it at a higher level of

Blooms, making use of the verbs associated with each level on the Blooms
handout.

Notes

Writing Questions Courtesy Science Education Initiative at the University of Colorado;

http://colorado.edu/sei
Writing Questions Courtesy Science Education Initiative at the University of Colorado;
http://colorado.edu/sei
 Question Cycle: Before / During / After
3

BEFORE
Setting up instruction DURING
Motivate Developing
Discover knowledge
Predict outcome Check knowledge
Provoke thinking Application
Assess prior knowledge Analysis
Evaluation
Synthesis
AFTER Relate to big picture Exercise skill
Assessing Demonstrate success Elicit misconception
learning
Review or recap
Exit poll
Credit: Rosie Piller and Ian Beatty.

Writing Questions Science Education Initiative at the University of Colorado;

http://colorado.edu/sei
 Tips for Writing Clicker
Questions
See also the Instructors Guide to the Effective Use of
Clickers, at http://STEMclickers.colorado.edu
Start with existing questions where possible: See lists of question
banks at http://STEMclickers.colorado.edu.
Dont agonize too much; after all, the perfect question doesnt solve
all problems: Its hard to tell in advance which questions will be
great, and a great question poorly facilitated can fall flat.
Dont make them too easy. This is a common mistake, and misleads
students as to your expectations. Challenge student thinking
(students prefer this!) rather than testing memorized facts. You can
write multiple choice questions that test higher levels of thinking
(e.g, Analyzing, Evaluating; see Blooms Taxonomy)!
Use questions that emphasize reasoning or process rather than the
correct answer; this is knowledge that is more generalizeable.
Use questions that will prompt discussion. Interesting questions
that students cant answer on their own are more likely to spur
productive discussion.
Use clear wording so that students understand what they are being
asked. Keep revising over time.
Write tempting distractors using your knowledge of student
difficulties. For example, look at student answers on exams or
quizzes, or first give the question as an open-ended question to
generate common wrong answers. Talk to other instructors who
have taught the course in the past, and talk to students one-on-one
in office hours.
Use a wide variety of creative question types. You can survey your
students on their experiences or beliefs, assess what students
already know about a topic, ask them to reflect on their own
understanding, ask students to predict an outcome, use a series of
questions to break problems into parts, stimulate discussion, or use
pictures or graphs in the answer choices.
Writing Questions Science Education Initiative at the University of Colorado;
http://colorado.edu/sei
 Good sources of questions:
Questions your students ask you or that you overhear
Common analogies you use as a teacher
A series of connected questions to lead students through
reasoning
Interpret graphs, data, pictures, etc.
Discussion questions where there is no one right
answer

Blooms Taxonomy: Cognitive Levels and

Verbs

(short handout #1)

Evaluatio
Knowledge Comprehension Application Analysis Synthesis
n

know restate translate distinguish compose judge

define discuss interpret analyze plan appraise
memorize describe apply differentiate propose evaluate
list recognize employ calculate design compare
recall explain demonstrate experiment assemble value
name identify dramatize compare construct select
relate locate practice contrast create choose
illustrate criticize design assess
operate solve organize estimate
examine manage measure

Writing Questions Science Education Initiative at the University of Colorado;

http://colorado.edu/sei
 Blooms Taxonomy: Cognitive Levels and
Verbs
(short handout #2)

Writing Questions Science Education Initiative at the University of Colorado;

http://colorado.edu/sei
 Bloom's Taxonomy Revised
Key Words, Model Questions, & Instructional Strategies

Blooms Taxonomy (1956) has stood the test of time. Recently Anderson & Krathwohl (2001) have
proposed some minor changes to include the renaming and reordering of the taxonomy.
This reference reflects those recommended changes.

I. REMEMBER (KNOWLEDGE)
(shallow processing: drawing out factual answers, testing recall and recognition)

Verbs for Objectives Model Questions Instructional Strategies

choose Who? Highlighting
describe Where? Rehearsal
define Which One? Memorizing
identify What? Mnemonics
label How?
list What is the best one?
locate Why?
match How much?
memorize When?
name What does It mean?
omit
recite
recognize
select
state

II. UNDERSTAND (COMPREHENSION)

(translating, interpreting and extrapolating)
Verbs for Objectives Model Questions Instructional Strategies
classify State in your own words. Key examples
defend Which are facts? Emphasize connections
demonstrate What does this mean? Elaborate concepts
distinguish Is this the same as. . .? Summarize
explain Give an example. Paraphrase
express Select the best definition. STUDENTS explain
extend Condense this paragraph. STUDENTS state the rule
give example What would happen if . . .? Why does this example. . .?
illustrate State in one word . . . create visual representations
indicate Explain what is happening. (concept maps, outlines, flow
interrelate What part doesn't fit? charts organizers, analogies,
interpret Explain what is meant. pro/con grids) PRO| CON
infer What expectations are there? NOTE: The faculty member can
judge Read the graph (table). show them, but they have to do it.
match What are they saying? Metaphors, rubrics, heuristics
paraphrase This represents. . .
represent What seems to be . . .?
restate Is it valid that . . .?
rewrite What seems likely?
select Show in a graph, table.
show Which statements support . . ?
summarize What restrictions would you add?
tell
translate

www.center.iupui.edu/ctl/idd/docs/Bloom_revised021.doc, February 8, 2006

III. APPLY

(Knowing when to apply; why to apply; and recognizing patterns of transfer to situations that are
new, unfamiliar or have a new slant for students)

Verbs for Objectives Model Questions Instructional Strategies

apply Predict what would happen if Modeling
choose Choose the best statements that Cognitive apprenticeships
dramatize apply Mindful practice NOT just a
explain Judge the effects routine practice
generalize What would result Part and whole sequencing
judge Tell what would happen Authentic situations
organize Tell how, when, where, why Coached practice
paint Tell how much change there Case studies
prepare would be Simulations
produce Identify the results of Algorithms
select
show
sketch
solve
use

IV. ANALYZE (breaking down into parts, forms)

Verbs for Objectives Model Questions Instructional Strategies

analyze What is the function of . . .? Models of thinking
categorize What's fact? Opinion? Challenging assumptions
classify What assumptions. . .? Retrospective analysis
compare What statement is relevant? Reflection through journaling
differentiate What motive is there? Debates
distinguish Related to, extraneous to, not Discussions and other
identify applicable. collaborating learning activities
infer What conclusions? Decision-making situations
point out What does the author believe?
select What does the author assume?
subdivide Make a distinction.
survey State the point of view of . . .
What is the premise?
State the point of view of . . .
What ideas apply?
What ideas justify the conclusion?
What's the relationship between?
The least essential statements are
What's the main idea? Theme?
What inconsistencies, fallacies?
What literary form is used?
What persuasive technique?
Implicit in the statement is . . .

www.center.iupui.edu/ctl/idd/docs/Bloom_revised021.doc, February 8, 2006

V. EVALUATE (according to some set of criteria, and state why)

Verbs for Objectives Model Questions Instructional Strategies

appraise What fallacies, consistencies, Challenging assumptions
judge inconsistencies appear? Journaling
criticize Which is more important, moral, Debates
defend better, logical, valid, appropriate? Discussions and other
compare Find the errors. collaborating learning activities
Decision-making situations

VI. CREATE (SYNTHESIS)

(combining elements into a pattern not clearly there before)

Verbs for Objectives Model Questions Instructional Strategies

choose How would you test. . .? Modeling
combine Propose an alternative. Challenging assumptions
compose Solve the following. Reflection through journaling
construct How else would you . . .? Debates
create State a rule. Discussions and other
design collaborating learning activities
develop Design
do Decision-making situations
formulate
hypothesize
invent
make
make up
originate
organize
plan
produce
role play
tell

Web References:
http://www.coun.uvic.ca/learn/program/hndouts/bloom.html
http://www.fwl.org/edtech/blooms.html
http://apu.edu/~bmccarty/curricula/mse592/intro/tsld006.htm
http://152.30.11.86/deer/Houghton/learner/think/bloomsTaxonomy.html
http://amath.colorado.edu/appm/courses/7400/1996Spr/bloom.html
http://www.stedwards.edu/cte/bloomtax.htm
http://quarles.unbc.edu/lsc/bloom.html
http://www.wested.org/tie/dlrn/blooms.html
http://www.bena.com/ewinters/bloom.html
http://weber.u.washington.edu/~krumme/guides/bloom.html
References:
Anderson, L. W. & Krathwohl, D. R. (2001). A Taxonomy for learning, teaching, and assessing.
Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals, by a committee of college
and university examiners. New York: Longmans.
John Maynard, University of Texas, Austin
Marilla Svinicki, University of Texas, Austin

Compiled by the IUPUI Center for Teaching and Learning, Revised December 2002

www.center.iupui.edu/ctl/idd/docs/Bloom_revised021.doc, February 8, 2006