The Assessment of Creativity: An Investment Based Approach

The Assessment of Creativity: An Investment-Based Approach
Robert J. Sternberg
Oklahoma State University
In this article, I review ideas about creativity and its assessment. I open with some gen-
eral remarks on the nature of creativity. Then I present the investment theory of crea-
tivity. Then I describe prompts my colleagues and I have used to measure creativity.
Next I describe some of the assessments we have used to measure creativity. The ulti-
mate goal is that assessments such as those described in this article eventually will make
their way into standardized tests of abilities, talents, and skills.
Creativity is a habit (Sternberg, 2006a, 2006b; Tharp,
2005). Behind all innovations one nds creativity, so
innovations arise from a habit. When I speak of a habit,
I refer to an acquired behavior pattern regularly fol-
lowed until it has become almost involuntary (rst de-
nition of habit according to www.dictionary.com).
That is, creativity becomes a way of life that one regu-
larly utilizes so that one is hardly aware one is engaging
in it. If we are to assess creativity, we need to assess it
as a habit of ordinary life, not merely as something
one can do at extraordinary times if challenged on a
standardized test.
Many people associate creativity with the greats, such
as Darwin, Picasso, Hemingway, Beethoven, and others
of that ilk. But creativity comes in various kinds
(Sternberg, Kaufman, & Pretz, 2002), as do theories of
creativity (Kozbelt, Beghetto, & Runco, 2010). For
example, Kaufman and Beghetto (2009) have dis-
tinguished among four levels of creativity: Big-C, the
kind found in Darwin and others of that level; little-c,
or everyday creativity; mini-c, or the kind found in the
learning process; and Pro-c, the kind represented in
the progression from little-c to Big-C. This article
focuses upon little-c or everyday creativity (Richards,
2010), but much of it applies to the other kinds of crea-
tivity as well.
Schools sometimes treat creativity as a bad habit
rather than as a good one. And the world of conven-
tional standardized tests we have invented does just that
(Sternberg, 1997a, 1997b). If students try being creative
on standardized tests, they will get slapped down just as
soon as they get their score. That will likely teach them
not to try again.
Oddly enough, a distinguished psychometric tester, J.
P. Guilford, was one of the rst to try to incorporate
creativity into the school curriculum, but his efforts
show little fruit today (Guilford, 1950). Disciples of
Guilford such as McKinnon (1965) and Torrance
(1962) had little more success, except among a relatively
small band of acionados.
It may sound paradoxical that creativitya novel
responseis a habita routine response. But creative
people are creative largely not as a result of any parti-
cular inborn trait, but, rather, through an attitude
toward life (Maslow, 1967; Schank, 1988; Sternberg,
2003b): They habitually respond to problems in fresh
and novel ways, rather than allowing themselves to
respond mindlessly and automatically (Sternberg,
Kaufman, & Pretz, 2002; Sternberg & Grigorenko,
2004; Sternberg & Lubart, 1995a, 1995b, 1995c).
Like any habit, creativity can either be encouraged or
discouraged. The main things that promote the habit are
(a) opportunities to engage in it, (b) encouragement
when people avail themselves of these opportunities,
and (c) rewards when people respond to such encourage-
ment and think and behave creatively. One needs all
three. Take away the opportunities, encouragement, or
rewards, and one will take away the creativity. In this
respect, creativity is no different from any other habit,
good or bad.
This may sound too simple. But creative people
routinely approach problems in novel ways (Albert &
Correspondence should be sent to Robert J. Sternberg, Ofce of
the Provost, Whitehurst 101, Oklahoma State University, Stillwater,
OK 74078. E-mail: robert.sternberg@okstate.edu
CREATIVITY RESEARCH JOURNAL, 24(1), 312, 2012
Copyright # Taylor & Francis Group, LLC
ISSN: 1040-0419 print=1532-6934 online
DOI: 10.1080/10400419.2012.652925
D
o
w
n
l
o
a
d
e
d

b
y

[
U
n
i
v
e
r
s
i
t
y

o
f

T
e
n
n
e
s
s
e
e
,

K
n
o
x
v
i
l
l
e
]

a
t

1
2
:
3
8

2
4

J
a
n
u
a
r
y

2
0
1
3

Runco, 1999; Baer & Kaufman, 2006). Creative people
habitually (a) look for ways to see problems that other
people dont look for, (b) take risks that other people
are afraid to take, (c) have the courage to defy the crowd
and to stand up for their own beliefs, and (d) seek to
overcome obstacles and challenges to their views that
other people give in to, among other things (Sternberg
& Lubart, 1995b, 1995c; see also Kaufman &
Sternberg, 2006, 2010; Sternberg, 1999; Sternberg &
Grigorenko, 2007).
Educational practices that seem to promote learning
may inadvertently suppress creativity, for the same rea-
sons that environmental circumstances can suppress any
habit (Sternberg & Williams, 1996). The result can be a
stultifying of creativity in development (Russ & Fiorelli,
2010). These practices often take away the opportunities
for, encouragement of, and rewards for creativity
(Beghetto, 2010; Smith & Smith, 2010). The increasingly
massive and far-reaching use of conventional standar-
dized tests is one of the most effective, if unintentional,
vehicles this country has created for suppressing creativ-
ity. I say conventional because the problem is not
with standardized tests, per se, but rather, with the kinds
of tests we use. And teacher-made tests can be just as
much of a problem.
Many conventional standardized tests encourage a
certain kind of learning and thinkingin particular,
the kind of learning and thinking for which there is a
right answer and many wrong answers (Gardner,
1983, 1991, 1993, 2006; Sternberg, 1997b, 2003a). To
create a multiple-choice or short-answer test, you need
a right answer and many wrong ones. Problems that do
not t into the right answerwrong answer format do
not well lend themselves to multiple-choice and
short-answer testing. Put another way, problems that
require divergent thinking are inadvertently devalued
by the use of standardized tests (Plucker & Makel,
2010; Runco, 2010). This is not to say that knowledge
is unimportant. On the contrary, one cannot think cre-
atively unless one has the knowledge with which to
think creatively. Creativity represents a balance
between knowledge and freeing oneself of that knowl-
edge (Johnson-Laird, 1988). Knowledge is a necessary,
but in no way sufcient, condition for creativity
(Sternberg & Lubart, 1995a). The problem is that
schooling often stops short of encouraging creativity;
teachers and testers are often content if students have
the knowledge. Teachers have no choice at this point
because they are beholden to the testers. They ignore
the testers at their peril.
Examples are legion (see Sternberg & Grigorenko,
2007; Sternberg, Jarvin, & Grigorenko, 2009; Sternberg,
Kaufman, & Grigorenko, 2008). If one is studying his-
tory, one might take the opportunity to think creatively
about how we can learn from the mistakes of the past to
do better in the future. Or one might think creatively
about what would have happened had a certain histori-
cal event not come to pass (e.g., the winning of the Allies
against the Nazis in World War II). But there is no one
right answer to such questions, so they are not likely to
appear on conventional standardized tests. In science,
one can design an experiment, but again, designing an
experiment does not neatly t into a multiple-choice for-
mat. In literature, one can imagine alternative endings to
stories, or what the stories would be like if they took
place in a different era. In mathematics, students can
invent and think with novel number systems. In foreign
language, students can invent dialogues with people
from other cultures. But the emphasis in most tests is
on the display of knowledge, and, often, inert knowl-
edge that may sit in students heads but may at the same
time be inaccessible for actual use.
Essay tests might seem to provide a solution to such
problems, and they might, but as they are typically used,
they dont. Increasingly, essay tests can be and are
scored by machine. Often, human raters of essays pro-
vide ratings that correlate more highly with
machine-grading than with the grading of other
humans. Why? Because they are scored against one or
more implicit prototypes, or models of what a correct
answer should be. The more the essay conforms to one
or more prototypes, the higher the grade. Machines
can detect conformity to prototypes better than humans,
so essay graders of the kind being used today succeed in
a limited form of essay evaluation. Thus, the essays that
students are being given often do not encourage creativ-
ityrather, they discourage creativity in favor of model
answers that conform to one or more prototypes. In the
end, essay tests can end up rewarding uncreative
students who spit back facts as well as creative ones
(Sternberg, 1994).
Oddly enough, then, accountability movements that
are being promoted as fostering solid education are, in
at least one crucial respect, doing the opposite
(Sternberg, 2004): It is discouraging creativity at the
expense of conformity. The problem is the very narrow
notion of accountability involved. But proponents of
this notion of accountability often make it sound as
though those who oppose them oppose any account-
ability, whereas, in fact, they instead may oppose only
the narrow form of accountability conventional tests
generate. The tests are not bad or wrong, per se, just lim-
ited in what they assess. But they are treated as though
they assess broader ranges of skills than they actually do
assess. In some environments, people who are creative
are treated as though they are mentally ill. Although
there are, in fact, associations between creativity and
mental illness (Kaufman, 2001a, 2001b; Silvia &
Kaufman, 2010), the overwhelming majority of creative
people are mentally well, not ill!
4 STERNBERG
D
o
w
n
l
o
a
d
e
d

b
y

[
U
n
i
v
e
r
s
i
t
y

o
f

T
e
n
n
e
s
s
e
e
,

K
n
o
x
v
i
l
l
e
]

a
t

1
2
:
3
8

2
4

J
a
n
u
a
r
y

2
0
1
3

If people want to encourage creativity, they need to
promote the creativity habit. That means they have
to stop treating creativity as a bad habit. They have to
resist efforts to promote a conception of accountability
that encourages students to accumulate inert knowledge
with which they learn to think neither creatively nor
critically. They need to assess not just analytical skills,
but creative ones. The investment theory provides one
theoretical basis for doing so.
THE INVESTMENT THEORY OF CREATIVITY
Together with Todd Lubart, I have proposed an invest-
ment theory of creativity as a means of understanding the
nature of creativity (Sternberg, 2006c; Sternberg &
Lubart, 1991, 1995a, 1995b). According to this theory,
creative people are ones who are willing and able to
metaphorically buy low and sell high in the realm of
ideas. Buying low means pursuing ideas that are
unknown or out of favor, but that have growth poten-
tial. Often, when these ideas are rst presented, they
encounter resistance. The creative individual persists in
the face of this resistance, and eventually sells high,
moving on to the next new, or unpopular, idea. In other
words, such an individual acquires the creativity habit.
The question is whether the creative thinker has the for-
titude to persevere and to go against the crowd (Simon-
ton, 1984, 1988, 1994, 2010). There are different kinds of
creativity (Sternberg et al., 2002) and the most difcult
kinds to win acceptance for are those that most defy
the crowd.
According to the investment theory, creativity
requires a conuence of six distinct, but interrelated,
resources: intellectual abilities, knowledge, styles of
thinking, personality, motivation, and environment.
Although levels of these resources are sources of individ-
ual differences, often the decision to use the resources is
the more important source of individual differences.
Ultimately, creativity is not about one thing, but about
a system of things (Csikszentmilhalyi, 1988, 1990, 1996,
1999; Locher, 2010).
Intellectual Abilities
Intellectual abilities are generally acknowledged to be
necessary but not sufcient for creativity (Kim,
Cramond, & VanTassel-Baska, 2010; Renzulli, 1986).
Three intellectual skills are particularly important: (a)
the synthetic ability to see problems in new ways and
to escape the bounds of conventional thinking; (b) the
analytic ability to recognize which of ones ideas are
worth pursuing and which are not; and (c) the practi-
calcontextual ability to know how to persuade others
ofto sell other people onthe value of ones ideas.
The conuence of these three abilities is also important.
Analytic ability used in the absence of the other two
abilities results in powerful critical, but not creative,
thinking. Synthetic ability in the absence of the other
two abilities results in new ideas that are not subjected
to the scrutiny required to make them work. And prac-
ticalcontextual ability in the absence of the other two
may result in the transmittal of ideas not because the
ideas are good, but, rather, because the ideas have been
well and powerfully presented. To be creative, one must
rst decide to generate new ideas, analyze these ideas,
and sell the ideas to others. A detailed analysis of cog-
nition and intellectual processes in creativity can be
found in Ward and Kolomyts (2010).
Knowledge
Concerning knowledge, on the one hand, one needs to
know enough about a eld to move it forward. One
cant move beyond where a eld is if one doesnt know
where it is. On the other hand, knowledge about a eld
can result in a closed and entrenched perspective, result-
ing in a persons not moving beyond the way in which he
or she has seen problems in the past (Adelson, 1984;
Frensch & Sternberg, 1989). Thus, one needs to decide
to use ones past knowledge, but also decide not to let
the knowledge become a hindrance rather than a help.
Thinking Styles
Thinking styles are related to creativity (Kogan, 1973).
With regard to thinking styles, a legislative style is parti-
cularly important for creativity, that is, a preference for
thinking and a decision to think in new ways (Sternberg,
1997c; Zhang & Sternberg, 2006). This preference needs
to be distinguished from the ability to think creatively:
Someone may like to think along new lines, but not
think well, or vice versa. It also helps, to become a major
creative thinker, if one is able to think globally as well as
locally, distinguishing the forest from the trees and
thereby recognizing which questions are important and
which ones are not.
Personality
Numerous research investigations have supported the
importance of certain personality attributes for creative
functioning (Barron, 1969, 1988; Feist, 2010). These
attributes include, but are not limited to, willingness to
overcome obstacles, willingness to take sensible risks,
willingness to tolerate ambiguity, and self-efcacy. In
particular, buying low and selling high typically means
defying the crowd, so that one has to be willing to stand
up to conventions if one wants to think and act in cre-
ative ways. Note that none of these attributes are xed.
INVESTMENT-BASED ASSESSMENT OF CREATIVITY 5
D
o
w
n
l
o
a
d
e
d

b
y

[
U
n
i
v
e
r
s
i
t
y

o
f

T
e
n
n
e
s
s
e
e
,

K
n
o
x
v
i
l
l
e
]

a
t

1
2
:
3
8

2
4

J
a
n
u
a
r
y

2
0
1
3

One can decide to overcome obstacles, take sensible
risks, and so forth.
Motivation
Intrinsic, task-focused motivation is also essential to
creativity (Hennessey, 2010). The research of Teresa
Amabile (1996, 1999) and others has shown the impor-
tance of such motivation for creative work, and has sug-
gested that people rarely do truly creative work in an
area unless they really love what they are doing and
focus on the work, rather than the potential rewards.
Motivation is not something inherent in a person: One
decides to be motivated by one thing or another.
Environment
Finally, one needs an environment that is supportive
and rewarding of creative ideas (Sternberg & Lubart,
1995a; Sternberg & Williams, 1996). One could have
all of the internal resources needed to think creatively,
but without some environmental support (such as a
forum for proposing those ideas), the creativity that a
person has within him or her might never be displayed.
Different cultures support creativity differentially and
may even have different conceptions of what constitutes
creativity (Lubart, 2010) so it is important to take the
environment into account when assessing creativity.
Conuence
Concerning the conuence of components, creativity is
hypothesized to involve more than a simple sum of a per-
sons level on each component (Sternberg & Lubart,
1991). First, there may be thresholds for some compo-
nents (e.g., knowledge) belowwhich creativity is not poss-
ible, regardless of the levels on other components. Second,
partial compensation may occur in which a strength on
one component (e.g., motivation) counteracts a weakness
on another component (e.g., environment). Third, inter-
actions may also occur between components, such as
intelligence and motivation, in which high levels on both
component could multiplicatively enhance creativity.
Creative ideas are both novel and valuable. But, they
are often rejected because the creative innovator stands
upto vested interests and dees the crowd. The crowd does
not maliciously or willfully reject creative notions. Rather,
it does not realize, and often does not want to realize, that
the proposed idea represents a valid and advanced way of
thinking. Society generally perceives opposition to the sta-
tus quo as annoying, offensive, and reason enough to
ignore innovative ideas. Creativity thus cannot be under-
stood fully outside its societal context (Moran, 2010).
Evidence abounds that creative ideas are often
rejected. Initial reviews of major works of literature
and art are often negative. Toni Morrisons Tar Baby
received negative reviews when it was rst published,
as did Sylvia Plaths The Bell Jar. The rst exhibition
in Munich of the work of Norwegian painter Edvard
Munch opened and closed the same day because of the
strong negative response from the critics. Some of the
greatest scientic papers have been rejected not just by
one, but by several journals before being published.
For example, John Garcia, a distinguished biopsycholo-
gist, was immediately denounced (see Garcia, 1981)
when he rst proposed that a form of learning called
classical conditioning could be produced in a single trial
of learning (Garcia & Koelling, 1986).
From the investment view, then, the creative person
buys low by presenting a unique idea and then attempt-
ing to convince other people of its value. After convinc-
ing others that the idea is valuable, which increases the
perceived value of the investment, the creative person
sells high by leaving the idea to others and moving on
to another idea. People typically want others to love
their ideas, but immediate universal applause for an idea
usually indicates that it is not particularly creative.
RESEARCH SUPPORTING THE
INVESTMENT THEORY
Research within the investment framework has yielded
support for this model (Lubart & Sternberg, 1995;
OHara & Sternberg, 20002001). This research has used
tasks such as (a) writing short-stories using unusual
titles (e.g., the octopus sneakers), (b) drawing pictures
with unusual themes (e.g., the earth from an insects
point of view), (c) devising creative advertisements for
boring products (e.g., cufinks), and (d) solving unusual
scientic problems (e.g., How could we tell if someone
had been on the moon within the past month?). This
research showed creative performance to be moderately
domain-specic, and to be predicted by the combination
of resources specied by the theory (see Baer, 2010;
Sternberg, 2009).
In another study, creativity was measured using
open-ended measures (Sternberg & the Rainbow Project
Collaborators, 2005, 2006). These performance tasks
were expected to tap an important aspect of creativity
that might not be measured using multiple-choice items
alone, because open-ended measures require more
spontaneous and free-form responses.
For each of the tasks, participants were given a
choice of topic or stimuli on which to base their creative
stories or cartoon captions. Each of the creativity
performance tasks were rated on criteria that were
determined a priori as indicators of creativity.
Participants were given ve cartoons, minus their cap-
tions, purchased from the archives of the New Yorker.
6 STERNBERG
D
o
w
n
l
o
a
d
e
d

b
y

[
U
n
i
v
e
r
s
i
t
y

o
f

T
e
n
n
e
s
s
e
e
,

K
n
o
x
v
i
l
l
e
]

a
t

1
2
:
3
8

2
4

J
a
n
u
a
r
y

2
0
1
3

The participants task was to choose three cartoons, and to
provide a caption for each cartoon. Two trained judges
rated all the cartoons for cleverness, humor, originality,
and task appropriateness on 5-point scales. A combined
creativity score was formed by summing the individual rat-
ings oneachdimensionexcept taskappropriateness, which,
theoretically, is not a pure measure of creativity per se.
Participants were further asked to write two stories,
spending about 15 minutes on each, choosing from the
following titles: A Fifth Chance, 2983, Beyond
the Edge, The Octopuss Sneakers, Its Moving
Backwards, and Not Enough Time (Lubart &
Sternberg, 1995; Sternberg & Lubart, 1995a). A team
of six judges was trained to rate the stories. Each judge
rated the stories for originality, complexity, emotional
evocativeness, and descriptiveness on 5-point scales.
Participants also were presented with ve sheets of
paper, each containing a set of 11 to 13 images linked by
a common theme (keys, money, travel, animals playing
music, and humans playing music). There were no restric-
tions on the minimumor maximumnumber of images that
needed to be incorporated into the stories. After choosing
TABLE 1
Creative Kaleidoscope Prompts: Classes of 20112014
1. Historys great events often turn on small moments. For example, what if Rosa Parks had given up her seat on that bus? What if Pope John
Paul I had not died after a month in ofce in 1978? What if Gore had beaten Bush in Florida and won the 2000 U.S. Presidential election? Using
your knowledge of American or world history, choose a dening moment and imagine an alternate historical scenario if that key event had
played out differently.
2. Create a short story using one of the following topics:
a. The End of MTV
b. Confessions of a Middle School Bully
c. The Professor Disappeared
d. The Mysterious Lab
3. The human narrative is replete with memorable characters like Americas Johnny Appleseed, ancient Greeces Perseus, or the Fox Spirits of
East Asia. Imagine one of humanitys storied gures is alive and working in the world today. Why does Eric the Red have a desk job? Would
Shiva be a general or a diplomat? Is Quetzalcoatl trapped in a zoo? In short, connect your chosen gure to the contemporary world and imagine
the life he=she=it might lead. (Analytical & Creative)
4. Engineers and scientists like astronomer Edwin Powell Hubble discover new solutions to contemporary issues. Equipped with his ve senses,
Hubble said, man explores the universe around him and calls the adventure Science. Using your knowledge of scientic principles, identify
an adventure in science you would like to study and tell us how you would design an investigation to address it. What solution do you hope to
nd and why? (Creative & Wisdom)
5. Create a short story using one of the following topics (Creative):
a) One Way Ticket
b) Do Not Push
c) Gorillas or Guerillas?
d) Toast
e) The Back Seat on the School Bus
6. Use an 8.5 11 inch sheet of paper to create something. You can blueprint your future home, create a new product, design a costume or a
theatrical set, compose a score or do something entirely different. Let your imagination wander. (Creative & Practical)
7. Use one of the following topics to create a short story:
a. The Spam Filter
b. Seventeen Minutes Ago. . .
c. Two By Two
d. Facebook
e. Now Theres the Rub. . .
f. No Whip Half-Caf Latte
g. The Eleventh Commandment
8. Its 1781 and the American colonies were just defeated by the British at Yorktown. Imagine history without the United States as we know it.
9. Are we alone?
10. French anthropologist Claude Levi-Strauss observed The scientist is not a person who gives the right answers; hes one who asks the right
questions. Using your knowledge of scientic and=or mathematical principles, identify a question whose answer you seek and tell us how you
might go about investigating it.
11. Kermit the Frog famously lamented Its not easy being green. Do you agree?
12. Share a one-minute video that says something about you, upload it to YouTube or another easily accessible Web site, and give us the URL.
What you do or say is totally up to you. (Unfortunately, we are unable to watch videos that come in any form other than a URL link.)
13. Write a short story using one of the following titles:
a) House of Cards
b) The Poor Sport
c) Drama at the Prom
d) Election Night, 2044
e) The Getaway
D
o
w
n
l
o
a
d
e
d

b
y

[
U
n
i
v
e
r
s
i
t
y

o
f

T
e
n
n
e
s
s
e
e
,

K
n
o
x
v
i
l
l
e
]

a
t

1
2
:
3
8

2
4

J
a
n
u
a
r
y

2
0
1
3

one of the pages, the participant was given 15min to for-
mulate a short story and dictate it into a cassette recorder.
Six judges were trained to rate the stories. As with the
written stories, each judge rated the stories for originality,
complexity, emotional evocativeness, and descriptiveness
on 5-point scales.
Rasch reliability indexes for the composite person
ability estimates for the Written and Oral Stories were
very good (.79 and .80, respectively). The judges for
both the Written and Oral Stories varied greatly in terms
of their severity of ratings for the stories. For the Writ-
ten Stories, the judges also ranged in their t to the
model, although the reliability was still sound (rater
reliability .94). For the Oral Stories, all the judges t
the model very well, so their differences could be reliably
modeled (rater reliability .97).
Creativity-based performance tests formed a unique
factor in a factor analysis. Furthermore, the creativity
tests signicantly and substantially increased prediction
of rst-year college grade-point averages for over 700
highly diverse students from 13 colleges and universities
across the United States that varied widely in quality
and geographic location. Prediction was almost
doubled. The tests also substantially decreased ethnic-
group differences. The reason is that different groups
are socialized to be intelligent in different ways. For
example, American Indians performed relatively poorly
in comparison with other ethnic groups on the analytical
measure of the battery but had the highest scores on
oral-story telling.
In a more recent project, Kaleidoscope (Sternberg,
2007a, 2007b, 2007c, 2010; Sternberg & Cofn, 2010;
Sternberg et al., 2009), over the last 3 years, we have pre-
sented to all of the roughly more than 45,000 undergrad-
uate applicants to Tufts University, application essays
designed to assess creative thinking (among other
things). For example, they might be invited to say what
the world would be like today if the Nazis had won
World War II, or to create a short story with a title such
as The End of MTV or Confessions of a
Middle-School Bully, or to draw a design for a new
product or to draw an advertisement. We found that
we did not obtain the ethnic-group differences that are
typical of university-admissions assessments when we
used our tests, and that students who were accepted with
excellent creativity scores on Kaleidoscope did just as
well as students accepted as outstanding in other ways.
Those who were rated for Kaleidoscope (i.e., who
showed gifts in creative and related dimensions) outper-
formed other students academically after controlling for
high school grades and standardized test scores. Those
who excelled in Kaleidoscope also excelled, on average,
in extracurricular and leadership activities during their rst
year of college.
In another project, Aurora, for children roughly 812
years of age, we are currently standardizing around the
world an assessment that includes creativity items in
the verbal, numerical, and gural domains (Chart,
Grigorenko, & Sternberg, 2008; see Sternberg, 2007a).
In this way, it will be possible to extend our work down-
ward to a younger level. Examples of Aurora items are
given later in the article.
PROMPTS FOR ASSESSING CREATIVITY
IN STUDENTS
In this section, we rst consider some prompts to assess
creative thinking.
Assessing creativity means evaluating students as they
(a) create, (b) invent, (c) discover, (d) imagine if . . . , (e)
suppose that . . . , or (f) predict (Sternberg & Grigorenko,
2007; Sternberg, Jarvin, & Grigorenko, 2009, 2011;
Sternberg & Lubart, 1995b; Sternberg & Williams, 1996).
TABLE 2
Examples of Two Stories Written in Response to Prompt to Imagine History Had Taken a Different Path
Story that is Highly Creative and also Well-Written
If the Trojans had heeded Laocoons advice and thrown Odysseus wooden horse into the sea, they would have defeated the Greeks at Troy.
Aeneas would then never have had reason to ee the city, and he would never have ventured to Italy to found Rome. Without Rome, neither the
Roman Republic nor a Roman Empire would have existed. Concrete, the arch, plumbing, and the sauna might never have been invented. The
modern implications of Rome never having existed are indeed drastic. Lacking even concrete oors, people would resort to sleeping in the mud, and,
without plumbing or saunas, they would be perpetually lthy and, generally, quite chilly. France could not have built the base of the Eiffel Tower
without arches, so tourists would be unable to purchase miniature collectible Towers in Parisian convenience stores.
Story that is Well-Written but not Quite as Creative
What if the ratication of the nineteenth amendment did not pass and women were never given the right to vote? What would life for women, like
me, be like in the United States? For one thing, I probably would not be writing this essay. If women were not given their right to vote, I probably
would stop going to school after this year and it would be unlikely that I would receive a college education. Without suffrage, my career options
would be limited, if a career were a possibility at all. My accepted practices would be limited to staying home and taking care of the family. Rather
than being equals, women would be subservient to men. I might not drive, I might not dress in the way in which I choose to, and I might not be able
to live my life the way that I can in the twenty-rst century.
8 STERNBERG
D
o
w
n
l
o
a
d
e
d

b
y

[
U
n
i
v
e
r
s
i
t
y

o
f

T
e
n
n
e
s
s
e
e
,

K
n
o
x
v
i
l
l
e
]

a
t

1
2
:
3
8

2
4

J
a
n
u
a
r
y

2
0
1
3

Consider some examples of instructional or assessment
activities that assess students skills in thinking creatively.
1. Create an alternative ending to the short story
you just read that represents a different way
things might have gone for the main characters
in the story. [Literature]
2. Invent a dialogue between an American tourist in
Paris and a French man he encounters on the
street from whom he is asking directions on
how to get to the Rue Pigalle. [French]
3. Discover the fundamental physical principle that
underlies all of the following problems, each of
which differs from the others in the surface
structure of the problem but not in its deep
structure . . . . [Physics]
4. Imagine if the government of China keeps evolv-
ing over the course of the next 20 years in much
the same way it has been evolving. What do you
believe the government of China will be like in 20
years? [Government=Political Science]
5. Suppose that you were to design one additional
instrument to be played in a symphony orchestra
for future compositions. What might that instru-
ment be like, and why? [Music]
6. Predict changes that are likely to occur in the
vocabulary or grammar of spoken Spanish in the
border areas of the Rio Grande over the next 100
years as a result of continuous interactions between
Spanish and English speakers. [Linguistics]
EXAMPLES OF ASSESSMENTS
It might be helpful actually to display some of the
assessments we have used and their results. In Table 1,
I present items from the Kaleidoscope Project
(Sternberg, Bonney, et al., 2010; Sternberg & Cofn,
2010) that are intended to assess creativity. Table 2 pre-
sents two stories in response to a prompt asking students
to speculate on what the world would be like if some
event in history had come out differently. Both
responses are high in quality. However, one is higher
in creativity than the other. Table 3 presents a sample
verbal creativity item from the Aurora Project (Chart
et al., 2008).
Figure 1 presents a response from Sternberg and
Lubart (1995a) to a prompt asking students to draw
the Earth from an insects point of view. The response
is high in quality but not in creativity. Figure 2 presents
a response to the same prompt that is highly creative,
TABLE 3
Example of Problem From the Aurora Project
Imagine that the world has changed so that almost everything can
speak. Write a little conversation describing what the two things listed
might say to each other if they could talk. Each thing must say at least
one thing. Before you write what they will say, circle who is speaking.
Be creative.
FIGURE 1 Drawing of Earth from an insects point of view: High
quality, good creativity. (Figure is provided in color online.)
FIGURE 3 From the Aurora Project: A colorful collage used to mea-
sure creativity. (Figure is provided in color online.)
FIGURE 2 Drawing of Earth from an insects point of view: High
quality, excellent creativity. (Figure is provided in color online.)
D
o
w
n
l
o
a
d
e
d

b
y

[
U
n
i
v
e
r
s
i
t
y

o
f

T
e
n
n
e
s
s
e
e
,

K
n
o
x
v
i
l
l
e
]

a
t

1
2
:
3
8

2
4

J
a
n
u
a
r
y

2
0
1
3

showing different facets of an insects eye, each
displaying a different threat. Figure 3 presents a collage
from the Aurora Project, in which students must use
colored tiles to produce a creative collage. This one
was judged as highly creative. Figure 4 presents a car-
toon that requires a caption. And Figure 5 presents a
book cover, for which students are invited to suggest a
book title.
CONCLUSION
In conclusion, my colleagues and I have used the invest-
ment theory of creativity as a leaping-off point for
designing assessments of creativity that vary widely.
The assessments are both verbal and nonverbal, and
are for ages ranging from later elementary school
through adulthood. Our results with these assessments
have been promising. But they are limited. They do
not sample all domains and have been on limited popu-
lations. Moreover, we have studied individual creativity
but creativity can function somewhat differently in
groups (Sawyer, 2010). Our hope is that, sooner or later,
assessments such as ours and going beyond ours will
become standard parts of batteries used to assess stu-
dent skills and talents.
REFERENCES
Adelson, B. (1984). When novices surpass experts: The difculty of a
task may increase with expertise. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 10(3), 483495.
Albert, R. S., & Runco, M. A. (1999). A history of research on
creativity. In R. J. Sternberg (Ed.), Handbook of creativity (pp.
1631). New York, NY: Cambridge University Press.
Amabile, T. M. (1996). Creativity in context. Boulder, CO: Westview.
Amabile, T. M. (1999). How to kill creativity. In Harvard business
review on breakthrough thinking (pp. 128). Cambridge, MA:
Harvard.
Baer, J. (2010). Is creativity domain-specic? In J. C. Kaufman & R. J.
Sternberg (Eds.), Cambridge handbook of creativity (pp. 321341).
New York, NY: Cambridge University Press.
Baer, J., & Kaufman, J. C. (2006). Creativity research in
English-speaking countries. In J. C. Kaufman, & R. J. Sternberg
(Eds.) The international handbook of creativity (pp. 1038). New
York, NY: Cambridge University Press.
Barron, F. (1969). Creative person and creative process. New York,
NY: Holt, Rinehart & Winston.
Barron, F. (1988). Putting creativity to work. In R. J. Sternberg (Ed.),
The nature of creativity (pp. 7698). New York, NY: Cambridge
University Press.
Beghetto, R. A. (2010). Creativity in the classroom. In J. C. Kaufman
& R. J. Sternberg (Eds.), Cambridge handbook of creativity (pp. 447
Chart, H., Grigorenko, E. L., &Sternberg, R. J. (2008). Identication: The
Aurora battery. In J. A. Plucker &C. M. Callahan (Eds.), Critical issues
and practices in gifted education (pp. 281301). Waco, TX: Prufrock.
Csikszentmihalyi, M. (1988). Society, culture, and person: A systems
view of creativity. In R. J. Sternberg (Ed.), The nature of creativity
(pp. 325339). New York, NY: Cambridge University Press.
Csikszentmihalyi, M. (1990). The domain of creativity. In M. A.
Runco & R. S. Albert (Eds.), Theories of creativity (pp. 190212).
Newbury Park, CA: Sage.
Csikszentmihalyi, M. (1996). Creativity: Flow and the psychology of
discovery and invention. New York, NY: HarperCollins.
Csikszentmihalyi, M. (1999). Implications of a systems perspective for
the study of creativity. In R. J. Sternberg (Ed.), Handbook of crea-
tivity (pp. 313335). New York, NY: Cambridge University Press.
Feist, G. J. (2010). The function of personality in creativity: The nature
and nurture of the creative person. In J. C. Kaufman & R. J.
FIGURE 4 From the Aurora Project: A cartoon for which the
participant must provide a caption.
FIGURE 5 From the Aurora Project: A potential book cover. What
might the book be about? (Figure is provided in color online.)
10 STERNBERG
D
o
w
n
l
o
a
d
e
d

b
y

[
U
n
i
v
e
r
s
i
t
y

o
f

T
e
n
n
e
s
s
e
e
,

K
n
o
x
v
i
l
l
e
]

a
t

1
2
:
3
8

2
4

J
a
n
u
a
r
y

2
0
1
3

Frensch, P. A., & Sternberg, R. J. (1989). Expertise and intelligent
thinking: When is it worse to know better? In R. J. Sternberg
(Ed.), Advances in the psychology of human intelligence. Vol. 5
(pp. 157188). Hillsdale, NJ: Lawrence Erlbaum Associates.
Garcia, J. (1981). Tilting at the paper mills of academe. American
Psychologist, 36(2), 149158.
Garcia, J., & Koelling, R. A. (1966). The relation of cue to conse-
quence in avoidance learning. Psychonomic Science, 4, 123124.
Gardner, H. (1983). Frames of mind: The theory of multiple
intelligences. New York, NY: Basic Books.
Gardner, H. (1991). The unschooled mind. New York, NY: Basic Books.
Gardner, H. (1993). Creating minds. New York, NY: Basic Books.
Gardner, H. (2006). Multiple intelligences: New horizons. New York,
NY: Perseus.
Guilford, J. P. (1950). Creativity. American Psychologist, 5, 444454.
Hennessey, B. A. (2010). The creativitymotivation connection. In J.
C. Kaufman & R. J. Sternberg (Eds.), Cambridge handbook of crea-
Johnson-Laird, P. N. (1988). Freedom and constraint in creativity. In
R. J. Sternberg (Ed.), The nature of creativity (pp. 202219).
Kaufman, J. C. (2001a). Genius, lunatics, and poets: Mental illness in
prize-winning authors. Imagination, Cognition, and Personality,
20(4), 305314.
Kaufman, J. C. (2001b). The Sylvia Plath effect: Mental illness in emi-
nent creative writers. Journal of Creative Behavior, 35(1), 3750.
Kaufman, J. C., & Beghetto, R. A. (2009). Beyond big and little: The
four C model of creativity. Review of General Psychology, 13, 112.
Kaufman, J. C., & Sternberg, R. J. (Eds.) (2006). International hand-
book of creativity. New York, NY: Cambridge University Press.
Kaufman, J. C., & Sternberg, R. J. (Eds.) (2010). Cambridge handbook
of creativity. New York, NY: Cambridge University Press.
Kim, H., Cramond, B., & VanTassel-Baska, J. (2010). The relationship
between creativity and intelligence. In J. C. Kaufman & R. J.
Kogan, N. (1973). Creativity and cognitive style: A life-span perspec-
tive. In P. B. Baltes & K. W. Schaie (Eds.), Life-span developmental
psychology: Personality and socialization (pp. 145178). New York,
NY: Academic Press.
Kozbelt, A., Beghetto, R. A., & Runco, M. A. (2010). Theories of crea-
tivity. In J. C. Kaufman & R. J. Sternberg (Eds.), Cambridge hand-
book of creativity (pp. 2047). New York, NY: Cambridge
University Press.
Locher, P. J. (2010). How does a visual artist create an artwork? In J.
Lubart, T. I. (2010). Cross cultural perspectives on creativity. In J. C.
Kaufman & R. J. Sternberg (Eds.), Cambridge handbook of creativ-
ity (pp. 265278). New York, NY: Cambridge University Press.
Lubart, T. I., & Sternberg, R. J. (1995). An investment approach to
creativity: Theory and data. In S. M. Smith, T. B. Ward & R. A.
Finke (Eds.), The creative cognition approach (pp. 269302).
Cambridge, MA: MIT Press.
MacKinnon, D. W. (1965). Personality correlates of creativity. In M. J.
Aschner & C. E. Bish (Eds.), Productive thinking in education (pp.
159171). Washington, DC: National Education Association.
Maslow, A. (1967). The creative attitude. In R. L. Mooney & T. A.
Rasik (Eds.), Explorations in creativity (pp. 4357). New York,
NY: Harper & Row.
Moran, S. (2010). The roles of creativity in society. In J. C. Kaufman
& R. J. Sternberg (Eds.), Cambridge handbook of creativity (pp. 74
OHara, L. A., & Sternberg, R. J. (20002001). It doesnt hurt to ask:
Effects of instructions to be creative, practical, or analytical on
essay-writing performance and their interaction with students
thinking styles. Creativity Research Journal, 13, 197210.
Plucker, J. A., & Makel, M. C. (2010). Assessment of creativity. In J.
Renzulli, J. S. (1986). The three-ring conception of giftedness: a devel-
opmental model for creative productivity. In R. J. Sternberg & J. E.
Davidson (Eds.), Conceptions of giftedness (pp. 5392). New York,
NY: Cambridge University Press.
Richards, R. (2010). Everyday creativity: process and way of lifefour
key issues. In J. C. Kaufman & R. J. Sternberg (Eds.), Cambridge
handbook of creativity (pp. 189215). New York, NY: Cambridge
University Press.
Runco, M. A. (2010). Divergent thinking, creativity, and ideation. In J.
Russ, S. W., & Fiorelli, J. A. (2010). Developmental approaches to
creativity. In J. C. Kaufman & R. J. Sternberg (Eds.), Cambridge
handbook of creativity (pp. 233249). New York, NY: Cambridge
University Press.
Sawyer, R. K. (2010). Individual and group creativity. In J. C.
Schank, R. C. (1988). The creative attitude. New York, NY:
Macmillan.
Silvia, P. J., & Kaufman, J. C. (2010). Creativity and mental illness. In
J. C. Kaufman & R. J. Sternberg (Eds.), Cambridge handbook of
creativity (pp. 381394). New York, NY: Cambridge University
Press.
Simonton, D. K. (1984). Genius, creativity, and leadership. Cambridge,
MA: Harvard University Press.
Simonton, D. K. (1988). Scientic genius. New York, NY: Cambridge
University Press.
Simonton, D. K. (1994). Greatness: Who makes history and why?
New York, NY: Guilford.
Simonton, D. K. (2010). Creativity in highly eminent individuals. In
Press.
Smith, J. K., & Smith, L. F. (2010). Educational creativity. In
Press.
Sternberg, R. J. (1994). Allowing for thinking styles. Educational Lead-
ership, 52(3), 3640.
Sternberg, R. J. (1997a). The concept of intelligence and its role in life-
long learning and success. American Psychologist, 52, 10301037.
Sternberg, R. J. (1997b). Successful intelligence. New York, NY:
Plume.
Sternberg, R. J. (1997c). Thinking styles. New York, NY: Cambridge
University Press.
Sternberg, R. J. (Ed.). (1999). Handbook of creativity. New York, NY:
Cambridge University Press.
Sternberg, R. J. (2003a). Teaching for successful intelligence: Princi-
ples, practices, and outcomes. Educational and Child Psychology,
20(2), 618.
Sternberg, R. J. (2003b). Wisdom, intelligence, and creativity, synthe-
sized. New York, NY: Cambridge University Press.
Sternberg, R. J. (2004). Good intentions, bad results: A dozen reasons
why the No Child Left Behind (NCLB) Act is failing our nations
schools. Education Week, 24(9), 4256.
Sternberg, R. J. (2006a). Creativity is a habit. Education Week, 25(24),
4764.
D
o
w
n
l
o
a
d
e
d

b
y

[
U
n
i
v
e
r
s
i
t
y

o
f

T
e
n
n
e
s
s
e
e
,

K
n
o
x
v
i
l
l
e
]

a
t

1
2
:
3
8

2
4

J
a
n
u
a
r
y

2
0
1
3

Sternberg, R. J. (2006b). Creative leadership: Its a decision. Leader-
ship, 36, 2224.
Sternberg, R. J. (2006c). Stalking the elusive creativity quark: Toward
a comprehensive theory of creativity. In P. Locher, C. Martindale,
L. Dorfman, V. Petrov, & D. Leontiev (Eds.), New directions in
aesthetics, creativity, and the arts (pp. 79104). Amityville, NY:
Baywood.
Sternberg, R. J. (2007a). Finding students who are wise, practical, and
creative. The Chronicle of Higher Education, 53(44), B11.
Sternberg, R. J. (2007b). The Kaleidoscope Project. SPIM Newsletter,
Summer Issue.
Sternberg, R. J. (2009). Domain-generality versus domain-specicity of
creativity. In P. Meusburger, J. Funke & E. Wunder (Eds.), Milieus
of creativity: An interdisciplinary approach to spatiality of creativity
(pp. 2538). New York, NY: Springer.
Sternberg, R. J. (2010). College admissions for the 21st century.
Cambridge, MA: Harvard University Press.
Sternberg, R. J., Bonney, C. R., Gabora, L., Jarvin, L., Karelitz, T.
M., & Cofn, L. (2010). Broadening the spectrum of undergraduate
admissions. College and University, 86, 217.
Sternberg, R. J., & Cofn, L. A. (2010). Admitting and developing
new leaders for a changing world. New England Journal of Higher
Education, 24, 1213.
Sternberg, R. J., & Grigorenko, E. L. (2004). Successful intelligence in
the classroom. Theory Into Practice, 43, 274280.
Sternberg, R. J., & Grigorenko, E. L. (2007). Teaching for successful
intelligence (2nd ed.). Thousand Oaks, CA: Corwin.
Sternberg, R. J., Jarvin, L., & Grigorenko, E. L. (2009). Teaching for wis-
dom, intelligence, creativity, and success. Thousand Oaks, CA: Corwin.
Sternberg, R. J., Jarvin, L., & Grigorenko, E. L. (2011). Explorations
of giftedness. New York, NY: Cambridge University Press.
Sternberg, R. J., Kaufman, J. C., & Grigorenko, E. L. (2008). Applied
intelligence. New York, NY: Cambridge University Press.
Sternberg, R. J., Kaufman, J. C., & Pretz, J. E. (2002). The creativity
conundrum: A propulsion model of kinds of creative contributions.
New York, NY: Psychology Press.
Sternberg, R. J., & Lubart, T. I. (1991). An investment theory of
creativity and its development. Human Development, 34(1), 131.
Sternberg, R. J., & Lubart, T. I. (1995a). Defying the crowd: Cultivat-
ing creativity in a culture of conformity. New York, NY: Free Press.
Sternberg, R. J., & Lubart, T. I. (1995b). Ten keys to creative
innovation. R & D Innovator, 4(3), 811.
Sternberg, R. J., & Lubart, T. I. (1995c). Ten tips toward creativity in
the workplace. In C. M. Ford & D. A. Gioia (Eds.), Creative action
in organizations: Ivory tower visions and real world voices (pp. 173
180). Newbury Park, CA: Sage.
Sternberg, R. J., & the Rainbow Project Collaborators (2005).
Augmenting the SAT through assessments of analytical, practical,
and creative skills. In W. Camara & E. Kimmel (Eds.), Choosing stu-
dents: Higher education admission tools for the 21st century (pp. 159
176). Mahwah, NJ: Lawrence Erlbaum Associates.
Sternberg, R. J., & the Rainbow Project Collaborators (2006). The
Rainbow Project: Enhancing the SAT through assessments of ana-
lytical, practical and creative skills. Intelligence, 34(4), 321350.
Sternberg, R. J., & Williams, W. M. (1996). How to develop student
creativity. Alexandria, VA: Association for Supervision and Cur-
riculum Development.
Tharp, T. (2005). The creative habit: Learn it and use it for life. New
York, NY: Simon & Schuster.
Torrance, E. P. (1962). Guiding creative talent. Englewood Cliffs, NJ:
Prentice-Hall.
Ward, T. B., & Kolomyts, Y. (2010). Cognition and creativity. In J. C.
Zhang, L. F., & Sternberg, R. J. (2006). The nature of intellectual
styles. Mahwah, NJ: Lawrence Erlbaum Associates.
12 STERNBERG
D
o
w
n
l
o
a
d
e
d

b
y

[
U
n
i
v
e
r
s
i
t
y

o
f

T
e
n
n
e
s
s
e
e
,

K
n
o
x
v
i
l
l
e
]

a
t

1
2
:
3
8

2
4

J
a
n
u
a
r
y

2
0
1
3

The Assessment of Creativity: An Investment Based Approach

Uploaded by

Document Information

Copyright

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

The Assessment of Creativity: An Investment Based Approach

Uploaded by

Copyright:

The Assessment of Creativity: An Investment-Based Approach

You might also like