This article was downloaded by: [Northeastern University]

On: 11 November 2014, At: 18:54

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

Creativity Research Journal

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/hcrj20

Application of the Rasch Model to the Measurement of

Creativity: The Creative Achievement Questionnaire
a a a a
Chia-Chi Wang , Hsiao-Chi Ho , Chih-Ling Cheng & Ying-Yao Cheng
a
National Sun Yat-Sen University , Taiwan
Published online: 26 Feb 2014.

To cite this article: Chia-Chi Wang , Hsiao-Chi Ho , Chih-Ling Cheng & Ying-Yao Cheng (2014) Application of the Rasch Model
to the Measurement of Creativity: The Creative Achievement Questionnaire, Creativity Research Journal, 26:1, 62-71, DOI:
10.1080/10400419.2013.843347

To link to this article: http://dx.doi.org/10.1080/10400419.2013.843347

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and Francis shall not be liable for
any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of
the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 CREATIVITY RESEARCH JOURNAL, 26(1), 62–71, 2014
Copyright © Taylor & Francis Group, LLC
ISSN: 1040-0419 print/1532-6934 online
DOI: 10.1080/10400419.2013.843347

Application of the Rasch Model to the Measurement of

Creativity: The Creative Achievement Questionnaire
Chia-Chi Wang, Hsiao-Chi Ho, Chih-Ling Cheng, and Ying-Yao Cheng
National Sun Yat-Sen University, Taiwan

This study was designed to provide multiple sources of evidence of the validity of the
Downloaded by [Northeastern University] at 18:54 11 November 2014

Creative Achievement Questionnaire (CAQ) and to clarify the hierarchy of creative

achievement using Rasch analyses. A total of 905 Taiwanese participants (345 men
and 558 women) completed the CAQ online. The Rasch model was used to assess
model–data fit. A differential item functioning (DIF) analysis was conducted to assess
the consistency of the ratings provided by males and females. The results revealed that
the 10 dimensions of the CAQ showed good model–data fit and supported the two-
factor classification (arts and sciences). Additionally, several items from each domain
exhibited substantial DIF across genders. Moreover, with the exception of dance, cre-
ative writing, architectural design, theater and film, and culinary arts, items in the 10
domains were hierarchically ordered. Suggestions for future research to revise the
CAQ were proposed.

Creative achievement can be defined as the sum of
creative products that are original, valuable, and pro-
duced by an individual over the course of his or her
lifetime (Carson, Peterson, & Higgins, 2005; Helson &
Pals, 2000). Common methods for measuring creative
achievement include the use of verifiable accomplish-
ments or honors, ratings of existing creative products
by experts, and self-report achievement inventories
(SAIs) as markers. Although time-consuming, SAIs
are regarded as more comprehensive than other meth-
ods because they can be designed to measure multiple
dimensions of achievement.
A number of self-report inventories have been
designed to assess creative achievement: (a) the Creative
Behavior Inventory developed by Hocevar (1979) is a list
of 90 creative actions in the domains of literature, music,
miscellaneous, math/science, art, and performing arts;
(b) the Creative Achievement Scale (Ludwig, 1992) mea-
sures remarkable creative achievements in terms of per-
sonal attributes, product qualities, and sociocultural
factors; (c) the Creative Achievement Questionnaire
(CAQ) developed by Carson et al. (2005) uses
Correspondence should be sent to Ying-Yao Cheng, Institute of
Education, National Sun Yat-sen University, 70 Lienhai Rd.,
Kaohsiung 80424, Taiwan. E-mail: chengyy@mail.nsysu.edu.tw
rank-ordered items to evaluate the level of creative
achievement across 10 domains including visual arts,
music, dance, creative writing, architectural design,
humor, invention, scientific inquiry, theater and film,
and culinary arts; and (d) the Biographical Inventory of
Creative Behaviours developed by Batey and Furnham
(2008), evaluates creative achievements in terms of 34
activities (e.g., writing a short story, producing one’s own
Web site, publishing research, designing and planting a
garden, composing a piece of music).
The CAQ is the most frequently used self-report inven-
tory of creative achievement and has been cited 49 times
in the Social Sciences Citation Index, 45 times in
PsycINFO, and 124 times in Google Scholar between
December 2005 and December 2012 (e.g., Batey, 2012;
Reiter-Palmon, Robinson-Morral, & Kaufman, 2012;
Simonton, 2012). The CAQ was based on a comprehen-
sive review of the creative accomplishment areas identi-
fied by previous research and included both arts and
science domains (Carson et al., 2005; Colangelo, Kerr,
Hallowell, Huesman, & Gaeth, 1992; Hocevar, 1979;
MacKinnon, 1962; Taylor & Ellison, 1967; Torrance,
1972). Furthermore, the CAQ was the first measure of
creative achievement to include the culinary arts as a
domain of creative achievement. Although Carson and
colleagues did not clearly explain why the culinary arts
 THE CREATIVE ACHIEVEMENT QUESTIONNAIRE 63

was regarded as an area for creative achievement, their METHOD

novel idea was supported by the findings of Horng and
Hu (2008), which showed that culinary creativity involves Participants
food, art, esthetics, and sciences such as physics, chemis-
Data collection proceeded in two phases involving two
try, and biology.
groups of participants. In the first phase, responses were
Despite the novel aspects of the CAQ, the instru-
obtained from 905 participants (92.6% of deliveries),
ment has several limitations with regard to validity.
including 345 men, 558 women, and 2 who did not spec-
First, the CAQ was developed using a homogeneous
ify their sex. Subject ages ranged from 14 to 78 years
sample of relatively gifted university students; thus, the
(M = 23.32 years, SD = 5.34). Participants were asked to
instrument may not be widely representative and have
complete the CAQ on a free Web site and their Internet
an inference bias. Second, the CAQ is a domain-specific
protocol (IP) addresses were recorded so repeated
tool used to assess 10 areas of creative achievement;
responses could be deleted from the database.
however, the total score for the 10 domains is used to
The CAQ asked participants to specify in which of
indicate overall creative performance, which appears to
13 creative areas they perceived themselves to be more
violate domain specificity, one of the major assump-
talented or competent than the average person. Of the
tions of the CAQ. Third, Carson et al. (2005) provided
905 participants, 31.93% (n = 289) chose visual arts
Downloaded by [Northeastern University] at 18:54 11 November 2014

evidence for construct validity by extracting common

(painting, sculpture), 37.46% (n = 339) music, 11.16%
factors using an exploratory factor analysis. As the data
(n = 101) dance, 19.12% (n = 173) individual sports (ten-
analysis was conducted on responses from participants
nis, golf), 23.87% (n = 216) team sports, 10.83% (n = 98)
and not on theory, the resulting factor structure might
architectural design, 10.17% (n = 92) entrepreneurship,
be inconsistent. Fourth, although the CAQ relies on a
30.61% (n = 277) creative writing, 37.35% (n = 338)
hierarchy of items, the hierarchy of creative achieve-
humor, 11.49% (n = 104) inventions, 18.45% (n = 167)
ments was based on consultations with only two domain
scientific inquiry, 25.64% (n = 232) theater and film, and
experts, which suggests a lack of objectivity. Moreover,
24.64% (n = 223) culinary arts.
like most of self-report creativity achievement measures
As the first phase of the study showed that the test
built upon classical test theory (CTT), the original test
results for five domains did not fit the theoretical hier-
scores of CAQ are unsuitable for data analysis since
archy, in the second phase, one expert in each of those
they are not based on interval scales and have biases
domains was invited to rerank the CAQ domain order
resulting from being dependent on test sample (Wang,
and provide suggestions for further investigations of the
2004).
order of items. Several criteria were used to select the
In recent years, Rasch models have been widely used
domain experts: more than 10 years of professional
to test the quality of questionnaires in fields such as
experience in the specific domain, receipt of awards in
human science, education, medical science, manage-
regional or national competitions, or certification from
ment (Wang, 2004), and creativity research (Teo &
national institutes. In the dance domain, Expert A was
Waugh, 2010; Wang & Cheng, 2000). Rasch models may
a female creative director of a dance company and a
overcome some of the limitations of psychological test-
professional dance instructor who had won several
ing associated with CTT; for example, when the observed
regional and national awards. In the architectural design
responses fit well with the model, the ordinal scale can
domain, Expert B was a female studying at an architec-
be converted into an interval scale using a Rasch analy-
tural institute who specialized in architectural design,
sis. Additionally, because Rasch measurement is based
urban design, green architecture, and indoor design.
on theoretical foundations, it can avoid the inconsistent
She had passed the national examinations for architec-
factor structure arising from exploratory factor analy-
ture in Taiwan. In the domain of creative writing,
sis. Furthermore, Rasch analysis not only examines the-
Expert C was a female editor of a local art magazine. In
oretical constructs, but also provides information
the theater and film domain, Expert D was a female
regarding item hierarchies and latent traits. Overall,
professional who had worked in an advertising com-
the quality of an assessment tool can be tested using
pany for more than 10 years. In the culinary arts domain,
the multiple sources of evidence offered by the Rasch
Expert E was a female Chinese cooking instructor and
measurement approach.
certified chef.
This study used the CAQ to demonstrate the use of
the Rasch model to validate a creativity measure. This
study also assessed the content validity, structural valid-
Instrument
ity, generalizability, substantive validity, and interpret-
ability (Messick, 1994, 1995a, 1995b; Wolfe & Smith, This study used the CAQ as the instrument. Regarding
2007) of the CAQ using a Rasch analysis, and suggestions back translation process of the CAQ, two bilingual
for CAQ revisions were made. experts were asked to translate the English version of the
 64 WANG ET AL.
questionnaire into Chinese, and one expert ensured con-
sistency between the versions.
The present study involved Parts One and Two of
the CAQ, yielding a total of 93 items. The CAQ is a
self-report tool with rank-ordered items designed to
evaluate the level of achievement across 10 domains of
creativity. Items in Part One measure talents in 13
areas, and participants were asked to select areas in
which they perceived themselves to be more talented or
competent than the average person. Items in Part Two
evaluate concrete achievements in the 10 domains of
visual arts, music, dance, creative writing, architectural
design, humor, inventions, scientific inquiry, theater
and film, and culinary arts. Each domain is assessed by
asking participants to rank eight items from lowest to
highest.
Downloaded by [Northeastern University] at 18:54 11 November 2014
Analyses
The Rasch model (Rasch, 1960) consists of two primary
parameters: person’s (n) ability (θn) and item (i) difficulty
(δi). When a person n responds to item i, the probability
of that person being correct on that item is:
exp(θ n − δ i )
Pni1 = , (1)
1 + exp(θ n − δ i )
where Pni1 is the probability of person n scoring 1 on item
i. Additionally, θn, which is the latent trait level of person
n, refers to the construct that is the target of the measure-
ment (e.g., personality, attitudes, interests, values, perfor-
mance), and δj is the overall difficulty of item i (e.g.,
difficulty or threshold value). Objective responses refer to
dichotomous scores (e.g., success or failure, agree or dis-
agree), and Pni1 and Pni0 denote the probabilities of per-
son n scoring 1 and 0, respectively, on item i. Thus,
Pni1 + Pni0 = 1.
The Rasch model was employed because each item
of the CAQ has binary response categories (yes/no).
Parameters in the Rasch model were estimated
using the ConQuest computer program (Wu, Adams, &
Wilson, 2007).
With regard to content and structural validity, the
Rasch model was used to examine the fit of each item
and the unidimensionality of each domain. When data
fit the expectation generated by the model, the infit
(weighted) mean square error (MNSQ) had an
expected value of unity. MNSQ values between 0.6
and 1.4 were set as the criteria for a reasonably good
model–data fit (Wright & Linacre, 1994). Additionally,
the two-factor structure identified by Carson et al.
(2005) was used to classify creative achievements in
terms of arts and sciences. The domains of visual arts,
music, dance, creative writing, humor, and theater
and film were categorized as arts, whereas inventions,
scientific discoveries, and the culinary arts were cate-
gorized as sciences. Using a multidimensional Rasch
analysis, this study examined whether the two-factor
construct of the CAQ fit the model’s expectations and
calculated the person separation reliability and
correlations.
With respect to generalizability, differential item
functioning (DIF; Holland & Wainer, 1993) analyses
were conducted across genders. A difference of 0.5 logits
in the overall difficulty of items across groups was
regarded as a substantial DIF (Wang, 2008). A multidi-
mensional Rasch analysis was conducted to obtain an
accurate estimate of the correlations among the 10 sub-
scales and to increase person reliability (Cheng, Wang, &
Ho, 2009). Finally, to evaluate the substantive aspect of
validity, the hypothesized item hierarchy was confirmed
by comparing it with the empirically derived hierarchy
of item calibrations using Spearman rank correlations
(Wilson, 2005).
RESULTS
Data Analysis
Content and structural validity. The Rasch model
was used to examine the unidimensionality of each of
the 10 CAQ subscales. The results revealed that all
items in each domain had acceptable infit MNSQs
(Table 1). Data from each domain fit the Rasch model
well, indicating that the items in each subscale meas-
ured a unidimensional construct. The multidimen-
sional Rasch analysis conducted within the 10 domains
of the CAQ indicated that not all items in each domain
fit the Rasch model well (Table 1). Furthermore, multi-
dimensional Rasch analysis conducted on the two-fac-
tor construct (arts and sciences) derived from the 10
domains exhibited good model–data fit, confirming
the two-factor structure of creative achievement
(Table 1). Table 2 shows the correlations among the 10
domains.
Generalizability validity. DIF analyses were con-
ducted separately for each of the 10 CAQ subscales. All
items in each domain, with the exception of theater and
film, were associated with substantial DIF between the
genders. A total of nine items in all domains exhibited
substantial DIF for men and women, indicating that the
meaning of these items differed for men and women.
The person separation reliabilities of the 10 subscales,
shown in Table 2, ranged from .63 to .77. Of these sub-
scales, the person separation reliability in the dance
domain (.63) was lower than were those in the other
 THE CREATIVE ACHIEVEMENT QUESTIONNAIRE 65

TABLE 1
Estimates of Item Difficulty and Goodness-of-Fit Values for the Arts and Sciences by Multidimensional Rasch Analysis for Each Domain by
Unidimensional and Multidimensional Rasch Analyses

Arts & Science Factor by Each Domain by Each Domain by

Multidimensional Rasch. Multidimensional Rasch Unidimensional Rasch

Factor Domain Item Estimate Infit MNSQ Estimate Infit MNSQ Estimate Infit MNSQ

Factor 1 Arts Visual arts I1 –3.04 0.96 –2.38 0.97 –2.41 1.01
I2 –3.25 1.05 –2.65 1.16 –2.68 1.13
I3 –2.29 1.00 –1.41 0.98 –1.43 0.99
I4 –1.75 1.03 –0.71 0.95 –0.73 0.90
I5 –1.09 1.03 0.12 1.01 0.12 0.90
I6 0.54 1.02 2.09 1.21 2.12 1.02
I7 0.44 0.97 1.97 1.10 2.00 1.01
I8 1.33 1.03 2.97 1.20 3.03 1.07
Music I1 –3.62 1.05 –4.76 1.02 –5.38 0.99
I2 –2.71 1.05 –3.58 0.99 –4.05 0.96
I3 –0.07 1.02 –0.35 1.07 –0.38 1.01
Downloaded by [Northeastern University] at 18:54 11 November 2014

I4 –0.53 1.03 –0.88 1.10 –0.98 1.02

I5 2.24 1.00 2.15 0.98 2.41 1.11
I6 1.61 1.04 1.51 1.00 1.69 1.04
I7 3.30 1.09 3.14 0.91 3.57 1.25
I8 2.88 1.06 2.77 0.98 3.12 1.20
Dance I1 –2.03 1.07 –4.33 1.13 –5.01 1.11
I2 1.17 1.03 0.49 1.33 0.56 1.03
I3 0.16 1.00 –1.00 1.05 –1.14 0.83
I4 0.05 1.01 –1.16 0.98 –1.33 0.79
I5 2.15 1.05 1.84 1.12 2.12 1.28
I6 1.17 1.02 0.48 1.13 0.56 0.85
I7 2.07 1.05 1.72 1.23 1.99 1.13
I8 2.25 1.05 1.96 1.15 2.25 1.27
Creative writing I1 –3.11 0.97 –2.92 1.08 –3.02 1.09
I2 –3.00 1.00 –2.78 1.01 –2.87 0.96
I3 –1.77 0.98 –1.20 0.99 –1.22 0.97
I4 –0.47 1.04 0.44 1.14 0.45 1.10
I5 1.42 1.02 2.58 1.10 2.68 1.04
I6 –0.07 1.03 0.92 1.08 0.94 0.99
I7 –0.22 1.00 0.74 1.00 0.76 0.91
I8 1.07 0.98 2.23 1.10 2.28 0.98
Humor I1 –4.08 1.03 –5.08 1.00 –5.46 0.98
I2 –2.61 1.08 –3.16 1.00 –3.37 0.95
I3 –2.11 1.06 –2.51 1.00 –2.66 0.97
I4 –0.30 1.05 –0.25 1.10 –0.23 1.04
I5 2.00 1.05 2.36 1.16 2.53 1.10
I6 1.92 1.03 2.29 1.15 2.45 1.11
I7 3.08 1.08 3.53 1.21 3.70 1.24
I8 2.46 1.07 2.83 1.16 3.04 1.17
Theater/film I1 –2.09 1.01 –3.03 1.08 –3.72 1.11
I2 –1.34 0.99 –2.17 1.04 –2.64 0.96
I3 1.78 1.03 1.32 1.15 1.62 1.18
I4 0.11 0.95 –0.53 0.97 –0.61 0.91
I5 1.17 1.00 0.65 1.05 0.82 1.01
I6 0.94 0.98 0.39 1.00 0.51 0.94
I7 1.73 1.01 1.26 1.09 1.54 1.00
I8 2.46 1.02 2.11 1.25 2.47 1.26
Factor 2 Science Invention I1 –3.54 0.94 –3.98 1.06 –4.17 1.04
I2 –3.15 0.96 –3.48 1.08 –3.63 1.05
I3 –1.59 0.95 –1.44 0.97 –1.49 0.89
I4 0.69 0.97 1.34 1.13 1.39 1.11
I5 –0.89 0.96 –0.55 1.00 –0.57 0.89
I6 1.76 1.03 2.53 1.11 2.65 1.13
I7 1.74 0.96 2.53 1.12 2.64 1.17

(Continued)
 66 WANG ET AL.

TABLE 1
(Continued)

Arts & Science Factor by Each Domain by Each Domain by

Multidimensional Rasch. Multidimensional Rasch Unidimensional Rasch

Factor Domain Item Estimate Infit MNSQ Estimate Infit MNSQ Estimate Infit MNSQ

I8 2.21 1.02 3.05 1.20 3.17 1.21

Scientific discovery I1 –3.42 1.01 –3.66 1.07 –3.98 1.04
I2 –3.23 0.98 –3.41 1.03 –3.71 1.06
I3 –1.13 0.98 –0.81 1.07 –0.83 1.01
I4 0.22 0.92 0.83 1.10 0.96 0.92
I5 0.07 0.93 0.65 1.17 0.76 0.93
a
I6 1.97 0.99 3.04 1.47 3.11 1.32
I7 0.54 0.84 1.21 0.99 1.38 0.85
I8 1.30 0.93 2.15 1.26 2.32 1.08
Culinary arts I1 –2.82 1.26 –3.00 1.05 –3.85 1.08
I2 –2.68 1.25 –2.89 1.04 –2.57 1.04
I3 2.35 1.09 1.25 0.73 –2.44 0.97
Downloaded by [Northeastern University] at 18:54 11 November 2014

I4 1.35 1.06 0.50 0.84 2.21 1.06

I5 2.21 1.05 1.15 0.75 1.28 1.03
I6 2.08 1.10 1.06 0.78 2.07 1.05
I7 1.43 1.06 0.56 0.85 1.95 1.05
I8 2.53 1.11 1.37 0.71 1.34 1.02
Other Architectural design I1 — — –3.93 1.34 –4.17 1.14
I2 — — –1.91 1.16 –2.04 0.94
I3 — — 0.58 1.23 0.62 1.07
a
I4 — — 1.14 1.48 1.22 1.33
a
I5 — — 1.14 1.46 1.22 1.32
I6 — — 1.02 1.36 1.09 1.25
I7 — — 0.79 1.22 0.84 1.14
a
I8 — — 1.16 1.40 1.22 1.37

Note. a = poor model–data fit.—= no values.

domains, indicating that items in the dance domain were
less consistent than were items in other domains.
Interpretability validity. Figures 1–10 show the per-
son measures and item difficulty for each CAQ domain.
Each X on the left side of the figure denotes a person
with a creative ability in that domain. The digit on the
right denotes the item number. Positive values indicate
higher levels of creativity achieved by that individual,
and more positive values for an item (i.e., higher levels)
indicate less likelihood that an individual reported a cre-
ative achievement for that item.
Overall, the items in the following domains were
higher than participants’ creativity: visual arts (M =
–1.96, SD = 1.89), music (M = –4.05, SD = 2.22), dance
(M = –6.23, SD = 3.16), architectural design (M =
–6.60, SD = 2.80), creative writing (M = –2.47,
SD = 1.97), humor (M = –3.52, SD = 2.16), invention
(M = –4.24, SD = 2.50), scientific inquiry (M = –4.19,
SD = 2.52), theater and film (M = –4.61, SD = 2.32),

TABLE 2
Reliabilities and Correlations Among 10 Domains

1 2 3 4 5 6 7 8 9 10

1. Visual arts (.72)

2. Music 0.42 (.67)
3. Dance 0.31 0.38 (.63)
4. Architectural design 0.57 0.47 0.69 (.69)
5. Creative writing 0.41 0.45 0.37 0.30 (.77)
6. Humor 0.24 0.25 0.32 0.34 0.38 (.73)
7. Inventions 0.31 0.19 0.18 0.55 0.26 0.47 (.77)
8. Scientific discovery 0.11 0.16 0.09 0.40 0.13 0.35 0.72 (.73)
9. Theater/film 0.44 0.46 0.54 0.68 0.79 0.49 0.34 0.18 (.73)
10. Culinary arts 0.54 0.61 0.66 0.59 0.65 0.39 0.56 0.38 0.55 (.69)

Note. ( ) = person separation reliability.


Downloaded by [Northeastern University] at 18:54 11 November 2014
FIGURE 1 Visual arts.
and culinary arts (M = –3.25, SD = 1.10). The average cre-
ativity of participants was lower than their creative
achievement in regard to items in each domain, indicat-
ing that most participants had low levels of creative
achievement.
FIGURE 2 Music.
THE CREATIVE ACHIEVEMENT QUESTIONNAIRE 67
FIGURE 3 Dance.
Substantive validity. Each domain in the CAQ
was composed of eight items arranged in order of low-
est to highest level of creative achievement. The
Spearman rank correlation of item estimates was used
to examine calibration invariance between the expected
FIGURE 4 Architectural design.
 68 WANG ET AL.
Downloaded by [Northeastern University] at 18:54 11 November 2014

FIGURE 5 Creative writing. FIGURE 7 Inventions.

(original) and the empirically derived item hierarchy. Correlations between the two sets of rankings for the
Correlations between the two sets of rankings for the dance, architectural design, creative writing, theater/
visual arts, music, humor, inventions, and scientific film, and culinary arts domains ranged from .59 to .88
discovery domains were greater than .91 (p < 0.01). (p > 0.01; Table 3), suggesting that the relative

FIGURE 6 Humor. FIGURE 8 Scientific discovery.

 THE CREATIVE ACHIEVEMENT QUESTIONNAIRE 69
Downloaded by [Northeastern University] at 18:54 11 November 2014

FIGURE 9 Theater/film. FIGURE 10 Culinary arts.

difficulty of the items in these five subscales required
modification.
Experts’ Ranking
To clarify the discrepancy in the item hierarchies of the
five domains, an expert in each of the domains was
asked to rank the items. Next, the rankings provided by
the domain experts, the empirically derived data, and
the data generated by the original hypothesis were com-
pared using Spearman rank correlations. In the dance
domain, the correlations between Expert A’s rankings
and the empirically derived data were significant (r = .91,
p < 0.01); however, the correlations between Expert A’s
rankings and those generated by the original hypothesis
were not (r = .79, p > 0.01). In the architectural design
domain, the correlations between Expert B’s rankings
and the empirically derived data (r = .59, p > 0.01) or
those of the original hypothesis (r = .38, p > 0.01) were
not significant. Similarly, the rankings provided by
Expert C for the creative writing domain were not sig-
nificantly correlated with the empirically derived data
(r = .59, p > 0.01) or those generated by the original
hypothesis (r = .38, p > 0.01). The rankings provided by
Expert D for the theater and film domain were not sig-
nificantly correlated with the empirically derived data
(r = .50, p > 0.01) or those generated by the original
hypothesis (r = .10, p > 0.01). Finally, the rankings of
Expert E for the culinary arts domain were not signifi-
cantly correlated with the empirically derived data
(r = .10, p > 0.01) or those generated by the original
hypothesis (r = .57, p > 0.01).

TABLE 3
Spearman Rank Correlations Between Expected and Empirically Derived Item Hierarchies

Domain Visual Arts Music Dance Architectural Design Creative Writing

Rank
Expected order .95(**) .93(**) .88 .59 .83

Domain Humor Invention Scientific Discovery Theater/Film Culinary Arts

Rank
Expected order .95(**) .95(**) .91(**) .74 .67

Note.**Correlation is significant at the 0.01 level (two-tailed).

 70 WANG ET AL.
DISCUSSION AND CONCLUSIONS
The CAQ is a practical and frequently used tool for mea-
suring and examining various components of creative
achievement. However, investigations that ignore the
assumptions and limitations of the CAQ may produce
biased results. This study examined the validity of the CAQ
using Rasch models to overcome these potential biases.
The first phase of the study involved 905 participants
aged between 14 and 78 years. This large and heteroge-
neous sample was designed to avoid the limitations arising
from a homogeneous sample, such as that used in the study
conducted by Carson et al. (2005), in which most partici-
pants were relatively gifted students attending Harvard
University. The results of our study revealed that the aver-
age creative achievement of participants was lower than
Downloaded by [Northeastern University] at 18:54 11 November 2014
the item difficulty and that most participants achieved
lower levels of creative achievement than expected. The
majority of respondents fell into the less-creative achieve-
ment group, with only a few in the high-creative achieve-
ment group (Carson et al., 2005; Eysenck, 1995). Similar
results were reported by Silvia, Kaufman, and Pretz (2009).
Second, Rasch analysis was utilized to validate the
construct components of the CAQ and provided multiple
evidence of validity. Previous studies have shown that cre-
ative ability differs across domains and is domain-specific
(e.g., arts vs. sciences; Carson et al., 2005; Gruber &
Wallace, 1999; Policastro & Gardner, 1999). Ivcevic (2009)
also reported that most creativity is domain-specific. The
Rasch analyses used in this study demonstrated that all
items in each domain fit their respective construct well
and supported the two-factor classification (arts and sci-
ences) proposed by Carson et al. (2005). Moreover, the
CAQ was transformed into an objective interval scale
using the Rasch analysis, which has important implica-
tions for future research. For instance, using the CAQ to
assess creative achievement allows each domain to be con-
sidered as an independent category, and several similar
domains may be integrated into a more general domain
(e.g., inventions, scientific discoveries, and culinary arts
can be combined into a single category of creative scien-
tific achievement). Additionally, a hierarchical relation-
ship may exist among different domains, allowing a test
of the domain hierarchy theory (Simonton, 2009), which
states that one dimension of the variability may coordi-
nate the disconnected data points of various domains.
Third, this study validated the generalizability of the
CAQ using DIF analysis. In theory, a single instrument
designed to measure a given latent trait or ability should
be applicable to all groups. The DIF reflects the degree to
which a given item holds a different meaning for partici-
pants with the same ability belonging to different groups.
The results of this study revealed that the items in most
of the CAQ domains (with the exception of theater and
film) may be interpreted differently by male and female
subjects with the same ability. This finding indicated that
the generalizability of the CAQ should be improved.
Additionally, the results for the architectural design
and culinary arts domains indicate that several overlap-
ping items should be reviewed (see Figures 4 and 10). In
terms of the architectural design domain, the creative
achievements described in items 3–8 were difficult for par-
ticipants to achieve. In other words, the ranking of items
in this domain appeared to be inconsistent with the
achievements of participants. This pattern was also
observed in the culinary arts domain. It is likely that few
participants in the study had creative abilities related to
architectural design or culinary arts; however, the repre-
sentative sample was too insufficient to recheck the item
rankings. Thus, the ranking of items in these two sub-
scales should be rechecked using participants with the
appropriate professional expertise. Finally, the Rasch
analysis revealed inconsistencies in the item hierarchy in
the dance, architectural design, creative writing, theater
and film, and culinary arts domains. To clarify the factors
underlying this discrepancy, an expert in each of the five
domains was asked to rank the items. The results revealed
that the hierarchical ranking provided by the five experts
were more consistent with the order of the empirical data
than that in the theoretical hierarchy. This finding sug-
gests that follow-up studies are necessary to reconfirm
item rankings in each CAQ domain. Furthermore, the
CAQ should be reexamined with respect to the number
and nature of items in each domain. To establish the lev-
els of creative achievement in different domains and to
validate and improve the quality of the CAQ, in-depth
interviews with focus groups and with experts in each
domain and tests of the validity of the instrument in a
larger and more representative sample than that used in
the present study are necessary.
Although the value of the Rasch model in obtaining
objective psychometric tests has been increasingly recog-
nized, the model is rarely applied to creativity measurement.
Using the CAQ (Carson et al. 2005) as an example, this
study has demonstrated that the Rasch model can be used
to validate and improve an existing creativity measurement
tool that was developed using CTT and can confirm the item
hierarchy and level of creativity through responses to indi-
vidual items. We hope that our study stimulates future
investigations that apply the Rasch model to improve the
quality and objectivity of creativity assessment tools.
REFERENCES
Batey, M. (2012). The measurement of creativity: From definitional
consensus to the introduction of a new heuristic framework.
Creativity Research Journal, 24, 55–65. doi: 10.1080/10400419.
2012.649181
Batey, M., & Furnham, A. (2008). The relationship between measures
of creativity and schizotypy. Personality and Individual Differences,
45, 816–821. doi: 10.1016/j.paid.2008.08.014

Carson, S. H., Peterson, J. B., & Higgins, D. M. (2005). Reliability, valid-
ity, and factor structure of the creative achievement questionnaire.
Creativity Research Journal, 17, 37–50. doi: 10.1207/s15326934crj1701_4
Cheng, Y. Y., Wang, W. C., & Ho, Y. H. (2009). Multidimensional
Rasch analysis of a psychological test with multiple subtests: A sta-
tistical solution for the bandwidth–fidelity dilemma. Educational and
Psychological Measurement, 69, 369–388. doi: 10.1177/
0013164408323241
Colangelo, N., Kerr, B., Hallowell, K., Huesman, R., & Gaeth, J.
(1992). The Iowa Inventiveness Inventory: Toward a measure of
mechanical inventiveness. Creativity Research Journal, 5, 157–163.
doi: 10.1080/10400419209534429
Eysenck, H. J. (1995). Genius: The natural history of creativity.
Cambridge, UK: Cambridge University Press.
Gruber, H. E., & Wallace, D. B. (1999). The case study method and
evolving systems approach for understanding unique creative people
at work. In R. J. Sternberg (Ed.), Handbook of creativity (pp. 93–115).
Cambridge, UK: Cambridge University Press.
Helson, R., & Pals, J. L. (2000). Creative potential, creative achieve-
Downloaded by [Northeastern University] at 18:54 11 November 2014
ment, and personal growth. Journal of Personality, 68, 1–27. doi:
10.1111/1467-6494.00089
Hocevar, D. (1979, April). The development of the Creative Behavior
Inventory. Paper presented at the annual meeting of the Rocky
Mountain Psychological Association. (ERIC Document Repro-
duction Service No. ED170350).
Holland, P. W., & Wainer, H. (Eds.) (1993). Differential item function-
ing. Hillsdale, NJ: Lawrence Erlbaum Associates.
Horng, J. S., & Hu, M. L. (2008). The mystery in the kitchen: Culinary
creativity. Creativity Research Journal, 20, 221–230. doi: 10.1080/
10400410802060166
Ivcevic, Z. (2009). Creativity map: Toward the next generation of theo-
ries of creativity. Psychology of Aesthetics, Creativity, and the Arts, 3,
17–21. doi: 10.1037/a0014918
Ludwig, A. M. (1992). The creative achievement scale. Creativity
Research Journal, 5, 109–124. doi: 10.1080/10400419209534427
MacKinnon, D. W. (1962). The nature and nurture of creative talent.
American Psychologist, 17, 484–495. doi: 10.1037/h0046541
Messick, S. (1994). The interplay of evidence and consequences in the vali-
dation of performance assessments. Educational Researcher, 23, 13–23.
Messick, S. (1995a). Standards of validity and the validity of standards
in performance assessment. Educational Measurement: Issues and
Practice, 14, 5–8.
Messick, S. (1995b). Validity of psychological assessment: Validation
of inferences from persons’ responses and performances as scientific
inquiry into score meaning. American Psychologist, 50, 741–749.
Policastro, E., & Gardner, H. (1999). From case studies to robust gener-
alizations: An approach to the study of creativity, In R. J. Sternberg
THE CREATIVE ACHIEVEMENT QUESTIONNAIRE 71
(Ed.), Handbook of creativity (pp. 213–255), New York, NY:
Cambridge University Press.
Rasch, G. (1960). Probabilistic models for some intelligence and attain-
ment tests. Copenhagen: Institute of Educational Research.
Reiter-Palmon, R., Robinson-Morral, E. J., & Kaufman, J. C. (2012).
Evaluation of self-perceptions of creativity: Is it a useful criterion?
Creativity Research Journal, 24, 107–114. doi: 10.1080/
10400419.2012.676980
Silvia, P. J., Kaufman, J. C., & Pretz, J. E. (2009). Is creativity domain-
specific? Latent class models of creative accomplishments and cre-
ative self-descriptions. Psychology of Aesthetics, Creativity, and the
Arts, 2, 139–148. doi: 10.1037/a0014940
Simonton, D. K. (2009). Varieties of (scientific) creativity: A hierarchi-
cal model of disposition, development, and achievement. Perspectives
on Psychological Science, 4, 441–452.
Simonton, D. K. (2012). Taking the US Patent Office criteria seriously:
A quantitative three-criterion creativity definition and its implica-
tions. Creativity Research Journal, 24, 97–106. doi: 10.1080/
10400419.2012.676974
Taylor, C. W., & Ellison, R. L. (1967). Biographical predictors of scien-
tific performance. Science, 155, 1075–1079. doi: 10.1126/
science.155.3766.1075
Teo, L. M. C., & Waugh, R. F. (2010). A Rasch measure of fostering
creativity. Creativity Research Journal, 22, 206–218. doi:
10.1080/10400419.2010.481534
Torrance, E. P. (1972). Career patterns and peak creative achievements
of creative high school students twelve years later. Gifted Child
Quarterly, 16, 75–88.
Wang, W. C. (2004). Rasch measurement theory and application in edu-
cation and psychology. Journal of Education and Psychology, 27,
637–694.
Wang, W. C. (2008). Assessment of differential item functioning.
Journal of Applied Measurement, 9, 1–22.
Wang, W. C., & Cheng, Y. Y. (2000). Development and item response
analysis of the creativity development inventory. Psychological
Testing, 47, 153–173.
Wilson, M. (2005). Constructing measures: An item response modeling
approach. Mahwah, NJ: Lawrence Erlbaum Associates.
Wolfe, E. W., & Smith, E. V. (2007). Instrument development tools and
activities for measure validation using Rasch models: Part
II-Validation activities. Journal of Applied Measurement, 8,
204−233.
Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit val-
ues. Rasch Measurement Transactions, 8, 370.
Wu, M. L., Adams, R. J., & Wilson, M. R. (2007). ConQuest [Computer
software and manual]. Camberwell, Australia: Australian Council
for Educational Research.