See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/233641629

Not only item content but also item format is important: Taxonomizing item
format approaches

Article  in  Social Behavior and Personality An International Journal · February 2011

DOI: 10.2224/sbp.2011.39.1.119

CITATIONS READS

4 216

1 author:

John F. Rauthmann
Universität zu Lübeck
109 PUBLICATIONS   1,997 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Situational Assessment - Person x Situation Interactions View project

All content following this page was uploaded by John F. Rauthmann on 23 May 2014.

The user has requested enhancement of the downloaded file.

SOCIAL BEHAVIOR AND PERSONALITY, 2011, 39(1), 119-128
© Society for Personality Research (Inc.)
DOI 10.2224/sbp.2011.39.1.119

NOT ONLY ITEM CONTENT BUT ALSO ITEM FORMAT IS

IMPORTANT: TAXONOMIZING ITEM FORMAT APPROACHES

JOHN F. RAUTHMANN
Leopold-Franzens University of Innsbruck, Innsbruck, Austria

In this article I argue that as well as item content, item formats (i.e., phrasing and response
formats) are also important. Most trait items can be mapped onto 4 dimensions: point of
reference (first person, possessive, others, indicator), general item format (staticity, frequency,
valency, frequency + valency), construct indicator (attributal, behavioral, mental, contextual),
and conditionality (unconditional, conditional). An item taxonomy tree for the first person
perspective is provided for an Openness to Experiences item, and NEO-PI-R Extraversion
items are analyzed according to the 4-item format dimensions. Future lines of research on
item phrasing are outlined.

Keywords: item format, response/answering format, item generation, scale construction,

psychological/personality/trait assessment.

Validly and reliably assessing people’s traits is crucial to personality psychology

and psychological assessment, and thus considerable effort has been put into
constructing self-report measures. Researchers usually seek to maximize content
validity of their measures, that is, each item’s content for a certain construct is
diligently chosen in order to optimally capture the construct. However, effects
of item wording, grammar, or syntax are generally neglected or underestimated,
and items – even from the same construct or scale – show a variety of formats.
Not only the content of items but also their formats (including answering scales)

John F. Rauthmann, Research Assistant, Department of Psychology, Leopold-Franzens University of

Innsbruck, Innsbruck, Austria.
Appreciation is due to reviewers including: Chao-Chien Chen, Office of Physical Education, Asia
University, No. 500, Liufeng Road, Wufeng, Taichung, Taiwan 413, ROC, Email: apeter72@
hotmail.com
Please address correspondence and reprint requests to: John F. Rauthmann, Department of Psychology,
Leopold-Franzens University of Innsbruck, Innrain 52, Bruno-Sander-Haus, A-6020, Innsbruck,
Austria. Phone: +43 512 507 5548; Fax: +43 512 507 2838; Email: j.f.rauthmann@gmx.de

119
120 PROPOSING AN ITEM FORMAT TAXONOMY

are important and influence psychometric properties of a scale, such as construct

and criterion validity.
In this article I seek to (a) formulate common item format approaches
and present an item taxonomy tree, (b) show how the proposed item format
taxonomization is applied to analyzing one of the most widely used trait
measures in personality psychology (the NEO-PI-R by Costa & McCrae, 1992)
for a central trait dimension (Extraversion), and (c) provide an outlook on future
lines of research concerning different item formats and how they might affect
psychometric properties.

A TAXONOMY FOR ITEM FORMATS

Items aimed at capturing trait content are various in their structures and formats,
yet they can be organized according to certain dimensions; namely, general item
format, construct indicators, conditionality/contextuality, and perspective taken/
point of reference.
In general, trait items use (a) static sentences or descriptions (for example, “I
am outgoing”, “I go out and talk with people”, “I think of other people”), (b)
frequency descriptions of behaviors and mental processes (for example, “I often
go out and talk with people”, “I usually think of people”), and (c) descriptions
concerning the valence of one’s feelings towards something (for example, “I
enjoy going out and talking with people”, “I love socializing”, “I do not like
being around too many people”). These three approaches can be deemed as
general item formats into which most items fit. Static descriptions use a staticity
approach, frequency descriptions a frequency approach, and valence descriptions
a valency approach. They need not be indicated within the item, however, but
can be found in the response scales (for example, when answering the item “I
go out and talk with people” on a 5-point Likert scale ranging from 1= almost
never to 5 = very often). Besides using a general approach to item formats, items
also contain different construct-relevant indicators (related to the item content)
which are mostly attributes (in other words, adjectives), behaviors, mental states
and processes, and contexts or situations. Further, items can be conditional or
unconditional: Conditional items use if- or when-clauses to give contextual
specifications under which certain mental processes and behaviors occur (cf.
if-then patterns of dispositions by Mischel & Shoda, 1995). Moreover, the point
of reference or perspective of items can be distinguished: There can be a first
person referring to his or her own attributes, mental processes, and behaviors
(“I am/think/feel/do/behave...”), but also can “possessively” refer to his or
her own attributes, mental processes, and behaviors (“My thoughts/feelings/
behaviors...”). Also, someone else can be referring to one’s own attributes,
mental processes, and behaviors (“Others think/say that I/my...”). Additionally,
the item need not explicitly refer to oneself in particular but rather to a construct-
 PROPOSING AN ITEM FORMAT TAXONOMY 121
relevant indicator (for example, when agreeing to the item “Most people impose
on others’ kindness”, one could be described as cynical although the item per
se does not refer in any way to the person answering the item but rather to the
person’s attitude).
Most items aimed at capturing trait-constructs can thus be described in
terms of a four-way interaction of the following dimensions (see Figure 1):
point of reference (first person, possessive, others, indicator) x general item
format (staticity, frequency, valency, frequency + valency) x construct indicator
(attributal, behavioral, mental, contextual) x conditionality (unconditional,
conditional). However, not all interactions are possible, and although most items
have a standing on all four of the dimensions, some items may possess two
standings within a dimension. The item taxonomy tree (ITT; Figure 1) gives an
overview of possible basic item categorizations for the first person perspective
along with examples of how a single item (“I enjoy thinking of new ways to solve
problems” from the FIRNI by Denissen & Penke, 2008) could be rephrased for
each branch.1

ANALYSES OF NEO-PI-R EXTRAVERSION ITEMS

All analyses were performed on the German version of Costa and McCrae’s
(1992) NEO-PI-R by Ostendorf and Angleitner (2003), and thus results should not
be rashly generalized to the English version although the basic steps of analysis
can also be applied to it. The NEO-PI-R2 was chosen due to its wide application
and comprehensiveness, the trait of Extraversion as it is a fundamental dimension
of human personality (Wilt & Revelle, 2009, p. 27).

METHOD AND PROCEDURE

Item format analyses were carried out with respect to the previously presented
four basic dimensions of items: reference point (about oneself in first person; about
indicators pertaining to oneself; others about indicators pertaining to oneself;
about indicators), general item approach (staticity; frequency; valency; frequency
+ valency), construct indicators (attributal; behavioral; mental; contextual), and
conditionality (unconditional; conditional). There were several criteria for each
dimension and its subdimensions that were used to evaluate each item’s standing
on these (sub)dimensions. All items with “I . . .” were classified as first person
reference items. Items with “My . . .” were classified as items about indicators

1 This particular item was chosen because it could be rephrased for nearly all branches and still remain
sensible at most points.
2 Analyses of the NEO-FFI are also included as the items for the NEO-FFI are taken from the NEO-
PI-R. NEO-PI-R Extraversion has 48 items (8 items for 6 facets), the NEO-FFI 12 Items.
122 PROPOSING AN ITEM FORMAT TAXONOMY

Reference point Item approach Construct indicator Conditionality Example

unconditional [1]
attributal
conditional [2]

unconditional [3]
Staticity behavioral
conditional [4]

unconditional [5]
mental
conditional [6]

unconditional [7]
attributal
conditional [8]

unconditional [9]
behavioral
conditional [10]
Frequency
unconditional [11]
mental
conditional [12]

unconditional [13]
contextual
conditional [14]
First person
unconditional [15]
attributal
conditional [16]

unconditional [17]
behavioral
conditional [18]
Valency
unconditional [19]
mental
conditional [20]

unconditional [21]
contextual
conditional [22]

unconditional [23]
attributal
conditional [24]

unconditional [25]
behavioral
conditional [26]
Frequency+Valency
unconditional [27]
mental
conditional [28]

unconditional [29]
contextual
conditional [30]

Figure 1. The item taxonomy tree (for the first person singular).
 PROPOSING AN ITEM FORMAT TAXONOMY 123
Figure 1 notes:
[1] I am a good and creative problem solver.
I am good and creative at problem solving.
I am someone who solves problems well and creatively.
[2] I am a good and creative problem solver if the problem is interesting.
I am good and creative at problem solving if the problem is interesting.
I am someone who solves problems well and creatively if the problem is interesting.
[3] I solve problems.
[4] I solve problems if they are interesting.
[5] I think of new ways to solve problems.
[6] I think of new ways to solve problems if they are interesting.
[7] I am usually a good and creative problem solver.
I am usually good and creative at problem solving.
I am someone who usually solves problems well and creatively.
[8] I am usually a good and creative problem solver if the problem is interesting.
I am usually good and creative at problem solving if the problem is interesting.
I am someone who usually solves problems well and creatively if the problem is interesting.
[9] I often solve problems.
[10] I often solve problems if they are interesting.
[11] I often think of new ways to solve problems.
[12] I often think of new ways to solve problems if they are interesting.
[13] I find myself often in problems which need to be creatively solved.
[14] I find myself often in problems if they are interesting.
[15] I like being a creative problem solver.
I like being creative at problem solving.
I am someone who likes solving problems well and creatively.
[16] I like being a creative problem solver if the problem is interesting.
I like being creative at problem solving if the problem is interesting.
I am someone who likes solving problems well and creatively if the problem is interesting.
[17] I enjoy solving problems.
[18] I enjoy solving problems if they are interesting.
[19] I enjoy thinking of new ways to solve problems.
[20] I enjoy thinking of new ways to solve problems if they are interesting.
[21] I enjoy (being in) interesting problems.
[22] I enjoy (being in) problems if they are interesting.
[23] I usually like being a creative problem solver.
I usually like being creative at problem solving.
I am someone who usually likes solving problems well and creatively.
[24] I usually like being a creative problem solver if the problem is interesting.
I usually like being creative at problem solving if the problem is interesting.
I am someone who usually likes solving problems well and creatively if the problem is
interesting.
[25] I often enjoy solving problems.
[26] I often enjoy solving problems if they are interesting.
[27] I often enjoy thinking of new ways to solve problems.
[28] I often enjoy thinking of new ways to solve problems if they are interesting.
[29] I often enjoy (being in) interesting problems.
[30] I often enjoy (being in) problems if they are interesting.
A staticity approach with contextual indicators was excluded as it cannot be formulated with
a first person perspective (in other words, the acting “I”). However, it could be formulated
somewhat like this with a construct indicator perspective (using the example item of
Openness to Experiences): Problems are [often] solved by me (if they are interesting).
124 PROPOSING AN ITEM FORMAT TAXONOMY

pertaining to oneself. Items such as “Others think/say/etc. that I/my . . .” were

classified as items with others (as the reference point) about indicators pertaining
to oneself. Also, there were items which were classified as being items that used
an “indicator reference point”: The indicator for Extraversion in one item was
that the person finds most people sympathetic – and this is the central point of
reference in the item. Any item containing adverbs of frequency (for example,
never, seldom, rarely, sometimes, usually, normally, often) were classified as
frequency items. Items containing words that reflect a certain positive versus
negative valence (for example, like, dislike, love, enjoy, prefer) were classified
as valency items. All other items were classified as staticity items as they had
neither indicators of a frequency nor a valency approach. Should an item contain
an adverb of frequency and a word reflecting valence, then it was classified as
an item using a frequency and valency approach at once. Attributal construct
indicators were adjectives, behavioral indicators any forms of overt behavior that
could be potentially perceived by others (without inference), mental indicators
any forms of mental processes and states (that need to be inferred as they are
not observable), and contextual indicators any forms of “circumstances” (for
example, “people around me”) and situations. Conditionality was judged upon
the presence or lack of if/when clauses: Any if/when denoted a conditional item;
if there was none, then the item was categorized as an unconditional item.

RESULTS

Each item provided a score for all four dimensions. For example, an item
saying “I am dominant and assertive” would be classified as an item in “staticity
approach with attributal indicators, in an unconditional format, and with first-
person reference”. In a similar manner, all NEO-PI-R items were classified. Table
1 shows the results of item format analyses for NEO-PI-R facets of Extraversion
(warmth, gregariousness, assertiveness, activity, experience seeking, positive
emotions), NEO-PI-R Extraversion domain, and NEO-FFI Extraversion.
Point of reference For NEO-PI-R Extraversion facets, the first person
perspective was found in most items (M = 91.67%), whereas the NEO-FFI
Extraversion had 100%. Only two items in NEO-PI-R Extraversion had other
people as the reference point and also only two items had construct indicators as
the reference point.
Item approach The general item approach used the most in NEO-PI-R
Extraversion facets was the staticity approach (M = 41.67%, range: 12.5 – 75%),
followed by the frequency approach (M = 31.25%, range: 0 – 50%)3, and then

3 The means reflect percentages of NEO-PI-R Extraversion domain.

 TABLE 1
PERCENTAGES OF PERSPECTIVES, GENERAL ITEM APPROACHES, CONSTRUCT INDICATORS, AND CONDITIONALITY IN NEO-PI-R AND NEO-FFI
EXTRAVERSION ITEMS

Scale Point of reference (perspective) in % General item approach in % Construct indicators in % Conditionality in %
1 2 3 4 S F V F+V A B M C No Yes

Warmth 87 0 12.5 12.5 75 0 25 0 25 37.5 37.5 0 100 0

Gregariousness 10 0 0 0 25 12.5 50 12.5 0 50 25 25 87.5 12.5
Assertiveness 87.5 0 12.5 0 25 50 25 0 12.5 87.5 0 0 100 0
Activity 10 0 0 0 62.5 37.5 0 0 25 62.5 12.5 0 87.5 12.5
Experience seeking 87.5 0 0 12.5 12.5 37.5 50 0 0 62.5 25 12.5 100 0
Positive emotions 100 0 0 0 50 50 0 0 37.5 37.5 25 0 100 0
Total (NEO-PI-R) 91.67 0 4.17 4.17 41.67 31.25 25 2.08 16.67 56.25 20.83 6.25 95.83 4.17
Total (NEO-FFI) 100 0 0 0 50 8.33 33.33 8.33 33.33 50 8.33 8.33 100 0
Notes: Facets of Extraversion (in the NEO-PI-R) = Warmth, Gregariousness, Assertiveness, Activity, Experience Seeking, Positive Emotions: 8 items
each; NEO-PI-R Extraversion: 48 items (6 x 8 items); NEO-FFI Extraversion: 12 Items (from the NEO-PI-R: 1 x Warmth, 2 x Gregariousness, 1 x
Assertiveness, 3 x Activity, 1 x Experience Seeking, 4 x Positive Emotions).
1 = about oneself in first person, 2 = (possessively) about own attributes, behaviors, and mental processes, 3 = others about oneself and one’s attributes,
PROPOSING AN ITEM FORMAT TAXONOMY

behaviors, and mental processes, 4 = about construct-relevant indicators; S = staticity approach, F = frequency approach, V = valency approach,
F + V = frequency and valency approach mixed together; A = attributal indicators, B = behavioral indicators, M = mental indicators, C = contextual
indicators.
125
126 PROPOSING AN ITEM FORMAT TAXONOMY

the valency approach (M = 25%, range: 0 – 50%). NEO-FFI Extraversion had

50% of staticity items, 33.33% of valency items, and 8.33% frequency items.
In both the NEO-PI-R and NEO-FFI, only one item had the frequency and
valency approaches combined. Obviously, the majority of items of NEO-PI-R
and NEO-FFI Extraversion are in a staticity format which makes no references
to frequencies or valencies of behaviors (or any other indicators).
Construct indicators Most indicators are behavioral in NEO-PI-R Extraversion
facets (M = 56.25%, range 37.5–87.5%) and in NEO-FFI Extraversion (50%).
NEO-PI-R Extraversion showed 16.67% of attributal (range 0–37.5%) and
20.83% of mental indicators (range 0–37.5%), whereas NEO-FFI Extraversion
showed 33.33% and 8.33% (in other words, only one item), respectively.
Contextual indicators were generally rare (6.25% in NEO-PI-R and 8.33% in
NEO-FFI Extraversion). Conditional items were also very rare (two items in
NEO-PI-R and no items in NEO-FFI Extraversion).

DISCUSSION

A special topic of classification should be raised critically: An item such as

“I enjoy talking to people”, for example, would have been classified here as an
item in “valency approach (enjoy) with a behavioral indicator (talking), in an
unconditional format (no if- or when-clause), and with first-person reference (I)”.
However, going strictly by the rules, the item could also have been classified
as an item in “staticity approach (static verb “enjoy”) with a mental indicator
(enjoy) and behavioral specification of the mental indicator (talking), in an
unconditional format (no if- or when-clause), and with first-person reference
(I)”. Although this possibility is also correct (and would have the advantage of
dissolving the valency approach and thus reducing the complexity of the item
format taxonomy proposed), this was not done for two reasons: First, the valency
approach may still be seen as a distinct form as it only refers to mental indicators
of positive versus negative valence, and the “valency” is a central point. Second,
the valency approach is used very commonly and should thus be distinguished.
Thus, although I recommend distinguishing the valency approach, it could
also be described as a special form of staticity approach with mental indicators
(denoting positive–negative valence), and future researchers might also use this
alternate type of classification.

PROSPECTS AND FUTURE LINES OF RESEARCH

Knowing which item phrasings are beneficial to which traits and which
psychometric abilities is very important for personality research and psychological
assessment as data quality could be improved. Thus, the ITT provides a first
guide in determining which kind of items a certain scale possesses (the example
 PROPOSING AN ITEM FORMAT TAXONOMY 127
used here was NEO-PI-R Extraversion) and which kinds of items there could
potentially be for a first person perspective (the example used here was a Five
Individual Reaction Norms Inventory (FIRNI) Openness to Experiences item).
Some traits might show affinities to certain branches of the ITT, and some
might even provide better or worse psychometric properties than others. Further
research should thus determine which ITT branches (a) are better suited for
which constructs (in other words, some constructs might be better captured
with certain branches than others), and (b) enhance or decrease psychometric
properties (reliability, validity) of a scale using either consistently one branch or
mixing branches.
Concerning branches and constructs, behaviorally manifestive traits such as
Extraversion are probably better captured by behavioral items in staticity or
frequency approaches, whereas more intrapersonal traits such as Neuroticism or
Intellect might better use items with mental indicators. Most traits may require a
first person perspective, but traits denoting attitudes, such as Machiavellianism,
might even be well suited to other- or construct-perspectives. Also, not all
branches fit equally well for a trait; indeed, most branches might make no sense,
but one, two, or three could be highly relevant for a certain construct. It will thus
be a future goal to determine optimal branches for certain constructs. In any case,
though, the researcher must decide, based on his or her knowledge, which item
content and format to use in a particular case.
Figure 1, depicting the ITT, and its notes show which possibilities in phrasing
there are for an Openness to Experiences item and how these possibilities can
be systematically organized in a taxonomy of item formats. Researchers aiming
to construct new scales should diligently choose the item formats for their items
as the format will likely have effects on psychometric properties of the scale4. It
is thus poorly understood how different item phrasing will affect reliability and
validity, and future researchers could start comparing branches systematically for
different constructs in order to evaluate which lead to better results and which do
not. The ITT provides a systematic framework for such comparative research.
With the emergence of large public-domain item pools such as the International
Personality Item Pool (IPIP; Goldberg, 1999; Goldberg et al., 2006)5, there is
a trend to employ short phrases (in a staticity approach) rather than complete
sentences as items. This gives the interesting opportunity to “outsource” the
general item approach into the answering format. For example, the item “I

4 Indeed, preliminary analyses of my own still ongoing research (with Jaap Denissen) show that a
frequency approach with behavioral indicators seems to have advantages over a valency approach
to extraversion concerning most psychometric properties. Thus, it may be safer to ask people how
often they engage in extraversion-relevant behaviors.
5 http://ipip.ori.org/ipip/, retrieved January, 2010.
128 PROPOSING AN ITEM FORMAT TAXONOMY

socialize with people” can be equipped with a frequency-answering format

(almost never to very often) or a valency-answering format (I dislike this very
much to I enjoy this very much). This has the advantage of making items easier to
understand and standardizing the frequency- or valency-based answering format
rather than relying on different frequency and valency indicators in the items that
might be perceived and interpreted idiosyncratically (for example, even though
semantically “rarely” and “seldom” are equal, they might not be in people’s
frequency perceptions and thus answering). Future researchers should determine
how such items with outsourced general item approaches function in relation to
those with the general item approach integrated into the items.

CONCLUSION

In this article, I provided an item taxonomy tree to systematize item

construction concerning their phrasing. It should be a goal of future researchers
to comparatively evaluate the different branches of the item taxonomy tree in
relation to different traits and different psychometric criteria. Not only item
content but also item formats are important.

REFERENCES

Costa, P. T., & McCrae, R. R. (1992). Revised NEO personality inventory (NEO PI-R) and NEO
Five-Factor Inventory (NEO-FFI): Professional manual. Odessa: Psychological Assessment
Resources.
Denissen, J. J. A., & Penke, L. (2008). Motivational individual reaction norms underlying the five
factor model of personality: First steps towards a theory-based conceptual framework. Journal of
Research in Personality, 42(5), 1285-1302.
Goldberg, L. R. (1999). A broad-bandwidth, public-domain, personality inventory measuring the
lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F.
Ostendorf (Eds.), Personality psychology in Europe (Vol. 7, pp. 7-28). Tilburg, The Netherlands:
Tilburg University Press.
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., et al. (2006).
The international personality item pool and the future of public-domain personality measures.
Journal of Research in Personality, 40(1), 84-96.
Mischel, W., & Shoda, Y. (1995). A cognitive-affective system theory of personality: Reconceptualizing
situations, dispositions, dynamics, and invariance in personality structure. Psychological Review,
102(2), 246-268.
Ostendorf, F., & Angleitner, A. (2003). NEO-Persönlichkeitsinventar nach Costa und McCrae,
Revidierte Fassung (NEO-PI-R). [NEO Personality Inventory by Costa and McCrae, revised
version]. Göttingen: Hogrefe.
Wilt, J., & Revelle, W. (2009). Extraversion. In M. Leary & R. Hoyle (Eds.), Handbook of individual
differences in social behavior (pp. 27-45). New York: Guilford.
Copyright of Social Behavior & Personality: An International Journal is the property of Society for Personality
Research and its content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email articles for
individual use.

View publication stats