PROFESSIONAL DEVELOPMENT
Getting it right:
Crafting the CPA Exam
Test developers aim to assess important CPA knowledge and skills
— and only that.
By Ken Tysiac
f you could assign a flavor to the questions on the
Uniform CPA Examination, it would undoubt-
ely be pure vanilla
‘Tales of adventure, romance, and violence can
be found at the bookstore or in the movie theater,
PA Exam, which is careflly
designed to avoid topies that could unnecessarily
distract CPA candidates from the actual purpose of
the exam.
“We try to give the candidates the opportunity
to provide clear evidence of their mastery of
accounting-related knowledge, skill, and abilities
(KSAs), and we try to exclude factors that have
nothing to do with those KSAs,” suid Jessica Lucas,
an associate manager for the AICPA Examina-
tions team. “For example, we're not trying to assess
their knowledge of unusual vocabulary words
tunless those words are commonly used in the field
of accounting or any of its specialized fields. So
a word like epine, which is part of the everyday
vocabulary of auditors, can and does appear on the
exam; but a word ike Afandisbment does not. We
dont test advanced reading skills either. Ques-
tions are designed to assess essential content only,
with no embellishment. We never try to trick the
candidates in any way. Each sentence is designed to
be as straightforward as possible and to have one
and only one meaning in context. The meaning has
to be clear to all readers.”
‘The CPA Exam is one of the most respected
professional licensure examinations in the world,
‘What's not so well known is that the rigor as-
sociated with exam development procedures is as
intense as the review process that many candidates,
undertake in their efforts to earn a passing score.
Each potential question undergoes scrutiny during
multiple rounds of review by specialists evaluating
it from different points of view. Ifthe question
survives those reviews, the next step is @ pretest
phase to perform a statistical analysis of actual test
takers’ responses to the question. (Pretest items
don't count toward a candidate's score.) If the ques-
tion fails to meet statistical standards, itis rejected
or revised
As laws, regulations, and other types of
guidance change, the exam also needs to change
Tt takes about eight months from the time the
Examinations team identifies a need for more
questions to the point when they are ready to be
pretested on the exam; in certain cases, the time
elapsed can be as short as four months.
In general, each question is designed to be
a successful act of communication. Linguistic
philosopher Paul Grice observed that successful,
cooperative communications typically follow four
basic rules:
Be rruthiul
Be clear
Be relevant.
Be appropriately informative
“The text of the CPA Exam obeys the rules and
conventions of international standard English so
that the questions will be easy for English speak-
crs around the world to understand. Likewise,
staff members use their knowledge of logic and
Tinguistics to make sure that the questions assess
only accounting content, not advanced skills in
logic or linguistics (see the sidebar “Questions That
Didrit Make the Cut”).
“We design the text of each question to be
easy to process. We avoid needlessty complicated
‘wordings and logical structures,” said Timothy
Habick, Ph.D, test development manager for the
AICPA Examinations team. "Now some people
might wonder how logic could be an issue relevant
to the CPA Exam in the first place,and that
About the
author
Ken Tysiacis the
JofA's editorial
director.To
comment on
this article or to
suggest an idea
for another article,
contact him at
Kenneth Tysiac@
aicpa:cima.com or
919-402-2112.
December 2018 | 33PROFESSIONAL DEVELOPMENT
is the point. We purposely craft the text so that
candidates will not need to use logic to understand
the intended meaning of any sentence in context.
‘Our goal is to ereate clear, coherent, and coopera-
tive communications. This tends to make the text
of our questions somewhat unremarkable, but that
is our intention.
“Candidates are supposed to focus on the essen
tial accounting content, not on our use of language.
For the same reason, we tend to avoid using n:
for individuals and companies that could attract.
the candidates’ attention. So whenever possible,
we refer to‘a taxpayer,"an auditor; or's company"
without using realistic names. Proper names can
contain allusions to irrelevant historical events
and individuals, which could be distracting, We're
hore to assess the candidates’ accounting-related
skills, not to entertain them. We try to include only
Questions that didn’t make the cut
Rejected early drafts demonstrate some important considerations in test development.
sight into the CPA Exam development process can be gained
by reviewing potential questions that were ejected. Here
are some excerpts from draft questions offered by writers that
illustrate why the questions did not make the cut.
‘Anna, a U.S. citizen, married Eduardo, a citizen of Mexico, in
October of the current year. Eduardo is an undocumented
alien in the U.S. Eduardo has no U.S. income for the year and
was fully supported by Anna.
Objection: There is no content-related need to refer to
undocumented aliens, and the casting of Eduardo as from
Mexico reinforces a stereotype that could be troubling to
candidates.
In year 1, Jack, Inc. acquired Jil, Inc.
Objection: The mention of nursery rhyme characters in atest
question could distract candidates who might be cued to recite
the thyme (“ack and Jill went up the hil." instead of focusing
on solving the question
After a devastating fire this year a city approved the
sale of the site of its former firehouse at $500,000 below
book value. Net cash proceeds from the sale amounted to
$40,000. The city council announced that it would build
a new firehouse at an alternate location. How should the
city report the transaction in its statement of revenues,
expenditures, and changes in fund balances for the year?
Objection: The mention ofa “devastating fire"is unnecessarily
distressing. Furthermore, although it’s certainly possible fr a
firehouse to catch fire, the thought ofa devastating fire at a
firehouse could be puzzling and distracting for candidates. The
Examinations team doesn't want the candidates to spend time
wondering wiy the firefighters couldnt quickly extinguish the
flames in their own firehouse.
Inemerging countries, as personal income grows,
‘consumers may buy a pack of gum or a soft drink. This is an
ample of which of the following types of spending?
Mandatory.
Entitlement.
Discretionary. (correct answer)
>. Staple.
one> eg
Objection: It could be considered insulting to imply that
consumers in emerging economies were unable to buy a
pack of gum before their personal income grew. And the term
“emerging countries‘is not needed to establish the concept of,
discretionary spending,
Source: ACPA Test Development Fimess Guidelines, Version 40 (Timothy Habick and
Unda Cook, 2018).
IN BRIEF
The process of creating a question
for the CPA Exam In many ways Is
as thorough and rigorous as the
preparation that many CPA candidates
Put into their study for the exam, Each
Question is correlated to material
fon the exam Blueprints; drafted by
34 | Jourslef
‘external CPA experts; reviewed by
Additional experts, including a CPA
panel; and given a trial run in front of
actual candidates.
Questions are carefully constructed to
avoid topics and material that would
‘cause an unnecessary emotional
response that could distract candidates