Accountin& OrganizationsandSocieO~,Vol. 18,No.6,pp. 559-574, 1993. 0361-3682/93 $6.00+.

00
Printed in Great Britain © 1993 Pergamon Press Ltd

THE RENEW PROCESS AS A CONTROL FOR DIFFERENTIAL RECAI.L

OF EVIDENCE IN AUDITOR JUDGMENTS*

ROBERT LIBBY
Cornell University

and

K E N T. T R O T M A N
University o f New South Wales

Abstract

"Dlis experiment examines whether there are systematic offsetting differences in the manner in which
initial decision makers and reviewers attend to information which ensure that evidence inconsistent with
initial judgments is given adequate consideration. Differences in attention are proposed, which result
in differential recall of evidence by the initial decision maker and reviewer and thus influence what knowledge
initial decision makers and reviewers bring to their discussions and subsequent decisions. The results suggest
that the review process can act as an effective control by increasing the chances that the implications of
inconsistent evidence are considered.

The hierarchical structure of the audit team and
the sequential and iterative review processes
w h i c h d o m i n a t e i n t e r a c t i o n s a m o n g its m e m b e r s
are well-recognized characteristics of the audit
d e c i s i o n - m a k i n g e n v i r o n m e n t ( s e e e.g. A s h t o n
et aL, 1 9 8 8 ; M a u t z & Sharaf, 1 9 6 1 ) . T h e i r r o l e
in q u a l i t y a s s u r a n c e is a l s o d e e p l y i n g r a i n e d i n
auditing standards (e.g. AU 230) and firm
policies. The potential effects of review on
judgment performance were explicitly recog-
nized in early audit judgment research (e.g.
J o y c e , 1 9 7 6 , p. 5 6 ) . H o w e v e r , it w a s n o t u n t i l
1 9 8 5 t h a t t w o s t u d i e s a i m e d p r i m a r i l y at
assessing these effects were published (see
Trotman, 1985; Trotman & Yetton, 1985). The
r e v i e w p r o c e s s w a s m o d e l e d i n t h e s e p a p e r s as
a statistical weighting of the initial decision
maker and reviewer's opinions I which yielded
two benefits: ( 1 ) the elimination of inconsistency
which results from aggregating multiple opinions
(Trotman & Yetton, 1985), and (2) reduction
in systematic bias resulting from more heavily
weighting the opinions of the more accurate,
more senior, team members (Trotman, 1985).
While these two papers successfully document
the gains resulting from review processes, they
have not led to significant additional research.
This may have occurred because the statistical

• The financial support of the KPMG Peat Marwick Foundation Research Opportunities in Auditing Program and the
Australian Research Council is gratefully acknowledged. We also wish to acknowledge useful comments by the participants
at the Boston Accounting Research Colloquium, and Connecticut, Cornell, Griflith, Macquarie, Penn State, Pittsburgh,
Queensland, and Washington accounting workshops, as well as Sarah Bonner, Vicky Heiman-Hoffman, John Luft, Mark
Nelson, Ira Solomon, and Arnie Wright. Finally, we wish to thank Zubaidah lsmaii and Joan Luft for their very capable
research assistance.

t For ease of exposition, we refer to the person whose work is being reviewed as the "initial decision maker" and the
one who is reviewing the prior judgment as the "reviewer".

559
560 R. LIBBYand K. T. TROTMAN

j u d g m e n t m o d e l e m p l o y e d in t h e p r i o r p a p e r s d e c i s i o n makers, this i n v o l v e s t h e t e n d e n c y for

( s e e also Libby et aL, 1 9 8 7 ) d o e s n o t d e s c r i b e
t h e u n d e r l y i n g c o g n i t i v e p r o c e s s e s in sufficient
d e t a i l to a l l o w f u r t h e r p r e d i c t i o n s o f t h e specific
benefits o f r e v i e w a n d t h e c i r c u m s t a n c e s w h e r e
t h e y will b e e n h a n c e d o r lost. As a result, it d o e s
n o t p r o v i d e a basis for p r o g r a m m a t i c r e s e a r c h
a i m e d at u n c o v e r i n g t h e sources o f gains ( o r
losses) produced by the review process or other
group processes.
T h e r e v i e w p r o c e s s , like o t h e r a s p e c t s o f
o r g a n i z a t i o n a l design, is v i e w e d h e r e , in part, as
an a t t e m p t to m i t i g a t e t h e effects o f b a s i c h u m a n
i n f o r m a t i o n p r o c e s s i n g limits (cf. Simon, 1947).
Since t h e s e l i m i t a t i o n s a r e b o t h s o u r c e s o f
decision error (see Bonner & Pennington,
1991 ) a n d d e t e r m i n a n t s o f t h e e f f e c t i v e n e s s o f
different c o n t r o l d e v i c e s , t h e y a r e f u n d a m e n t a l
to u n d e r s t a n d i n g e n t i t i e s s u c h as t h e a u d i t team.
In this p a p e r , w e s u g g e s t a p a r t i c u l a r s o u r c e o f
gain b a s e d o n differences b e t w e e n initial d e c i s i o n
m a k e r s a n d r e v i e w e r s in t h e p o i n t at w h i c h t h e y
e n t e r t h e j u d g m e n t p r o c e s s a n d in t h e i r
o b j e c t i v e s . Initial d e c i s i o n m a k e r s e n t e r t h e
p r o c e s s p r i o r to w h e r e an initial j u d g m e n t is
m a d e a n d h a v e i n c e n t i v e s t o justify t h e i r
p o s i t i o n s ( E m b y & Gibbins, 1988). R e v i e w e r s
e n t e r t h e p r o c e s s after an initial j u d g m e n t has
b e e n m a d e a n d w o u l d b e e x p e c t e d to b e m o r e
c o n c e r n e d w i t h d e t e r m i n i n g w h e t h e r t h e judg-
m e n t m a k e s s e n s e g i v e n t h e facts. In p a r t i c u l a r ,
w e e x a m i n e w h e t h e r t h e a b o v e r e s u l t s in
s y s t e m a t i c offsetting d i f f e r e n c e s in t h e m a n n e r
in w h i c h initial d e c i s i o n m a k e r s a n d r e v i e w e r s
a t t e n d to i n f o r m a t i o n 2 w h i c h e n s u r e s that
e v i d e n c e i n c o n s i s t e n t w i t h initial j u d g m e n t s is
given adequate consideration.
D i f f e r e n c e s in a t t e n t i o n a r e p r o p o s e d w h i c h
r e s u l t in differential r e c a l l o f e v i d e n c e b y t h e
initial d e c i s i o n m a k e r a n d r e v i e w e r . For initial
a t t e n t i o n a n d r e l a t i v e recall t o b e b i a s e d in t h e
d i r e c t i o n o f facts t h a t a r e c o n s i s t e n t w i t h t h e i r
initial judgments. F o r reviewers, this involves the
t e n d e n c y to d e v o t e g r e a t e r relative attention to
and thus shift relative recall in t h e d i r e c t i o n o f
facts i n c o n s i s t e n t w i t h those judgments.3 Import-
antly, t h e r e v i e w e r ' s t e n d e n c y should offset part
o f t h e initial d e c i s i o n maker's p r o p e n s i t y to ignore
o r forget conflicting information and thus m o d e -
r a t e a n y effect o n s u b s e q u e n t decisions.
T h e c u r r e n t s t u d y has t h r e e specific p u r p o s e s .
First, w e i n t r o d u c e c o n c e p t s that b e a r o n
w h e t h e r t h e s e offsetting t e n d e n c i e s m a y b e
e x p e c t e d a n d an e x p e r i m e n t a l d e s i g n a n d
m e a s u r e m e n t m e t h o d a p p r o p r i a t e for d e m o n -
s t r a t i n g t h e s e effects. S e c o n d , w e p r e s e n t an
experiment which demonstrates the difference
in r e c a l l s u g g e s t e d for t h e initial d e c i s i o n m a k e r
a n d r e v i e w e r . T h e l a c k o f any s t u d i e s o f a t t e n t i o n
a n d m e m o r y effects in a d e c i s i o n r e v i e w c o n t e x t
and c o n f l i c t i n g r e s u l t s in w e a k l y a n a l o g o u s
settings n e c e s s i t a t e d e s t a b l i s h i n g t h e s e c o n d i -
t i o n s in an a u d i t r e v i e w setting. Third, w e
present a second experiment which investigates
t h e effect o n r e v i e w e r b e h a v i o r o f r e c o r d i n g
f e w e r d e c i s i o n i n c o n s i s t e n t d a t a in t h e w o r k -
p a p e r s to b e r e v i e w e d . This e x p e r i m e n t b e g i n s
to establish t h e g e n e r a l i t y o f t h e r e v i e w e r
p h e n o m e n o n a n d tests t h e p o t e n t i a l effect o f
the under-recording of inconsistent information
w h i c h m a y r e s u l t f r o m t h e initial d e c i s i o n
m a k e r ' s t e n d e n c y to r e c a l l c o n s i s t e n t items.
In t h e n e x t s e c t i o n , t h e specific h y p o t h e s e s
e x a m i n e d a r e d e v e l o p e d fi'om t h e r e l a t e d
literature. This is f o l l o w e d b y sections d e s c r i b i n g
t h e m e t h o d a n d r e s u l t s o f e x p e r i m e n t s 1 a n d 2.
T h e final s e c t i o n d i s c u s s e s p o s s i b l e i n t e r a c t i o n s
o f t h e effects e x a m i n e d h e r e w i t h o t h e r k e y
attributes of the audit team environment.

2 Following Kahneman (1973), we define attention as a nonspecific cognitive resource necessary for the functioning of
mental activities. This resource is limited, but may vary with task demands and across individuals.

The term relative recall (proportion of facts recalled supporting one alternative compared to those supporting the other)
is appropriate here since overall attention may differ across tasks and individuals. Also, independent of the decision, facts
consistent and inconsistent with a judgment alternative may not be equally memorable. As a result, the effects discussed
below only predict a shift in relative recall.
 REVIEW PROCESS 561

RECALL OF AUDIT EVIDENCE AND THE accumulated in an audit will partially be a

REVIEW PROCESS function of the individual's long-term memory.
The potential source of gain from review
The heavy reliance placed by the auditing investigated here involves offsetting tendencies
profession on review processes as the principal in relative attention to and recall of evidence
audit control p r o c e d u r e is noted in the original consistent and inconsistent with the initial
Research Opportunities in Auditing published decision maker's judgments. 4 In examining this
by Peat Marwick in 1976. As indicated above, source of gain, two factors determine our
despite this recognition, research to date on the approach. First, we analyze relative recall as
role of the review process is quite limited opposed to levels of recall for particular item
(Solomon, 1987). Little insight has been provided types because total attention may vary across
c o n c e r n i n g the sources of performance gains or different tasks or among different individuals
the circumstances in which they will be (see footnote 3). For example, in the current
enhanced or lost. Given the significant resources experiments, recall is examined in the context
e x p e n d e d on the review process, understanding of evaluating a company's future viability. If
h o w it enhances effectiveness is essential to individuals are presented with the task of
understanding the efficient design of the audit reviewing a prior judgment of failure, they may
approach. pay greater or lesser overall attention to the
Recent research has recognized the role of the evidence than individuals reviewing a judgment
experienced accountant's accumulated know- of viability. Similarly, individuals w h o make
ledge in professional judgment (e.g. Birnberg & failure predictions may pay greater or lesser
Shields, 1984; Bonner & Pennington, 1991; overall attention to the evidence than those
Heiman, 1990; Libby, 1985; Marchant, 1989; w h o make viability predictions. As a result,
Weber, 1980). In addition to this general audit predicting and interpreting levels of recall of
knowledge, auditors must rely on their memories facts indicating failure or viability in any
for audit evidence encountered in the current condition is problematic. Computing the pro-
audit (Frederick, 1986; Moeckel & Plumlee, portion of facts recalled indicating failure minus
1989; Plumlee, 1985). Because of the volume those indicating viability (relative recall) for
of evidence typically recorded, and the time each subject controls for the effects of overall
frame over which it is gathered, auditors must attention. 5
rely on retrieval from long-term m e m o r y or re- Second, we analyze differences in relative
reading ofworkpapers when testing the implica- recall between the two groups of initial decision
tions of further evidence. Efficiency issues will makers (reviewers) w h o made ( r e v i e w e d )
often lead the auditor to rely on long-term judgments of failure and viability. We do not
m e m o r y as opposed to a thorough re-reading of predict the sign of relative recall in any
the workpapers. In situations where workpapers particular treatment because, other things equal,
are re-read, the process of searching for the facts consistent and inconsistent with a judgment
relevant information and remembering that alternative may not be equally memorable
information still will require use of long-term (footnote 3). For example, in the current
memory. As a result, the impact of the evidence setting, prior research found that auditors better

4 S i n c e a t t e n t i o n is difficult to m e a s u r e a n d w e d o n o t a t t e m p t t o d o so, w e a s s u m e h e r e that, all o t h e r t h i n g s e q u a l ,

c h a n g e s in r e l a t i v e r e c a l l reflect c h a n g e s in t h e r e l a t i v e i n t e n s i t y o r d u r a t i o n o f a t t e n t i o n . S u c h a n a s s u m p t i o n is c o m m o n
in s t u d i e s o f t h e a s s o c i a t i o n o f m e m o r y a n d j u d g m e n t (e.g. H a s t i e & K u m a r , 1 9 7 9 ) .

5 T h i s a p p r o a c h is s i m i l a r t o B o n n e r ' s ( 1 9 9 0 ) c o m p a r i s o n o f d i f f e r e n c e s in p e r f o r m a n c e o n t w o t a s k s a c r o s s e x p e r i e n c e
levels t o c o n t r o l f o r o v e r a l l ability, d i f f e r e n c e s b e t w e e n n o v i c e s a n d e x p e r t s .
562 R. L1BBYand K. T. TROTMAN

r e m e m b e r e d facts i n d i c a t i n g failure t h a n t h o s e ease s u b s e q u e n t justification o f a d e c i s i o n ,

i n d i c a t i n g v i a b i l i t y (Kida, 1984; T r o t m a n & Sng,
1989). This w o u l d also bias t h e d i r e c t i o n o f
r e l a t i v e r e c a l l for all initial d e c i s i o n m a k e r s a n d
r e v i e w e r s a n d m a k e p r e d i c t i o n s o f t h e sign o f
r e l a t i v e r e c a l l in a n y c o n d i t i o n p r o b l e m a t i c .
Comparison of relative recall between the
t w o g r o u p s o f initial d e c i s i o n m a k e r s ( p r e d i c t i n g
failure a n d v i a b i l i t y ) o r t h e t w o g r o u p s o f
r e v i e w e r s ( r e v i e w i n g failure a n d v i a b i l i t y pre-
d i c t i o n s ) c o n t r o l s for this bias.
As a c o n s e q u e n c e , t h e d i s c u s s i o n a n d t h e
h y p o t h e s e s w h i c h f o l l o w r e l a t e to differential
recall of inconsistent compared to consistent
i n f o r m a t i o n o r r e l a t i v e recall. W e c o m p a r e t h e
effect o f t y p e o f j u d g m e n t (fail o r r e m a i n v i a b l e )
o n r e l a t i v e r e c a l l b e t w e e n t h e t w o r o l e s (initial
decision maker or reviewer). The key prediction
in o u r first h y p o t h e s i s is t h a t t h e effect o f t y p e
o f j u d g m e n t will b e in o p p o s i t e d i r e c t i o n s for
initial d e c i s i o n m a k e r s a n d r e v i e w e r s , p r o d u c i n g
t h e a b o v e - m e n t i o n e d offsetting effects.
A t t e n t i o n a n d recall by the initial decision
maker
Systematic investigation o f t h e relation b e t w e e n
m e m o r y a n d j u d g m e n t is a r e c e n t p h e n o m e n o n
in t h e p s y c h o l o g y l i t e r a t u r e . D e c i s i o n m a k e r s '
s u p e r i o r recall o f facts s u p p o r t i n g their decisions
has b e e n d o c u m e n t e d in s o m e studies. M o s t
r e l e v a n t to t h e c u r r e n t s t u d y a r e e x a m i n a -
tions of whether making a decision increases
t h e s u b s e q u e n t a c c e s s i b i l i t y in m e m o r y o f
facts s u p p o r t i n g a c h o s e n p o s i t i o n ( r e l a t i v e to
facts o p p o s i n g t h e p o s i t i o n ) . D i r e c t e v i d e n c e
s u p p o r t i n g this p r o p o s i t i o n is p r o v i d e d b y
D e l l a r o s a & B o u r n e ( 1 9 8 4 ) , w h o s u g g e s t that
d e c i s i o n s h a v e this effect b e c a u s e t h e y p r o d u c e
a r e o r g a n i z a t i o n o f m e m o r y 6 w h i c h facilitates
r e c a l l o f d e c i s i o n c o n s i s t e n t facts a n d / o r i n h i b i t s
r e c a l l o f i n c o n s i s t e n t facts. In t h e i r m o d e l , to
s u p p o r t i n g facts are d i r e c t l y l i n k e d to t h e
d e c i s i o n in m e m o r y a n d i n c o n s i s t e n t facts a r e
not. 7 This i n c r e a s e in t h e n u m b e r o f linkages to
t h e s u p p o r t i n g facts i n c r e a s e s t h e p r o b a b i l i t y
t h a t t h e s u p p o r t i n g facts w i l l b e r e c a l l e d . In t h e i r
e x p e r i m e n t s , s u b j e c t s w h o m a d e d e c i s i o n s after
r e a d i n g fact p a s s a g e s r e c a l l e d r e l a t i v e l y m o r e
facts c o n s i s t e n t w i t h t h e i r d e c i s i o n s t h a n i n c o n -
sistent. Similar findings h a v e b e e n r e p o r t e d in a
s h o r t s t u d y b y T i m m e et al. ( 1 9 8 7 ) a n d a s t u d y
o f t h e effects o f a c c o u n t a b i l i t y o n d e c i s i o n s b y
T e t l o c k ( 1 9 8 3 ) . If t h e initial d e c i s i o n itself is
a f f e c t e d b y t h e differential a c c e s s i b i l i t y o f facts
supporting or opposing a certain position (often
c a l l e d an availability effect), this r e l a t i o n s h i p
would be accentuated.
T h e ability to justify has b e e n i d e n t i f i e d as a
defining a t t r i b u t e o f j u d g m e n t q u a l i t y (e.g. E m b y
& G i b b i n s , 1988). If, as D e l l a r o s a & B o u r n e
( 1 9 8 4 ) suggest, t h e n e e d for justification c a u s e s
t h e a b o v e relationship, similar effects o f decisions
o n m e m o r y m i g h t b e e x p e c t e d in t h e a u d i t
setting. W e establish t h e relationship o f decisions
to the relative recall of decision consistent and
i n c o n s i s t e n t i n f o r m a t i o n in t h e a u d i t setting, to
p r o v i d e a basis for c o m p a r i s o n for o u r m a j o r
c o n c e r n w h i c h is t h e r e v i e w e r ' s p e r f o r m a n c e .
Attention a n d recall by the reviewer
The consideration of competing explanations
a n d m a t t e r s t h a t t h e initial d e c i s i o n m a k e r m i g h t
h a v e m i s s e d h a v e b e e n i d e n t i f i e d as k e y r e v i e w
activities in b o t h s u r v e y r e s p o n s e s a n d t e x t b o o k
t r e a t m e n t s (e.g. B a m b e r et aL, 1988, p. 141;
G r o b s t e i n et al., 1985, p. 58). In a similar vein,
t h e n e e d to m a i n t a i n an i n d e p e n d e n t c r i t i c a l
v i e w p o i n t o r a p p r o p r i a t e l e v e l o f s k e p t i c i s m in
r e v i e w activities is e m p h a s i z e d in b o t h n o r m a t i v e
t r e a t m e n t s a n d a u d i t m a n u a l s (e.g. Mautz &
Sharaf, 1961; D e l o i t t e , Ross, T o h m a t s u , 1991,

6 Lingle & Ostrum (1979) also suggest that memory is re-structured by the judgment formulation process.

7 Whether this reorganization is a cognitive or motivational phenomenon is an open question. The need for later justification
of decisions suggests a motivational cause. The authors also provide a cognitive argument based on the strength of the
decision as a retrieval cue. These explanations are not inconsistent since the cognitive and motivational effects of decisions
are confounded both in our experiment and in the natural environment.
 REVIEW PROCESS
p. 222). C o n c e r n that this maxim is not followed
in p r a c t i c e has b e e n e x p r e s s e d by p r a c t i t i o n e r s
(e.g. Defliese eL aL, 1984; Peat Marwick, 1976)
and r e s e a r c h e r s ( B a m b e r et aL, 1 9 8 8 ) w h o fear
that, as a result, similarities in the reasoning
strategies of initial decision makers and reviewers
may reduce the effectiveness of review processes.
However, no research has b e e n u n c o v e r e d
w h i c h addresses this issue in a decision r e v i e w
setting in any domain.
Firms' attempts to e n s u r e r e v i e w e r indepen-
d e n c e through training may be aided by
a general human t e n d e n c y for novelty or
inconsistency to attract attention (Kahneman,
1973; Lynch & Srull, 1982). If the initial decision
maker's j u d g m e n t affects the r e v i e w e r ' s expect-
ations, after the r e v i e w e r has b e e n informed of
the initial decision maker's j u d g m e n t (e.g. the
firm will remain viable), this effect w o u l d
increase relative attention to and m e m o r y for
s u b s e q u e n t e v i d e n c e which is inconsistent with
that judgment. Yet, again, no e v i d e n c e bearing
on this p h e n o m e n o n in closely analogous
settings is available.
The most closely related evidence comes from
the impression formation literature. Research in
this area suggests that p r e s e n t a t i o n of an initial
categorization, such as an ethnic or occupational
stereotype, changes the salience of subsequently
v i e w e d information resulting in s u p e r i o r recall
for inconsistent information (e.g. Bodenhausen,
1988; Hastie & Kumar, 1979; Srull, 1981; Srull
et aL, 1985). Various mechanisms have b e e n
p r o p o s e d to a c c o u n t for this effect. Most
relevant to the c u r r e n t study is Hastie &
Kumar's ( 1 9 7 9 ) suggestion that integration of
inconsistent items requires d e e p e r p r o c e s s i n g
at the e n c o d i n g stage ( s e e also Hastie, 1984).
The increase in attention results in s u p e r i o r
r e c a l l s Although most of this research also
h y p o t h e s i z e s the e x i s t e n c e of a schematic
m e m o r y structure, virtually any t a x o n o m i c or
s c h e m a t i c s t r u c t u r e w h e r e the firm type (e.g. a
viable or failing firm) is a relevant d i m e n s i o n of
categorization or associated attribute w o u l d
8 Similarpredictions are made by a model of memory by Graesser & Nakamura (1982).
563
p r o v i d e sufficient basis for the p h e n o m e n o n . In
the r e v i e w process, r e v i e w e r s often r e c e i v e an
initial j u d g m e n t ( i n this case, fail or v i a b l e )
w h i c h has s o m e similarities to the initial
categorization in the impression formation
literature. Following Hastie & Kumar ( 1 9 8 4 ) w e
suggest that items inconsistent with the senior's
initial j u d g m e n t will r e q u i r e d e e p e r p r o c e s s i n g
leading to superior recall. However, the existence
of this p h e n o m e n o n in decision r e v i e w settings,
and in audit r e v i e w in particular, is an o p e n
question.
Recall by the reviewer given biased workpapers
The arguments above assume that the initial
decision maker and r e v i e w e r evaluate the
same evidence. However, in situations w h e r e
u n s t r u c t u r e d w o r k p a p e r formats are used, initial
decision makers and r e v i e w e r s might e x a m i n e
different information sets as a result of selective
r e c o r d i n g of e v i d e n c e in the workpapers. In
particular, if initial decision makers recall m o r e
consistent than inconsistent items and then use
these items in the w o r k p a p e r s to s u p p o r t their
decision, the r e v i e w e r will r e c e i v e a biased set
of facts. Warnings in firm manuals to eliminate
d o c u m e n t a t i o n s u p p o r t i n g d i s c a r d e d conclu-
sions and not to c o n v e y inconclusiveness may
p r o m o t e this behavior.
It was argued above that information that is
novel or u n e x p e c t e d receives relatively m o r e
attention, is p r o c e s s e d m o r e extensively, and
thus its s u b s e q u e n t m e m o r a b i l i t y is increased
c o m p a r e d to information that is e x p e c t e d
to appear in a given context. It has also
b e e n p r o p o s e d that decreasing the n u m b e r of
inconsistent items p r e s e n t e d further increases
their novelty and thus the attention they receive.
In the audit r e v i e w setting, this effect w o u l d
e n h a n c e the gain from the r e v i e w by partially
offsetting the impact of the r e p o r t i n g of a larger
n u m b e r of consistent items. The most closely
related e x p e r i m e n t a l w o r k bearing on the
r e v i e w e r ' s response to smaller sets of incon-
sistent data is again the impression formation
564 R. LIBBYand K. T. TROTMAN
literature. In these studies, the p r o p o r t i o n of
inconsistent items recalled increased compared
to the p r o p o r t i o n of c o n s i s t e n t i t e m s as the size
of t h e i n c o n s i s t e n t set d e c r e a s e d ( B e l m o r e &
H u b b a r d , 1987; Hastie & Kumar, 1979; Srull,
1981).
Hypotheses
T h e a b o v e d i s c u s s i o n suggests that shifts in
relative a t t e n t i o n to c o n s i s t e n t a n d i n c o n s i s t e n t
i n f o r m a t i o n p r o d u c e t w o t e n d e n c i e s in relative
recall, w h i c h m a y have partially offsetting effects
in the adult r e v i e w c o n t e x t . T h e s e effects
p r o v i d e t h e basis for t h e j o i n t h y p o t h e s i s t e s t e d
in e x p e r i m e n t 1.
Hla. Initial decision makers' judgments result in a
tendency for their relative recall to be in the direction
of facts consistent with their judgments.
Hlb. Initial decision makers' judgments result in a
tendency for reviewers' relative recall to be in the
direction of facts which are inconsistent with the initial
decision maker's judgments.
I n c r e a s e s in relative a t t e n t i o n to n o v e l i t e m s
are suggested to r e s u l t in f u r t h e r i n c r e a s e s in
the r e v i e w e r s ' t e n d e n c y w h e n t h e n u m b e r of
i n c o n s i s t e n t i t e m s available decreases. This
effect results i n t h e s e c o n d h y p o t h e s i s w h i c h is
e x a m i n e d in e x p e r i m e n t 2.
H2. As the proportion of inconsistent items in an
information set decreases, there will be a greater
tendency for the reviewers' relative recall to be in the
direction of inconsistent facts.
T h e e x p e r i m e n t s d e s i g n e d to e x a m i n e these
h y p o t h e s e s are d e s c r i b e d in t h e n e x t section.
Experimental overview
Subjects participated as initial decision makers
o r r e v i e w e r s in a financial viability p r e d i c t i o n
task b a s e d o n Kida ( 1 9 8 4 ) . This task was
9 Tests reported later in terms of a ratio measure produced similar results.
zoBased on prior research, this variable should be positive, all things equal. Lynch & SruU(1982) discuss studies in other
fields which find superior recall for items suggesting a negative outcome.
s e l e c t e d b e c a u s e of its r e c o g n i z e d i m p o r t a n c e
(cf. AICPA, 1 9 9 1 ) a n d the e x i s t e n c e of p r i o r
r e s e a r c h to g u i d e t h e d e v e l o p m e n t of experi-
m e n t a l stimuli. T h e p a r t i c i p a n t s in the s t u d y
w e r e Australian a u d i t o r s for w h o m the task is
o f g r e a t e r r e l e v a n c e b e c a u s e of an a d d e d legal
r e q u i r e m e n t . T h e y m u s t gather e v i d e n c e a n d
attest to each c l i e n t ' s j u d g m e n t , e x p r e s s e d in
the D i r e c t o r ' s Statement, of its ability to pay
d e b t s w h e n due.
As n o t e d above, the h y p o t h e s e s are stated in
t e r m s o f relative recall of facts s u g g e s t i n g failure
(fail i t e m s ) versus facts s u g g e s t i n g viability
( v i a b l e items). E x p e r i m e n t 1 c o m p a r e s relative
recall b e t w e e n initial d e c i s i o n m a k e r s and
r e v i e w e r s w h o m a k e / r e v i e w failure a n d viability
p r e d i c t i o n s . E x p e r i m e n t 2 c o m p a r e s relative
recall b e t w e e n r e v i e w e r s in c o n d i t i o n s w h e r e
the p r o p o r t i o n of failure items p r e s e n t e d is
varied. It also tests t h e r e l a t i o n s h i p of relative
recall a n d the r e v i e w e r ' s o w n s u b s e q u e n t
j u d g m e n t of viability. Relative recall is opera°
t i o n a l i z e d h e r e as the p r o p o r t i o n of fail items
less the p r o p o r t i o n of viable items recalled. 9
Using t h e s e relative m e a s u r e s c o n t r o l s for the
p r e v i o u s l y s u g g e s t e d s u p e r i o r recall of fail
items m a n d p o s s i b l e overall effort differences
b e t w e e n the initial d e c i s i o n m a k e r a n d r e v i e w e r
c o n d i t i o n s a n d the c o n d i t i o n s w h e r e t h e initial
d e c i s i o n m a k e r p r e d i c t s that t h e firm will r e m a i n
viable o r fail. No m e a s u r e m e n t s w e r e m a d e
a i m e d at d i r e c t l y t e s t i n g the p r o p o s e d u n d e r -
lying m e c h a n i s m s (e.g. a t t e n t i o n ) or d e t e r m i n -
ing w h e t h e r t h e effects e x a m i n e d h e r e are
n a t u r a l c o g n i t i v e effects, the r e s u l t o f training,
o r j o i n t effects.
Subjects p a r t i c i p a t i n g as r e v i e w e r s in b o t h
e x p e r i m e n t s w e r e p l a c e d in a s i t u a t i o n w h e r e
the initial d e c i s i o n m a k e r ' s j u d g m e n t s w e r e
k n o w n before any related e v i d e n c e was evaluated.
This s i t u a t i o n o f t e n o c c u r s in the b u s y season
w h e r e staff w o r k m a y b e i g n o r e d u n t i l late in
 REVIEW PROCESS
t h e i n f o r m a t i o n g a t h e r i n g p r o c e s s o r in t h e case
of independent partner or manager review. The
e m p h a s i s g i v e n t o t h e n e e d for an i n d e p e n d e n t
a t t i t u d e in r e v i e w s u g g e s t s t h a t this is t h e
s i t u a t i o n w h e r e t h e r e v i e w e r effects w o u l d
m o r e likely b e in e v i d e n c e . It is left to f u t u r e
r e s e a r c h to d e t e r m i n e t h e d e g r e e t o w h i c h p r i o r
i n v o l v e m e n t ( l a c k o f i n d e p e n d e n c e ) affects t h e
reviewer's behavior.
EXPERIMENT 1
T h e first e x p e r i m e n t w a s d e s i g n e d to c o m p a r e
t h e effect o f t h e initial d e c i s i o n m a k e r s ' judg-
ment on both their and the reviewers' relative
recall. This r e q u i r e d that all factors e x c e p t t h e
subject's role (initial d e c i s i o n m a k e r o r r e v i e w e r )
b e h e l d constant o r controlled. As a result, s u b j e c t
experience and order of presentation were
r a n d o m i z e d and a s s u m e d rank ( n e w l y a p p o i n t e d
supervisor), item content, recall procedures,
and coding procedures were held constant
a c r o s s t h e initial d e c i s i o n m a k e r a n d r e v i e w e r
t r e a t m e n t s . T h e combination o f n u m b e r o f
i n f o r m a t i o n items, t i m e available for reading,
a n d t i m e i n t e r v a l b e f o r e r e c a l l w a s c h o s e n so
that a v e r a g e m e m o r y p e r f o r m a n c e w o u l d n o t
a p p r o a c h t h e c e i l i n g a n d floor w h i c h c o u l d
artifactually p r o d u c e s o m e o f t h e i n t e r a c t i o n s
w e p r e d i c t . T h e s e levels w e r e e s t a b l i s h e d b a s e d
o n p r e t e s t i n g . M e m o r y d e m a n d s on s u b j e c t s
(information load) were well below those
e n c o u n t e r e d o n a c t u a l audits. Since t h e m e m o r y
effects d e s c r i b e d a b o v e are said to r e s u l t f r o m
h i g h e r m e m o r y d e m a n d s , any bias s h o u l d b e
against t h e h y p o t h e s i s .
Design
The experiment employed a 2 × 2 between-
s u b j e c t s d e s i g n w h e r e t h e i n d e p e n d e n t vari-
a b l e s w e r e r o l e (initial d e c i s i o n m a k e r o r
r e v i e w e r ) a n d initial d e c i s i o n m a k e r ' s j u d g m e n t
( v i a b l e o r fail). S u b j e c t s w e r e r a n d o m l y a s s i g n e d
nThere were three minor variations of terms, given that the experiment used Australian auditors. This involved replacing
"the sale of stock" by "the issue of shares". "labor representatives" by "union representatives" and "stockholders' dividends"
by "shareholders' dividends".
565
to t h e initial d e c i s i o n m a k e r a n d r e v i e w e r
t r e a t m e n t s . S u b j e c t s in b o t h t r e a t m e n t s r e a d t h e
s a m e 2 0 - i t e m firm d e s c r i p t i o n w h i c h s e r v e d as
t h e e x p e r i m e n t a l stimuli. In t h e initial decision°
maker treatment, subjects were asked to make
a judgment of the future viability of the company
after r e a d i n g t h e d e s c r i p t i o n . That j u d g m e n t
b e c a m e t h e i r observed o r s e l f - s e l e c t e d level for
t h e s e c o n d i n d e p e n d e n t variable. T h e r e v i e w e r
treatment subjects were presented with a
s e n i o r ' s j u d g m e n t o f f u t u r e viability o r failure,
b a s e d o n r a n d o m a s s i g n m e n t , before t h e y r e a d
t h e s a m e firm d e s c r i p t i o n . This r a n d o m l y
assigned j u d g m e n t d e t e r m i n e d t h e i r manipu-
lated level for t h e s e c o n d i n d e p e n d e n t variable.
These were the only differences between the
initial d e c i s i o n m a k e r a n d r e v i e w e r t r e a t m e n t s .
After c o m p l e t i n g a c o m m o n d i s t r a c t o r task, t h e y
w e r e a s k e d to r e c a l l t h e facts p r e s e n t e d in t h e
firm d e s c r i p t i o n . T h e p r o p o r t i o n s o f i t e m s
r e c a l l e d i n d i c a t i n g failure a n d v i a b i l i t y w e r e
compared between the subjects who judged (in
t h e "initial d e c i s i o n m a k e r t r e a t m e n t " ) o r w e r e
t o l d ( i n t h e " r e v i e w e r t r e a t m e n t " ) that t h e firm
w a s m o r e likely to fail o r b e v i a b l e in t h e future.
Experimental stimuli
All s u b j e c t s w e r e g i v e n t h e s a m e b a c k g r o u n d
i n f o r m a t i o n , w h i c h s t a t e d that ( 1 ) XYZ w a s a
medium-sized manufacturing company which
h a d b e e n a u d i t e d b y t h e i r firm for t h e last five
years, ( 2 ) t h e y w e r e to a s s u m e t h e y w e r e a
n e w l y a p p o i n t e d s u p e r v i s o r for XYZ, and ( 3 )
a t t a c h e d w a s s o m e i n f o r m a t i o n o n XYZ w h i c h
w a s c o l l e c t e d b y t h e senior. In a d d i t i o n all
s u b j e c t s r e c e i v e d a firm d e s c r i p t i o n . R e v i e w e r s
w e r e t o l d that t h e i n f o r m a t i o n w a s m a d e
available for t h e i r r e v i e w .
T h e firm d e s c r i p t i o n p r o v i d e d w a s t h e o n e
u s e d b y Kida ( 1 9 8 4 ) a n d is p r e s e n t e d in Table
1. l~ It c o n s i s t e d o f 20 i t e m s w h i c h a u d i t o r s
c o n s i d e r i m p o r t a n t in m a k i n g g o i n g - c o n c e r n
d e c i s i o n s a n d assessing i n h e r e n t risks. A statistic-
ally p o w e r f u l test o f H y p o t h e s i s l a r e q u i r e d as
566 R. LIBBY and K. T. TROTMAN

TABLE 1. Firm description given to subjects

1. Management indicates that there is a good chance of losing a major customer.
2. The technology of the company is competitive with other firms in the industry.
3. The competence of the firm's management has been questioned by outside observers.
4. The firm's major product is generally considered to be of good quality.
5. The company has significantly less working capital than the average firm in the industry.
6. Management states that it is possible that a key patent may be obtained in the near future.
7. Discussions with management indicate that a material liability from litigation is likely this year.
8. The debt-to-equity ratio of the company is around the industry average.
9. Management believes that additional equity capital can be raised through the issue of shares if needed.
10. This year the firm reported a significant loss from operations.
11. Management and union representatives indicate that there is a chance that labour will strike this year.
12. This year the company reported a positive cash flow from operations.
13. Management indicated that new legislation may make it difficult to market one of the firm's major products.
14. In general, suppliers of the firm indicate that usual trade credit to the firm will be available.
15. It appears that if needed it will be difficult to obtain additional debt capital.
16. The economic oudook for the industry is stable.
17. The company has not been paying its preferred shareholders dividends in recent years.
18. The market share of the firm is below average for the industry.
19. Management has indicated that there is no chance of losing a major supplier.
20. An analysis of accounts receivable revealed the collection time to be around the industry average.

c l o s e as p o s s i b l e t o a 5 0 - 5 0 s p l i t i n t h e i n i t i a l was determined for each subject in the initial

decision maker subjects' judgments. To this end,
the items were selected such that one half
pointed to failure and the other half to continued
v i a b i l i t y . 12 A s e c o n d c o n s e q u e n c e o f t h i s c h o i c e
was testing the joint hypothesis in situations
n e a r its l i m i t w h e r e t h e e x i s t i n g l i t e r a t u r e
suggests the effects should be at their weakest.
T h e i n i t i a l d e c i s i o n - m a k e r e f f e c t s h o u l d b e a t its
w e a k e s t w h e r e t h e r e is m a x i m u m u n c e r t a i n t y
w h i c h s h o u l d r e s u l t f r o m t h e e q u a l s e t sizes.
The impression formation literature referenced
earlier also implies that the reviewer effect may
b e a t its w e a k e s t i n t h i s s i t u a t i o n .
Kida (1984) had the firm description reviewed
by eight partners and managers to ensure that
the case was realistic and there were data
equally supportive of both failure and viability.
Eleven other partners and managers rated these
i t e m s as p r o v i d i n g s u p p o r t f o r e i t h e r f a i l u r e o r
viability. Of the 220 responses, 218 were
consistent with original classifications. In dev-
eloping the research instruments, a different
random order of presentation for the 20 items
decision-maker treatment and the same orders
of presentation were used in the reviewer
treatment.
Subjects
The subjects were 56 auditors from two
A u s t r a l i a n o f f i c e s o f a " B i g 6 " a u d i t firm. T h e
average experience level was 3.95 years. The
subjects consisted of one manager, 26 supervisors,
2 8 s e n i o r s a n d o n e a c c o u n t a n t III. A c c o r d i n g t o
B a m b e r et al. ( 1 9 8 8 ) , a u d i t o r s at t h i s l e v e l a r e
r e g u l a r l y i n v o l v e d as b o t h i n i t i a l d e c i s i o n
m a k e r s a n d r e v i e w e r s . I n a d d i t i o n , as n o t e d
earlier, Australian seniors and supervisors are
heavily involved in attesting to the client's
viability assessment expressed in the Director's
Statement.
All s u b j e c t s w e r e a t t e n d i n g o n e o f t h e
firm's training sessions for seniors/supervisors.
Twenty-eight subjects were randomly allocated
to the initial decision maker and reviewer
treatments.

t2 For firms using unstructured working papers it is unlikely that there would be a balanced set of information documented
in the working papers. This external validity threat is treated in experiment 2. It is also noted that where structured
working papers are used such a split could occur.
 REVIEW PROCESS
Scoring o f the recall data
Similar to D e l l a r o s a & B o u r n e ( 1 9 8 4 ) , a
s t a t e m e n t w a s s c o r e d as c o r r e c t if b o t h t h e
s u b j e c t m a t t e r ~tnd t h e d i r e c t i o n w e r e c o r r e c t l y
r e c o r d e d . That is, r e c a l l o f a g i v e n i t e m d i d n o t
n e e d to b e v e r b a t i m , b u t t h e m e a n i n g n e e d e d
to b e e q u i v a l e n t to t h e o r i g i n a l passage, a n d t h e
d i r e c t i o n n e e d e d to b e c o r r e c t . T h e recall
p r o t o c o l s w e r e classified b y t w o i n d e p e n d e n t
r e s e a r c h assistants. T h e c o r r e l a t i o n b e t w e e n t h e
t w o i n d i v i d u a l classifications w a s 0.98 ( f o r b o t h
failure a n d v i a b i l i t y i t e m s ) w h i c h is c o n s i s t e n t
with the high correlation found by previous
r e s e a r c h u s i n g this m e t h o d ( C h o o & T r o t m a n ,
1991; G r a e s s e r et al., 1980). T h e K a p p a
coefficient ( C o h e n , 1 9 6 0 ) indicates a n o n c h a n c e
a g r e e m e n t o f 0.96 ( p < 0 . 0 0 1 ) . T h e small
number of differences were adjudicated by one
of the researchers before he became aware of
which treatment group each subject belonged
to.
Procedures
Initial d e c i s i o n - m a k e r treatment. S u b j e c t s
were told to assume they were the newly
a p p o i n t e d s u p e r v i s o r for XYZ a n d w e r e t h e n
p r o v i d e d w i t h t h e list o f 20 i t e m s d e s c r i b i n g t h e
company which they were told was collected
b y t h e senior. T h e y w e r e i n s t r u c t e d to r e a d t h e
i n f o r m a t i o n c a r e f u l l y in o r d e r t o familiarize
themselves with the company because they
would later be asked some questions about the
c o m p a n y . T h e y w e r e i n f o r m e d that t h e informa-
t i o n w o u l d b e c o l l e c t e d in 5 m i n u t e s . N o
m e n t i o n w a s m a d e o f a s u b s e q u e n t r e c a l l test
o r t h e n e e d to m a k e a specific j u d g m e n t s u c h
as w h e t h e r t h e c o m p a n y w o u l d r e m a i n a g o i n g
c o n c e r n . This a p p r o a c h is c o n s i s t e n t w i t h t h e
p s y c h o l o g y l i t e r a t u r e (e.g. D e l l a r o s a & B o u r n e ,
1984; G r a e s s e r et al., 1980; Srull, 1981).
After t h e i n f o r m a t i o n w a s c o l l e c t e d , s u b j e c t s
w e r e g i v e n f o u r d e m o g r a p h i c q u e s t i o n s to
a n s w e r ( e x p e r i e n c e , age, rank, i n d u s t r i e s o f last
t w o a u d i t s w o r k e d o n ) . As in D e l l a r o s a & B o u r n e
( 1 9 8 4 ) , this d i s t r a c t o r task w a s u s e d to c l e a r
13Since 20 of 28 subjects chose the "confident" category, no further analysis was performed based on the confidence scale.
567
short-term memory before a judgment was
m a d e . O n a s e p a r a t e page, s u b j e c t s w e r e t h e n
a s k e d to m a k e a j u d g m e n t o n w h e t h e r t h e
c o m p a n y w o u l d fail w i t h i n t w o y e a r s o r r e m a i n
v i a b l e for t w o years. S u b j e c t s w e r e also a s k e d
t h e i r level o f c o n f i d e n c e in this j u d g m e n t . T h e
c o n f i d e n c e scale u s e d w a s a t h r e e - p o i n t scale,
r a n g i n g f r o m n o t v e r y c o n f i d e n t to v e r y confi-
d e n t . It w a s t h e s a m e scale u s e d in Libby
(1975)fl 3
Subjects w e r e t h e n given a 30-minute d i s t r a c t o r
task. This d i s t r a c t o r task w a s s i m i l a r to t h e
A s h t o n ( 1 9 7 4 ) i n t e r n a l c o n t r o l task a n d w a s
t o t a l l y u n r e l a t e d to t h e p r e s e n t e x p e r i m e n t a l
task. T h e d i s t r a c t o r task w a s u s e d to c l e a r s h o r t -
t e r m m e m o r y p r i o r t o t h e r e c a l l test. S u b j e c t s
w e r e t h e n g i v e n 10 m i n u t e s t o c o m p l e t e a r e c a l l
test. T h e y w e r e a s k e d to list all t h e i n f o r m a t i o n
that t h e y c o u l d r e m e m b e r a b o u t XYZ. T h e y
w e r e t o l d that e v e n if t h e y h a d d o u b t s a b o u t
the completeness or the importance of their
s t a t e m e n t s , t h e y s h o u l d i n c l u d e them.
Reviewer t r e a t m e n t S u b j e c t s w e r e t o l d to
assume that they were the newly appointed
s u p e r v i s o r for XYZ a n d that a t t a c h e d w a s s o m e
i n f o r m a t i o n o n XYZ w h i c h h a d b e e n c o l l e c t e d
b y t h e s e n i o r a n d m a d e available f o r their
review ( e m p h a s i s a d d e d ) . Half w e r e t o l d t h a t
t h e s e n i o r c o n c l u d e s , o n t h e basis o f t h e
i n f o r m a t i o n given, that t h e c o m p a n y is likely to
r e m a i n v i a b l e for t h e n e x t t w o years. T h e o t h e r
half w e r e t o l d that t h e s e n i o r c o n c l u d e s that t h e
firm is likely to fail w i t h i n t h e n e x t t w o years.
As in t h e initial d e c i s i o n - m a k e r t r e a t m e n t ,
s u b j e c t s w e r e t h e n a s k e d t o familiarize t h e m -
s e l v e s w i t h this i n f o r m a t i o n as t h e y w o u l d l a t e r
be asked some questions about the company.
After t h e a b o v e i n f o r m a t i o n w a s c o l l e c t e d ,
subjects were asked the same four demographic
q u e s t i o n s as in t h e initial d e c i s i o n - m a k e r treat-
m e n t b u t w e r e n o t a s k e d for a d e c i s i o n o n
viability. So t h a t it w a s n o t o b v i o u s to p a r t i c i -
p a n t s t h a t t h e y w e r e i n v o l v e d in different
t r e a t m e n t s , all s u b j e c t s in t h e r o o m ( w h e t h e r
a s s i g n e d to t h e initial d e c i s i o n m a k e r o r
568 R. L1BBYand K. T. TROTMAN
reviewer treatments) received the same number
o f p a g e s for e a c h p a r t o f t h e e x p e r i m e n t . This
w a s a c c o m p l i s h e d b y s p r e a d i n g the d e m o g r a p h i c
q u e s t i o n s o v e r t w o p a g e s in t h e r e v i e w e r
treatment. Subjects then received the same
d i s t r a c t o r task a n d r e c a l l t e s t as d i d t h e initial
d e c i s i o n - m a k e r subjects.
Results
Initial d e c i s i o n - m a k e r s u b j e c t s w e r e classified
b y w h e t h e r t h e y j u d g e d t h e firm t o b e m o r e
l i k e l y to fail o r b e v i a b l e o v e r t h e n e x t t w o years.
T h i r t e e n s u b j e c t s j u d g e d t h e firm t o b e m o r e
likely to r e m a i n v i a b l e a n d fifteen to fail. As n o t e d
earlier, this o b s e r v e d v a r i a b l e b e c a m e t h e
s e c o n d i n d e p e n d e n t v a r i a b l e in t h e analysis for
t h e initial d e c i s i o n makers. T h e r e w a s n o
d i f f e r e n c e in y e a r s o f a u d i t e x p e r i e n c e b e t w e e n
t h e s u b j e c t s j u d g i n g t h e firm to b e m o r e likely
to r e m a i n v i a b l e a n d m o r e likely to fail ( t =
O. 16, p > 0.87, t w o -tailed ). Each initial d e c i s i o n -
maker subject's recall protocol was then scored,
as d e s c r i b e d a b o v e , a n d t h e n u m b e r o f i t e m s
i n d i c a t i v e o f failure a n d v i a b i l i t y w e r e c o u n t e d .
Each r e v i e w e r s u b j e c t ' s r e c a l l p r o t o c o l w a s
s c o r e d in t h e s a m e fashion.
H y p o t h e s i s l a p r e d i c t s initial d e c i s i o n makers'
r e l a t i v e r e c a l l to b e in t h e d i r e c t i o n o f informa-
t i o n c o n s i s t e n t w i t h t h e i r d e c i s i o n s . This h y p o -
thesis, r e s t a t e d in t h e f o r m o f a c o n t r a s t b e t w e e n
m e a n s , is p r e s e n t e d in e q u a t i o n ( 1 ) :
(% Fail -- %Viable)judged ~able
< ( % F a i l -- %Viable)juaged va~ (1)
w h e r e % Fail = p r o p o r t i o n o f i t e m s r e c a l l e d
i n d i c a t i n g failure; %Viable = p r o p o r t i o n o f
i t e m s r e c a l l e d i n d i c a t i n g viability; %Fail -
%Viable = relative recall;Judged viable = mean
o f s u b j e c t s w h o judged t h e firm t o m o s t likely
b e viable; a n d J u d g e d fail = m e a n o f s u b j e c t s
w h o judged the firm to m o s t likely f a i l
H y p o t h e s i s l b s u g g e s t s t h a t initial d e c i s i o n
m a k e r s ' j u d g m e n t s r e s u l t in a t e n d e n c y for
t4 On average, preparers (reviewers) recalled 68% (67%) of failed items and 60% (60%) of viable items. As suggested
by prior studies (e.g. Kida, 1984), failure items were more memorable overall than viable items.
r e v i e w e r s ' r e l a t i v e recall t o b e in t h e d i r e c t i o n
o f i n f o r m a t i o n i n c o n s i s t e n t w i t h t h e initial
d e c i s i o n m a k e r ' s j u d g m e n t . This h y p o t h e s i s can
b e r e s t a t e d in t h e s a m e fashion as H y p o t h e s i s
la. H o w e v e r , t h e d i r e c t i o n o f t h e p r e d i c t i o n is
t h e o p p o s i t e o f that in t h e initial d e c i s i o n - m a k e r
t r e a t m e n t , a n d is p r e s e n t e d in e q u a t i o n ( 2 ) :
(% Fail -- % Viable)Tola viable
> (% Fail -- %Viable)told f~u (2)
where Told viable = mean of subjects who were
told t h a t t h e s e n i o r h a d j u d g e d t h e firm to m o s t
likely b e viable; a n d T o l d fail = m e a n o f s u b j e c t s
w h o w e r e told t h a t t h e s e n i o r h a d j u d g e d t h e
firm t o m o s t likely f a i l
T h e s e e q u a t i o n s p r e d i c t o p p o s i t e effects o f
j u d g i n g o r b e i n g t o l d ( i n t h e initial d e c i s i o n -
maker and reviewer treatments, respectively)
t h a t t h e c o m p a n y w i l l m o r e likely fail o r b e
viable, a n d c a n b e j o i n t l y t e s t e d b y t h e
i n t e r a c t i o n t e r m in a t w o - w a y ANOVA. I n d e p e n -
d e n t v a r i a b l e s in t h e A_NOVA a r e ( 1 ) w h e t h e r
the subjects judged (were told) the company
w o u l d m o r e likely fail o r b e viable, a n d ( 2 )
w h e t h e r t h e y w e r e p a r t i c i p a t i n g as initial
decision makers or reviewers. The dependent
variable was relative recall (the proportion of
failure i t e m s r e c a l l e d m i n u s t h e p r o p o r t i o n o f
v i a b l e i t e m s r e c a l l e d ) . T h e i n t e r a c t i o n is h i g h l y
significant ( F = 8.65, p = 0 . 0 0 5 ) , s u p p o r t i n g
Hypotheses la and lb. Mean relative recall
statistics in t h e f o u r t r e a t m e n t c o m b i n a t i o n s
w e r e in t h e e x p e c t e d d i r e c t i o n s a n d a r e
p r e s e n t e d g r a p h i c a l l y in Fig. 1. ~4 Relative r e c a l l
b y initial d e c i s i o n m a k e r s rose f r o m 2% t o 12%
( t = 1.81, p = 0 . 0 4 1 ) b e t w e e n t h o s e j u d g i n g
t h e firm to m o r e likely b e v i a b l e a n d fail.
R e v i e w e r s ' r e l a t i v e r e c a l l declined f r o m 16% to
-2% between those reviewing judgments of
viability a n d failure ( t = 2 . 3 2 , p = 0 . 0 1 4 ) . W h e n
t h e o v e r a l l analysis w a s c o n d u c t e d u s i n g t h e
p r o p o r t i o n o f failure i t e m s in total r e c a l l as
t h e d e p e n d e n t variable, t h e significance o f t h e
 REVIEW PROCESS 569

16
14

" 12

10
8
+ Initial Decision Maker
%Fail - % Viable 6
• Reviewer
4
2
o
-2
\
-4
Viable Fail
Subjects Predicting or Told
Fig. 1. Relative recall for initial decision maker and reviewer.

TABLE 2. Mean number of items recalled

Initial decision maker Reviewer
Item Chose Chose Told Told
type viable failure viable failure
Failure 6.8 6.7 7.2 6.2
Viable 6.5 5.5 5.6 6.4

interaction was largely unchanged (F = 9.52, p d e c i s i o n m a k e r a n d r e v i e w e r w i l l f o c u s o n as

= 0 . 0 0 3 ) . 15
T h e a b o v e a n a l y s i s is b a s e d o n r e l a t i v e r e c a l l
f o r t h e r e a s o n s s t a t e d a b o v e . As a d d i t i o n a l
information, Table 2 provides the mean number
o f v i a b l e a n d f a i l u r e i t e m s r e c a l l e d i n e a c h cell.
It c a n b e s e e n t h a t t h e e f f e c t f o r t h e i n i t i a l
decision makers results from a difference in the
n u m b e r o f v i a b l e i t e m s r e c a l l e d . F o r t h e re-
viewers, both the number of viable and ~il items
recalled changes in the expected directions but
a g a i n t h e v i a b l e i t e m s h a v e t h e l a r g e s t effect.
Discussion
The current study hypothesizes two effects
which determine what information the initial
t h e y b e g i n t h e i r d i s c u s s i o n s . As p r e d i c t e d , i n i t i a l
decision makers' judgments resulted in a ten-
dency for their relative recall to be in the
direction of information consistent with their
judgments, while initial decision makers' judg-
ments resulted in the reviewers' relative recall
being in the direction of information inconsistent
with that decision. The reviewer's propensity
c o u l d a c t as a n e f f e c t i v e c o n t r o l b y i n c r e a s i n g
the chances that the implications of inconsistent
evidence are considered.
Using subjects with the same rank and general
experience levels in both the initial decision
maker and reviewer treatments insures that the
findings presented here result from the difference

t s Twenty-five and eighteen items in the recall protocols of the initial decision makers and reviewers, respectively, were
scored by the raters as false recalls. These items included those which were judged as having not been presented, only
vaguely similar to those presented, or in the opposite direction as those presented. To test whether the exclusion of these
items affected the results, the analysis was repeated with the false recalls included in the data. The two raters scored these
items as "inferred" failure or viability responses, and the total proportion of failure minus proportion of viable items
recalled was re-computed for each subject. With these items included, the significance of the interaction was basically
unchanged (F = 6.87,p = 0.01 ). Items recalled that indicated neither failure nor viability were excluded from all analysis.
570 R. LIBBYand K. T. TROTMAN
in the roles that the subjects p l a y e d as o p p o s e d
to differential training and e x p e r i e n c e .
EXPERIMENT 2
E x p e r i m e n t 2 was designed to e x a m i n e the
effect of d e c r e a s i n g the size of the inconsistent
data set on r e v i e w e r s ' relative recall. Such a test
helps establish the generality of the r e v i e w e r
effect identified in e x p e r i m e n t 1 and tests the
possible effect of under-reporting of inconsistent
data in the workpapers. It also e x a m i n e d the
relationship b e t w e e n reviewers' recall and their
judgments. The same p r o c e d u r e s and scoring
m e t h o d s w e r e e m p l o y e d as in e x p e r i m e n t 1
e x c e p t for t w o small changes n o t e d below.
Design
The p r o p o r t i o n of inconsistent items was
varied across three levels: 12 inconsistentdl2
consistent, 8 inconsistent/16 consistent, and
4 inconsistent/20 consistent. Subjects w e r e
randomly assigned to each treatment. All subjects
w e r e told that the senior c o n c l u d e s on the basis
of the information given that the c o m p a n y is
likely to remain viable for the n e x t t w o years.
Stimuli
An additional four items ( t w o fail, t w o viable)
w e r e a d d e d to the 20 items e m p l o y e d in
e x p e r i m e n t 1. These items w e r e taken from
C h o o & T r o t m a n ( 1 9 9 1 ) w h o s e pilot testing
i n d i c a t e d that the t w o failure items and t w o
viable items w e r e a p p r o x i m a t e l y of equal
strength. The resulting 24 items c o n s i s t e d of
12 fail and 12 viable items. Because s o m e
t r e a t m e n t s r e q u i r e d m o r e viable items, the
d i r e c t i o n s of the 12 failure items w e r e c h a n g e d
to p r o d u c e an additional 12 ( a total of 2 4 ) viable
items.
The following p r o c e d u r e s w e r e used to
form the 8/16 and 4/20 fact sets. First, to
balance the p r e s e n t a t i o n of the fall items
across the t r e a t m e n t s to c o n t r o l for differential
m e m o r a b i l i t y across items, the 12 fall items
w e r e r a n d o m l y assigned to t h r e e g r o u p s of four.
In the 8/16 condition, w e first f o r m e d the t h r e e
possible pairs of the three groups of four fail
items. Each one-third of the subjects then
r e c e i v e d a different pair. In the 4/20 condition,
each one-third of the subjects r e c e i v e d a
different o n e of the t h r e e groups of four fail
items. As a result, each fail item was read by an
equal n u m b e r of subjects w i t h i n each treatment.
Second, to c o m p l e t e the set of viable items, in
the 8/16 treatment, the viable versions of the
four o m i t t e d failure items w e r e a d d e d to
c o m p l e t e the set of 24 items. Likewise, in the
4/20 treatment, the viable versions of the
o m i t t e d eight failure items w e r e a d d e d to
c o m p l e t e the set of 24 items. To c o n t r o l for
differential m e m o r a b i l i t y of the old and n e w
viable items, the analyses w e r e c o n d u c t e d
w i t h o u t the n e w viable items.
Subjects
The subjects w e r e 83 auditors from two
Australian "Big 6" audit firms, o t h e r than that
p r o v i d i n g participants in e x p e r i m e n t 1. The
average e x p e r i e n c e level was 4.2 years. The
subjects c o n s i s t e d of 14 managers, 63 seniors,
and six e x p e r i e n c e d staff accountants. All
subjects e x c e p t nine w e r e attending o n e of the
firm's training sessions. The remaining nine
c o m p l e t e d the task as a single g r o u p in
the training r o o m of their office. Subjects
w e r e r a n d o m l y allocated to each of the t h r e e
treatments.
Procedure
The same p r o c e d u r e followed in the r e v i e w e r
t r e a t m e n t in e x p e r i m e n t 1 was e m p l o y e d with
t w o exceptions. First, after c o m p l e t i n g the
recall, subjects w e r e asked to make a j u d g m e n t
on w h e t h e r the c o m p a n y w o u l d m o r e likely
remain viable/fall w i t h i n the n e x t t w o years and
the p r o b a b i l i t y that the c o m p a n y w o u l d fail
within t w o years on an 11-point scale ranging
from 0 to 1.0. Second, the d i s t r a c t o r task was
s h o r t e n e d firom 30 to 20 minutes to hold total
time to the allotted hour.
Results
Each subject's recall p r o t o c o l was s c o r e d in
the same m a n n e r as in e x p e r i m e n t 1. Hypothesis
 REVIEW PROCESS 571

26

24

22

20

% Fail- % Viable 18

16

14

12

10
12 - 12 8 - 16 4 - 20
N u m b e r Fail a n d V i a b l e I t e m s Presented

Fig. 2. Effect of set size on reviewer (all told viable).

2 p r e d i c t s that as t h e p r o p o r t i o n o f i n c o n s i s t e n t ( t = 2.07, p = 0.04), i n d i c a t i n g an a s s o c i a t i o n

facts in an i n f o r m a t i o n s e t d e c r e a s e s , r e v i e w e r s ' of recall with subsequent judgment.
r e c a l l o f i n c o n s i s t e n t facts r e l a t i v e to c o n s i s t e n t
facts will i n c r e a s e . This h y p o t h e s i s c a n b e t e s t e d
Discussion
b y c o m p a r i n g t h e p r o p o r t i o n of failure i t e m s
D e c r e a s e s in t h e a m o u n t o f i n c o n s i s t e n t
recalled minus the proportion of viable items
e v i d e n c e a p p e a r to i n c r e a s e its n o v e l t y a n d
r e c a l l e d b e t w e e n t h e s u b j e c t s in t h e t h r e e set
r e s u l t in i n c r e a s e d a t t e n t i o n to a n d subse-
size conditions. H y p o t h e s i s 2 restated in c o n t r a s t
q u e n t r e c a l l o f i n c o n s i s t e n t items. This finding
f o r m is p r e s e n t e d in e q u a t i o n 3.
e s t a b l i s h e s t h e g e n e r a l i t y o f t h e r e v i e w e r finding
(%Fail - %Viable)ix/t2 < (%Fail -- %Viable)a/t6 to e n v i r o n m e n t s w h e r e f e w e r i n c o n s i s t e n t d a t a
< (% Fail - %Viable)4/2o (3) i t e m s a r e in e v i d e n c e . It also s u g g e s t s that a n y
b i a s e s against r e c o r d i n g i n c o n s i s t e n t e v i d e n c e
This c o n t r a s t w o u l d p r o d u c e a m a i n effect o f
in t h e w o r k p a p e r s will b e offset in p a r t b y t h e
set size in an ANOVA w h i c h w a s significant
i n c r e a s e d a t t e n t i o n g i v e n to t h e r e m a i n i n g
( F = 3.09, p = 0.05). T h e J o n c k h e e r e - T e r p s t r a
i n c o n s i s t e n t items. H o w e v e r , this effect w o u l d
t e s t ( H o l l a n d e r & Wolfe, 1 9 7 3 ) for o r d e r e d
o n l y fully offset t h e bias in t h e w o r k p a p e r s if
l o c a t i o n s w h i c h tests t h e specific c o n t r a s t
t h e i n c r e a s e d a t t e n t i o n l e d to f u r t h e r s e a r c h for
e x p r e s s e d in e q u a t i o n ( 3 ) w a s also significant
t h e o m i t t e d data.
( p = 0.02), s u p p o r t i n g H y p o t h e s i s 2. T h e m e a n
d i f f e r e n c e s in p r o p o r t i o n s a r e p r e s e n t e d in
Fig. 2. B e t w e e n t h e largest a n d s m a l l e s t set o f
i n c o n s i s t e n t items, r e l a t i v e r e c a l l i n c r e a s e d f r o m GENERAL DISCUSSION
11 t o 2 5 % .
To test w h e t h e r the recall differences r e p o r t e d In g e n e r a l , w e v i e w t h e i n t e r n a l s t r u c t u r e o f
h e r e w o u l d affect t h e r e v i e w e r s ' s u b s e q u e n t an o r g a n i z a t i o n as a r e s p o n s e to t h e c o m p l e x
judgments, the subjects' probability judgments e c o n o m i c a n d b e h a v i o r a l factors facing it. M o r e
w e r e r e g r e s s e d o n t h e i r set size t r e a t m e n t specifically, w e s u g g e s t that t h e o r g a n i z a t i o n a l
(represented by two dummy variables) and h i e r a r c h y a d o p t e d for t h e p e r f o r m a n c e o f t h e
t h e n u m b e r o f fail m i n u s n u m b e r o f v i a b l e a u d i t is a r e s p o n s e , in part, to t h e i n f o r m a t i o n
items recalled (recall tendency). The regression p r o c e s s i n g c a p a b i l i t i e s a n d d e f i c i e n c i e s o f its
c o e f f i c i e n t o n r e c a l l t e n d e n c y w a s significant m e m b e r s . This v i e w p r o v i d e s a basis for
572 R. LIBBYand K. T. TROTMAN

h y p o t h e s i z i n g specific m e c h a n i s m s and resulting t h e r e is u s u a l l y a c a r r y o v e r o f p e r s o n n e l from

benefits o f p a r t i c u l a r d e s i g n e l e m e n t s , t6
It w a s n e c e s s a r y to e s t a b l i s h t h e initial
d e c i s i o n - m a k e r a n d r e v i e w e r effects in e x p e r i -
m e n t 1 b e c a u s e o f t h e lack o f a n y s t u d i e s
in a d e c i s i o n r e v i e w c o n t e x t in a n y d o m a i n a n d
t h e s o m e t i m e s c o n f l i c t i n g findings in w e a k l y
a n a l o g o u s settings. Establishing t h e s e first-
o r d e r effects p r o v i d e s a basis for e x a m i n i n g
i n t e r a c t i o n s w i t h a v a r i e t y o f i m p o r t a n t fea-
tures o f t h e audit e n v i r o n m e n t w h i c h w e r e n o t
i n c o r p o r a t e d in t h e c u r r e n t e x p e r i m e n t s . Experi-
m e n t 2 e x a m i n e d the first o f these interactions:
t h e effect o f t h e size o f t h e set o f inconsistent
i n f o r m a t i o n in t h e w o r k p a p e r s o n t h e r e v i e w e r ' s
b e h a v i o r . It also e s t a b l i s h e d t h e effect o f t h e
reviewer's recall on subsequent judgments.
T h e m a i n l i m i t a t i o n s o f t h e p r e s e n t s t u d y are
as follows. First, o n l y o n e f o r m o f w o r k p a p e r
d o c u m e n t a t i o n w a s used. T h e u s e o f d o c u -
m e n t a t i o n s u c h as r a t i o n a l e m e m o s w h i c h
primarily involve the repetition and recording
o f the consistent items c o u l d actually e x a c e r b a t e
t h e initial d e c i s i o n m a k e r ' s t e n d e n c y to r e m e m b e r
o n l y d e c i s i o n c o n s i s t e n t i n f o r m a t i o n ( s e e also
A n d e r s o n & Sechler, 1986; Ross e t a L , 1977).
This p r o b l e m m a y i n c r e a s e t h e d e m a n d s o n t h e
r e v i e w e r b e c a u s e m i s s i n g - d a t a r e c o g n i t i o n is a
difficult task. Since s u c h o m i s s i o n s i n c r e a s e t h e
c o g n i t i v e difficulty o f t h e r e v i e w e r ' s task, t h e
e f f e c t i v e n e s s o f t h e r e v i e w is likely t o d e c l i n e .
Second, the current study does not consider the
effects o f a s e c o n d i m p o r t a n t i n s t i t u t i o n a l
c h a r a c t e r i s t i c o f t h e a u d i t setting: t h e pre-
p o n d e r a n c e o f r e p e a t a u d i t e n g a g e m e n t s . Since
p r i o r years, o f t e n in m o r e r e s p o n s i b l e p o s i t i o n s
in t h e team, a n d p r i o r y e a r s ' d e c i s i o n s a r e often
f o l l o w e d in s u c c e s s i v e years, it is n o t c l e a r
w h e t h e r r e v i e w e r s in t h e field h a v e t h e fresh
p e r s p e c t i v e that t h e r e v i e w e r s u b j e c t s h a d in
t h e c u r r e n t e x p e r i m e n t s . This effect has implica-
t i o n s for t h r e e issues r e l a t e d to t h e o r g a n i z a t i o n
of review processes: (1) the importance of
r o t a t i o n o f p e r s o n n e l , ( 2 ) t h e effect o f i n t e r i m
e x p o s u r e o f t h e r e v i e w e r to r e l a t e d e v i d e n c e ,
and (3) the effectiveness of "cold" second-
p a r t n e r r e v i e w . Third, t h e r e v i e w e r s u s e d
in this e x p e r i m e n t w e r e at t h e s u p e r v i s o r level.
Alba & H u t c h i n s o n ( 1 9 8 7 ) a n d Higgins & Bargh
( 1 9 8 7 ) s u g g e s t that, b e c a u s e o f t h e i r m o r e w e l l -
d e v e l o p e d m e m o r y structures, m o r e e x p e r i e n c e d
r e v i e w e r s will b e b e t t e r a b l e t o d e t e c t a n d
r e m e m b e r inconsistent information. This suggests
that e x p e r i e n c e c o u l d b e o f benefit to t h e r e v i e w
process, and may provide part of the explanation
for t h e h i e r a r c h i c a l n a t u r e o f r e v i e w . Fourth,
consistent with the psychology literature the
d i s t r a c t o r tasks u s e d in this s t u d y t o c l e a r short-
t e r m m e m o r y w e r e r e l a t i v e l y brief. T h e s t u d y
d i d n o t e x a m i n e t h e effects w h e n t h e t i m e
interval between examining information and
recall o f i n f o r m a t i o n is e x t e n d e d as will b e t h e
case in s o m e a u d i t situations. It is s u g g e s t e d that
each of the above important features of the audit
e n v i r o n m e n t , t h a t w e r e n o t i n c o r p o r a t e d in t h e
c u r r e n t e x p e r i m e n t s , p r o v i d e s o p p o r t u n i t i e s for
future research which examines how these
interactions m a y b e a r d i r e c t l y o n t h e effectiveness
of review processes.

BIBLIOGRAPHY

AICPA, The Auditor's Consideration o f an EntiO~'s Ability to Continue as a Going Concern, AU Section
341 (New York: A1CPA, 1991).
Alba, J. W. & Hutchinson, J., Dimensions of Consumer Expertise,Journal o f Consumer Research (March
1987) pp. 411--454.

16We believe that this provides benefits similar to those gained by the inclusion in agency theory of a reactive human as
opposed to a machine as the focus of control and reporting mechanisms (cf Demski & Kreps, 1982). In both cases,
inclusion of human factors in the model leads to additional insights into the purposes of control mechanisms.
 REVIEW PROCESS 573

Anderson, C. A. & Sechler, E. S., Effects of Explanation and Counter Explanation on the Development
and Use of Social Theories,Journal o f Personality and Social Psychology (January 1986) pp. 24-34.
Ashton, R. A., An Experimental Study o f Internal Control Judgments, Journal o f Accounting Research
(Spring t 9 7 4 ) pp. 143-147.
Ashton, R. A., Kleinmuntz, D. N., Sullivan, J. B. & Tomassini, L. A., Audit Decision Making, in Abdel-khalik,
A. R. & Solomon, I. (eds), Research Opportunities in Auditing: The Second Decade ( Sarasota, FL: AAA,
1988).
Bamber, E. M., Bamber, L. S. & Bylinski, J. H., A Descriptive Study of Audit Managers' Working Paper
Review, Auditing: A Journal o f Practice and Theory (Spring 1988) pp. 137-149.
Belmore, S. M. & Hubbard, M. L., The Role of Advance Expectancies in Person Memory, Journal o f
Personality and Social Psychology (July 1987) pp. 61-70.
Birnberg, J. G. & Shields, M. D., The Role of Attention and Memory in Accounting Decisions, Accounting
Organizations and Society (1984) pp. 365-382.
Bodenausen, G. V., Stereotypic Biases in Social Decision Making and Memory: Testing Process Models of
Stereotype Use,Journal o f Personality and Social Psychology (November 1988) pp. 726--737.
Bonner, S., Experience Effects in Auditing: The Role of Task-specific Knowledge, Accounting Review
(January 1990) pp. 72-92.
Bonner, S. & Pennington, N., Cognitive Process and Knowledge as Determinants of Auditor Expertise,
Journal o f Accounting Literature ( 1991 ) pp. 1-50.
Choo, F. & Trotman, K., The Relationship between Knowledge Structures and Judgments for Experienced
and Inexperienced Auditors, Accounting Review (July 1991 ) pp. 464-485.
Cohen, J., A Coefficient of Agreement for Nominal Scales, Educational a n d Psychological Measurement
(Spring 1960)pp. 37-46.
Defliese, P. L., Jaenicke, H. R., Sullivan, J. D. & Gnospelius, R. A., Montgomery's Auditing ( N e w York:
Ronald Press, 1984).
Dellarosa, D. & Bourne, L E., Decisions and Memory: Differential Retrievability of Consistent and
Contradictory Evidence, Journal o f Verbal Learning and Verbal Behavior ( D e c e m b e r 1984) pp.
669-682.
Deloitte, Ross, Tohmatsu, Audit Manual (Sydney: De|oitte, Ross, Tohmatsu, August 1991 ).
Demski, J. S. & Kreps, D. M., Modeis in Managerial Accounting,Journal o f Accounting Research (Supplement
1982) pp. 117-148.
Emby, C. & Gibbins, M., Good Judgment in Public Accounting: Quality and Justification, Contemporary
Accounting Research ( Spring 1988) pp. 287-313.
Frederick, D., Auditors' Representation and Retrieval of Knowledge in Internal Control Evaluation,
Unpublished Doctoral Dissertation, University of Michigan, Ann Arbor (1986).
Graesser, A. C. & Nakamura, G. V., The Impact of a Schema on Comprehension and Memory, Psychology
o f Learning and Motivation ( 1982 ) pp. 60-109.
Graesser, A. C., Woll, S. B., Kowalski, D. J. & Smith, D. A., Memory of Typical and Atypical Actions in
Scripted Activities, Journal o f Experimental Psychology: H u m a n Learning and Memory (September
1980) pp. 503-515.
Grobstein, M., Loeb, S. E. & Near),, R. D., Auditing.. A Risk Analysi$Approach (Homewood, IL: Irwin, 1985 ).
Hastie, R., Causes and Effects of Causal Attribution J o u r n a l o f Personality and Social Psychology (January
1984) pp. 45-56.
Hastie, R. & Kumar, P. A., Person Memory: Personality Traits as Organizing Principles in Memory for
Behavior, Journal o f Personality and Social Psychology (january 1979) pp. 25--38.
Heiman, V. B., Auditor's Assessments of the Strength of Analytical Review Explanations, Accounting
Review ( O c t o b e r 1990) pp. 875--890.
Higgins, E. T. & Bargh, J. A., ,Social Cognition and Social Perception, Annual Ret~eu, o f Psychology (1987)
pp. 369--425.
Hollander, M. & Wolfe, D. A., Non.parametric Statistical Methods (New York: Wile},, 1973).
Joyce, E. J., Expert Judgment in Audit Program Planning, Journal o f Accounting Research (Supplement
1976) pp. 29-60.
Kahneman, D., Attention and Effort (Englewotxl Cliffs, NJ: Prentice-HaU, 1973).
Kida, T., The Impact of Hypothesis-testing Strategies on Auditors' Use of Judgment Data, Journal o f
Accounting Research ( Spring 1984) pp. 332-340.
Libby, IL, Accounting Ratios and the Prediction of Failure: Some Behavioral Evidence, Journal o f
Accounting Research (Spring 1975) pp. 150-161.
574 R. LIBBY and K. T. TROTMAN

Libby, R., Availability and the Generation of Hypotheses in Analytical Review, Journal of Accounting
Research (Autumn 1985) pp. 648-667.
Libby, R., Trotman, K. & Zimmer, 1., Member Variation, Recognition of Expertise, and Group Performance,
Journal of Applied Psychology (February 1987) pp. 81--87.
Lingle, J. H. & Ostrom, T. M., Retrieval Selectivity in Memory-based Impression Judgment.,,. Journal of
Personality and Social Psychology (January 1979) pp. 180-194.
Lynch, J. G. & Srull, T. K., Memory and Attentional Factors in Consumer Choice: Concepts in Research
Methods, Journal of Consumer Research (June 1982) pp. 18-37.
Marchant, G., Analogical Reasoning and Hypothesis Generation in Auditing, Accounting Review (July 1989)
pp. 50(O513.
Mautz, R. K. & Sharaf, H. A., The Philosophy ofAuditing (Sarasota, FL: AAA, 1961).
Moeckel, C. & Plumlee, R. D., Auditors' Confidence in Accurate and Inaccurate Recognition of Audit
Evidence, Accounting Review (October 1989) pp. 665--679.
Peat, Marwick, Mitchell & Co., Research Opportunities in Auditing (New York: Peat, Marwick, Mitchell
& Co., 1976).
Plumlee, R. D., The Standard of Objectivity for Internal Auditors: Memory and Bias Effects, Journal of
Accounting Research (Autumn 1985) pp. 683-699.
Ross, l., Lepper, M. R., Strack, F. & Steinmetz, J., Social Explanation and Social Expectation: Effects of
Real and Hypothetical Explanations on Subjective Likelihood, Journal of Personality and Social
Psycho/ogy (November 1977) pp. 817-829.
Simon, H. A., Administrative Behavior (New Yorle Macmillan, 1947).
Solomon, I., Multi-auditor Judgment/Decision-makingResearch,Journal of Accounting Literature (1987)
pp. 1-25.
Srull, T. K., Person Memory: Some Tests of Associative Storage and Retrieval Models, Journal of
Experimental Psychology: Human Learning and Memory (November 1981 ) pp. 440-463.
Srull, T. K., LicbtenStein, M. & Rothbart, M., Associative Storage and Retrieval Processes in Person Memory,
Journal of Experimental Psychology: Learnin~ Memory and Cognition ( April 1985 ) pp. 316-345.
Tetlock, P. E., Accountability and the Perseverance of First Impressions, Social Psychology Quarterly
(December 1983)pp. 285-292.
Timme, V., Corkill, A. & Glover, J. A., Decisions Following Reading and Bias in Recall, Bulletin of the
Psychonomic Society (March 1987) pp. 77-78.
Trotman, K. T., The Review Process and the Accuracy of Auditor Judgments, Journal of Accounting
Research (Autumn 1985) pp. 740-752.
Trotrnan, K. T. & Sng, J., The Effect of Hypothesis Framing, Prior Expectations and Cue Diagnosticity on
Auditors' Information Choices,Accountin~ Organizations andSociety (December 1989) pp. 565-576.
Trotman, I~ T. & Yetton, P. W., The Effect of the Review Process on Auditor Judgments, Journal of
Accounting Research (Spring 1985) pp. 256-267.
Weber, IL, Some Characteristics of the Free Recall of Computer Controls by EDP Auditors, Journal of
Accounting Research (Spring 1980)pp. 214-241.