CHAPTER
Evaluating System
Effectiveness
Chapter Outing
Muliple-Choice Questions
rerciss and Cases
Anse f9 Mutifle-Choice Questions
References
888INTRODUCTION
it be subjected 10.8
After a system has beer operational for some
fol
postimplementation review. A postimplementation reve
can be used to determine how well the system
eto be continued, whether it should be modi
wo Meter. Seeond, the review can be used to evaluate the adequacy of
‘fom development proses wed to design and implement the syste. nih
yh evaluation, sStem development stndards might be changed or system
oe tSoment personnel might be provided with feedback on how fo improve
the tasks they undertake.VALUATION PROCESS
OVERVIEW OF THE EFI
are maintained routinely by an organization.
4. Obtain ex ante values for measures. When audivors have identified the me
tem Unless these ex ate values were collected prior to the implementation of
the system, itcould be difficult to obtain them after the system is operationPTER 22 Evalusing System Enecivenees 893 W
tunately, these
have made some proutess, much
obiain a thorough u
res and the fa
mand the qv
fel whether users perceive the system 10 be both useful and
tema
affects their performance
performance of the organiza
ion with the syste
: etwen satisfaction ond individu aipact To the extent users are mone
: stip yh ystems ely Te havea greater effect on ther, Smal,
iEVALUATING SYS}
‘HAPTER 2
wring Syne ENectveness 885
Me iso associated with software effectiveness. To
assis auditors (0 make a jadgment about software etfectvenes, they should
‘Rmine fou attributes ofthe software used to support the system:
4. History of eat maintenance The bistory of program repair maintenance i
cates the quality of progr’ logic, Recall that repair maintenance is ar-
die out oootteet logic errs Extensive repair maintenance means inappro-
rate sin, coding resting echaropies have bee used to pment the
poem.896 PARTY Eider
pethaps theSystm Etectvenees 887 a
rent usersmight have
ed by a System,
‘we use 0 understand the world. One person's model of the
sane & another person’. If the infor
it produces be high,
efor thatthe attributes of information qual
jays “objective.” The ratings provided by users can vary depend
1 information represents the ways they conceive ths real
herefore, if they obtain widespread
‘information produced by a system
Uifferent perceptions of the world that the information, system is seeking
to represent
Rea.
ema | “Fpesens vs
sem emoEVALUATING PERCEIVED USEFULNESS
he peret
fet sis Yo use the
for any interviews they con
‘uitors should be mind
ofthe usefulness ofan
Linfavorable effects on
might view the systent
thet job performance,
tion system,
the quality of users’ working
‘negatively even though it has the
EVALUATING PERCEIVED EASE OF USE
Formation sytem,
|) Users peresie that itis easy for them to learn to operate the information
‘system,
2 Users peresive thatcacy might not be a major concer in
tobe elle, For example,
be dealing wih wets who have extensive expeicce with sin
lac sors of computer systems tothe one they ate evaluating. Where sh cond
tions do nt ext, however, autor shoul! be mindful of the impact that
users’ peteptions of computer sel-ficacy might hae onthe effectiveness of
an infomation system.
EVALUATING INFORMATION SYSTEM USE
formation system is useful and that itis easy to use,
sing th
em ih ter (Dave. 198).amount of use that enight be help
ditors could use the Following:
database,
5. Number of reports generated, and
6, Size of chargeout costs for system ue.
‘Voluntary Avolany
oe we
‘Actal —
me: (nose)
ean""22 Evaluating System Eecivances 901
a rin fhe aye
sem to sole probes They tent
ved mph ber roblem
Gineberg (1978)
types of change (level
er
gion he nate ofthe system (Figure
srppor is argument (Ginmerg 17S),
Por Gould be careful herefore, no wo cole that a stem Ha
pig pina eflesnens son he aft nut frequency of
haeA
raetion most likely
users’ task accomp!
ia system improves users’ quality of working li
task accomplishment. For example, if a system allows users greater 0
is to use their abilities on the job, it might improve thet task aesomplsh
. ee oojshment might be
nd wellte y+
pis
1. Namib of urits. ope,
of defecive units eworkd
ms, some manures of task accomplisiment thot audits
5, Number
§ Number ofsles made to od cstomets8 904 PARTY Evidence Eva
‘among employees. In maay
User perfomance1 auditors ry 10 use these faciors to assess the impact ofan information|
feof ts users,
in ponses by persons to the same: example,a
person who has had several jobs is likely to have a higher level of satisfaction
Stith a particular job than a person who is employed (or the first time. The
Time span for measurement also must be chosen. Employees wi
to poe working cand 3 report @ high quality of ifthey
fave high expectations of better things to come. The measures also must be
pavable and nol sabjest to manipulation. Otherwise, responses could be bi-
Jy intentionally by a particular person oF interest group to further their
Saiecause of these problems. ofc approach auditors can adopt (o assess the
quality of working If 0 Use surogate measures—tha i, measures that act
ie aia ,i = 906 PARTY Evidance|
absenteeism can be measured by calclating the cost of wages and
fits of replacement workers, the opportunity cost of profit lost durin
placement proces, and the eost of the personnel department's ime i
with the absenteeism.
‘The major disadvantage of using surrogate measures is
‘not always know why the ity of working life has been lov
‘aninformation system, What atibutes ofthe quality of wor
affecied by the implementation of a system sill must be dete
wise, if the quality of working life has heen lowered, there is little basisrent t0 the specific type of information system they are evs
iments that have been developed for a batch general ledger system, for
might not provide valid-and reliable measures of user salislaction
support system. Although questions about usor satisfaction with
‘withthe former system. .
uditots shoud note, also, thatthe distinetion between information system
‘and other measures ike perceived usefulness and perceived ease of
formation sysiem is not clear-cut, Some of the items included in
rumen for measuting information sytem satisfaction ae sii
Taded in instruments fo measure perceived usefulness and per
‘use of an information sytem. When auditors evaluate the effec-
eine an information sjiemthefefor, they need to be eteumspestebout
the potential overlap between measures:
es a, 908 PARTY Evi
EVALUATING ORGANIZATIONAL IMPACT
Evaluation Approach
Rational goal model22 Coating Sytem Etecveness 908
focus
sais cone!
thug information
‘is communiaen910. PARTY Evidence Evaluation
erupt ight ave
ic meh cae aj ii
different goals among di
vary, and in some ease the goals might even cot
ampi p of dono
work vi
might be more concerned with haw
to reach »
‘the information system
1d cover
and their
Economie Effectiveness
One particular type of elfeetiveness that management is likely to ask auitoes
{6