You are on page 1of 15



Introduction I think, to be perfectly candid, the objective isn't to support managers, buttoeliminatethem.Managementisaveryinteresting,butpossiblynot averyimportantjobinthefuture.Wehaveusedtechnologybrilliantlyto eliminate primarily clerical jobs. Suddenly, the beast is going to turn on thosepeoplewhohavefundedprojectsandfeltasiftheirskillsetwasnot areplaceableskillset.Themanagerialskillset*1 The implementation of Expert Systems (ES) has broadened from the automation of structured and repetitive business processes to systems that require a very high degree of expertise and control, generating fundamental changes in the way businesses operate. As evidenced in Randall Fields statement, some corporations are moving into a domain in which humans are replaced by and subject to the intelligence of a machine. There is a growing sense that "more than beinghelpedbycomputers,companieswilllivebythem,shapingstrategy and structure to fit new information technology"2 While the importance

RandallFields(CEOandPresidentofParkCityGroup,Inc.onthefirmsdevelopmentof ActionManagement,aretailingexpertsystem)

- 114 -

of the technical design and implementation of ES is evidenced by the considerable literature on the subject, there is a lack of comprehensive analysis of the ethical issues that arise from employee interaction with thesesystems. Organizations are today facing new types of ethical dilemmas associated with the information technologies they are increasingly depending upon. An ethical dilemma, according to Abramson, is as a situation in which there are conflicts and tensions concerning the right andthegood,whenchoosingonecourseofactionwillupholdonemore principle while violating another3 Hence, it is important to understand the ethical implications of a gradually more common situation in which manyemployeesareincreasinglydependinguponautomatedsystemsfor decision making purposes. When acting as passive acceptors, employees can practice moral disengagement4 which may have dangerousorganizationalandsocialconsequences.Atthesametime,itis criticaltounderstandthattherearenoethicallyneutralESandidentify potential sources of conflict between the users and the systems moral standards. This paper aims to open the debate on the ethics of employee interaction with Expert Systems. While the importance of the technical design of ES is evidenced by the considerable literature on the subject, press and academic attention has virtually ignored the ethical issues pertaining to the actual users interaction with the system. The core problem arises when employees perceive these intelligent systems as legitimate authorities, disengaging morality from their conduct5 and decreasingtheirautonomy. OnthenatureofExpertSystems

- 115 -

Expert Systems, also called Knowledgebased systems (KBS), are computer programs or systems which emulate the decisionmaking and problem solving abilities of a human expert.6 More formally defined, ES are computer programs which use nonnumerical domainspecific knowledgetosolveproblemswithacompetencecomparablewiththatof human experts.7 By using human knowledge stored in the systems database, ES intend to solve problems that would otherwise require an expert.8 Expertsystemshavethreemaincomponents:aknowledgebase, a reasoning or inference engine, and an interaction interface.9 The knowledgedatabasewillprovidetherawmaterialandwilldefinewhat information the system will use. It is defined as a finite stock of informationthatESwilluseforproblemsolving.Theinferenceenginewill determine how to use that information thus designing the systems reasoning chains. Although all the elements are important for the usability and quality of the system, the knowledge base occupies a fundamental role. Some authors go even further claiming that the knowledge base is what will define the success or failure of an ES.10 Nonetheless, the component that makes the system intelligent is not the database or the ability to store very large amounts of data but the reasoningorinferenceengine.Thisenginewilldeterminetherulesthat the system will utilize for solving a specific problem. One of the most commonproblemsolvingmodelsinvolvesthechainingofIFTHENrulesto formalineofreasoning.Ifthechainingstartsfromasetofconditionsand movestowardsomeconclusion,themethodiscalledforwardchaining.If theconclusionisknownbutthestepstothatconclusionremainunknown, thenthemethodisbackwardchaining.

- 116 -

Because expert systems intend to simulate human intelligent

behavior, ES development occurs within the Artificial Intelligence (AI) domain. The latter is defined as the art of creating machines that perform functions which require intelligence when performed by people11 and it arose from the first AI conference, held at Dartmouth Collegeintheyear1956.OneofthemostprominentresearchersintheAI domain, Robert Feigenbaum, defines expert systems as intelligent computer programs that use knowledge and inference procedures to solve problems that were difficult enough to require significant human expertisefortheirsolutions.12 Currently, ES are able to complete a number of tasks that were traditionally accomplished by human managers. These systems can schedule crews, interview applicants, produce sales projections, manage inventory, administer skill tests and even assess employee knowledge. Hence, ES are taking over more comprehensive functions within the organization,increasingtheirspanofinfluencesignificantly.Formerly,as decision makers, managers applied their own reasoning and ethical standards when resolving a problem. Thus, if ES replace managers as decisionmakers,willthemanagersbeabletoinfluencetheethicsofthe systemsrecommendation? CaseStudy:Mrs.FieldsCookies TheuseofexpertsystemsinMrs.FieldsCookiesoperationsisthe key of their business success.13 14 The ES at Mrs. Fields have been developed by Park City Group owned by her former husband, Randall Fields and is called ROIRetail Operations Management. Its designers

- 117 -

claimthatthissoftwarehasreducedadministrativeworkbyalmost70%, liberatingmanagersfromlesschallengingtasks15 ByanalyzingMrs.Fieldsoperations,itisevidentthatmanagement has not been eliminated but instead has been limited to the domain of execution. This shift has left most of the reasoning and judgment considerationstotheES.Thus,thestoresbusinessdecisionssuchasthe numberofcookiedoughbatchestomix,thecookiesperhourtobakeor thesupplies orders are made bythe interactive,interadjusted system thateachMrs.Fieldretailshophas.Hence,assoonasthestoremanager opens the store, (s)he will enter basic information such as the weather conditions, and theprogram will respond with an outline of the day's schedule and a sales projection.In this way, the companys ES standardizes thegoals and type of operations managementof each of themorethan700stores.16 Mrs. Fields ES also operates as an important Database Asset,allowing the managers and especially the store controllers (at the headquartersinUtah)tomonitorfinancialrecords,lookatthecomputer reports of sales at each store and identify current and potential problems.17 But most importantly, this holistic management information systemallowseachMrs.Fieldsstoretobakethesamequalityofcookies withequaltaste,whetheryouarepurchasetheminNewYorkorLondon. Initially, the implementation of ES was to maintain control over the companys different stores as it expanded its operations. The companyfounder,DebbyFields,wantedtoremainpersonallyinvolvedin the various retail outlets. Her extraordinary amount of control over the stores operations is accomplished by the systems standardization and preprogrammed decision making features. So, day to day, Mrs.

- 118 -

Fieldscanwatch over all of her kitchens worldwide and keeps the chocolatechipsmeltingintheexactwayineveryshop.18 With the automation of the production process and standardizationofdailymyriaddecisions,thecompanyisabletoreduce the average cost for each cookie. This enables them to bake the most optimum number of cookies per hour, taking into account storespecific variables. Thus, the system will ask the store manager to input information related to the weather and special events to project productionandsales.Forinstance,thestoresexpectedsaleswillnotbe projected high when the there is an intense snowstorm in town. The interesting fact is that the program will also recommend marketing explicitmoves,suchasfreesampleswhensalesarelowerthanexpected (basedonthecomputersforecast)Thus,theprogrameliminatesthecost associatedwithhavinganexpertmanagerthatcanforecastdemand. This decisionmaking programis a competitive advantage to the firm,asitsetsthestandardsofqualityandperformanceofalltheirstores, reducing inefficiencies and saving time by nearly eliminating the bureaucratic issues arising frompaperwork and personal interviews. The system will schedule employees timetables and record their work hours.Byenablingthestafftopunchinandout,thetimeclockfacilitates the payroll process.All the applicants considered for raises and promotions are exposed toa variety of multiple choice tests thatcreate an "equalopportunity" workpolicy.When its time to hire new employees, the company has a specific program that compares the potential worker's skills with present workers.Basically, the program filters the most eligible candidates through a number of interactive interviews and tests. The final decision of whether to hire or not the employeeisultimatelybasedontheprogram'scriteria.

- 119 -

MoralDisengagement The importance of preventing the user from becoming a passive acceptororlettingthesystembeinchargeofa decisionisnotalways evident.However,ignoringtheconsequencesoftheseattitudestowards ES can create various sources of conflict. Decision making in an organization cannot be delegated solely to a computer because they cannot take responsibilities or liabilities from the consequences of their conclusions. Hence, businesseshavetoimplementthenecessarybuffers to preventing moral disengagement from occurring. For instance, if a physician misdiagnoses a patient after using an ES, and the latter dies, who is going to be held liable? Before the introduction of ES, the responsibilityremainedvestedinthephysician.19Today,thereisnoclear answer to that issue as other people could be held responsible, such as the expert systems developers and even the experts who provide the information in the first place. Hence, reducing the workers autonomy hasmajorimplicationsasittriggersadangerousdisengagementfromthe consequencesoftheirdecisions.Withthecommonlyacceptedfallacythat thecomputerisalwaysright,workersmightbecomeoverdependenton the decisions suggested by the system. When assuming an intrinsic reliabilityintheESsdecisions,workerswillseparatethemselvesfromthe consequencesoftheiractions. The concept of Moral Disengagement includes a number of psychological processes by which a person can disengage morality from their conduct and thus prevent selfcondemnation when acting against individual ethical standards20 According to Bandura, individuals adopt moral standards in order to selfsanction and control their behavior, allowingforadistinctionbetweenrightandwrong.Overall,heconsiders

- 120 -

this selfmonitoring process as part of a persons development of individual morality. In this selfregulatory practice, he claims that individuals are motivated to act in ways that promote satisfaction and a sense of selfworth; this is because people act according to their moral beliefstopromotepositiveselfsanctions. At the same time, Bandura states that people will refrain from behavinginwaysthatviolatetheirmoralstandardsbecausesuchconduct will bring selfcondemnation Hence, the underlying let motive in Bandurasresearch,istryingtocomprehendhowpeopleactagainsttheir own moral standards but still avoid negative the otherwise applied self sanctions. After many years of research, Bandura identified nine basic mechanisms through which moral selfsanctions were selectively activatedanddisengagedwithinapersonsselfregulatoryprocess.21 In the case of ES, employees exercise of moral control is weakenedbytwomechanismsthatBanduradefinesasDisplacementof ResponsibilityandDiffusionofResponsibility.22Inthefirstcase,when employees regard ES as authorities, and their recommendations as dictates,theirsenseofpersonalaccountabilitywillerodeorinmanycases simply inexistent. This means that the users of the ES do not recognize themselvesaspersonallyresponsiblefortheiractionswhenfollowingthe systems advice or recommendations. This displacement of responsibility hasthepotentialofallowingemployeestoactunethicallywhileclaiming thattheywerejustfollowingordersfromthesystem. With the standardization of business processes and decision making,ESpromotesthediffusionofemployeesresponsibilityovertheir conduct.ThefactthatESprovidesuggestionsorrecommendationsfrom whichallemployeeshavetoactupon,operatesindetrimentofpersonal moralcontrol.Thismeansthatwhenemployeescandiffersomeorallof

- 121 -

their personal accountability to the system, they have the potential to behave more cruelly or inhumanely. Bandura states that Where everyoneisresponsiblenooneisresponsibleidentifyingoneofthemost dangerous unintended consequences of ES implementation in the workplace. LossofAutonomy In the process of facilitating decision making, ES also erode employeesautonomy.Autonomyisthedegreetowhichajobprovides an employee with the discretion and independence to schedule their work and determine how it is to be done.23 To operate smoothly, organizationsneedtolimitemployeesautonomytoacertainextentwith thepurposeofestablishingcommonworkpracticesandrules.However, whenthelimitstoemployeejudgmentaretoohigh,anumberofethical issuesarise. First, autonomy is correlated to responsibility. If employees utilizing the ES are to be held morally responsible for the decisions they make,thesystemmustbedevelopedtoprovidefortheusersautonomy. The underlying principle behind this position is that employees can only beheldaccountablewhenactingasautonomousagents.Hence,whenES excessively restrict or limit the users autonomy, a corresponding employeelimittoresponsibilitywillresult. Although predesigned decision making processes provide consistency and homogeneity to the businesses operations24 they are double edged swords. On the one hand they may reduce and avoid commonhumanerrors.Ontheother,EShavethepotentialtoconvertthe userintoapassiveacceptorofthecomputerssuggestions.Asamatter

- 122 -

of fact, ES are purposely designed to enable employees rapid endorsement with the automated solution25

When employees are

encouraged to perceive the computers reasoning and decisions as the right ones they will diminish their individual reasoning capabilities. AccordingtoAristotles,reasoningiswhatmakesashumansandtheonly waytoreacheudaimoniaorhappiness.26Thus,replacinghumanjudgment for automated reasoning has consequences at the employees personal levelaswell.Whenworkersnoticethattheirreasoninghasnoimportance in the ES decisionmaking process, they numb themselves, becoming somethingotherthanahumanbeing. InMrs.Fieldscase,thestoremanagerwasreducedtoanexecutor of the systems recommendations, (s)he had little to no control over. There is no place for breakthrough innovations and managersencounter avery lowdegree of flexibility and personal "addons"when it comes to hiring, retaining and managing their staff. In many ways, the expert system is the store manager's supervisor. This advanced computer programdictateswhatthemanagershoulddo,everyday,everyhour,and every minute. It is no surprise, with that degree of paternalism builtin theexpertsystems,thatMrs.Fieldshas100%turnoverrateamongstore managers. Generally people, and especially managers, want to make a difference in the place they work at; they want to make important decisionsthatwillinfluenceconsiderablyinthefirm'sstrategy.However, thatispracticallyimpossibleforFieldsstoremanagers,whoarenoteven entitledtodecidewhentomixthebatchesorwhentodiscardthecookie dough.27 It is precisely this automated influence over employees decisions and performance that workers ultimately perceive as restrictions and limitations to their autonomy.28 A study by Klein and

- 123 -

Jiang examining novice decision makers using an expert system showed that although employees had a positive impression of the system, they didnotfeelasenseofaccomplishmentwhenutilizingES.KleinandJiang also found out that employees tend to view ES favorably during their trainingperiod.29Inmanyways,theESbecomestherealstoremanager. ES can indeed provide great advantages for novice practitioners of a particular field who have not yet acquired enough experience to make confident decisions. Oz et Al claims that ES could potentially narrow the knowledge gap between novices and experts.30 However, as employees become more knowledgeable, they are confronted by the fact that the solution was not a product of their own reasoning process but the computersone. Sometimesemployeesarewillingtogiveuptheirautonomywhen acknowledging that they can separate themselves from the systems decisions. This means that a user could be attracted by the level of support that these paternalistic systems offer. ES remove the burden of decidingwhatisimportantandwhatisnot;andCraigetal.suggestthat bytellingtheuserswhattheyshouldbelookingat,thedesignerofthe standardreportsremovestheburdenofdecidingwhatisimportantand whatisnot31 Aspiring to foster morally neutral employees will only strengthen the ES ethical standards. The programmers desire to design anobjectivesystemavoidingtheethicaldimensionswithinthesystems inferenceengineisanunobtainablegoal.Therearenoethicallyneutral ES.32Waldroparguesthatexpertsystemsrepresentvalues,assumptions, and purposes from its designer/s33 Thus, the ethical standards of ES developersarebuiltinthesystemsfunctionsandinferencerules.Atthe same time, there is no universal set of values that could apply to all

- 124 -

employees.Byimposingonewayofthinkingandresolvingaproblem,ES will operate on the basis of specific ethical assumptions and approaches tosolvingamoralproblem.Thissituationcangenerateapotentialconflict with the user as their moral standards could differ from those of the system.34Hence,itisimportanttounderstandthatalthoughEScurrently designed do not include explicit ethical decision models, they are still applyingmoralstandardstotheirinferencemechanisms.35 FinalThoughts ExpertSystemsaredesignedtoenhancehumandecisionmaking and should never replace individual judgment. Organizations need to be aware off to prevent potential organizational conflicts arising from employees autonomy loss and moral disengagement. Although ES have multipleeconomicbenefitsasseeninMrs.Fieldscookiesexample,these systems can also operate as moral inhibitors when interacting with employees.WhendecidingtoimplementanESitisimportanttoconsider thepotentialsourcesforemployeesmoraldisengagementandautonomy loss. Once the roots are identified, organizations will be able to design mechanismstoavoidanaccountabilityvacuumfromemployeesactions.

- 125 -

1.Fields,R.(2004,April9).RandyFieldsInterview:Automating 'Adminitrativia'Decisions.(D.Power,Interviewer) 2.Main,J.(1988,September26).TheWinningOrganization.Fortune,50 52.Newman,M.(1988).Professionalsandexpertsystems:ameetingof minds?ComputersandSociety,18(3),1427. 3.Abramson,M.(1990).EthicsandTechnologicalAdvances:Contributions ofSocialWorkPractice.SocialWorkinHealthCare,15(2),516. 4.Bandura,A.(1999).MoralDisengagementinthePerpetuationof Inhumanities.PersonalityandSocialPsychologyReview,3(3),192209. 5.Ibid4. 6.Durkin,J.(1994).Expertsystems.DesignandDevelopment.London: PrenticeHallInternational. 7.Doran,J.(1988).ExpertsystemsandArcheology:Whatliesahead?InR. &.(eds),ComputerandQuantitativeMethodsinArchaeology(pp.235 241).BARInternationalSeries. 8.Turban,E.,&Aronson,J.(2001).DecisionSupportSystemsand IntelligentSystems.UpperSaddleRiver,NJ:PrenticeHall. 9.Feigenbaum,E.,&Engelmore,R.(1993,May).KnowledgeBased SystemsinJapan.RetrievedfromWorldTechnologyEvaluationCenter: 10.Feigenbaum,E.(1977).TheArtofArtificialIntelligence:I.Fifth InternationalJointConferenceonArtificialIntelligence(pp.10141029). Massachusetts:MassachusettsInstituteofTechnology. 11.Kurzweil,R.(2001).TheAgeofIntelligentMachines:A(Kindof)Turing Test.RetrievedfromKurzweilCyberArtTechnologies: 12.Ibid8. 13.Hopper,M.(1990).RattlingSABREnewwaystocompeteon

- 126 -

information.HarvardBusinessReview,64(4),118. 14.Gill,G.(1995).HighTechhidebound:Casestudiesofinformation technologiesthatinhibitedorganizationallearning. Accounting,ManagementandInformationTechnologies,5(1),4160. 15.Ibid1. 16.Fitzsimmons,J.,&Fitzsimmons,M.(2006).Service Management:Operations,Strategy,InformationTechnology.NewYork: McGrawHill. 17.Ibid16. 18.Ibid16. 19.Newman,M.(1988).ProfessionalsandExpertSystems:AMeetingof Minds?Computers&Society,18,1427. 20.Ibid4. 21.Ibid4. 22.Bandura,A.(2002).Selectivemoraldisengagementintheexcerciseof moralagency.JournalofMoralEducation,31(2),101116. 23.Simmering,M.(2005).Autonomy.Encyclopediaof Management.October42007. <>. 24.Harmon,P.,&King,D.(1985).ExpertSystems:Artificialintelligencein business.NewYork:Wiley&Sons. 25.Holsapple,C.,&Whinston,A.(1985).Managementsupportthrough artificialintelligence.Humansystemsmanagement,163171. 26.Biondi,C.(2005).Aristotle'sMoralExpert:ThePhronimos.InL. Rasmussen,EthicsExpertise:History,ContemporaryPerspectives,And Applications(pp.125132).NewYork:Springer. 27.Ibid13. 28.Argote,L.,&Goodman,P.(1986).Theorganizationalimplicationsof robotics.InD.(.Davis,Managingtechnologicalinnovation: Organizationalstrategiesforimplementingadvancedtechnologies(pp. 127153).SanFrancisco:JosseyBass.

- 127 -

29.Klein,G.,&Jiang,J.(1999).Userperceptionofexpertsystemadvice. JournalofSystemsandSoftware,48(2),155161. 30.Oz,E.,Fedorowicz,J.,&Stapleton,T.(1993).Improvingquality,speed andconfidenceindecisionmaking:measuringexpertsystemsbenefits. InformationandManagement,24(2),7182. 31.Craig,R.,Berkovich,D.,&Vivona,J.(1999).MicrosoftData Warehousing:BuildingDistributedDecisionSupportSystems.NewYork: Wiley. 32.Carlson,J.,Carlson,D.,&Wadsworth,L.(1999).Ontherelationship betweenDSSdesigncharacteristicsandethicaldecisionmaking.Journal ofManagerialIssues,180197. 33.Waldrop,M.(1987).Manmademinds:Thepromiseofartificial intelligence.NewYork:Walker&Co. 34.Boland,R.(1987).TheInFormationofInformationSystems.InR. Boland,&R.Hirschheim,CriticalIssuesinInformationSystemsResearch (pp.363370).NewYork:Wiley. 35.Chae,B.,Paradice,D.,Courtney,J.,&Cagle,C.(2005).Incorporating anethicalperspectiveintoproblemformulation:implicationsfor decisionssupportsystemdesign.DecisionSupportSystems,40(2),197 212.

- 128 -